What is ActivityPub?

This was my first question when I joined to MoodleNet team. Now, after studying it deeply for a few weeks, I feel able to give a proper answer. So I thought of writing a blog post, with my goal being to offer a nice introduction to ActivityPub for technical people. If you’re interested in my more general impressions on the fediverse and our approach to building a federated app, I recommend you to read my previous blog post about our goals.

ActivityPub is a protocol for decentralized social networks, which can also be extended to create all kinds of federated apps. So if you are thinking of building one it is very interesting to know how this standard works. It is important to notice it has two different parts:

  • Client to Server API
  • Server to Server API

A service can decide if it wants to implement only one or both of them. This decision just depends on the needs of the project. If you’re a frontend developer, you might use the Client to Server API. You will be focused on the Server to Server API when you only care about the federation part though.

ActivityStreams 2.0

The ActivityPub message format is JSON-LD, which can be used just like regular JSON and we’ll do this for this blog post. The vocabulary used in this protocol is defined by ActivityStreams 2.0 and ActivityPub just adds a couple of requirements more on top of it. In this vocabulary only exists two kind of entities: Link and Object. Here we will focus on Object and its subtypes. An Object has several fields and only two of them are required by ActivityPub: id and type. The most important properties are:

  • type: which defines the type of object. It can be a string or an array if the object implements more than a type. This is mandatory in ActivityPub.
  • id: a unique identifier for the object. For ActivityPub this field should be a URL. The interesting part is that you can retrieve the full object information making a GET request to this URL with the ActivityStreams accept header. This allows us to send just the id instead the full object information in many cases.
  • content: The content or textual representation of the Object, usually an HTML string.
  • name: A simple, human-readable, plain-text name for the object.
  • summary: A natural language summary of the object encoded as HTML.
  • image: A link to an image that represents an object.
  • @context: This is the only necessary property to accomplish with JSON-LD specification. In our examples we just set to "https://www.w3.org/ns/activitystreams".

Natural Language Values

So any object can have those fields. It is interesting that any field likely to be translated, like content, name or summary, can be renamed to a “map variant” to include different languages, ie:

{
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Note",
"id": "https://mycoolserver.com/notes/1",
"content": "This is a translated note"
}

{
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Note",
"id": "https://mycoolserver.com/notes/1",
"contentMap": {
"es": "Esto es una nota traducida",
"en": "This is a translated note"
}
}

“Data Container” Objects

You may have noticed that the type of the previous object is "Note" instead of "Object". A Note is a subtype of object defined by ActivityStreams too, and it is used to represent any short written work. A subtype means that share the same fields, but it may define more if needed.

There are some more defined types like Article, Image, Page, Document, Video, and more. While it may feel strange to not see something like comment or similar, ActivityStreams is very generic and we have to keep this in mind. The recommendation is to use Note because it “Represents a short written work”.
Those objects may be considerated “data container” objects. They don’t define more fields and they are used to transmit information.

Activity

In addition to those “data container” objects, ActivityStream defines Activity objects. The type of those objects may be more intuitive:

  • Create
  • Delete
  • Update
  • Add
  • Remove
  • Join
  • Follow
  • etc

They are the type of actions that you can expect from a social network. Special attention should be paid to the Announce activity, as it used to call attention to particular objects or notifications. It can be understood like a “retweet” on Twitter, or a simple way to share an already created object in your social network.

Activities may have 6 additional properties compared to a simple object:

  • object
  • actor
  • target
  • origin
  • result
  • instrument

Of course, not all of them make sense in every type of activities, so they are optional. So imagine you want to represent that Luke created the previous note, you can write:

{
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Create",
"id": "https://mycoolserver.com/activity/1",
"actor": "https://mycoolserver.com/actor/luke",
"object": "https://mycoolserver.com/note/1"
}

Here, you only used actor and object. The more common ones. You don’t need to use the rest of them if they do not make sense in your activity.

One important thing to keep in mind is that the ids of the objects will not always appear, and the complete specification of them may appear instead. The previous example could be written in the following way, both being completely valid:

{
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Create",
"id": "https://mycoolserver.com/activity/1",
"actor": {
"type": "Person",
"id": "https://mycoolserver.com/actor/luke",
"name": "Luke"
}
"object": {
"type": "Note",
"id": "https://mycoolserver.com/notes/1",
"contentMap": {
"es": "Esto es una nota traducida",
"en": "This is a translated note"
}
}

Actor

If you take a close look at the previous example you will see that Luke is a Person, this is a type of Object we haven’t yet mentioned. Person is one of the 5 types of actors that ActivityStreams defines:

  • Person
  • Group
  • Organization
  • Application
  • Service

Actors are objects that may execute the different activities, this means they can be used as the actor field on an Activity. An actor has two mandatory properties as well:

  • inbox
  • outbox

The previous two properties are mandatory for any actor. The following properties are not but they are recommended to be implemented:

  • following
  • follower
  • liked

Collection

All these five properties are from a core type we haven’t talked about yet: Collections. As you’d expect, a Collection is simply a group of objects. They also have additional properties, ie: totalItems with the number of elements in the Collection or items where you’ll find the objects in the collection.

So this means that the inbox of an actor is just a Collection containing all the activities received by the actor. The outbox contains all the activities sent by the actor. Any time an actor likes an object, this is added to their liked collection.

Obviously, after a while, those collections cannot be managed with just an array. It will be impractical to manage all the followers of a popular person in a simple array. For this reason, we have the last core type of ActivityStreams: the CollectionPage which should be used to paginate a full Collection.

Core types

So those are the 5 core object types that ActivityStreams defines:

  • Object
  • Actor
  • Activity
  • Collection
  • CollectionPage

All of them are also an Object, so they share most of the fields. Once you understand this vocabulary it becomes easier to work with the ActivityPub protocol.

Targeting

A big difference from working with other private social networks is the “Audience Targeting”. All objects can use 5 optional fields for this:

  • to
  • bto
  • cc
  • bcc
  • audience

Not only that, but it is the client application that fills in these fields. It can be just an actor, but maybe is your followers collection or it is just “public”. This can be better understood with the following examples.

Examples

Sending a private message

First of all, messages are always activities in ActivityPub. So if you want to send a private message to Ben, you have to build a Create activity with a Note. It is equally important to set the field to to only Ben’s id, because is a private message between Alyssa and Ben. You don’t want to make it public or to share with your followers. The JSON could be something like the following:

{
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Create",
"id": "https://social.example/alyssa/posts/a29a6843-9feb-4c74-a7f7-081b9c9201d3",
"to": [
"https://chatty.example/ben/"
],
"actor": "https://social.example/alyssa/",
"object": {
"type": "Note",
"id": "https://social.example/alyssa/posts/49e2d03d-b53a-4c4c-a95c-94a6abf45a19",
"attributedTo": "https://social.example/alyssa/",
"to": [
"https://chatty.example/ben/"
],
"content": "Say, did you finish reading that book I lent you?"
}
}

This is a Create activity because, of course, the message didn’t already exist. As you can see, the actor property is just a Link with Alyssa’s ID. Not need to send the full object. Similarly with the property to with Ben’s ID, but in this case in an array.

ActivityPub defines that a client should always create new activities talking to the server where the user’s account resides. In this example, Alyssa account resides at “https://social.example/”. Once the message is received by the server it is persisted so it can be requested at any time, but also, it detects that the recipient of the message does not belong to this server. This means that it’s now the server which should communicate this message to Ben’s server, in this case: “https://chatty.example/ben/”. It will send a post request to Ben’s inbox. It should then be Ben’s server that notifies Ben that he has a new message.

Add an image to an album

In this example we are working with images. The user Alyssa just move an image from her “Camera Roll” album to “My Cat Pictures”. The client application generates the following activity to represent the action that just happened:

{
"@context": "https://www.w3.org/ns/activitystreams",
"id": "https://social.example/alyssa/activities/10239"
"summary": "Alyssa added a picture of her cat to her cat picture collection",
"type": "Add",
"actor": "https://social.example/alyssa",
"to": "https://social.example/alyssa/followers",
"object": {
"type": "Image",
"name": "A picture of my cat",
"id": "http://example.org/img/cat.png"
},
"origin": {
"type": "Collection",
"name": "Camera Roll"
"id": "https://social.example/alyssa/collection/1"
},
"target": {
"type": "Collection",
"name": "My Cat Pictures",
"id": "https://social.example/alyssa/collection/278"
}
}

This activity is public only for the Alyssa’s followers. The client application send this activity to the Alyssa’s server. Now it is the server which has to locate all her followers and make sure that they receive this activity in their inbox. Maybe they are all locals, they resides on the same server, or maybe it has to send this activity to many external servers.

Follow request

In this scenario, Sally asks John to be friends in our application:

{
"@context": "https://www.w3.org/ns/activitystreams",
"id": "http://example.org/connection-requests/123",
"summary": "Sally requested to be a friend of John",
"type": "Offer",
"actor": "https://sally.example.org",
"object": {
"summary": "Sally and John's friendship",
"id": "http://example.org/connections/123",
"type": "Relationship",
"subject": "https://sally.example.org",
"relationship": "http://purl.org/vocab/relationship/friendOf",
"object": "https://john.example.org"
},
"target": "https://john.example.org",
"to": "https://john.example.org",
"cc": "https://www.w3.org/ns/activitystreams#Public"
}

We make this request using the Offer activity type with a Relationship object and with the target John. Now John can accept the offering with:

{
"@context": "https://www.w3.org/ns/activitystreams",
"summary": "John accepted Sally's friend request",
"id": "http://example.org/activities/122",
"type": "Accept",
"actor": "https://john.example.org",
"object": "http://example.org/connection-requests/123",
"inReplyTo": "http://example.org/connection-requests/123",
"context": "http://example.org/connections/123",
"to": "https://sally.example.org/",
"cc": "https://www.w3.org/ns/activitystreams#Public"
}

It is interesting to see that in this application, the creator decided to make public this kind of interactions. To make public an object we have to add to any “targeting property” the value "https://www.w3.org/ns/activitystreams#Public", in this case in the cc property.

Conclusions

It is important to remember that ActivityPub is a protocol. It defines the expected behaviour but never the implementation details behind it. The client sends activities to the server. The server federates the same activities to other servers. The client can fetch for activities and the server should provide them. The way you do it is completely up to you.

There are a lot more things around ActivityPub, here we only scratched the surface. There are more types in the vocabulary, the possibility of extending it with your own types and properties, security recommendations, side effects, client and server responsibilities, etc. The goal of this article was to give a small introduction without getting lost in technical formalities. I hope you enjoyed.


This post was also published in Alex Castaño’s blog.

Photo by Marius Christensen

MoodleNet: more than a technical project


As many of you already know, I have been hired by Moodle to work on the MoodleNet project. Moodle is the world’s most popular learning platform with more than 120 million users around the world. It is used by many universities on every continent. In fact, I used it myself during my time as a student of Computer Engineering. And, of course, it’s an open source project!

The MoodleNet project is a little bit different, but of course, related to the Moodle main project. Our mission is to empower communities of educators to share and learn from each other to improve the quality of education. We are building a new open social media platform for educators, focused on professional development and open content.

For now, I have only worked on the project for two weeks, but I have already noticed big differences from previous clients and jobs. So I’d like to use this post to explain these differences and how they influence the technical aspects of the project.

“Make the world a better place” is more than a slogan

A lot of workplaces write these kind of slogans on their walls. They think it is motivating for their workers. But do you know what is really motivating? Actually doing it. It is easier to feel like that when you write open source code, but we are pushing this mantra even further.

We must have a prototype of the MoodleNet project ready in January. For this very reason, I just wanted to start coding immediately to make that possible. You know, if there is something that you learn in the world of technology, it is “build fast, fail fast”. This is applied almost everywhere for MVPs, proof of concepts, prototypes, etc. However, things are a little different with MoodleNet. We’re not just building for one company, but for a whole community.

I have been learning about ActivityPub, which is an open protocol developed by the W3C for social networks. We are not just implementing this protocol for MoodleNet, we want to create an ActivityPub generic implementation for the community. In this way, the future developers will be able to create their own open social network more easily. How cool is that?

This way, we cannot fail. Even if the MoodleNet project does not achieve all our goals, we’ll leave a legacy and we’ll collaborate with other open social networks. This is very important and brings us to the next point.

Philosophy in practice

We can architect MoodleNet in many possible ways. We could just create a centralized service where people log in and share their content, just like Facebook or Twitter do. This is much easier to code and to maintain. We didn’t choose this path because of our philosophy.

We are not building just a simple project, we are building a more democratic world. Learning is an essential value in any society. We want to make MoodleNet in the same way, and as  democratically as possible.

This is the reason we bet on open and federated social networks, sometimes called just the “Fediverse” (Federation + Universe). This allows interacting with users that are hosted in other servers in the same way you can do it with users which are in the same server.

A great example is the email service. We all use it on a daily basis and it is a federated protocol. When you send an email from your ProtonMail address I can receive it in my Fastmail. And they are not the same server, but it works! Now, it is the same with the social networks, you can follow me from your ProtonSocial server, you can receive my messages from your ProtonSocial server and you can like my posts from your ProtonSocial server. You don’t have to worry about if I’m using FastSocial, or ProtonSocial or anything else.

Our goal is you can set up your own server, so you own your data. When you use Facebook you don’t own your data. If tomorrow Google+ decided to close its social network, all its users would lose their data. They would not be able to keep using even if they want it.

In an open federated social network like MoodleNet, you will have the code and you can install it on your own servers. So, you can always keep using it with all the other users which are using your server, and with all the other users in other servers around the world. You don’t depend on a big organization anymore! You are absolutely free! And all of this without the need to data-mine and sell your data 🙂

Another big problem for the democracy that is solved by the open social networks is censorship. Corporate owners of social networks decide what should and what should not go inside of their platforms. Most of them motivated by economic reasons such as providing shareholder value and you cannot do anything about it.

Well, here, in the fediverse, the rules change. It is more, “your server your rules”. So, if you don’t like a server you can move to another. If in the future they change their rules, you can move to another one again. Finally, if you really need it, you can create your own server only by yourself. You can make censorship a thing of the past.

A Pluriform Social Network

Right now, we go to Youtube if we want to see videos, to Instagram to see photos and Medium to read something. While there is nothing wrong with have multiple accounts in different platforms, it may be a fragmented and frustrating. experience.

Thanks to the ActivityPub protocol, we have many more possibilities – more than could have imagined before I started this project! ActivityPub defines several entities, ie: “actors”, “follower”, “likes”, “notes”, “article”, “videos”, “photos”, etc. If two servers implement ActivityPub, they know how to speak each other even if they are from different platforms!

This can be explained better with an example. Imagine you have an account in Mastodon, which is best described as a kind of “Twitter clone” in the Fediverse and you love the shows of one guy on Peertube (a Youtube clone). The ActivityPub protocol means you can just follow him from your Mastodon account, instead of having to create a separate PeerTube account. When your Mastodon server receives a “video” from Peertube, it may not only notify you, but actually present the message with an embedded video! The same is true of PixelFed (an Instagram clone), if you follow people there, any new photos can appear in your Mastodon timeline. Or imagine a blog which implements ActivityPub. Blog posts can appear as a regular message in Mastodon with the introduction and a link to read more! The same data can be presented in so many different ways! I have given the example of everything appearing in Mastodon here, but of course the reverse is true.

The great thing about Open Source is that you always have a multiple options. Let’s say that the Mastodon interface is not for you. You can use another client, which implements ActivityPub, to read your data and updates. Even more, you can actually use different ones. You can have one that is very useful for community managers, another focus on a very nice UX for reading and the last one is just useful to listen to your favorite podcasts. If you are interested, you can find an alternative interface for Mastodon in Pinafore.

MoodleNet wants to be part of this Fediverse and we are going to support it creating as many tools as possible to encourage people to join us. For this very reason, we are doing the other way around, we are going to build a nice and generic platform for ActivityPub and later build, on top of it, our MoodleNet project.

Work openly

This is something I liked from the very beginning. Even before I was hired! When I was in the selection process, I did a little research about the company and the project. I was surprised they had a blog, and actually, it was regularly updated. So I saw many videos about the UX for the future prototypes. Then I could read about why they chose Elixir for the project and the next steps for the future backend developer, including the open source project they wanted to reuse. In addition,  there were details about frontend technologies, why some decisions were made, diagrams, a lot of interesting stuff.

I didn’t realize but I spent like 3 hours reading content about the project. I feel like I knew everything about it!  It was funny then, in the interview, because I had to explain that I didn’t have questions, not because I wasn’t interested in the job, but because I had already done super research as a result of the team working openly.

What I mean is I felt like I was in the team and I understood why some decisions were made. This is a really important thing and I’ve missed in many open source projects that don’t share this knowledge with the rest of the world. Sometimes I don’t understand why some projects took one direction instead of another. After all, we can learn a lot from other project experiences, technical and not technical things.

Now that I’m on the other side, I see it is not easy at all. For example, I don’t know a lot of stuff about the Fediverse but I have to share with the world that I actually don’t know what I’m doing! Or maybe, I changed my opinion about something and I have to publish in the public issue tracker that my first decision was completely wrong.

It’s not easy to work so openly, especially at the beginning. You’re exposing yourself to the world. On the other hand, I see how the people actually understand your process and they empathize with you. It’s also cool to see how some people, which don’t work in Moodle, are interested in the project and collaborates with you. They give us their perspective and it is very enlightening, to see their perspective at the beginning of building a new project. So it is hard but it’s worth.

Conclusion

So, as you can see, Moodle’s core values affect our development process deeply. It is more difficult to code a federated system that a centralized one, but at the same time, makes it more exciting, and more importantly, we can have a broader impact. We are creating a tool to empowering teachers to share content, impossible a better goal. Besides this, we are collaborating with open source community exposing our code and creating new open source libraries. Equally important, we are promoting technologies and protocols to make the world a better and more democratic place. All of this in just one project. And this is one of the reason because I love this job.

If you are interested in our project you can also collaborate with us, although a simple comment is more than enough.


This post was also published in Alex Castaño’s blog

Photos by Jon Tyson, Clark Tibbs, Ilyass SEDDOUG and “My Life Through A Lens”

Why did we choose Elixir for the backend development of MoodleNet?

Welcome to a new series of technical posts on the MoodleNet blog! These won’t show up on the front page but will be included in the RSS feed. We’ll also post links to them to the MoodleNet Twitter account, and to the moodle.org forum.


Hi everyone, I’m Alex Castaño, the newest member of the MoodleNet team! Mayel de Borniol, our technical architect, is currently on holiday, so I thought I’d get the ball rolling here.

Back in August, Mayel did some research around languages for both frontend and backend development. After talking with people and reading a lot of information, he decided that we should use Elixir for backend development. I am obviously very pleased that he made this decision, as I’ve been working with Elixir for the past few years!

Infrastructure

Some people, both within Moodle HQ and the Moodle community, have asked why we’re using Elixir for MoodleNet. Fortunately, Mayel has written a very detailed wiki page explaining the benefits and drawbacks of Elixir, and in particular using the Phoenix framework. Here are some of the benefits:

  1. Performance – “The Phoenix web framework has been found to perform better than Rails, PHP, and Python. One benchmark showed Phoenix handling over 10x more requests than Rails in a given period. Phoenix was also much more consistent under load – Rails was more prone to have some requests bog down. This can cause a “chain reaction” because Rails apps are configured with a fixed number of application processes, so if some of them are slow, it can mean that others have to wait in line, which dramatically increases the app’s response time.”
  2. Scalability – “The Erlang VM’s model of concurrency is great for multi-core CPUs, but it was created before they existed. Its original purpose was to support concurrency and fault-tolerance via the use of many different machines. This makes it an excellent tool for building systems that can handle more load by simply adding more servers.”
  3. Developer productivity & happiness – “When people argue about programming languages, often the performance card is pulled (“this language is so much faster in these benchmarks”). But performance isn’t the only consideration: while it’s possible to just add more servers, developer time often costs more than servers. More important then is which framework helps write software faster, of a better quality, and which benefits the programmer’s wellbeing.”
  4. Reliability – “While Elixir is a fairly new language, it is mostly a friendly interface to the Erlang virtual machine, which has been used since the 80s to build some of the most reliable systems in the world: telephone systems. It is made for running a complex system with almost no downtime, like having multiple levels of “supervising” processes in a system to reboot parts that have errors. Basically, microservices before microservices were cool.”
  5. Concurrency – “CPUs are no longer getting dramatically faster each year. Instead, we get machines with more cores. Our code can run faster, but only if it can run concurrently – meaning, different bits of code run simultaneously on different cores.”
  6. Simplicity – “For modern web applications (written in languages like PHP or Ruby on Rails) to do everything we require of them, they depend on a lot of other pieces running on the server… In Elixir, on the other hand, it is possible to spin up a nearly limitless number of processes as needed.”
  7. Flexibility – “Using Phoenix and Elixir opens the door to building applications that are not possible with other web frameworks.”
  8. Correctness – “Elixir’s Ecto database library embraces database constraints, with built-in support for adding them, catching constraint violations, and turning them back into friendly user-facing error messages.”

As Mayel points out, there are some downsides to Elixir, but these are mainly to do with its relative newness and potential difficulty in finding developers. However, the MoodleNet team has already solved the second of these problems in finding me!

Read morehttps://docs.moodle.org/dev/MoodleNet/tech/stack