From the editor
In the last update, I mentioned that the ongoing ruma-events revamp was impeded by an issue with types from ring which don't implement Clone and PartialEq.
It turns out that there is a reasonable explanation for this, and so it was decided to work around it.
I restructured the ruma-signatures types in question so that they don't contain ring types, but simply construct them and use them as-needed in the relevant methods.
While I was working on ruma-signatures, I decided to fill in the missing functionality—signing and verifying events.
In the process of doing that, I ended up with a significantly revised API for the crate, which has now been released as version 0.5.0.
In a somewhat amusing development, the new ruma-signatures API doesn't include the Signatures and SignatureSet types that were causing the issue in ruma-events in the first place.
As a result, ruma-events is not only unblocked, but doesn't have depend on ruma-signatures at all.
I made a little bit more progress on ruma-events.
There is only one module remaining to be updated to the new API, so we're on the home stretch for a new ruma-events release.
From the editor
Work continues on the major revamp of ruma-events mentioned in the last update.
Only a few modules remain to be converted to the new API.
It's not hard work, but it's a bit tedious, so I've been dragging on getting it done.
There are also a few modules that are somewhat blocked on an issue in ring.
Some of the types in ruma-events contain types from ruma-signatures which don't implement Clone and PartialEq because they contain types from ring which don't.
I want all event types in ruma-events to implement these traits.
I say this issue is "somewhat blocked" because I could always just modify ruma-events-macros to derive these traits conditionally and then just leave them out for the few types in question, but this would be an unfortunate workaround, so I'd prefer to have the issue solved upstream.
Unfortunately, the issue hasn't received a reply from any maintainers since I opened it a few weeks ago.
No hard feelings—that's how open source works sometimes.
Rust at large
The big news in Rust since the last update is that Rust 1.36 was released, and it includes stabilization of the Future trait, one of the long-awaited building blocks for first-class async support in Rust.
As readers probably already know, the biggest reason for Ruma's development hiatus is waiting for async networking in Rust to mature, and this is one of the final pieces of foundational support we've been waiting for.
The remaining pieces are async/await syntax, which is expected in either the next version or the one following it, and finally, waiting for important libraries like Hyper and Tokio, as well as web frameworks, to adopt the new stuff.
I also consider support for impl Trait in traits to be an important feature that is not yet supported, but at this point Ruma doesn't seem to need this, so while it's a major missing component of Rust's async support, it doesn't seem likely to block Ruma development.
From the editor
This week was spent working on a big revamp of ruma-events, the library that defines Rust types for the "events" used in Matrix.
The previous week, I did a pass through the entire library to bring it up to date with version r0.5.0 of the Matrix specification.
After some discussion in #ruma:matrix.org, I decided to make a move towards treating ruma-events as a higher-level library.
Previously, ruma-events has more or less offered Rust types that are exact representations of the JSON structures used by Matrix.
However, by representing events this way, it would be possible for users to easily create values that, while valid JSON, would be invalid events according to the specification.
The way we're approaching this problem is by separating serialization/deserialization of JSON from validation of events.
Rather than directly implementing serde::Deserialize, event types will be converted from a string of JSON data using std::str::FromStr.
This implementation will deserialize the data internally and then validate it, returning either a valid event, or a raw serde_json::Value along with an error message about why the event was invalid.
The latter form is necessary because invalid events will always be present in the Matrix system, created via bugs or other problems in homeserver implementations.
Once an event is in the event graph, other servers will receive these invalid events over federation, and must still persist them and deal with them as they are.
Similar to the new interface around deserialization, the serialized form of an event created by ruma-events may not directly match the structure of the Rust type.
ruma-events will instead expose variants of each event to ensure they can only be created in valid states.
To reduce the boilerplate that this approach necessarily causes (because for many events there will now be two structures: the public one and the private one used for converting to and from JSON), I started a new procedural macro, ruma-events-macros, to make the most common cases require minimal code in ruma-events.
It's been a lot of work, and there's still probably another week or more to go to finish implementing ruma-events using the new approach.
But when it's done, the result will be a Rust library for Matrix events that leaves less room for users to accidentally do the wrong thing, while also supporting interoperation with other Matrix software that isn't so strict.
From the editor
Work is underway to bring Ruma up to date with version r0.5.0 of the Matrix specification.
Starting with the most foundational libraries and working up towards the higher-level ruma-client, this work should be done in the next week or two.
The bulk of the work since the last update has been on ruma-events, adding all of the events that were previously missing, and doing a full pass through existing events to make sure our definitions match the specification.
In addition, community members Florian and Luca identified an issue regarding the use of Rust's u64 and i64 for numeric values and potential interoperability problems with JavaScript.
After some discussion, Ruma team member jplatte created a new crate called js_int to address this.
It has already been integrated with ruma-client-api and ruma-events, and will be used for all integer values in Ruma from now on.
Notable changes to ruma-api
- Released version 0.8.0.
- Simplified the
Error type.
- Updated to the latest ruma-identifiers (0.13).
- Released version 0.5.0
- Generated documentation now includes the names and descriptions of API endpoints.
- Updated to the latest ruma-api (0.5).
- Integrated the new js_int crate to ensure numeric types conform to the requirements stated in the specification.
- Added new events and updated existing events to cover everything in version r0.5.0 of the client-server specification.
Notably, this includes events for end-to-end encryption and push notifications.
- Integrated the new js_int crate to ensure numeric types conform to the requirements stated in the specification.
- Released version 0.13.0
EventId::hostname and EventId::port now return Option because they are only applicable for the original event ID format.- All methods named
opaque_id are now named localpart.
EventId now supports multiple formats according to the different room versions.RoomVersionId was added.DeviceId and a function for generating a DeviceId were added.
- Released version 0.13.1
- Added support for room version 5 with
RoomVersionId::version_5 and RoomVersionId::is_version_5.
Matrix at large
In the last post I mentioned that Matrix 1.0 was coming soon, and it has now happened!
A hearty congratulations to everyone who works on Matrix!
This is a huge milestone.
You can read the details about Matrix 1.0, as well as the new Matrix.org Foundation, over on the Matrix blog: Introducing Matrix 1.0 and the Matrix.org Foundation.
From the editor
Ruma has largely been on hiatus while we wait for async networking in Rust to mature.
We now have a project status page which gives the full story.
Please refer people here if they wonder about the project's activity.
The stabilization of async/await syntax is now on the horizon, so this week I worked on bringing the Ruma homeserver up to date.
I updated all its dependencies to their latest versions, which took a bit of work, primarily due to updating from pre-1.0 Diesel to Diesel 1.4, as well as some breaking changes in ruma-events.
The good news is that the homeserver is now a Rust 2018 edition crate and compiles on stable Rust!
That means that the entire Ruma project is now running on stable Rust.
I made some other relatively minor updates to the other Ruma libraries, adding rustfmt and clippy to their builds and addressing lots of clippy warnings.
Several of the Ruma libraries now have substantially more internal documentation, which should help anyone who wants to work on them in the future.
ruma-client also has some new crate-level documentation which should make it much more friendly to new users.
Matrix at large
The Matrix team is getting ready to release 1.0 versions of the Matrix specifications.
This is great news, as it means things are going to stabilize.
When this happens, we'll be able to bring the various Ruma libraries up to date without having to worry about any more major changes.
We'll also begin implementing the other APIs (e.g. federation) after the 1.0 specs are released.
Rust at large
As noted above, there is now an expected date for the stabilization of async/await syntax.
It's expected to become stable with the release of Rust 1.37 in August.
Once this happens, the Rust ecosystem will begin to adopt the new syntax, as well as the new version of futures provided by the standard library.
Eventually, the dust will settle, and we'll be in a good position to choose a new web framework to replace Iron, which is what the Ruma homeserver currently uses but is no longer an active project.
New contributors