Collaboration Engine architecture - summer 2021 edition

[Legioth]

This post describes key aspects of Collaboration Engine from an architectural perspective. This description is of a visionary kind that attempts to show what Collaboration Engine might be in the future rather than only focusing on its current state.

We are sharing this overview to help the community evaluate how Collaboration Engine can fit into their application and to get feedback on any aspects that we should reconsider.

Purpose

The main goal with Collaboration Engine is to make collaboration universally used in Vaadin applications. It should be possible to create a UI with collaborative features without deep knowledge in areas like concurrent programming, system administration, or distributed systems. Wide adoption of collaborative functionality is also facilitated by enabling common use cases with minimal amount of application code to implement and maintain. It is a secondary priority to enable power users with deep knowledge and a dedicated budget to implement highly sophisticated custom UI and interaction designs.

Use cases

The focus is specifically on collaboration through a UI. This means that parts of the UI are rendered based on a shared state that can be updated in real time from various sources. There might be some technical similarities with persistence or caching technologies through the focus on data and event or message processing through the real time aspect, but Collaboration Engine is not intended to be a generic tool for either of those domains.

Since the shared state is intended for human interaction, the total volume of data handled for a specific use case is typically low enough to not overwhelm the human brain. Acceptable end-to-end update latency is expected to be in the range between 100ms and 1000ms. This isn't a 60 fps multiplayer game and it isn't a data warehouse ingestion pipeline.

Some typical UI-level use case categories are:

• Collaboratively editing a draft copy of a business entity (can in DDD terms be an aggregate, an entity, or an individual value object). The actual business logic of the application is typically based on snapshot versions that can be exported from the collaboratively edited state, but there may also be cases where the collaborative state is the main representation.
• Viewing live data updates without actively editing the data, e.g. a stock ticker, a monitoring dashboard, or a listing of current business entity snapshots. This is technically not "collaboration", but it's still close enough to be meaningful.
• Messaging and event notifications that are not in themselves business data, but that supplement working with the actual business data. (If the primary function of the application itself is communication, then the messages in a chat room can be seen as a business entity or aggregate that is being collaboratively drafted by participants appending messages.)
• Transient presence state that shows who is currently active and what they are currently focused on.

Concurrency model

Business constraints are typically more strict than the underlying data model. This means that there can be logical conflicts in the data even though the data model itself is without conflicts. A familiar example of this is how a git merge without conflicts can still lead to broken code even though both the merged branches were functional.

Collaboration Engine helps the application developer deal with potential logical or physical conflicts caused by concurrent updates by providing a concurrency model that is similar to transactional memory. The way application code accesses shared state is arranged so that changes to the data cannot be observed concurrently with application code that uses the data, but only between invocations and only in association with delivering change events to application code. Behind the scenes, data updates are processed concurrently, but the application code has a view into the data that can be updated separately.

There is special integration with Vaadin Flow that integrates with UI::access to control when data changes are made visible and to deliver the corresponding change events to application code. Subscribing and unsubscribing is handled automatically based on the attached state of an associated Flow component, which means that no special consideration is needed to avoid memory leaks that would be caused by object references captured in listener closures. This enables a familiar programming model where the developer can handle change events for concurrently modified data in the same way that they handle interaction events from the user.

Application code can submit atomic data changes with last-write-wins or compare-and-swap as well as inside an explicit transaction callback that can be automatically retried if there were concurrent changes to any data read by the transaction.

There is by default a separation between writes and reads so that not even the writer can read their own submitted modifications until they are confirmed to be committed. Writes can optionally be made with latency compensation which keeps a local version of the state with the suggested changes immediately available and newly received changes are applied as long as there are no signs of potential conflicts. If a potential conflict is detected, then synchronization with new updates stops until authoritative update ordering can be used to determine the actual state.

Concurrent text editing is a special case where there's a greater need to accept pending changes before they can be confirmed. This case is supported by a sequence CRDT (conflict-free replicated data type) solution such as LSEQ (tempting based on academic merits) or Yjs (tempting because of the wide ecosystem). The eventually consistent model inherent to CRDT is very suitable for the more free-form type of changes performed when editing text, but it might be confusing or even limiting when applied to business entities (especially for developers with a casual view on causality).

Why not CRDT all the way?

CRDT is a way of dealing with potential conflicts through elaborate data operations that cannot lead to data-level conflicts. This in turn makes it possible to avoid data conflicts even if the event log is only eventually consistent (i.e. older changes might suddenly show up and rewrite history).

With an immediately consistent lock, application logic can take that lock into account just by choosing which parts of the code to run. With an eventually consistent lock, every data modification operation that depends on the lock needs to be expressed as such so that other nodes can independently know which operations would no longer be valid if it later turns out that someone else had acquired the lock earlier. This means that application logic would become more verbose since there's more ceremony needed to define how different operations relate to each other.

Another problem with eventual consistency is that every business case that needs an atomic data change needs to be implemented as a custom low-level message if it's necessary to ensure the operation is carried out. If you need to atomically increment a counter, then you need special mechanism for that (e.g. a G-Counter). A pessimistic lock gives certainty that a lock is acquired before performing a sequence of transactional operations so you can just do x = x + 1;. With an optimistic lock, you can with certainty detect whether the operation was accepted and retry if necessary (i.e. a CAS loop). With an eventually consistent lock, you can submit changes that are conditional on the lock being acquired, but you cannot guard against the possibility that the eventual result is that the lock was never acquired.

Data model

The transactional nature of the regular data model makes it prone to contention for high change volumes. To deal with this, the data is partitioned into independent units, named topics. Total ordering of changes is preserved within each topic, but no guarantees are made between different topics. Each topic is created on demand and identified by an arbitrary application-provided string. An existing topic can be explicitly deleted or configured to be deleted after being unused for a certain period of time.

It is desirable to keep topics small to reduce contention and to avoid firing redundant change messages. A good rule of thumb is to follow the same logical structure for topics as for URLs to views in the application, so that the collaborative state shared between multiple persons navigated to https://myapp.com/person/1 would be managed in a topic with the id person/1. Cross-cutting application concerns such as a notification center or a global chat popup might also have their own topics even if they don't have an associated URL. The notification center is a slightly special case since each user would have their own notification center topic with the notifications that have been delivered to them. There might also be a separate shared notification topic for system-wide messages such as upcoming maintenance breaks.

Access to topic data is provided through a topic connection that handles life cycle events and provides a view into a specific version of the shared data. Different topic connection instances for the same topic can be on a different version of the state, but the actual data for each topic is shared within the JVM to avoid duplicating data when there are multiple connections to the same topic.

Shared state in a topic is mainly based on nested lists and maps, similarly to how e.g. JSON or XML is structured. The list abstraction is presented as a linked list where each entry is addressable by a unique id. Offset based access is not provided since the offset is highly volatile for a concurrently updated list. Map or list entries can be marked to be automatically removed when an associated topic connection is disconnected. This can be used to manage state about presence so that it can be automatically cleared even in the case of abrupt disconnects.

Topic data is internally managed as an append-only log of change messages. The map and list support is based on message change processors that incrementally update an internal multiversion state repository that works as a cache to enable data lookups without traversing the full message log. The change processors can be seen as reducers over the stream of messages. There can also be other change processors, e.g. to process the sequence CRDT for concurrent text editing or a stateless change processor used to deliver transient notifications that should only be handled by currently active topic connections.

Collaboration Engine assumes that topic data is not persisted over server restarts. Otherwise, the user would have to deal with data management issues such as backups specifically for Collaboration Engine. Instead, it is assumed that data important to the application is to be persisted in the application's regular database. Various high-level APIs are specifically designed to allow integration with the application's data layer.

Clustering

The concurrent data model is trivially implemented using JVM locks and atomics for using Collaboration Engine on a single JVM (actually a single classloader since static fields are used for coordination). Usage in a cluster with multiple application server nodes requires separate backend support to ensure topic change messages are delivered to all nodes (and in the same order) and that a new node can load the message log for an existing topic.

There are provided integrations for the most common backend types but it is also possible to use a custom integration. Collaboration Engine multiplexes all access to a given topic within a JVM (or actually a classloader) so that the integration logic only needs to deal with a limited number of subscriptions to the actual backend resource.

Collaboration Engine can create a snapshot of topic state and distribute it through the backend. The first time a node accesses an existing topic, it can load the snapshot first and then only need to load individual messages that have been added after the snapshot was created. This can significantly reduce the time needed to access a topic with a long history. It can also help reduce the memory use by the backend infrastructure since messages older than the snapshot can be trimmed from the message log.

Collaboration Engine can be configured to use a local in-memory backend in development mode while using a cluster-ready backend integration in production. The development mode backend can be configured to introduce random latencies to reduce the risk of logic errors in application code that would otherwise only show up in production.

It is possible to use a wide variety of shared infrastructure solutions as a backend without any custom logic running on the shared infrastructure. In this way, Collaboration Engine can be seen as a peer-to-peer system that only relies on dumb pipes. It would even be possible to run in a cluster without relying on any shared infrastructure at all by using an embedded clustering solution such as Akka or Hazelcast IMDG to directly coordinate between the nodes.

As a backend example, a JCache (JSR-107) implementation can be used by creating a new Cache for each topic and implementing the message log as a linked list of cache entries with each entry referencing the generated key used by the next entry. A HEAD key is updated using replace and retried in case of a race. Nodes can listen to new topic messages by registering a cache entry listener that processes updates to the HEAD key.

The backend integration implementation can use various security mechanisms depending on the security level of the underlying infrastructure. This might include obfuscating topic ids to thwart rogue message injection or to provide end-to-end encryption of messages for a backend hosted by a third party.

Collaboration Engine could provide a separately deployable standalone backend server that can be used for simple clusters where no suitable shared infrastructure is available (e.g. only a conventional SQL database without support for listening to changes). Vaadin might also provide a PaaS backend as a simple solution that doesn't require any shared infrastructure.

Access control

Collaboration Engine should be treated in the same way as a regular database when it comes to access control. The application itself has full access and applies its own mechanisms to control how users read and write data. Ownership of data is not automatically recorded, but the application is free to use its own data scheme to record such information as appropriate, similar to typical createdBy columns in a database. High level use case APIs are designed to enable delegating access control checks to a service layer.

This access control model can in some cases lead to complications since it conflicts with the conventional understanding of how data is accessed. As an example, two users with different access levels might be collaborating to edit a business object. The user with higher privileges makes changes to sensitive fields that the other user is not allowed to edit. If the user with lower privileges proceeds to save the collaboratively edited state, then it might trigger access control checks in the service layer that prohibits that user from making changes to the sensitive fields. Working around this requires a more complicated service layer API that isn't just based on passing an entity or POJO to a save() method, or some scheme where the changes are applied stepwise with different authentication contexts.

Use with Fusion has two different access control modes. The simpler approach is that server-side application logic creates a token that grants direct access to a specific topic and this token is passed to the client e.g. as the return value from an endpoint call. The client can then use the token to directly interact with Collaboration Engine without needing any further help from application code on the server. To restrict which parts of the topic can be written by a client, the server logic can provide a token that only grants read access, while requiring that any change requests are performed by making a regular endpoint request to application logic that goes on to submit the change to Collaboration Engine after performing access checks. It could also be possible to route reads through application code to e.g. filter out sensitive data that the client shouldn't have access to, even though that mode of operation might be quite complicated to implement since application code would need some way of filtering individual messages from the topic's change log without breaking any invariants.