Scaling out with Xbus

Performance, robustness and scaling out capabilities are essential qualities for a solution which purpose is to interface a lot of applications. To get those capacities, a certain type of protocol and architecture need to be implemented.

In a connection-oriented architecture, if an application delivers data to the bus, one server has to be connected during the whole operation to receive it. A single operation mobilizes resources. What is more, data has to be received in order. Whenever the connection is lost, all the data has to be sent again.

Xbus is a set of microservices. The communications protocol implemented for the microservices is datagram-oriented. It is a stateless protocol. Therefore, the incoming messages can be processed by different servers which are not aware of the route of each message.

The implementation of a datagram-oriented internal architecture made it easier to implement a distributed architecture. It is a complex system made up of a set of servers (the microservices), realizing simple, very specialized tasks. Each microservice has a role of its own. For instance, the broker microservice receives all the Xbus messages and another one brings them to their destination. Xbus also has distributed interfaces with the database used, PostgreSQL. Each microservice is connected to one or several database interfaces.

The choice of a datagram-oriented protocol for microservice communications, of a distributed architecture and of distributed interfaces with the database are of benefit, bringing Xbus:

Scalability : Xbus distributed architecture enables to balance the load related to each microservice, for example to the broker, across several servers, for example across several brokers. That architecture paves the way for a geographical distribution across several datacenters, for instance for an international organization.

Performances and robustness:
1. Xbus datagram-oriented internal architecture enables the messages to be processed by different servers within the bus and received in disorder. Therefore, whenever a server stops, message transmission is not systematically interrupted, and all the messages do not systematically have to be sent again. Whenever there is a server failure, data exchanges quickly resume after the server restarts. This is made easier by the datagram-oriented internal architecture and the fact messages are saved in the database, which enables to get them back whenever you need. This is particularly convenient for asynchronous messaging.
2. Handling sizeable envelopes: the choice of a datagram-oriented internal architecture makes streaming easier: each message can be divided into several pieces sent separately even if the emitter and the bus are disconnected between two deliveries. This improves Xbus performance.

The possibility of implementing interfaces optimized for each use case : Xbus distributed interfaces with the database are flexible to call. The interfaces are use-oriented, that is to say specifically adapted to each use case. For instance, the interface dedicated to storing the envelopes is flow-oriented. This enables it to store very quickly pieces of envelopes that are received in disorder and to send them back in order. It is possible to set use-oriented requirements for each interface, for example guarantees increasing the resilience of the system.

NATS was chosen as Xbus network protocol. This makes the following elements easier :

Actor reconnection after a network failure,
Publish/subscribe protocol,
Load balancing with clusters,

Those features guarantee the bus performance, robustness and resilience.

Xbus is developed in Golang (Go language), which is particularly efficient to manage concurrency. Program units and components are broken down into sub-units and sub-components that perform independently, without need for order. Go makes asynchrony easier, enabling the program components to perform another task while waiting for an answer from a component. The program execution speed is thus increased.

Therefore, Xbus is an efficient, robust, flexible and scalable tool and its architecture makes resilience easier. All those characteristics are included in the Open Source solution.