This section provides more details about some features of the MigratoryData server.

Security

The security of the MigratoryData server is assured by:

  • Use of the industry standards TLS/SSL for encrypted communication with the clients
  • Use of TLS/SSL encryption and authentication for JMX and HTTP monitoring
  • Configurable list of TLS/SSL ciphers
  • Password protected inter-cluster communication
  • Being configurable to run as a normal non-privileged user
  • Ability to deploy using dual firewall and DMZ policy
  • Use of Entitlement for data access protection
  • Message publication enabled from a configurable list of IP addresses

Communication Ports

MigratoryData can listen for client connections on one or more ports. If the machine hosting the MigratoryData server is multi-homed (i.e. it has multiple IP addresses associated either with multiple network interfaces or with a single network interface but using multiple IP aliases), then the MigratoryData server can be configured to listen on one or more ports of one or more IP addresses of the machine.

Moreover, the ports can be configured to accept encrypted connections via HTTP Secure (https) or WebSocket Secure (wss). Note that MigratoryData Server can be configured to accept normal connections, encrypted connections, or both encrypted and normal connections.

Both protocols HTTP and WebSocket use the same standard port numbers: 80 for normal connections and 443 for encrypted connections.

For production deployment, the recommendation is to configure the MigratoryData server to use encrypted connections. In this way, your data will be securely delivered. Encrypted connections might also help to avoid the interference with certain security solutions.

For example, when using normal connections, certain antivirus software might block the data streaming between the MigratoryData server and a client. It might wrongly interpret the data streaming as a potential security attack. Using encrypted connections, the antivirus software is unable to inspect the data.

MigratoryData Server should be typically configured to accept client connections on a public address, say push.example.com. Ideally, it should be configured to accept only encrypted client connections via the standard https/wss port 443. Thus, its configuration should be as follows:

ListenEncrypted = push.example.com:443

The following diagram shows how both subscribers and publishers securely connect to a single open port of the MigratoryData server.

While it’s perfectly valid and beneficial to use a single network address and a single port to accept all clients, there are setups when the MigratoryData server is deployed in the DMZ and the publisher clients are deployed behind the second firewall of the DMZ to integrate with the backend servers. In this setup, the publishers typically are not allowed to access Internet addresses, thus they will not be allowed to connect to push.example.com:443. For such a setup, a secondary LAN address, say 192.168.1.1, should be configured on the machine hosting the MigratoryData server. For this local address, you can configure any port available to accept connections from publishers, provided however that the port is allowed by the firewall. As in the case of the client port, you can configure the publisher port to accept either normal or encrypted connections.

The following diagram shows the ports used by the MigratoryData server to communicate with its clients and publishers.

A new port (not shown in the diagram) should be opened if you enable the JMX or HTTP monitoring feature, and up to three other ports (not shown in the diagram) are used for inter-cluster communication.

DMZ Deployment ~~~~~~~~~~~~~

The following diagram shows a secure dual firewall DMZ deployment of MigratoryData Server.

Entitlement

The goal of the Entitlement feature is to offer a data control mechanism such that every client will access only messages with the subjects for which it was authorized to subscribe to and will publish messages only for the subjects for which it was authorized to publish on.

The entitlement should be enabled as follows:

  • Configure the parameter Entitlement of the MigratoryData server
  • Use the API call setEntitlementToken() to assign an entitlement token to the client
  • If the Entitlement is Custom, use the MigratoryData Extension API for Entitlement to build an extension which implements your entitlement rules

Monitoring

You can monitor the MigratoryData server using the Java Management Extensions (JMX) technology and/or HTTP requests. You can access the monitoring services, either JMX or HTTP, with or without password authentication, via normal or encrypted connections depending on the configuration of the MigratoryData server.

The indicators which can be monitored are:

  • The number of connected users to the MigratoryData server
  • The number of connections per second established with the MigratoryData server
  • The number of disconnections per second from the MigratoryData server
  • The number of incoming messages per second received from the publishers
  • The number of outgoing messages per second sent to the subscribers
  • The number of incoming bytes per second received from the publishers
  • The number of outgoing bytes per second sent to the subscribers

The following statistics are computed for the parameters above:

  • Maximum
  • Average
  • Standard Deviation

At each moment, the values of the statistics above are available for the following periods of time:

  • Last minute, last 5 minutes, and last 15 minutes
  • Last hour, last 5 hours, and last 15 hours
  • Last day, last 5 days, and last 15 days
  • Last month, last 5 months, and last 15 months

Clustering

You can deploy multiple instances of MigratoryData Server as an active/active cluster to achieve:

  • Fault tolerance with no single point of failure
  • Horizontal scalability through load balancing

Therefore, the MigratoryData solution does not require specialized appliances or sophisticated application delivery network services to achieve load balancing and fault tolerance because all of these are built-in.

There are two clustering modes offering different qualities of service for message delivery:

Clustering Model Server configuration

Standard Message Delivery ClusterDeliveryMode = Standard Guaranteed Message Delivery ClusterDeliveryMode = Guaranteed

To enable Standard Message Delivery, you will need to deploy a cluster of at least two MigratoryData servers. To enable Guaranteed Message Delivery, you will need at least three MigratoryData servers for your cluster.

Both clustering modes offer reliable message delivery and fault tolerance, including automatic client reconnection if the connection between a client and a cluster member goes down or if a cluster member goes down. However, Guaranteed Message Delivery offers a better quality of service as detailed in the Guaranteed Message Delivery section below.

Guaranteed Message Delivery

All entities which communicate with the MigratoryData server use the TCP protocol at the transport layer, which is a reliable protocol. Also, with any of the two clustering modes, the client will automatically reconnect to the cluster and will continue to function even if a sudden failure occurs. This self-healing feature and the use of the reliable TCP protocol offer reliable message delivery for both clustering modes. However, Guaranteed Message Delivery offers something more as follows:

  • With Standard Message Delivery, when a client reconnects to another cluster member after a failure, it will get only the snapshot messages available for its subscribed subjects (as well as any subsequent message for its subscribed subjects)
  • With Guaranteed Message Delivery, when a client reconnects to another cluster member after a failure, it will get not only the snapshot messages available for its subscribed subjects, but also the messages received by the cluster for its subscribed subjects during the fail-over period (as well as any subsequent message for its subscribed subjects)

The following diagram shows an example of fail-over recovery with Standard Message Delivery enabled. Note that when the client reconnects to the server B at 10:12:20, it will only get the snapshot message available for the subject X at that time (which is the latest message for the subject X with the flag retained on true). Therefore, it will not get one of the two messages which occurred during the fail-over recovery period, i.e. the message at 10:12:05.

The following diagram shows an example of fail-over recovery with Guaranteed Message Delivery enabled. Note that when the client reconnects to the server B at 10:12:20, it will get not only the snapshot message available for the subject X at that time, but all messages received during the fail-over recovery period, including the message at 10:12:05.

Elasticity

A MigratoryData cluster is defined by the ClusterMembers parameter as an ordered list of nodes using the same order for each node. To achieve elasticity, you might start with a number of cluster members smaller than the total number of cluster members defined by ClusterMembers. Then, as the load of your cluster increases, you can add one or more cluster members, up to the total number of cluster members you defined. If the load of your cluster decreases, you can remove one or more cluster members, up to the minimum number of members started initially.

Consequently, the cluster can be scaled up and down according to the load of the system. This can be done either manually or by using a feature like autoscaling made available by the automatic deployment systems like Kubernetes.

That being said, there are specific requirements related to the cluster members to be started initially depending on the clustering mode used as detailed below.

Standard Message Delivery

For clustering using Standard Message Delivery, the minimum number of cluster members to be started initially is one. Moreover, this member can be any member of the ordered list of members defining the cluster.

A cluster using Standard Message Delivery can tolerate `N` members down in a cluster with `N + 1` members.

Guaranteed Message Delivery

For clustering using Guaranteed Message Delivery, the cluster members to be started initially should include the seed members. So, the minimum number of members to be started initially should be equal or higher than the number of seed members of the cluster. A seed member is defined as follows. For clusters with three and four members, the first 3 members of the ordered list of members defining the cluster are seed members. For the clusters with five and more members, the first 5 members are the seed members. The number of seed members can be changed to either 3 or 5 with the parameter ClusterSeedMemberCount.

The StatefulSet controller will start the initial cluster members in the order they are defined, therefore the seed members will be part of the cluster members started initially. Also, the autoscaling mechanism will add and remove cluster members in the correct order. For example, the cluster will scale down by removing the latest nodes added to the cluster, and therefore not seed members.

For example, a cluster with 12 members will have by default 5 seed members. So, the first 5 cluster members should be initially started as the minimum number of members. Then, the nodes from six to twelve can be added to the cluster or removed from the cluster anytime according to the load of the cluster. The parameter ClusterSeedMemberCount can be used to decrease the number of seed member to 3, and therefore allowing to start with a smaller number of initial members.

The seed members use a quorum-based algorithm. So, a majority of the seed members must be up at any moment. Therefore, a cluster with `3` seed members can tolerate `1` seed member down, as well as any number of non-seed members. And a cluster with `5` seed members can tolerate `2` seed members down, as well as any number of non-seed members.

In the example above, the cluster of 12 members with the default number of 5 seed members can tolerate 9 members down: 2 seed members and 7 non-seed members.

If the number of seed members is reduced to 3 with the parameter ClusterSeedMemberCount, then the cluster of 12 members can tolerate 10 members down: 1 seed member and 9 non-seed members.

Internals

The paper Reliable Messaging to Millions of Users with MigratoryData explains the internals of the MigratoryData technology and how concepts like replication, in-memory distributed caching, coordinators, subscribers partitioning, subject groups, sequence numbers, epoch numbers, two-layer architecture: I/O and workers, message ordering are used to achieve high vertical scalability, horizontal scalability, clustering, and guaranteed message delivery at scale. The final version of this paper is available in the proceedings of the 18th ACM/IFIP/USENIX Middleware International Conference from ACM.