This chapter describes the core concepts of the MigratoryData server:

  • Message
  • Publish/Subscribe Model

Messages

A MigratoryData message consists of several pieces of information as follows:

Property Description
subject the subject of the message
content the content of the message
closure optional metadata of the message (e.g. the ID of the message)
retained indicate whether or not the message should be/was
retained by the cluster
compressed indicate whether or not the message should be/was compressed
qos indicate the quality of the service of the message, which can be either:
- QOS_STANDARD or
- QOS_GUARANTEED
type the type of the message, which can be one of the following:
- snapshot
- update
- historical
- recovered
replySubject the subject to reply to the received request message

MigratoryData Client API provides methods to create messages from application-specific data, publish messages, and retrieve the application-specific data from messages when received by the client.

Subjects

A message subject can be used by the subscriber clients to listen for messages with that subject. Also, it can be used by the publisher clients to publish messages with that subject.

A subject is a string of characters that respects a syntax similar to the Unix absolute paths. It consists of an initial slash (/) character followed by two or more character strings, called segments, separated by the single slash (/) character. Within a segment, the slash (/) character is reserved. Each subject must have two or more segments.

For example, the following character string, composed by the segments Stocks, NYSE, and IBM, is a valid subject for the MigratoryData server /Stocks/NYSE/IBM.

Here are some examples of invalid subjects:

Invalid Subject Reason
/Stocks//IBM/BID The slash (/) is not allowed in a segment or the 2nd segment is empty
/Stocks The second segment is empty
Stocks/IBM/BID The subject does not start with a slash (/) character
/Stocks/IBM/BID/ The last segment is empty
/ Subject is formed from a single empty segment
/Stocks Subject has only one segment (two or more are required)

Snapshot Messages

For each subject X, the MigratoryData server maintains a snapshot message which is the most recent message received by the MigratoryData having the property retained on true.

The following table shows the snapshot message of the subject [/Stocks/NYSE/IBM]{.title-ref} as new messages are received by the MigratoryData server.

Time Received Message Snapshot Message
10:12
(first message)
subject=/Stocks/NYSE/IBM
content=140
retained=false
No snapshot available
for /Stocks/NYSE/IBM
10:13 subject=/Stocks/NYSE/IBM
content=150
retained=false
No snapshot available
for /Stocks/NYSE/IBM
at this time
10:15 subject=/Stocks/NYSE/IBM
content=141
retained=true
subject=/Stocks/NYSE/IBM
content=141
retained=true
10:25 subject=/Stocks/NYSE/IBM
content=144
retained=false
subject=/Stocks/NYSE/IBM
content=141
retained=true
10:40 subject=/Stocks/NYSE/IBM
content=142
retained=true
subject=/Stocks/NYSE/IBM
content=142
retained=true

When a client subscribes to a subject, the MigratoryData server first sends the snapshot message of that subject (if available) to that client, then it sends the subsequent real-time messages for that subject as they are received by the MigratoryData server from the publisher clients.

Retrieve Snapshots via HTTP Requests

You can also get the snapshot message of a subject from the MigratoryData server via a simple HTTP request. Supposing the MigratoryData server is deployed at [https://push.example.com]{.title-ref}, then you can retrieve the snapshot message of a particular subject X via the following HTTP request provided that no entitlement rule is configured:

https://push.example.com/snapshot?subject=X

Otherwise, if the entitlement/authorization feature is enabled for the MigratoryData server, then you should also include the entitlement token in the HTTP request. Supposing the entitlement token of the user is U and the user identified by the token U is allowed to subscribe to the subject X, then, to retrieve the snapshot of the subject X, use:

https://push.example.com/snapshot?subject=X&token=U

Batching

Batching is the process of collecting several messages together for a period of time or until a total size is reached before sending them in a single I/O operation to a client.

To enable the Batching feature, a period of pre-configured time and/or a pre-configured size should be configured by the MigratoryData server using the parameter MaxBatchingTime and MaxBatchingSpace.

Once enabled, MigratoryData will not send every message individually to the client, instead it will send messages in batches, thus MigratoryData will perform a single I/O network operation for a single batch (that contains a number of messages).

Depending on your use case, especially if subjects are systematically updated at a high frequency (multiple messages per second), then batching can optimize the network I/O.

The following diagram shows the circulation of messages without batching.

The following diagram shows the circulation of messages with batching enabled.

Character Encoding

The UTF-8 character encoding is used for all components of MigratoryData messages including for the message content and message subject. Thus, MigratoryData Server is able to handle messages with any international character set including ASCII.

Publish-Subscribe Model

The Publish-Subscribe Model is defined as follows: a client connects to a MigratoryData server and subscribes to a subject X. Depending on whether the subject X is already subscribed by other clients or not, one of the following two situations will happen:

  • If X is already subscribed, then the MigratoryData server will send to that client the snapshot message of the subject X (if available). Also, it will send to the client any subsequent message with the subject X received from the publishers
  • If X is not subscribed by any other client, then the MigratoryData server will send to the client any message with the subject X once received from the publishers

When the client is not interested anymore by the messages with the subject X, it can unsubscribe from the subject X.

The following diagram shows an example of publish-subscribe interaction. Note that Subscriber 1 which subscribes to the subject A receives only messages with the subject A. It does not receive messages with subject B as it does not subscribe to the subject B.