This chapter describes the core concepts of the MigratoryData server:
- Message
- Publish/Subscribe Model
Messages
A MigratoryData message consists of several pieces of information as follows:
| Property | Description |
|---|---|
subject |
the subject of the message |
content |
the content of the message |
closure |
optional metadata of the message (e.g. the ID of the message) |
retained |
indicate whether or not the message should be/was |
| retained by the cluster | |
compressed |
indicate whether or not the message should be/was compressed |
qos |
indicate the quality of the service of the message, which can be either: |
| - QOS_STANDARD or | |
| - QOS_GUARANTEED | |
type |
the type of the message, which can be one of the following: |
| - snapshot | |
| - update | |
| - historical | |
| - recovered | |
replySubject |
the subject to reply to the received request message |
MigratoryData Client API provides methods to create messages from application-specific data, publish messages, and retrieve the application-specific data from messages when received by the client.
Subjects
A message subject can be used by the subscriber clients to listen for messages with that subject. Also, it can be used by the publisher clients to publish messages with that subject.
A subject is a string of characters that respects a syntax similar to
the Unix absolute paths. It consists of an initial slash (/) character
followed by two or more character strings, called segments, separated
by the single slash (/) character. Within a segment, the slash (/)
character is reserved. Each subject must have two or more segments.
For example, the following character string, composed by the segments
Stocks, NYSE, and IBM, is a valid subject for the MigratoryData
server /Stocks/NYSE/IBM.
Here are some examples of invalid subjects:
| Invalid Subject | Reason |
|---|---|
/Stocks//IBM/BID |
The slash (/) is not allowed in a segment or the 2nd segment is empty |
/Stocks |
The second segment is empty |
Stocks/IBM/BID |
The subject does not start with a slash (/) character |
/Stocks/IBM/BID/ |
The last segment is empty |
/ |
Subject is formed from a single empty segment |
/Stocks |
Subject has only one segment (two or more are required) |
Snapshot Messages
For each subject X, the MigratoryData server maintains a snapshot
message which is the most recent message received by the MigratoryData
having the property retained on true.
The following table shows the snapshot message of the subject /Stocks/NYSE/IBM as new messages are received by the
MigratoryData server.
| Time | Received Message | Snapshot Message |
|---|---|---|
| 10:12 (first message) |
subject=/Stocks/NYSE/IBM content=140 retained=false |
No snapshot available for /Stocks/NYSE/IBM |
| 10:13 | subject=/Stocks/NYSE/IBM content=150 retained=false |
No snapshot available for /Stocks/NYSE/IBM at this time |
| 10:15 | subject=/Stocks/NYSE/IBM content=141 retained=true |
subject=/Stocks/NYSE/IBM content=141 retained=true |
| 10:25 | subject=/Stocks/NYSE/IBM content=144 retained=false |
subject=/Stocks/NYSE/IBM content=141 retained=true |
| 10:40 | subject=/Stocks/NYSE/IBM content=142 retained=true |
subject=/Stocks/NYSE/IBM content=142 retained=true |
When a client subscribes to a subject, the MigratoryData server first sends the snapshot message of that subject (if available) to that client, then it sends the subsequent real-time messages for that subject as they are received by the MigratoryData server from the publisher clients.
Retrieve Snapshots via HTTP Requests
You can also get the snapshot message of a subject from the
MigratoryData server via a simple HTTP request. Supposing the
MigratoryData server is deployed at
https://push.example.com, then you can retrieve the
snapshot message of a particular subject X via the following HTTP
request provided that no entitlement rule is configured:
https://push.example.com/snapshot?subject=X
Otherwise, if the entitlement/authorization feature is enabled for the MigratoryData server, then you should also include the entitlement token in the HTTP request. Supposing the entitlement token of the user is U and the user identified by the token U is allowed to subscribe to the subject X, then, to retrieve the snapshot of the subject X, use:
https://push.example.com/snapshot?subject=X&token=U
Batching
Batching is the process of collecting several messages together for a period of time or until a total size is reached before sending them in a single I/O operation to a client.
To enable the Batching feature, a period of pre-configured time and/or a
pre-configured size should be configured by the MigratoryData server
using the parameter MaxBatchingTime and MaxBatchingSpace.
Once enabled, MigratoryData will not send every message individually to the client, instead it will send messages in batches, thus MigratoryData will perform a single I/O network operation for a single batch (that contains a number of messages).
Depending on your use case, especially if subjects are systematically updated at a high frequency (multiple messages per second), then batching can optimize the network I/O.
The following diagram shows the circulation of messages without batching.
The following diagram shows the circulation of messages with batching enabled.
Character Encoding
The UTF-8 character encoding is used for all components of MigratoryData messages including for the message content and message subject. Thus, MigratoryData Server is able to handle messages with any international character set including ASCII.
Publish-Subscribe Model
The Publish-Subscribe Model is defined as follows: a client connects to a MigratoryData server and subscribes to a subject X. Depending on whether the subject X is already subscribed by other clients or not, one of the following two situations will happen:
- If X is already subscribed, then the MigratoryData server will send to that client the snapshot message of the subject X (if available). Also, it will send to the client any subsequent message with the subject X received from the publishers
- If X is not subscribed by any other client, then the MigratoryData server will send to the client any message with the subject X once received from the publishers
When the client is not interested anymore by the messages with the subject X, it can unsubscribe from the subject X.
The following diagram shows an example of publish-subscribe interaction. Note that Subscriber 1 which subscribes to the subject A receives only messages with the subject A. It does not receive messages with subject B as it does not subscribe to the subject B.
