While MigratoryData deals with real-time messages, some messages are persisted in memory for a configurable period of time as detailed below.

Snapshot Messages

For each subject X, each server of a MigratoryData cluster maintains a snapshot message which is the most recent message received by the MigratoryData cluster with the property retained on true. This applies for both types of MigratoryData clustering, using Standard Message Delivery and Guaranteed Messaging Delivery.

The following table shows the snapshot message of the subject /Stocks/NYSE/IBM as new messages are received by the MigratoryData cluster.

Time Received Message Snapshot Message
10:12
(first message)		 subject=/Stocks/NYSE/IBM
content=140
retained=false		 No snapshot available
for /Stocks/NYSE/IBM
at this time
10:13 subject=/Stocks/NYSE/IBM
content=150
retained=false		 No snapshot available
for /Stocks/NYSE/IBM
at this time
10:15 subject=/Stocks/NYSE/IBM
content=141
retained=true		 subject=/Stocks/NYSE/IBM
content=141
retained=true
10:25 subject=/Stocks/NYSE/IBM
content=144
retained=false		 subject=/Stocks/NYSE/IBM
content=141
retained=true
10:40 subject=/Stocks/NYSE/IBM
content=142
retained=true		 subject=/Stocks/NYSE/IBM
content=142
retained=true

Whenever a client connects or reconnects to a MigratoryData server within a MigratoryData cluster and subscribes to a specific subject, the server will send the snapshot message of that subject (if one is available) to the client first. After that, the server will immediately send any subsequent real-time messages for that subject to the client as soon as the MigratoryData cluster receives them from publishers.

Additionally, if the REST Client Interface is enabled, you can retrieve a snapshot message for a subject via a HTTP Request as further detailed here.

By default, the snapshot messages are persisted in memory for an indefinite period. However, you can specify a limit on the retention time of the snapshot messages using the SnapshotExpireTime parameter. This feature is especially helpful when new subjects are continuously added to the system, but they are not updated after a certain time. Without a time limit, the snapshot messages of these inactive subjects would accumulate in memory, leading to potential memory issues.

Historical Messages

For MigratoryData clustering using Guaranteed Message Delivery only, each server of the cluster maintains a history of messages for each subject. If a failure is detected by a subscriber, it will automatically reconnect to the cluster and will get the missed messages occurred during the reconnection time from this history of messages. These missed messages will be labeled with recovered message type, see the Messages section.

Moreover, the client API exposes the following method to get a number most recent historical messages prior to getting real-time messages for as subject:

void	subscribeWithHistory(
  List< String > subjects, 
  int numberOfHistoricalMessages
)

Additionally, if the REST Client Interface is enabled, you can retrieve historical messages for a subject via a HTTP Request as further detailed here.

By default, the historical messages of a subject are preserved in memory up to 180 seconds, a time which will suffice for a typical reconnection of a client. However, you can modify this default time using the parameter CacheExpireTime. Additionally, you can limit the number of historical messages for a subject using the parameter MaxCachedMessagesPerSubject.