MigratoryData Hub introduces a fresh and thrilling feature to the MigratoryData product. Embracing the concepts of API management, which deals with request/response REST APIs, and extending them into the realm of publish/subscribe streaming APIs, MigratoryData Hub enables developers to publish, discover, monitor, and securely connect to real-time streaming APIs. Similar to API management solutions, MigratoryData Hub empowers organizations to build, and possibly monetize, collaborative hubs for internal, partner, or external developers through published streaming APIs, offering similar business benefits but tailored specifically for real-time use cases.
What is a Streaming API?
A streaming API is a logical collection of related endpoints that allow clients to both publish and receive real-time messages. Consequently, there are two communication methods for an endpoint:
| Method | Description |
|---|---|
SUBSCRIBE |
Clients utilize this method to express their interest in receiving real-time messages associated with an endpoint |
PUBLISH |
Clients utilize this method to send messages to an endpoint, feeding it with real-time messages |
Example of a Streaming API
Here is an example of a streaming API named stocks designed to receive in real-time the updated list of the
traded symbols, receive real-time price quotes and news about symbols, and receive and publish comments about news or
other comments:
| Method | Endpoint | Description |
|---|---|---|
SUBSCRIBE |
/stocks/symbols/list |
Receive in real-time the updated list of the traded symbols |
SUBSCRIBE |
/stocks/quote/{symbol} |
Receive real-time price quotes about a traded symbol |
SUBSCRIBE |
/stocks/news/{symbol} |
Receive real-time news about a traded symbol |
SUBSCRIBE PUBLISH |
/stocks/news/comments/* |
Receive and publish real-time comments about news or other comments |
An endpoint follows the syntax of a MigratoryData subject, i.e. a UTF-8 string resembling a Linux absolute file path. This means that any MigratoryData subject can serve as a streaming API endpoint.
Furthermore, we offer support for both single-level and multi-level wildcards in streaming API endpoints. When the last
segment of an endpoint takes the form of {segment} or *, it signifies a single-level wildcard and multi-level
wildcard. Endpoint /stocks/quote/{symbol} is an example of a single-level wildcard. It allows
subscription to subjects such as /stocks/quote/IBM and /stocks/quote/MSFT. However, it does not permit subscription
to /stocks/quote/IBM/bid. To enable this, a multi-level wildcard such as /stocks/quote/* should be utilized.
Streaming API vs REST API
Both Streaming API and REST API deal with endpoints. Moreover, the communication methods of Streaming APIs
SUBSCRIBE and PUBLISH are similar to the communication methods of REST APIs GET
and POST. However, they use totally different communication models which are appropriate for different use
cases. Streaming APIs use an asynchronous publish/subscribe interaction model, while REST APIs use a synchronous
request/reply interaction model.
The following example illustrates the advantages of a Streaming API over a REST API when it comes to scaling and
providing users with a real-time experience. Let’s consider a sports web app that delivers real-time scores over an
endpoint /sports/scores/match to football fans. This diagram shows the communication
interaction using both types of APIs:
During a regular 90-minute football match, there are 180 API calls made using a REST API, whereas only one API call is made using a Streaming API for each user. Assuming there are one million concurrent sports fans connected, this 1:180 ratio means that one million API calls are required with a Streaming API, as opposed to 180 million API calls with a REST API.
Last but not least, the real-time experience offered by the REST API is inferior to that provided by the Streaming API. In this example, the scores are relayed to users with a latency of up to 30 seconds when employing the REST API. However, by utilizing the Streaming API, the latency is reduced to mere milliseconds. This improvement is due to the fact that the client uses the asynchronous publish/subscribe model and remains connected to the Streaming API throughout the duration of the match, utilizing a persistent WebSocket connection.
API Management
While REST APIs are not ideal for real-time use cases, applying their API Management concepts — undoubtedly proven during the last decade — to real-time use cases makes a lot of sense. MigratoryData adopts the API Management concepts as follows:
Therefore, MigratoryData Hub allows MigratoryData’s customers and prospectors to abstract and document the MigratoryData subjects used in their real-time apps as streaming APIs, share those streaming APIs with other in-house developers, design new real-time APIs and apps based on streaming APIs approach, and launch real-time API programs for partner and external developers.
Free Public Preview
MigratoryData Hub is now accessible through a free public preview service at:
https://cloud.migratorydata.com
You can easily sign in using your Gmail or GitHub account, or register for free to begin using MigratoryData’s streaming API management. We encourage you to share your feedback with us, and let us know which features you would like to see added next.
Drinking our own champagne
Our approach is to “drink our own champagne” by utilizing MigratoryData Hub to provide you with a range of real-time demos, higher level constructs, and recipes in a central place and a consistent way, via AsyncAPI-documented streaming APIs , live demos, and source code.