Building realtime web and mobile apps that scale to millions of users with Flutter and Kafka

Flutter and Kafka are two popular technologies. Flutter is a framework for building multi-platform native apps, including web apps and mobile apps both for Android and iOS, from a single codebase written in the Dart language. Kafka is a distributed event streaming platform for interconnecting the backend systems of a company, typically across a data center network. Kafka is not designed to address the challenges of connecting lots of devices across the public Internet. Therefore, building a realtime web or mobile app with lots of users to communicate with Kafka over the Internet needs a complementary technology. This blog post takes a look at the technologies available to extend Kafka across the Internet, and discusses the challenges faced to scale to millions of users. Finally, we show how to build a realtime sports scores app that scales using Flutter and Kafka.

Read more

MigratoryData Cloud Unveiled

Today MigratoryData unveiled MigratoryData Cloud, a cloud-native edition of its ultra-scalable WebSockets pub/sub message broker that has been powering realtime web and mobile apps with millions of users for over a decade.

Read more

Accelerate Kafka messaging over WebSockets to millions of devices using F5 BIG-IP or Citrix ADC

Integrating Apache Kafka with millions of Internet facing devices is challenging. Typically used to interconnect backend systems, Kafka needs to delegate its messaging over the Internet to another service. WebSockets is the modern standard for implementing realtime messaging over the Internet. In this article, we discuss how to scale a WebSockets service to extend Kafka messaging to and from millions of connected devices. We also show how an Application Delivery Controller (ADC), such as F5 BIG-IP or Citrix ADC, can be used to accelerate Kafka messaging across the Internet.

Read more

Building a Realtime Gamification Feature that Scales to Millions of Devices using MigratoryData, Kafka, and WebSockets: A look at Watch’NPlay Interactive Game

In this post we show how to create an interactive gamification feature that scales. We demonstrate that it is feasible and cost effective to build such kind of interactive features using MigratoryData to engage in realtime with millions of users. To this end, we perform a benchmark study that demonstrates that MigratoryData scales vertically up to one million concurrent users on a single instance in the given scenario, and scales horizontally in a linear fashion, with a shared-nothing clustering architecture, by simply adding more instances to the cluster.

Read more

Connecting Millions of End-User Devices to your Kafka Pipelines over WebSockets

In this article we provide a step-by-step guide to getting started with MigratoryData with its Kafka Native Add-on enabled, an ultra-scalable Websocket publish/subscribe broker which integrates natively with Apache Kafka. Thanks to its huge vertical scalability (1000x more scalable than the C10K problem) and linear horizontal scalability both for publishers and subscribers, MigratoryData can enable bidirectional messaging communication between Apache Kafka and millions of Web and Mobile users or IoT devices - and do so cost-effectively.

Read more

Monitoring MigratoryData with Prometheus and Grafana

We've built a live demo of MigratoryData simulating a realtime stock trading platform. In this article, we provide a step-by-step guide on how to monitor MigratoryData with Prometheus and Grafana, and present the MigratoryData metrics of the live demo as Grafana dashboard charts.

Read more

MigratoryData Integration with Apache Kafka

MigratoryData just released the integration of MigratoryData with Apache Kafka. This is a significant step forward both for Kafka users and MigratoryData users. On one hand, MigratoryData is going to provide to Kafka a scalable extension for Kafka data pipelines across the Web while preserving its key feature like real-time messaging, high availability, guaranteed delivery, message ordering. On the other hand, Kafka is going to provide to MigratoryData a reliable layer for message storage and integration with Kafka data pipelines like stream processing, databases, microservices. In this blog post, we outline the compatibility and complementarity of MigratoryData and Apache Kafka.

Read more

Users go offline, but you can still keep them updated – Use MigratoryData Presence API to Implement your Fallback from Realtime Messaging to Notifications: Push Notifications, Email, SMS, and more

In this paper we present a fault-tolerant mechanism for sending notifications to offline users, to complement MigratoryData’s realtime messaging to online users. Hence, MigratoryData’s unified client API can be used to achieve both realtime messaging to online users and notifications – such as Push Notifications, Email, SMS, and more – to offline users. We exemplify the new mechanism with a chat mobile app for Android and iOS.

Read more

Fast C10M: MigratoryData running on Zing JVM achieves near 1 Gbps messaging to 10 million concurrent users with 15 millisecond consistent latency

In our previous world-record-setting C10M benchmark, we showed that MigratoryData Server can achieve extreme high-scalability by delivering real-time messaging to 10 million concurrent users on a single 1U machine. Here, we show that MigratoryData Server can achieve simultaneously both extreme high-scalability and consistent low latency.

Read more

How MigratoryData solved the C10M problem: 10 Million Concurrent Connections on a Single Commodity Server

The C10M problem relates to the classic C10K Internet scalability problem, which originally occurred in the context of Web servers. The C10K problem consists of successfully supporting 10,000 concurrent connections on a single machine. While the C10K problem is currently solved by certain Web servers, the C10M problem, which consists in supporting 10 million concurrent connections on a single machine, remains a challenging problem not only for Web servers, but for any Internet server in general. In this article, we demonstrate that MigratoryData Server is able to handle 10 million concurrent connections on a single commodity machine. Moreover, it is able to push almost 1 Gbps live data to these 10 million users (each user receiving a 512-byte message per minute) with an average latency of under 100 milliseconds.

Read more

Scaling to 12 Million Concurrent Connections: How MigratoryData Did It

Massive scalability is the biggest challenge we undertake at MigratoryData, a provider of an enterprise publish-subscribe messaging system for building very scalable real-time web and mobile applications. We recently published a blog post demonstrating 12 million concurrent connections with MigratoryData WebSocket Server running on a single 1U server. We are going to share some lessons learned while pushing the boundaries of scalability with MigratoryData WebSocket Server.

Read more

MigratoryData Demonstrates Record-Breaking 8X Higher WebSocket Scalability than Competition

This benchmark shows that MigratoryData achieves 8X higher scalability than the record obtained by the competition in the same benchmark category; reaffirming it is the most scalable WebSocket server. This benchmark result also demonstrates that, using MigratoryData WebSocket Server, it is feasible and affordable to build real-time web applications delivering high volumes of real-time information to a high number of concurrent users.

Read more