Cluster management considerations
These apply to bare metal, virtualization, hypervisor, containers and other technologies.
With a single-node MongooseIM, one can set up a vertically scalable system, that is a function of the server resources. MongooseIM can scale from hundreds to tens of thousands of concurrent users.
Note that in a single-node MongooseIM, there is no load distribution, and no fallback or failover in case of a failure.
This architecture is suitable for low-scale deployments, such as testing and development environments on embedded devices, personal computers, or servers.
With a dual-node MongooseIM, one can set up a vertically scalable system, that is a function of the servers' resources. We recommend that servers with the same power are used. Both nodes can handle different sets of services, given that these non-MongooseIM services consume roughly the same resources on both servers. In this setup, MongooseIM can scale up to hundred of thousands of concurrent users.
In a dual-node MongooseIM, there is a 50-50 load distribution - there is a possible fallback or failover in case of a node failure. Please keep in mind that to avoid degrading the service the remaining node should be able to handle the full load when necessary.
This setup is applicable to low to mid-scale deployments used f.e. for functional and load testing. We recommend using real dedicated servers, although MongooseIM could run in a cluster mode with low-power machines, such as embedded devices.
This setup provides better fault tolerance and robustness than the single-node but it's not recommended for production environments. The minimum recommended production setup is 3 nodes.
With a multi-node MongooseIM, one can set up a system that is highly scalable both vertically and horizontally and that is still a function of the servers' resources. We recommend that servers with the same power are used. We also recommend that all the nodes handle the same set of services. In this setup, MongooseIM can scale up to tens of millions of concurrent users.
In a multi-node MongooseIM, with
n nodes, there is a
1/n load distribution - there is a possible fallback or failover in case of a node failure.
To avoid degrading the service the remaining nodes should be able to handle
1/(n-1) load when necessary.
This setup fits mid and large-scale deployments, such as production environments. We recommend using real dedicated, powerful servers.
With a multi-datacenter MongooseIM, one can set up a system that is highly scalable both vertically and horizontally. The MongooseIM clusters are simply distributed across continents for local, low-lag client connections, and the clusters are interconnected via high-speed links. In this setup, MongooseIM can scale up to hundreds of millions of concurrent users.
This applies to large and very large-scale deployments.
We advise contacting us in case of such a big deployment.
Setup: reflects the number of nodes in your cluster.
Purpose: is the goal and use of this cluster.
Low-end: number of concurrent users on low-power machines, such as laptops, embedded devices, entry-level cloud or bare metal.
High-end: number of concurrent users on powerful machines, with lots of memory, multi-core CPU, whether in cloud or bare metal.
|Single-node||Functional testing, development||100 to 10k||100k to 500k|
|Dual-node||Low-end system, load testing||1k to 100k||1M to 3M|
|Multi-node||High-end production system||10k to 1M||2M to 10M|
|Multi-datacenter||Very large scale production system||100k to 10M||10M to 100M|
Scalability highly depends on variables such as:
- machine's power, such as memory, CPU, I/O
- the type of concurrent users:
- most iOS apps are not connected in the background, they use APNS to push info to the device
- web clients use websockets, with fallback on BOSH (HTTP long-polling)
- client-side and backend-side REST API
- how much archiving is needed and the latency for storage and querying, which depends a lot on storage backend architecture
- message throughput:
- MUC light
- HTTP notifications (may include queuing systems such as RabbitMQ or Kafka)
- latency of messaging, both real-time and archived messages
To achieve high scalability you have to adjust the configuration of your operating system.
First, set some network related parameters - this is what we use for load testing:
|net.ipv4.tcp_mem||16777216 16777216 16777216|
|net.ipv4.tcp_wmem||4096 87380 16777216|
|net.ipv4.tcp_rmem||4096 87380 16777216|
Then, you have to increase the number of file descriptors allowed for the user running your MongooseIM server process.
In Linux, this is most commonly done in
You should remember, though, that there is a limit to it — you can't increase it above an upper bound which is set by the
fs.file-max kernel parameter.
And there is a limit to a possible increase in
fs.file-max as well — you can't increase it beyond 1048576, which is 2^20 and is set by another kernel parameter,
Once you increase that one, you are good to go.