18 Jul 2017

[Blog] Load balancing especially for media servers

Hello streaming media enthusiasts! Jaron here, with a new background article. In this blog post, I'll detail the why and how of our load balancing solution, which is currently being beta-tested in some of our clients' production deployments.

Quick primer on load balancing and how it relates to media

The concept of load balancing is not new; not by a long shot. It is probably one of the most researched and experimented-upon topics in computing. So, obviously, the load balancing problem has already been thoroughly solved. There are many different techniques and algorithms, both generic and specific ones.

Media is a very particular type of load, however: it highly benefits from clustering. This means that if you have a media stream, it's beneficial to serve all users for this stream from the same server. Serving the same stream from multiple servers increases the load much more than serving multiple users from a single server. Of course that changes when there are so many users connected that a single server cannot handle the load anymore, as you will be forced to spread those users over multiple servers. Even then, you will want to keep the amount of servers per stream the lowest possible, while spreading the users more or less equally over those servers.

Why is this important?

Most traditional load balancing methods will either spread randomly, evenly, or to whatever server is least loaded. This works, of course. However, it suffers greatly when there is a sudden surge of new viewers coming in. These viewers will either all be sent to the same server, or spread over all servers unnecessarily. The result of this is sub-optimal use of the available servers... and that means higher costs for a lesser experience for your end-users.

Load balancing, MistServer-style

MistServer's load balancing technique is media-aware. The load balancer maintains constant contact with all servers, receiving information on bandwidth use, CPU use, memory use, active streams, viewer counts per stream and bit rates per stream. It uses these numbers to preemptively cluster incoming requests on servers while making predictions on bandwidth use after these users will connect. This last trick in particular allows MistServer to handle surges of new users correctly without overloading any single server. Meanwhile, it constantly adjusts its predictions with new data received from the servers, accounting for dropped users and changes in bandwidth patterns over time.

A little extra

Our load balancer also doubles as an information source for the servers: they can make queries as to what server to best pull a feed from for the lowest latency and highest efficiency. As an extra trick, the load balancer (just like MistServer itself) is fully capable of making any changes to its configuration without needing to be restarted, allowing you to add and remove servers from the list dynamically without affecting your users. Last but not least, the load balancer also provides merged statistics on viewer counts and server health for use in your own monitoring and usage tracking systems.

Want to try it?

As mentioned in the introduction, our load balancer is still in beta. It's fully stable and usable; we just want to collect a little bit more data to further improve its behavior before we release it to everyone. If you are interested in helping us make these final improvements and/or in testing cool new technology, contact us and let us know!

Wrapping up

That was it for this post. Next time Balder will be back with a new subject!