1 Jun 2017

[Blog] Fantastic protocols and where to stream them

Hello everyone, Balder here. As mentioned by Carina I will talk about streaming protocols and their usage in streaming media. As the list can get quite extensive I will only focus on protocols in common use that handle both video and audio data.

Types of protocols

To keep it simple, a protocol is the method used to send/receive media data. You can divide the protocol in two types: - Stream based protocol - File based protocol

Stream based protocol

Stream based protocols are true streaming protocols. It has two-way communication between the server and the client and maintain an open connection. This allows for faster, more efficient delivery and lower latency. These properties make it the best option, if available. On the downside you need a player that supports the protocol.

File based protocol

File based protocols use media containers to move data from one point to another. The two main methods within file based protocols are progressive delivery and segmented delivery. Progressive delivery has shorter start times and latencies, while segmented delivery has longer start times and latencies.

Progressive delivery

With progressive delivery a single file is either simulated or present and transmitted in one large chunk to the viewer. The advantage of this is that when downloaded it becomes a regular file, the disadvantage is that trick play (seeking, reverse playback, fast forward, etc.) is only possible if the player supports it for the container format being used.

Segmented delivery

With segmented delivery a single multiple smaller files are transmitted to the viewer in a smaller chunk each. The advantage of this is that trick play is possible even without direct support for the container and it is more suited to infinite duration streams, for example live streams.

Short summary per protocol

Streaming protocols

Flash: RTMP

Real-Time Messaging Protocol, also known as RTMP, is used to deliver Flash video to viewers. One of the true streaming protocols. Since Flash is on the decline it’s less used as delivery method to viewers and more as a streaming ingest for streaming platforms.


Real Time Streaming Protocol, also known as RTSP. The oldest streaming protocol, in its prime it was the most used protocol for media delivery. Nowadays it sees most of its use in IoT devices such as cameras and has uses for LAN broadcasts. Internally it uses RTP for delivery. Because of its excellent latency and compatibility with practically all codecs it is still a popular choice.


Web Real-Time Communications, also known as WebRTC. This is a set of browser APIs that make them compatible with secure RTP (SRTP). As such it has all the properties of RTSP with the added bonus of browser compatibility. Because it is still relatively new it hasn’t seen much use yet, but there is a good chance that this will be the most prevalent protocol in the near future.

Progressive file delivery

Flash: Progressive FLV

Flash Video format, also known as FLV. It used to be the most common delivery format for web-based streaming, back when Flash was the only way to get a playing video in a browser. With the standardization of HTML5 and the decline of Flash installations in browsers it is seeing decreasing use.

Progressive MP4

MPEG-4, also more commonly known as MP4. Most modern devices and browsers support MP4 files, which makes it an excellent choice as protocol. The particularities of the format make it relatively high overhead and complicated for live streams, but the excellent compatibility still makes it a popular choice.

MPEG-Transport Stream (MPEG-TS)

Also known as MPEG-TS. It is the standard used for Digital Video Broadcast (DVB), the old-and-proven TV standard. Because it was made for broadcast it is extremely robust and can handle even high levels of packet loss. The downside to this is almost 20% of overhead.


Ogg was one of the first open source and patent unencumbered container formats. Due to the open nature and free usage it has seen some popularity, but mostly as an addition to existing solutions because the compatibility is not wide enough to warrant it being used as the only delivery method.


Also known as MKV. It is a later open source container format that enjoys widespread adoption, mainly because of its excellent compatibility with almost all codecs and subtitle formats in existence. Because of the wide compatibility in codecs it is very unpredictable whether it will play in-browser however.


WebM is a subset of Matroska, simplified for use on the web. Made to solve the browser compatibility issues of Matroska, it plays in almost all modern browsers. The downside is that the codecs are very restricted.

Segmented file delivery

HTTP Smooth Streaming (HSS)

Also known as HSS. Created by Microsoft as adaptive streaming solution for web browsers. The only downside is that it has no support outside of Windows systems, and is even dropped in their latest browser Edge in favor for HLS.

HTTP Live Streaming (HLS)

Also known as HLS. This uses segmented TS files internally and is the streaming protocol developed for iOS devices. Due to the many iOS devices it sees widespread use. The biggest downsides are the high overhead and high latency.

Flash: HTTP Dynamic Streaming (HDS)

HTTP Dynamic Streaming, also known as HDS. This uses segmented F4S (FLV-based) files internally and is the last Flash protocol. It was created as a response to HLS, but never saw widespread use.

Dynamic Adaptive Streaming over HTTP (MPEG-DASH)

More commonly known as MPEG-DASH. It was meant to unify the splintered segmented streaming ecosystem under DASH, instead it standardized all of the existing protocols under a single name. Their current focus is on reducing complexity and latency.

Which protocol should you pick?

If available and an option you should always pick a stream based protocol as these will give you the best results. Sadly, most devices/browsers do not support most stream based protocols. Every device has its own preferred delivery format, which complicates matters.

Some protocols look like they’re supported by every device, so why not just stick to those? Well, every protocol also comes with their own advantages and disadvantages. Below is a table that attempts to clear up the differences:

Protocol Type Platforms Trick play Latency Overhead Video codecs* Audio codecs* Status
RTMP Streaming Flash Yes Low Low H264 AAC, MP3 Legacy
RTSP Streaming Android, native players Yes Low Low Practically all Practically all Legacy
WebRTC Streaming Browsers Yes Low Low H264, VP8, VP9 Opus, AAC, MP3 Active development
FLV Progressive file Flash No Low Medium H264 AAC, MP3 Legacy
MP4 Progressive file Browsers, native players No Low High H264, HEVC, VP8, VP9 AAC, MP3 Maintained
MPEG-TS Progressive file TV, native players No Low Very high H264, HEVC AAC, MP3, Opus Maintained
Ogg Progressive file Browsers, native players No Low Medium Theora Opus, Vorbis Maintained
Matroska Progressive file Native players No Low Low Practically all Practically all Maintained
WebM Progressive file Browsers, native players No Low Low VP8, VP9 Opus, Vorbis Active development
HSS Segmented file Scripted players, Silverlight Yes High High H264, HEVC AAC, MP3 Legacy
HLS Segmented file iOS, Safari, Android, scripted players Yes Very high Very high H264, HEVC AAC, MP3 Active development
HDS Segmented file Flash Yes High Medium H264 AAC, MP3 Legacy
DASH Segmented file Scripted players Yes High Varies Varies Varies Active development

*Only codecs still in common use today are listed.

In the end you can pick your preferred protocol based on the features and support, but to truly reach everyone you will most likely have to settle for a combination of protocols.

Relating all of the above to MistServer, you can see that we are still missing some of the protocols mentioned above. We are adding support for those in the near future. For the next blog post Erik will write about AV1, the upcoming (open) video codec.