News

13 Sep 2018

[News] Now at IBC2018

Hello everyone! Like last year we are exhibiting at IBC2018 this year. If you happen to attend IBC please don't forget to drop by and say hi - we'd be more than happy to meet you! We'll be available at booth C10...

Hello everyone!

Like last year we are exhibiting at IBC2018 this year. If you happen to attend IBC please don't forget to drop by and say hi - we'd be more than happy to meet you!

We'll be available at booth C10 in Hall 14. Though if you want to talk to a specific person it is a good idea to contact us to schedule a meeting.

I'm also sorry to say that since most of our team will be attending the IBC, some of the more difficult e-mail support questions we receive might be answered a bit slower.

read moreless
23 Aug 2018

[Blog] How to build a Twitch-alike service with MistServer

Hey all! First of all, our apologies for the lack of blog posts recently - we've been very busy with getting the new 2.14 release out to everyone. Expect more posts here so that we can catch back up to...

Hey all! First of all, our apologies for the lack of blog posts recently - we've been very busy with getting the new 2.14 release out to everyone. Expect more posts here so that we can catch back up to our regular posting pace!

Anyway - hi 👋! This is Jaron, not with a technical background article (next time, I promise!) but with a how-to on how you can build your own social streaming service (like Twitch or YouTube live) using MistServer. We have more and more customers running these kind of implementations lately, and I figured it would be a good idea to outline the steps needed for a functional integration for future users.

A social streaming service, usually has several common components:

  • A login system with users
  • The ability to push (usually RTMP) to an "origin" server (e.g. sending your stream to the service)
  • A link between those incoming pushes and the login system (so the service knows which stream belongs to which user)
  • A check to see if a viewer is allowed to watch a specific stream (e.g. paid streams, password-protected streams, age restricted stream, etc)
  • The ability to record streams and play them back later as Video on Demand

Now, MistServer can't help you with the login system - but you probably don't want it to, either. You'll likely already have a login system in place and want to keep that and its existing database. It's not MistServer's job to keep track of your users anyway. The Unix philosophy is to do one thing and do it well, and Mist does streaming; nothing else.

How to support infinite streams without configuring them all

When you're running a social streaming service, you need to support effectively infinite streams. MistServer allows you to configure streams over the API, but that is not ideal: Mist start to slow down after a few hundred streams are configured, and the configuration becomes a mess of old streams.

Luckily, MistServer has a feature that allows you to configure once, and use that stream config infinite times at once: wildcard streams. There's no need to do anything special to activate wildcard mode: all live streams automatically have it enabled. It works by placing a plus symbol (+) behind the stream name, followed by any unique text identifier. For example, if you configured a stream called "test" you could broadcast to the stream "test", but also to "test+1" and "test+2" and "test+foobar". All of them will use the configuration of "test", but use separate buffers and have separate on/off states and can be requested as if they are fully separate streams.

So, a sensible way to set things up is to use for example the name "streams" as stream name, and then put a plus symbol and the username behind it to create the infinite separate streams. For example, user "John" could have the stream "streams+John".

Receiving RTMP streams in the commonly accepted format for social streaming

Usually, social streaming uses RTMP URLs following a format similar to rtmp://example.com/live/streamkey. However, MistServer uses the slightly different format rtmp:example.com/passphrase/streamname. It's inconvenient for users to have to comply with Mist's native RTMP URL format, so it makes sense to tweak the config so they are able to use a more common format instead.

The ideal method for this is using the RTMP_PUSH_REWRITE trigger. This trigger will call an executable/script or retrieve an URL, with as payload the RTMP URL and the IP address of the user attempting to push, before MistServer does any parsing on it whatsoever. Whatever your script or URL returns back to MistServer is then parsed by MistServer as-if it was the actual RTMP URL, and processing continues as normal afterwards. Blanking the returned URL results in the push attempt being rejected and disconnected. Check MistServer's manual (under "Integration", subchapter "Triggers") for the documentation of this trigger.

An example in PHP could look like this:

<?PHP
//Retrieve the data from Mist
$payload = file_get_contents('php://input');
//Split payload into lines
$lines = explode("\n", $payload);
//Now $lines[0] contains the URL, $lines[1] contains the IP address.

//This function is something you would implement to make this trigger script "work"
$user = parseUser($lines[0], $lines[1]);
if ($user != ""){
  echo "rtmp://example.com//streams+".$user;
}else{
  echo ""; //Empty response, to disconnect the user
}
//Take care not to print anything else after the response, not even any newlines! MistServer expects a single line as response and nothing more.

The idea is that the parseUser function looks up the stream key from the RTMP URL in a database of users, and returns the username attached to that stream key. The script then returns the new RTMP URL as rtmp://example.com//streams+USERNAME, effectively allowing the push as well as directing it to a unique stream for the authorized user. Problem solved!

How to know when a user starts/stops broadcasting

This one is pretty easy with triggers as well: the STREAM_BUFFER trigger is ideal for this purpose. The STREAM_BUFFER trigger will go off every time the buffer changes state, meaning that it goes off whenever it fills, empties, goes into "unstable" mode or "stable" mode. Effectively, MistServer will let you know when the stream goes online and offline, but also when the stream settings aren't ideal for the user's connection and when they go back to being good again. All in real-time! Simply set up the trigger and store the user's stream status into your own local database to keep track. Check MistServer's manual (under "Integration", subchapter "Triggers") for the documentation of this trigger.

Access control

Now, you may not want every stream accessible for every user. Limiting this access in any way, is a concept usually referred to as "access control". My colleague Carina already wrote an excellent blog post on this subject last year, and I suggest you give it a read for more on how to set up access control with MistServer.

Recording streams and playing them back later

The last piece of the puzzle: recording and Video on Demand. To record streams, you can use our push functionality. This sounds a little out of place, until you wrap your head around the idea that MistServer considers recording to be a "push to file". A neat little trick is that configuring an automatic push for the stream "stream+" will automatically activate this push for every single wildcard instance of the stream "stream"! Combined with our support for text replacements (detailed in the manual in the chapter 'Target URLs and settings'), you can have this automatically record to file. For example, a nice target URL could be: /mnt/recordings/$wildcard/$datetime.mkv. That URL will sort recordings into folders per username and name the files after the date and time the stream started. This example records in Matroska (MKV) format (more on that format in my next blog post, by the way!), but you could also record in FLV or TS format simply by changing the extension.

If you want to know when a recording has finished, how long it is, and what filename it has... you guessed it, we have a trigger for that purpose too. Specifically, RECORDING_END. This trigger fires off whenever a recording finishes writing to file, and the payload contains all relevant details on the new recording. As with the previous triggers, the manual, under "Integration", subchapter "Triggers", has all relevant details.

There is nothing special you need to do to make the recordings playable through MistServer as well - they can simply be set up like any other VoD stream. But ideally, you'll want to use something similar to what was described in another of our blogposts last year, on how to efficiently access a large media library. Give it a read here, if you're interested.

In conclusion

Hopefully that was all you needed to get started with using MistServer for social streaming! As always, contact us if you have any questions or feedback!

read moreless
22 Aug 2018

[Release] Stable release 2.14 now available!

Hello everyone! Stable release 2.14 of MistServer is now available! The full change log is available here and downloads are here. Our Pro-edition customers with active licenses will receive a new build notification in their e-mail automatically. Here are some highlights: Full MKV/WebM support...

Hello everyone! Stable release 2.14 of MistServer is now available! The full change log is available here and downloads are here. Our Pro-edition customers with active licenses will receive a new build notification in their e-mail automatically.

Here are some highlights:

  • Full MKV/WebM support for both input and output
  • (Pro) MKV recording support
  • Websocket versions of many data feeds, to improve on polling-based techniques
  • (Websocket-based) JSON metadata track input and output
  • RTSP, RTMP and JSON timestamps are automatically synced together when send to the same stream
  • (Pro) Added USER_END trigger which triggers when an access log entry is written
  • Significant improvements to RTSP and HTTP(S) handling (mostly relevant for Pro edition)
  • More precise track selection mechanic
  • Greatly improved overall system stability
  • Many other small fixes/improvements/etc. See changelog for full list!
read moreless
5 Jun 2018

[Blog] Generating a live test stream from a server using command line

Hello everyone! Today I wanted to talk about testing live streams. As you will probably have guessed: in order to truly test a live stream you'll need to be able to give it a live input. In some cases that...

Hello everyone! Today I wanted to talk about testing live streams. As you will probably have guessed: in order to truly test a live stream you'll need to be able to give it a live input. In some cases that might be a bit of a challenge, especially if you only have shell access and no live input available. It's for those situations that we've got a script that uses ffmpeg to generate a live feed which we call videogen. The script itself is made for Linux servers, but you could take the ffmpeg command line and use it for any server able to run ffmpeg.

What is videogen

Videogen is a simple generated live stream for testing live input/playback without the need for an actual live source somewhere. It is built on some of the examples available at the ffmpeg wiki site. It looks like this:

picture of videogen in action

ffmpeg

As you might've suspected, in order to use videogen you'll need ffmpeg. Make sure to have it installed or have the binaries available in order to run videogen.

Installing videogen

Place the videogen file in your /usr/local/bin directory, or make your own videogen by pasting this code in a file and making it executable:

 #!/bin/bash

 ffmpeg -re -f lavfi -i "aevalsrc=if(eq(floor(t)\,ld(2))\,st(0\,random(4)*3000+1000))\;st(2\,floor(t)+1)\;st(1\,mod(t\,1))\;(0.6*sin(1*ld(0)*ld(1))+0.4*sin(2*ld(0)*ld(1)))*exp(-4*ld(1)) [out1]; testsrc=s=800x600,drawtext=borderw=5:fontcolor=white:fontsize=30:text='%{localtime}/%{pts\:hms}':x=\(w-text_w\)/2:y=\(h-text_h-line_h\)/2 [out0]" \
 -acodec aac -vcodec h264 -strict -2 -pix_fmt yuv420p -profile:v baseline -level 3.0 \
 $@

Using Videogen

Videogen is rather easy to use, but it does require some manual input as you need to specify the output, but you can specify any of the codecs inside as well incase you want/need to use something else than our default settings.

The only required manual input is the type of output you want and the output URL (or file). For MistServer your output options are:

RTMP

 videogen -f flv rtmp://ADDRESS/APPLICATION/STREAM_NAME

RTSP

 videogen -f rtsp rtsp://ADDRESS:PORT/STREAM_NAME

TS Unicast

 videogen -f mpegts udp://ADDRESS:PORT

TS Multicast

 videogen -f mpegts udp://MULTICASTADDRESS:PORT

As it's all run locally it doesn't really matter which protocol you'll be using except for one point. RTMP cannot handle multi bitrate using this method, so if you want to create a multi bitrate videogen you'll usually want to use TS.

Additional parameters

You'll have access to any of the additional parameters that ffmpeg provides for both video and audio encoding simply by just adding them after the videogen command. Ffmpeg handles the last given parameters if they overwrite previously given parameters. For all the ffmpeg parameters we recommend checking the ffmpeg documentation for codecs, video and audio.

Some of the parameters we tend to use more often are:

-g NUMBER

This determines when keyframes show up. This sets the amount of frames to pass before inserting a keyframe. When set to 25 you'll get one keyframe per second, as videogen runs at 25fps.

-s RESOLUTIONxRESOLUION

This changes the resolution. The default of videogen is 800x600, so setting this to 1920x1080 will make it a "HD" stream, though the quality is barely noticeable with this script. We tend to use screen resolutions to verify a track is working correctly.

-c:v hevc or -c:v h264

This changes the video codec. The default is h264 baseline profile of 3.0, which should be compatible with any modern device. Changing the codec to H265 (HEVC) or "default" h264 changes things and might be exactly what you want to find out. Do note that HEVC cannot work over RTMP, use RTSP or TS instead!

-c:a mp3 -ar 44100

This changes the audio codec. The default is aac, so knowing how to set mp3 instead can be handy. Just be sure to add an audio rate as MP3 tends to bug out when it's not set. We tend to use 44100 as most devices will work with this audio rate.

Multibitrate videogen

Obviously you would want to try out a multi bitrate videogen as well, which you can do but will want to use TS for instead of RTMP as RTMP cannot handle multi bitrate through a single stream as push input.

You can find our multi bitrate videogen here.

You can also make an executable file with the following command in it:

 #!/bin/bash

 #multibitrate videogen stuff if you want to edit qualities or codecs edit the parameters per track profile. If you want to add qualities just be sure to map it first (as audio or video depending on what kind of track you want to add). Videotracks will generally need the -pix_fmt yuv420p in order to work with this script.

 exec ffmpeg -hide_banner -re -f lavfi -i "aevalsrc=if(eq(floor(t)\,ld(2))\,st(0\,random(4)*3000+1000))\;st(2\,floor(t)+1)\;st(1\,mod(t\,1))\;(0.6*sin(1*ld(0)*ld(1))+0.4*sin(2*ld(0)*ld(1)))*exp(-4*ld(1)) [out1]; testsrc=s=800x600,drawtext=borderw=5:fontcolor=white:fontsize=30:text='%{localtime}/%{pts\:hms}':x=\(w-text_w\)/2:y=\(h-text_h-line_h\)/2 [out0]" \
 -map a:0 -c:a:0 aac -strict -2 \
 -map a:0 -c:a:1 mp3 -ar:a:1 44100 -ac:a:1 1 \
 -map v:0 -c:v:0 h264 -pix_fmt yuv420p -profile:v:0 baseline -level 3.0 -s:v:0 800x600 -g:v:0 25 \
 -map v:0 -c:v:1 h264 -pix_fmt yuv420p -profile:v:1 baseline -level 3.0 -s:v:1 1920x1080 -g:v:1 25  \
 -map v:0 -c:v:2 hevc -pix_fmt yuv420p -s:v:2 1920x1080 -g:v:2 25 \
 $@ 

This will create a multi bitrate video stream with aac and mp3 audio and a 800x600, 1920x1080 h264 video stream and a single 1920x1080 h265 (HEVC) stream. That should cover "most" multi bitrate needs.

You will always want to combine this with the ts output for ffmpeg, so using it will come down to:

 multivideogen -f mpegts udp://ADDRESS:PORT

Using videogen or multivideogen with MistServer directly

Of course you can also use videogen or multivideogen without a console, you will still have to put the scripts on your server (preferably the /usr/local/bin folder) however.

To use them together with MistServer just use ts-exec and the mpegts output of ffmpeg like this:

MistServer source:

 ts-exec:videogen -f mpegts -
 ts-exec:multivideogen -f mpegts -

Example of how to fill in MistServer in the Interface

You can put the streams on always on to have a continuous live stream or leave them on default settings and only start the live stream when you need it. Keep in mind that as long as they're active they will use CPU.

read moreless
25 Apr 2018

[Blog] Repushing to social media and streaming services using MistServer

Hey everyone, Balder here. Today I wanted to talk about repushing to social media like Youtube and Facebook or streaming services like Twitch and Picarto using MistServer. Why would you want to put a media server in the middle of...

Hey everyone, Balder here. Today I wanted to talk about repushing to social media like Youtube and Facebook or streaming services like Twitch and Picarto using MistServer. Why would you want to put a media server in the middle of this, what would the benefit be and how do you set it up? Let's find out shall we.

When do you want to use MistServer to repush?
There could be multiple reasons why you would want to use MistServer to push your live stream to your social media or streaming service, but the most likely answer will be that you're pushing to multiple platforms at the same time. You will want to use MistServer once you stream to more than one platform as it saves your local PC the trouble of creating several live streams at the same time and sending them out. Besides needing less resources (bandwidth, CPU and RAM) because you'll only have one stream outgoing, MistServers unique buffering feature allows it to reach all the stream targets from the same buffer. Your viewers will get the exact same stream no matter their preferred platform of watching.

Other reasons to put MistServer in between would be using MistServer to record or using MistServer's input to transcode the stream before sending it to your targets. Those cases are usually best handled on your first encoding machine unless your machine is having trouble keeping up to real-time when doing everything at the same time.

Repushing a stream through MistServer
Repushing through MistServer unsurprisingly goes through the "Push" panel. There's two options: push or automatic push. In order to push a stream using MistServer you will need to have a stream created, so be sure to make your live stream first.

Push starts a push of your chosen stream immediately and once it's done it will stop and remove itself. Activating a push on a stream that is offline will close again almost immediately so don't use this option on streams that aren't active.

Automatic push will do a push of your chosen stream everytime it becomes active, so it will wait until you start pushing and immediately pushes the stream to the given target as well under its default settings. You can also set it up to wait a few seconds and then start pushing, but that's generally not something you would use for live streaming. In general it's the option you want when you want to automatically push towards your other platforms from a single point.

Once you've chosen your method you just need to fill in the stream name used within MistServer and the target to push to.

An example of streaming to Picarto would be:
Stream name: {mylivestream}
Target: rtmp://live.eu-west1.picarto.tv/golive/{my_picarto_stream_key}

As you can see it'll be as easy as knowing the push target and if you want to push to multiple platforms at the same time just add another one. I'll leave a short explanation for Youtube, Facebook, Twitch and Picarto below.

Pay attention to the "?" symbol
MistServer assumes that the last "?" symbol will be used to include additional parameters like track selection or push mode. If your given stream target has a question mark included in the path, make sure you add an additional "?" at the end of the given stream target. The question mark itself will be removed don't worry.

YouTube
Youtube has everything you need in your live dashboard overview. You can get at the right panel by going to your creator studio and selecting live streaming and stream now. At the bottom of that page will be the RTMP url and your stream key. Just use those in MistServer and you should be good.

Your stream target will be something like:
rtmp://a.rtmp.youtube.com/live2/{youtube_stream_name_key}

Facebook
Facebook finally has a permanent stream key so that makes things easier; there's no longer a need to prepare a stream key before going live. You'll need to go to the facebook live page and tick the box for a permanent key. After that just fill in the URL and stream key, but do note that stream keys from Facebook always contain a "?" symbol, so you need to add one at the end in order to work properly with MistServer.

Your stream target will be something like:
rtmp://live-api.facebook.com:80/rtmp/{stream_key}?

Twitch
Streaming to Twitch requires some look up, but once set it'll work until you change your settings so that's pretty great as it allows you to use Automatic pushes. You'll need to look up two things: The recommended Twitch ingest server and your stream key.

Recommended twitch ingest server, this is most easily found here: Twitch ingest servers just pick one that works.

Stream key: This is found in your dashboard, never share it with anyone as they can hijack your channel with this. Obviously it'll be safe to fill into your stream target in MistServer, but make sure no one is watching while you're setting this up.

Once done your stream should look something like this:
rtmp://live-ams.twitch.tv/app/{stream_key}

The Amsterdam Twitch server has been chosen here as I'm located in The Netherlands, the {stream_key} part just needs to be replaced with whatever is given by Twitch at your dashboard panel. Once that's done you're all set.

Picarto
Streaming to Picarto is a little bit easier than Twitch as a single page contains all the information you need and unless you reset your stream key you'll be done.

Once you've chosen your ingest server and stream key it should look something like this:
rtmp://live.eu-west1.picarto.tv/golive/{stream_key}

I've chosen the Europe Picarto server for this example as I'm still in The Netherlands.

That's all for this blog post, I hope it makes it clear why and when you would want to use MistServer as a repushing server for other platforms and gives you a starting point in setting it up.

read moreless
Latest 2018 2017 2016 2015 2014 2013 2012