Hey all!

As most of you probably know, about a year and a half ago in October 2021, Livepeer acquired DDVTech (the company that is building MistServer).

The partnership enabled the open sourcing of a decentralized media server, combining the two technologies, which serves as the backbone for Livepeer’s streaming platform. By early 2023 Livepeer was seeing traction within its core web3-focused market, and decided to lean into building for and serving only this market. This meant potentially winding down the MistServer team and project. To prevent this from happening, Livepeer offered the MistServer team the opportunity to spin back out the project and focus on their own core activities: MistServer-related services and technologies. Not wanting to let the project die, and not wanting to let down all our customers and users that rely on MistServer for their day-to-day operations, we naturally decided to take this option.

So, I'm happy to now formally announce that as of March 2023, DDVTech has a new majority shareholder: "OptiMist Video". OptiMist Video is entirely owned by the engineers on the MistServer team. DDVTech will continue to provide services and products exactly as it has been, and the MistServer team continues to be employed by DDVTech.

This allows us to fully focus on MistServer and related services and technologies once more. We're looking forward to continuing to help our customers, including Livepeer, with their integrations/operations/products based on MistServer. We've enjoyed working on widely varied applications such as streaming platforms, rack encoders, self-driving cars and low latency videochat applications and are eager for more challenges in the future.

If you have any questions, you can reach us through all the usual channels, like our contact form.

Hey everyone,

We've had quite some questions about how to use tokens with live streaming and decided to give another easy example. Do note that this wouldn't be a recommended way for larger platforms, but for something small or quick and dirty this will work and is pretty much done in 5 minutes. For a longer and proper explanation please look at this post.

What is live streaming with tokens

Live streaming with tokens allows you to give your streamers some security in their RTMP push url while at the same time allowing for a more user friendly stream name for viewers/channels. The default for RTMP is that the stream name returns in the push url. While handy this does open it up for people to "hijack" the stream as the push url is easily guessed.

Using tokens allows you to use complicated stream keys to push, while keeping the stream name within MistServer easy to use & user friendly. To get this done we will be using Triggers and a bash script.

Requirements

• MistServer 3.0+ on Linux
• Minor knowledge of Linux

Steps we go through in this guide

• 1 we will have a look at the bash script
• 2 we will look at the MistServer settings
• 3 we will be editing the bash script to your streams
• 4 we will be changing the push URLS for your pushing application for RTMP
• 5 we'll then look at what changes when we use SRT
• Finally we'll look at how this works for a wildcard setup

1. The bash script

#!/bin/bash

#log incoming data for debugging/logging. Uncomment below
#TRIGGERDIRECTORY="./"
#cat >> "${TRIGGERDIRECTORY}push_rewrite.log"

#Collect the trigger payload
DATA=`cat`

#Split up 2nd and 3rd line to IP and KEY given by trigger
IP=`echo "$DATA" | sed -n 2p`
KEY=`echo "$DATA" | sed -n 3p`

#variables to match with IP & KEY.
#PASS = KEY
#ORI = IP
#STREAM = Stream name to redirect towards
function checkStream {
        PASS=$1
        ORI=$2
        STREAM=$3

        if [ "$IP" = "$ORI" ]; then
                if [ "$KEY" = "$PASS" ]; then
                        echo -n "$STREAM"
                        exit 0
                fi
        fi
}

#checkstream fills in $1,$2,$3 with: "password" "IP" "stream_name" and checks info below for matches
#To add streams copy below checkStream and fill in pass/IP/stream_name to allow any IP use $IP
#checkStream "key" "$IP" "streamname"

Save the following text in your folder of preference under your preferred name. I would recommend saving it something easily recognizable like: push_rewrite.sh

After saving it don't forget to make it executable with the terminal command:

chmod +x file

2. MistServer settings

Stream Setting

Live streams will be set up normally, simply make a stream with any stream name and as source use:

push://

It does not matter how many live streams you set up, but do mind that every live stream needs to be set up later on in the bash script.

Trigger setting

This is where the magic happens. We will use a bash script below to rewrite the stream keys to their easier counterparts.

Use the trigger

PUSH_REWRITE
Handler (URL or executable): /path/to/bash.sh
Blocking: YES
Default response: false

The /path/to/bash.sh will be the path/filename you used when setting up the bash script.

3. Editing the bash script to your streams

Now all that is left will be editing the bash script to work with your live streams.

Generating stream tokens

Now this is something I highly recommend generating yourself, it can be pretty much as long as you like. To avoid annoyances I would also recommend avoiding special characters as some applications might not like those when pushing out an RTMP stream.

For this example I will be using md5sum. Keep in mind that the whole point of the token is that they cannot be easily guessed, so don't just only hash the stream name. Add something that only you know in front and behind the stream name to generate your token.

Example:

md5sum <<< supersecret1_live_supersecret2

This will give me the output:

8f4190132f1b6f1dfa3cf52b6c8ef102

Using the stream name in there will make the token generation different every time and the randomly chosen words before and after will keep it impossible to guess for outsiders. Use a token your comfortable with or you feel is random enough. One could also use a MD5HASH_streamname_MD5HASH as token making it longer.

Editing the bash file

The only thing to add is a line at the bottom of the bash file. The line needs to follow this syntax:

checkStream "key" "$IP" "streamname"

Where:

• key is your Stream token
• streamname is your set up stream name within MistServer
• $IP is either $IP to skip IP verification or the IP you want to white list as the only one able to push towards this stream. Usually tokens are enough, but this is another extra security step you can take.

Using the example above and assuming the stream live within MistServer would be:

checkStream "8f4190132f1b6f1dfa3cf52b6c8ef102" "$IP" "live"

To add streams simply keep adding lines below your last, make sure to use a new & correct key and streamname every time.

4. Editing your pushing application

Instead of pushing towards stream live your pushing application would now require the token as stream key/name.

So you would be using 8f4190132f1b6f1dfa3cf52b6c8ef102 instead. Making the full RTMP address to use rtmp://mistserveraddress/live/8f4190132f1b6f1dfa3cf52b6c8ef102

or

rtmp://mistserveraddress/live/ stream key: 8f4190132f1b6f1dfa3cf52b6c8ef102

Depending on whether your application wants a full url or a partial with stream key.

When pushing towards this url you should see your stream come online, with the shorter live stream name. While anyone trying to push towards MistServer using the stream name instead of the token will be blocked.

5. Using this set up with SRT

Using this set up with SRT will work almost the same way as RTMP. The major differences are that you'll be setting up an incoming SRT port or use MistServers default ports.

MistServer Configurations

If you're using the defaults of MistServer you can skip this step and your default port will be 8889.

You will need the SRT protocol active in the protocol panel. Look for TS over SRT. For the port you can choose any valid port you like, we would recommend setting the 'Acceptable connection types' to Allow only incoming connections only as this speeds up the input side of things.

Stream Configurations

No changes here, keep the source input on "push://". This will work for both RTMP and SRT after all!

Trigger Configurations

Nothing changes for the trigger script or the trigger itself. Generate a token for every stream name you'd like to use and when it matches it will be accepted.

Pushing application configurations

In order to push towards MistServer you will have to use a newer syntax for pushing SRT. We're going to include the value streamid. This value tells MistServer what stream should be matched with your push attempt, and is exactly what we need for the token matching.

SRT URL

Your SRT urls would work like this: srt://mistserveraddress:port?streamid=TOKEN You can follow up the url with any other parameters you want to use in SRT. Using the same token as the RTMP example you'd get: srt://mistserveraddress:port?streamid=8f4190132f1b6f1dfa3cf52b6c8ef102 MistServer will grab the streamid, match it to the streamnames set in the trigger and forward it to the matching stream name.

6. Wildcard setup

A wildcard setup is where you set up a single live stream and use that configuration for all other live streams that will be going to your platform. It's best use is for platforms that have to deal with a significant amount of live streams from users that might be added on the fly. It's a set up that allows you to make one major stream name and add secondary streams using the same setup. It works by placing a plus symbol (+) behind the stream name, followed by any unique text identifier. For example, if you configured a stream called "test" you could broadcast to the stream "test", but also to "test+1" and "test+2" and "test+foobar". All of them will use the configuration of "test", but use separate buffers and have separate on/off states and can be requested as if they are fully separate streams.

Trigger changes

The only change here is that you need to make the new streams by using mainstream+uniquestream where the uniquestream is the new stream name for every new push.

For example: checkStream "8f4190132f1b6f1dfa3cf52b6c8ef102" "$IP" "live+uniquestream1" The token 8f4190132f1b6f1dfa3cf52b6c8ef102 would then create the stream live+uniquestream1 which saves you the setup of making the stream uniquestream1, just adding new lines to this and creating a uniquestream identifyer after the plus every time will allow you to instantly use it.

Conclusion

That is pretty much it for a simple bash method to stream tokens and live streaming. It all comes down to setting up a trigger and adding every new stream to the bash script and giving them an unique token. Now, just to mention it again: we wouldn't recommend something like this for a bigger setup, but it would get you started for a small server.

Sorry for the long wait since 3.1, but 3.2 is now finally available! Downloads can be found here.

A big under-the-hood improvement in this release is full support for the Meson build system, which will take care of automatically building all the needed dependencies as well. If you're one of our users that likes building their own binaries, definitely take it for a spin! (Those that simply download builds from our website, you will not notice any difference.)

Official Windows and Mac releases are still not available for this version while we're in the middle of upgrading our build system, but users wanting a MacOS build can already compile from source from this version onward (3.2 code is fully MacOS compatible again).

We also temporarily reverted support for LLHLS (just plain HLS in this build), due to many reported issues with inter-track syncing. We hope to bring full support back somewhere in the coming months. For those that only need to use single-track playback (without adaptive bit rate), you can reach out to us for a 3.2 build that has LLHLS re-enabled, or simply keep using the 3.1 build of course.

The full changelog can be found here.

Noteworthy features

New sessions system

What: Until now, Mist has already had a concept of a “session”, but it was hardcoded to our predetermined definition: a session is considered to be a single view action, over a single protocol, a single IP address, and for a single stream name. This new sessions system makes this definition configurable, and splits out the session management into a new MistSession binary that collates connection stats into session stats, and passes those on to the rest of the system as a single thing.

Why: Somebody’s definition of “session” might not match ours, and the work of matching connections to sessions was quite a bit of strain on the MistController binary, which turned this into a single point of failure in the system. This change not only allows more use cases to work without needing to make changes to the code, but also speeds up the whole system and increases reliability and predictability.

Custom variables support

What: MistServer already supports using various variables in places, mostly covering the current date and time, stream name, etcetera. This functionality would add support for custom-defined variables, which may be either static or the result of running some command or calling some URL (similar to the trigger system) at a configurable interval.

Why: This makes re-using paths, hostnames, and other similar types of configuration while allowing them to change in the future easy. It's also a step up for another upcoming feature: the ability to use these variables to configure arbitrary recording times/intervals, allowing for much more powerful advance scheduling of recordings and pushes.

Scheduled Pushes

What: So far, MistServer's ability to "push" streams (which is a single concept that encompasses both recording to disk as well as sending the stream elsewhere over various protocols for replication) has been limited to two styles of pushes: Regular pushes, which are started whenever they are configured and disappear once they end. Automatic pushes, which are started whenever their corresponding stream is active, and automatically restarted as needed while the stream is still active/available, even if it fails or gets disconnect. The new scheduled pushes system lets you use arbitrary variable comparisons to determine when a push should and shouldn't be active, giving extra flexibility on top of and alongside the existing features.

Why: The perfect use case for this is a stream that may be active continuously, but only should be pushed out and/or recorded at set intervals or times. The flexibility of this new system lets you call out to external sources to set variables and then write rules to interpret those variables into a decision to push or not. For example, MistServer could be linked to a calendar that decides when pushes are active, or some kind of external toggle/switch system that can be polled for the current wanted state.

WebRTC WHIP/WHEP/WISH support

What: Support for WebRTC ingest using WHIP/WISH and egress using WHEP.

Why: Our WebRTC support required a proprietary WebSocket connection for signalling. This adds support for industry standard WHIP/WISH ingest and WHEP egress in addition to our existing signalling WebSocket (which is still supported, and usually preferred because it supports more features). WHIP/WISH/WHEP can (for example) be interesting if there is an expected change in the IP address of the user, as it can survive such changes while the existing WebSocket-based method cannot.