News

8 Feb 2018

[News] Introducing subscriptions

Hello everyone! We're happy to announce our shop now supports subscriptions. When you select the subscription payment plan, you'll be charged for the first month. Then, when the month is up, a new license will automatically be created and paid...

Hello everyone! We're happy to announce our shop now supports subscriptions.

When you select the subscription payment plan, you'll be charged for the first month. Then, when the month is up, a new license will automatically be created and paid for.
This means you will no longer have to worry about your licenses expiring.

You can cancel your subscription at any time. Your license will remain valid until its end date, but it will not be renewed and you will not be charged again.

If you currently have an active license and would like to subscribe from now on, you can click the Renew button from your Invoices overview and select the subscription payment plan. That will configure the cart to be filled with the settings of your previous order, starting at its end date.

If you have any questions or encounter any problems, please feel free to contact us.

read moreless
1 Feb 2018

[Blog] Setting up Analytics through Prometheus and Grafana

Hey Everyone! Balder here, this time I wanted to talk about using Prometheus and Grafana to set up analytics collection within MistServer. There’s actually quite a lot of statistics available and while we do tend to help our Enterprise customers...

Hey Everyone! Balder here, this time I wanted to talk about using Prometheus and Grafana to set up analytics collection within MistServer. There’s actually quite a lot of statistics available and while we do tend to help our Enterprise customers to set this up it’s actually available for our non-commercial users as well and easily set up too.

Best practises for setting up your analytics server

As you might have guessed using Prometheus and Grafana will require some resources and we recommend running it on a different device than you are running MistServer on. This is for a few reasons, but the most important being that you would want your analytics collection to keep going if your MistServer instance goes dark for some reason or has run into trouble.

As such we would recommend setting up a server whose sole focus is to get the analytics from your MistServer instances. It can be any kind of server, just make sure it has access to all your MistServer instances.

Operating system choice

While this can be run in almost every operating system, a clear winner is Linux.

Under Linux both Prometheus and Grafana work with little effort and will become available as a service with their default installs. Mac comes in second as Prometheus works without too much trouble, but Grafana requires you to use the homebrew package manager for MacOS. Windows comes in last as I couldn’t get the binaries to work without a Linux simulator like Cygwin.

Installing Prometheus and Grafana

Linux

Installing both Prometheus and Grafana under linux is quite easy as they're both quite popular. There's a good chance they're both available as a standard package immediately. If not I recommend checking their websites to see how the installation would go for your Linux Operating System of choice.

Starting them once installed is done through your service system which is either:

systemctl start grafana.service

or

service grafana start

depending on your Operating system.

MacOS

Installing Prometheus can be done rather easy. The website provides Darwin binaries that should work on your Mac. It can also be installed through Homebrew which we will be using for Grafana. Which method you use is up to you, but I prefer to work with the binaries as it made using the configuration file easier for me.

Install Homebrew as instructed on their website.

Then use the following commands in a terminal:

brew update
brew install prometheus

Installing it as a service would be preferred, but I would recommend leaving that until after you've set everything up.

Installing Grafana can also be done through Homebrew. The Grafana website offers some excellent steps to follow in order to install it properly.

For Prometheus you will have to make your own service to have it automatically start on boot. Installing Grafana through Homebrew will make it available as a service through Homebrew.

Windows

Both Prometheus and Grafana offer Windows binaries, however I could not get them to work natively in Windows 10. They did instantly work when I tried running them in the Cygwin terminal.

Because of the added difficulty here I would just run them both in a Cygwin terminal and be done with it, though you could try to run them as a system service. The combination of Cygwin and Windows Services tend to cause odd behaviour however, so I can't exactly recommend it.

Setting up Prometheus and Grafana

01: Editing the Prometheus settings file

This is done by editing prometheus.yml, which may be stored on various locations. You will either find it in the folder you've unpacked, or when installed in Linux, at /etc/prometheus/prometheus.yml or /etc/prometheus.yml

You need to add the following to the scrape_configs:

scrape_configs:
  - job_name: 'mist'
    scrape_interval: 10s
    scrape_timeout: 10s
    metrics_path: '/PASSPHRASE'
    static_configs:
      - targets: ['HOST:4242']

To add multiple MistServers just keep adding targets with their respective HOST.

An example minimal prometheus.yml would be:

scrape_configs:
  - job_name: 'mist'
    scrape_interval: 10s
    scrape_timeout: 10s
    metrics_path: '/PASSPHRASE'
    static_configs:
      - targets: ['HOST01:4242', 'HOST02:4242', 'HOST03:4242']

We did notice that if there's a bad connection between your analytics server and a MistServer instance the scrape_timeout of 10 seconds could be too short and no data will be received. Setting a higher value for the scrape time could help in this scenario.

You can check if this all worked by checking out http://HOST:9090 at the machine you've set this up after you've started Prometheus. Within the Prometheus interface at StatusTargetsyou can inspect whether Prometheus can find all the MistServer instances you've included in your settings.

02: Starting Prometheus

For Linux

systemctl start prometheus.service

or

service prometheus start

or

Use a terminal to go to the folder where you have unpacked Prometheus and use:

./prometheus --config.file=prometheus.yml

For MacOS

Use a terminal and browse to the folder where you have unpacked Prometheus. Then use:

./prometheus --config.file=prometheus.yml

For Windows

Use a command window to browse to the folder where you have unpacked Prometheus. Then use:

prometheus.exe --config.file=prometheus.yml

03: Setting up Grafana

Through your installation method Grafana should be active and available as a service, or if you are using Windows you will need to boot Grafana by starting grafana-server.exe.

Once active Grafana will have an interface available at http://HOST:3000 by default. Open this in a browser and get started on setting up Grafana.

Adding a data source

The next step is to add a data source. As we're running Grafana and Prometheus in the same location, this is quite easy. All we need to set is the Name, Type and URL all other settings will be fine by default.

Image of adding data to Grafana

  • Name can be anything you'd want.
  • Type has to be set to: Prometheus
  • URL will be the location of the Prometheus interface: http://localhost:9090

Add those and you're ready for the next step.

Adding the dashboard

We've got a few Dashboards available immediately which should give the most basic things you'd want. You can add a dashboard by following these steps:

Click on the grafana icon in the top left corner → hover Dashboards → SelectImport`.

You should see the following
Image of the import dashboard option in Grafana

Fill in the Grafana.com Dashboard number with our preset dashboards (for example our MistServer Vitals: 1096)

If recognised you will see the following
Image of importing the MistServer Vitals Dashboard into your Grafana

Just add that and you should have your first basic dashboard. Our other dashboards can be added in the same manner. More information about what each dashboard is for can be found below.

MistServer provided dashboards

All of the dashboards can be found here on Grafana Labs as well.

MistServer Vitals: 1096

Image of the MistServer Vitals Dashboard in grafana

This is our most basic overview which includes pretty much all of the statistics you should want to see anyway. It covers how your server is doing resource and bandwidth wise.

You switch between given MistServers at the top of given panels by clicking and selecting the server you want to inspect.

MistServer Stream Details: 4526

Image of the MistServer Stream Details Dashboard in grafana

This shows generic details per active stream. Streams and Servers are selected at the top of the panel. You'll be able to see the amount of viewers, total bandwidth use and amount of log messages generated by the stream.

MistServer All Streams Details: 4529

Image of the MistServer All Streams Details Dashboard in grafana

This shows the same details as the MistServer Stream Details Dashboard, but for all streams at the same time. This can be quite a lot of data, and will become unusable if you have a lot of streams. If you have a low amount of streams per server this gives an easy to use overview however.

Well that's it for this blogpost, I hope it's enough to get most of you started on using Prometheus and Grafana in combination with MistServer.

read moreless
23 Jan 2018

[Blog] Metadata format

Hello readers, this is Erik, and today we are going to be diving in-depth into some important updates we have been making to our internal metadata systems and communication handling. Over the last couple of years "low latency" streaming has become...

Hello readers, this is Erik, and today we are going to be diving in-depth into some important updates we have been making to our internal metadata systems and communication handling.

Over the last couple of years "low latency" streaming has become more and more important, with viewers no longer accepting long buffering times or being delayed in their stream in any way. To achieve this all processes in your ecosystem will need to be able to work with the lowest latency possible, and having a media server that aids in this aspect is a large step in the right direction.

With this in mind we have been working on creating a new internal format for storing metadata, that allows multiple processes to read while having a single source process generate and register the incoming data. By doing this directly in memory we can now bring our internal latency down to 2 frames direct throughput, and this post is an overview of how we do this.

Communication in a modular system

Because MistServer is a multi-process environment - a separate binary is started for each and every viewer - efficiency is mostly dependent on the amount of overhead induced by the communication between the various processes. Our very first version used a connection between each output and a corresponding input, which has been replaced a couple of years ago by a technique called shared memory.

Shared memory is a technique where multiple processes - distributed over any number of executables - can access the same block of memory. By using this technique to distribute data, all source processes need to only write their data once, allowing any output process to read it simultaneously.

The main delaying factor in its current implementation, is that the metadata for a live stream only gets written to memory every second. As all output processes read once per second as well, this yields a communication delay of up to 2 seconds.

For our live streams we also have the additional use case where multiple source processes can be used for a single stream in order to generate a multi bitrate output. All source processes write their own data to memory pages and a separate MistInBuffer process handles the communication, authorization and negotiation of all tracks. Next to this it will make sure DVR constraints are met, and inactive tracks get removed from the stream.

During this it will parse the data written to a page by a source process, only to regenerate the metadata that was already available in the source to begin with. This in itself adds a delay as well, and moreover it demands processing power to recalculate information that was already known.

Synchronisation locks

To make matters worse, in order to maintain an up to date view on all data, all executables involved in this system will need to 'lock' the metadata page in its entirety to make sure it is the only process with access. Though the duration of this lock is generally measured in fractions of milliseconds, having a stream with hundreds or thousands of viewers at the same time does put a strain on keeping output in realtime.

Reliable Access

For the last couple of months we have been busy with a rework of this structure to improve our metadata handling. By using the new RelAccX structure we can generate a lock-less system based on records with fixed-size data fields.

If the field sizes do need to be changed a reload of the entire page can be forced to reset the header fields. By doing so all processes that reload the structure afterwards will be working with the new values, as these are stored in a structured header. This also allows us to add fields and maintain consistency going forward.

About latency

By using the above described structure we can assign a single page per incoming track, and make the source process write its updates to the same structure that is immediately available for all other processes as well. By setting the 'available' flag after writing all data, we can make sure that the data on the page matches the generated metadata. By doing this we have measured a latency of 2 frames end-to-end in our test environment.

In the same way we can set a record to 'unavailable' to indicate that while the corresponding data might actually still exist at the moment, it is considered an unstable part of the stream and will be removed from the buffer in the near future.

Communications

Besides having implemented this technique for the metadata, we have also upgraded the stability and speed of our internal communications. The key advantages here are that our statistics API can now give even more accurate data, and that output processes can now select any number of tracks from a source up from the previous limit of 10 simultaneous tracks - yes we have had customers reaching this limitation with their projects.

Conclusion

By updating the way we have structured our internal communications, we have been able to nearly remove all latency from the system, as well as attaining a reduced resource usage due to not having to recalculate 'known' data. This system will be added in our next release, requiring a full restart of the system. If you have any question on how to handle the downtime generated by this, or about the new way we handle things, feel free to contact us

read moreless
4 Jan 2018

[Blog] Using MistServer through a reverse proxy

Sometimes, it can be convenient to direct requests to MistServer through a reverse proxy, for example to limit the amount of ports that will be used or to use a prettier url. In this blog, I'll take you through the steps...

Sometimes, it can be convenient to direct requests to MistServer through a reverse proxy, for example to limit the amount of ports that will be used or to use a prettier url.
In this blog, I'll take you through the steps to configure your web server and MistServer to work in with this. For this example I'll consider the use case of having an existing website that is running on port 80, with MistServer on the same machine, and that we want MistServer's HTTP endpoint (by default that's port 8080) to be reached through http://example.com/mistserver/.

Using a reverse proxy for MistServer does have a cost to efficiency, however.
Using this method means that instead of once, output data needs to be moved four times. Furthermore, MistServer tries to reuse connections, but it can't if they're behind a proxy server.

Configure your web server

The first step is to configure your web server to reverse proxy to MistServer's HTTP port.

For Apache:
Enable the mod_proxy and mod_proxy_http modules and add the following to your configuration file:

<Location "/mistserver/">
  ProxyPass "http://localhost:8080/"
</Location>

For Nginx:
Add the following to your configuration's server block, or nested into your website's location block:

location /mistserver/ {
  proxy_pass http://localhost:8080/;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_buffering off;
}

For Lighttpd:
Add mod_proxy to your list of server modules, or add the following to your configuration file:

server.modules = ("mod_proxy")

Also, to disable response buffering, add:

server.stream-response-body = 2

and for the reverse proxy itself:

$HTTP["url"] =~ "(^/mistserver/)" {
  proxy.server  = ( "" => ( "" => ( "host" => "127.0.0.1", "port" => 8080 )))
  proxy.header = ( "map-urlpath" => ( "/mistserver/" => "/" ))
}

You should now be able to watch your stream directly at http://example.com/mistserver/STREAMNAME.mp4 (MP4) or http://example.com/mistserver/hls/STREAMNAME/index.m3u8 (HLS).

Configure MistServer

MistServer also outputs information about its streams, including the various urls under which the stream can be accessed. However, if reverse proxying is being used, these urls cannot be accessed externally. That can be fixed by configuring the "Public address"-setting of the HTTP protocol through the MistServer Management Interface.

You should now also be able to watch your stream at http://example.com/mistserver/STREAMNAME.html.

Edit your web pages

Lastly, the urls with which streams are embedded on your webpage will need to be updated. If you are using MistServer's embed code, update any urls.

<div class="mistvideo" id="nVyzrqZSm7PJ">
  <noscript>
    <a href="http://example.com/mistserver/STREAMNAME.html" target="_blank">
      Click here to play this video
    </a>
  </noscript>
  <script>
    var a = function(){
      mistPlay("STREAMNAME",{
        target: document.getElementById("nVyzrqZSm7PJ")
      });
    };
    if (!window.mistplayers) {
      var p = document.createElement("script");
      p.src = "http://example.com/mistserver/player.js"
      document.head.appendChild(p);
      p.onload = a;
    }
    else { a(); }
  </script>
</div>

That's all folks! Your website should now access MistServer through the reversed proxy.

On our next blog, Erik will talk about the upcoming lock-free internal metadata structure Jaron mentioned last time.

read moreless
20 Dec 2017

[Blog] Live streaming with Wirecast and MistServer

Hey everyone! As Jaron mentioned I would do the next blog post. Since our blog post about OBS Studio and MistServer is quite popular I figured I would start adding other pushing applications, this time I'll talk about Telestream Wirecast. Wirecast Wirecast...

Hey everyone! As Jaron mentioned I would do the next blog post. Since our blog post about OBS Studio and MistServer is quite popular I figured I would start adding other pushing applications, this time I'll talk about Telestream Wirecast.

Wirecast

Wirecast is an application meant for live streaming, their main focus is to easily allow you to create a live stream with a professional look and feel. It's a great piece of software if you want to go for a professional feel and want a piece of software that makes it easy to do so.

Basic RTMP information

This information will be very familiar to those who read how to push with OBS Studio to MistServer, so feel free to skip it.

Most popular consumer streaming applications use RTMP to send data towards their broadcast target. The most confusing part for newer users is where to put which address, mostly because the same syntax is used for both publishing and broadcasting.

Standard RTMP url syntax

rtmp://HOST:PORT/APPLICATION/STREAM_NAME

Where:

  • HOST: The IP address or host name of the server you are trying to reach
  • PORT: The port to be used; if left out it will use the default 1935 port.
  • APPLICATION: This is used to define which module should be used when connecting, within MistServer, this value will be ignored or used as password protection. The value must be provided, but may be empty.
  • STREAM_NAME: The stream name of the stream: used to match stream data to a stream id or name.

This might still be somewhat confusing, but hopefully it will become clear as you read this post. These will be the settings I will be using in the examples below.

  • Address of server running Wirecast: 192.168.137.37
  • Address of server running MistServer: 192.168.137.64
  • Port: Default 1935 used
  • Application: not used for mistserver, we use live to prevent unreadable URLs.
  • Stream name: livestream

Set up the stream in MistServer

Setting up the stream in MistServer is easy, just go to the stream window and add a stream. For the stream name just pick anything you like, but remember it you will need it in Wirecast later on. For the source select push://ADDRESS_OF_SERVER_RUNNING_WIRECAST. In this example I will go with:

  • Stream name: livestream
  • Source: push://192.168.137.37

Settings used in MistServer for this blog

Booting Wirecast

First you will enter the boot screen, here you can pick your previously saved templates or start with a new one. We will just start with a new one, so just click continue in the bottom right corner.

Image of the Wirecast start screen

And you should see the start interface.

Image of the Wirecast start interface

Setting up Sources

Luckily Wirecast is quite easy to set up. You add sources to your layers, sources could both be audio, video or both at the same time. For this example I'll just add a simple media file, but you could add multiple sources to multiple layers and switch between presets. It's one of the reasons to use Wirecast so I would recommend checking out all the possibilities once you've got the chance.

Adding a stream to Wirecast

Setting up the output

Setting up the output can be done through the output settings menu in the top left.

Choosing output settings in the Outputs menu

Choose a custom RTMP server when setting everything up. Most important are the Address and Stream. You will need to fill in the address of MistServer and the stream you have set up previously. We will go with the following:

  • Address: rtmp://192.168.137.64/live/
  • Stream: livestream

setting up the encoder

now Wirecast has a lot of presets, but they're all a bit heavy to my tastes. If you just want to be done fast I would recommend the 720p x264 2Mbps profile as it's the closest to what you'll need if you're a starting user and unsure what you will need. If you do know what you need or want feel free to ignore this bit. Just be aware that Wirecast tends to set not that many key frames which can drastically change latency.

If you want to tweak the settings a bit I recommend the following settings:

  • encoder: xh264
  • width: 1280
  • width: 720
  • frames per second: 30
  • average bitrate: 1200
  • quality: 3 (very fast encoding)
  • profile: high
  • keyframe every: (30 -)150

The rest of the settings on default.

Recommended encoder settings for normal streaming in Wirecast

This profile should work for most streams you will want to send over a normal internet connection without being in the way of other internet traffic.

Setting the layers to live

By pressing the go button your current stream will transition towards your preview to the left following the rules to the left of the button. Only if it's on the live preview will it be pushed out towards your chosen server, so be sure that you're happy with your live preview.

Push the streams to the live preview so they will be pushed when you start broadcasting

Push towards MistServer

You push by pressing the broadcast button, it's to the top left and looks a bit like a wifi button. Alternatively you could click output and select start / stop broadcasting. If it lights up green you're pushing towards MistServer and it should become available within moments, if not you will have to go through your settings as you might have made a typo.

Start your stream push by pressing broadcast

Check if it is working in MistServer

To check if it is working in MistServer all you will have to do is press the preview button when at the stream menu. If everything is setup correctly you will see your stream appear here. If you would like to stop your live stream just stop the broadcast in Wireshark by pressing the broadcast button or the start / stop broadcasting option.

Check the live preview to see if the push is coming into MistServer

Getting the stream to your viewers

As always MistServer can help you with this. At the Embed panel (found at the streams panel or stream configurations) you can find some embed-able code or the direct stream URLs. Use the embed-able code to embed our player or use the stream codes for your own preferred player solution and you should be all set! Happy streaming!

The embed information for this stream preview

The next blog post will kick off the 🎆new year🎆 and be made by Carina. She will write about how to combine MistServer with your webserver.

read moreless
Latest 2018 2017 2016 2015 2014 2013 2012