Using WebRTC with Mattermost

2017-10-28 12:16 by Thomas Urban

This post is a rough protocol of what we've done to set up Janus WebRTC Gateway and coturn service in an LXC container for use with Mattermost.

You might prefer testing WebRTC using the officially provided container, instead. That container requires docker.

Setting Up Container

We've started with creating a new LXC container running Ubuntu Xenial (16.04 LTS). The container is running behind some NAT firewall and was assigned IP 10.0.3.194. The host's public IP is assumed to be 1.2.3.4.

Please remember to replace any further occurrence of 1.2.3.4 by your host's public IP and any further occurrence of 10.0.3.194 by the local IP of your container to be created next. Finally, webrtc.mycompany.com and mycompancy.com are fake domain names used to address the host from public internet requiring them to be replaced, too.

lxc-create -n mattermost-webrtc -t ubuntu -- -r xenial -u usertobedropped
lxc-start -dn mattermost-webrtc
lxc-attach -n mattermost-webrtc

The last command is entering container and thus all succeeding commands are executed in context of that container.

First of all container was updated:

apt update && apt upgrade

Next some basically required software is installed:

apt install nano git wget

As we are going to access this container via its host it's okay to remove SSH service and custom user created before for security measures:

apt remove openssh-server
userdel usertobedropped

At this point we are read to start with installing custom software.

Installing coturn

Ubuntu Xenial includes recent-most version of coturn (which is 4.5.0.6 as of late 2017), thus installing it is as simple as this:

apt install coturn

The service must be enabled explicitly. Open file /etc/default/coturn with nano (or any other editor you like) and remove # at start of line reading

#TURNSERVER_ENABLED=1

The configuration of coturn is available in /etc/turnserver.conf. This file requires some tweaking. The following options must be set:

external-ip=1.2.3.4/10.0.3.194
This rule is required due to running LXC container behind NAT firewall. The first IP preceding slash is the host's public IP (here 1.2.3.4 for example) and the IP succeeding slash is the container's local IP in virtual network running on host (here 10.0.3.194). You need to adjust both IPs to match your particular scenario.
lt-cred-mech
This option is commented-out by default and must be enabled to support authenticated access on STUN/TURN service.
use-auth-secret
This option must be enabled, too, to enable user-based authentication.
realm=mycompany.com
This option selects some default realm. Here it is the domain name of your company, for example.
cert=/etc/ssl/some-certificate.pem
and
pkey=/etc/ssl/some-private-key.key
These options are required to support STUN/TURN over TLS. All rules common to setting up TLS certificates apply here, too:
- Paste all chaining certificates into the certificate file.
- Limit file access on private key file at least. However, the same file will be used for setting up Janus WebRTC Gateway below, thus you should obey that on limiting access here.

Finally coturn should be restarted:

systemctl restart coturn

Check the output of

systemctl status coturn

to see if it's running properly.

Currently, Mattermost works without TURN service and some of the previously set options and the following steps using turnadmin might not be required accordingly. Nonetheless they are given here for sake of completing the setup.

It's time to set some shared secret for your service:

turnadmin -s "someVerySecretString" -r mycompany.com

Use your realm instead of mycompany.com and replace the very secret string with an arbitrary sequence of letters, digits and special characters.

turnadmin -A -u myusername -p "mypassword" -r mycompany.com

is now adding some admistrative user. Use some different username, password and realm here, too.

When using bash or similar shell the two preceding commands might be saved in plain text in history file of container's root user. You should fix your history e.g. by pressing Cursor Up key to recall either command, then remove the recalled input with Backspace key and press Cursor Up key once again. Finally press Cursor Down key once and press Enter key so the history is clean. You should double-check here by pressing Cursor Up key again.

Don't forget to enable coturn service with

systemctl enable coturn

so it is restarted on system startup.

Installing Janus WebRTC Gateway

This service isn't included with Ubuntu and thus must be compiled from source. The README file provides full documentation so you should read it. Here comes are short version of what we've done in our case:

apt install libmicrohttpd-dev libjansson-dev \
    libnice-dev libssl-dev libsofia-sip-ua-dev \
    libglib2.0-dev libopus-dev libogg-dev libcurl4-openssl-dev \
    pkg-config gengetopt libtool automake cmake

This is installing all required dependencies but libsrtp as Ubuntu comes with version 1.4.x which is claimed to have flaws in README.

mkdir /root/build
cd /root/build/

Here we are creating a separate subfolder in /root to build applications from source. Next latest version of libsrtp is build from source:

wget "https://github.com/cisco/libsrtp/archive/v2.1.0.tar.gz"
tar xzf v2.1.0.tar.gz
cd libsrtp-2.1.0/
./configure --prefix=/usr --enable-openssl
make shared_library && make install

Mattermost relies on WebSockets when interacting with WebRTC so required library for supporting websockets must be built as another dependency, first:

cd /root/build
git clone git://git.libwebsockets.org/libwebsockets
cd libwebsockets/
git checkout tags/v2.4.0
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr -DCMAKE_C_FLAGS="-fpic" ..
make && make install

Here we have installed v2.4.0 of websockets library as it was related to latest branch named stable listed here. You might try different version according to that list.

Finally it's time to build and install Janus WebRTC Gateway in folder /opt/janus:

cd /root/build
git clone https://github.com/meetecho/janus-gateway.git
cd janus-gateway/
sh autogen.sh
./configure --prefix=/opt/janus --enable-websockets
make && make install
make configs

For running Janus on system start we need to create a unit description for systemd to be saved in /etc/systemd/system/janus.service:

[Unit]
Description=Janus WebRTC gateway
After=network.target

[Service]
Type=simple
ExecStart=/opt/janus/bin/janus -o
Restart=on-abnormal
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

Next have systemd discovering this new file:

systemctl daemon-reload

Adjusting Configuration

Default configuration requires some extensive tweaking. All configuration files are found in /opt/janus/etc/janus. The main configuration resides in /opt/janus/etc/janus/janus.cfg. Adjust at least the following options using nano editor:

log_to_stdout = true
Due to using systemd starting Janus in simple mode logging can be written to stdout safely.
token_auth = yes
Token-based authentication must be enabled for Mattermost.
admin_secret = anotherVerySecretString
This secret key is required for administrative / monitoring requests.
server_name = webrtc.mycompany.com
You might want to set your public FQDN of the container here.
cert_pem=/etc/ssl/some-certificate.pem
and
cert_key=/etc/ssl/some-private-key.key
These options in section [certificates] are selecting same files used with coturn before. Please see notes there for security considerations.
stun_server = webrtc.mycompany.com
and
stun_port = 3478
These options in section [nat] are selecting STUN server to use. Here the public name of STUN server is used, but you might prefer using 127.0.0.1 or map this FQDN to loopback device in /etc/hosts since STUN service is set up on same machine.
turn_server = webrtc.mycompany.com
turn_port = 3478
turn_type = udp
turn_user = myusername
turn_pwd = mypassword
These options set up TURN service to be used. myusername and mypassword must be replaced with the username and password used with turnadmin before on setting up coturn. Regarding turn_server the same rules apply as given before on stun_server.

This configuration is affecting core functionalities, but doesn't cover APIs exposed for interacting with the gateway and controlling it. Thus two more files must be adjusted to configure HTTP RESTful API and support for WebSockets as used by Mattermost. Those files are /opt/janus/etc/janus/janus.transport.http.cfg and /opt/janus/etc/janus/janus.transport.websockets.cfg.

Some ports are exposed by default, other must be enabled. Actually the default list of ports used looks rather strange, thus we've decided to adjust ports to match some pattern:

regular requests: 8xxx
administrative requests: 7xxx
HTTP RESTful API: x0xx
WebSockets API: x1xx
unencrypted: xx88
encrypted: xx89

Let's start with HTTP RESTful API to be configured in file /opt/janus/etc/janus/janus.transport.http.cfg.

Configuration of general requests API is found in section [general]:
- port = 8088
  This option selects different port by default, but we change it to match port schema described above.
- https = yes
  This option enables support for HTTPS on regular requests.
- secure_port = 8089
  Regular requests over HTTPS are to be exposed on port 8089.
Configuration of API for administrative and monitoring requests if found in section [admin]:
- admin_http = yes
  This option enables support for unencrypted HTTP requests. Basically you shouldn't enable this unless your Mattermost is working unencrypted as well.
- admin_port = 7088
  The port for unencrypted requests is selected in accordance with port scheme given before.
- admin_https = yes
  You always need to enable support for request over HTTPS when your Mattermost is working over HTTPS, too.
- admin_secure_port = 7089
  Again, it's possible to explicitly choose port for those requests, so we select the one matching port scheme.
On enabling HTTPS for either API related certificates and private key must be given. This happens in section [certificates]:
- cert_pem = /etc/ssl/some-certificate.pem
  and
  cert_key = /etc/ssl/some-private-key.key
  These two options are required for setting up HTTPS, again. They select the same files as before on adjusting configuration.

Mattermost relies on support for WebSockets on using regular requests. The related configuration file must be created from sample first:

cd /opt/janus/etc/janus
cp janus.transport.websockets.cfg.sample janus.transport.websockets.cfg

The resulting file /opt/janus/etc/janus/janus.transport.websockets.cfg must be adjusted using nano. It's structure is mostly equivalent to the one used for setting up HTTP RESTful API before.

The API for regular requests is configured in section [general].
- ws = yes
  and
  ws_port = 8188
  enable unencrypted WebSockets requests on port 8188.
- wss = yes
  and
  wss_port = 8189
  enable encrypted WebSockets requests on port 8189.
Accordingly, administrative API over WebSockets is configured in section [admin]:
- admin_ws = yes
  and
  admin_ws_port = 7188
  enable unencrypted WebSockets requests for administration on port 7188. Don't enable this unless your Mattermost is working over unencrypted HTTP, too.
- admin_wss = yes
  and
- admin_wss_port = 7189
  expose support for encrypted WebSockets requests on port 7189.
Again, files containing proper certificates and private key must be configured in section [certificates]:
- cert_pem = /etc/ssl/some-certificate.pem
  and
  cert_key = /etc/ssl/some-private-key.key
  point to the same files as used in previous configurations.

Fixing Janus Talking to STUN

Due to having selected same machine for STUN service using its FQDN you need to map this to your container's local IP so requests from Janus WebRTC Gateway to STUN service are seen by STUN service coming from container's local IP which is configured to be related to the host's public IP in coturn configuration option external. You need to adjust /etc/hosts for that by adding this line

10.0.3.194 webrtc.mycompany.com

Time for Testing

Try starting Janus WebRTC Gateway using systemd:

systemctl start janus

Again, you should run

systemctl status janus

to check if service was starting properly. This might fail after some timeout if STUN/TURN service can't be accessed initially for testing its availability.

If service is running you should enable it with

systemctl enable janus

so it is started on system startup, too.

Exposing Ports

When running both janus and coturn some ports are by these tools. Using netstat for listing used ports becomes slightly difficult due to coturn running quite a lot of listeners. But you might filter to list all actually used ports like this:

netstat -tulpen | awk '$1~/^tcp|^udp/{print $4}' | \
    awk -F : '{print $NF}' | sort | uniq

This commonly lists all open ports on your server in ascending order.

coturn is listening on these ports for communication over TCP and UDP:

3478 and 3479 are plain STUN/TURN ports for unencrypted communication.
5349 and 5350 are STUN/TURN ports for TLS-encrypted communication.

Another TCP-only port is used:

5766 is used locally, only, for interacting with CLI tools of coturn.

Janus is providing its APIs on several TCP ports explicitly configured before. Just to remember:

8088 and 8089 handle reqular request to HTTP RESTful API.
7088 and 7089 do the same for administrative requests to HTTP RESTful API.
8188 and 8189 handle regular requests to WebSockets API.
7188 and 7189 are used for administrative requests using WebSockets API.

In addition it is listening on ports 50336, 5002 and 5004 for UDP packets. These ports are used for actually sending RTP data for transmitting and distributing audio/video.

Let regroup these lists by UDP and TCP for selecting ports to be published on host's firewall:

UDP: 3478, 3479, 5349, 5350, 50336, 5002, 5004
TCP: 3478, 3479, 5349, 5350, 7088, 7089, 8088, 8089, 8188, 8989, 7188, 7189

Leaving container by pressing Ctrl+D the following commands are executed on host running LXC container.

Both lists of ports are published by forwarding incoming traffic on these ports to container using iptables:

iptables -t nat -A PREROUTING -p udp -m multiport -d 1.2.3.4/32 \
    --dports 3478,3479,5349,5350,50336,5002,5004 \
    -j DNAT --to-destination=10.0.3.194
iptables -t nat -A PREROUTING -p tcp -m multiport -d 1.2.3.4/32 \
    --dports 3478,3479,5349,5350,8088,8089,7088,7089,8188,8989,7188,7989 \
    -j DNAT --to-destination=10.0.3.194

For sake of security you should not expose administrative API on unencrypted ports 7088 and 7188 unless you need it actually, e.g. due to using Mattermost over HTTP, too.

Replace 1.2.3.4 in either command with the public IP of your host.

These commands highly depend on the way your host is running LXC containers and what other rules have been declared in table nat of iptables. Thus you might need to insert the rules rather than appending. In addition you might need to save the added rules manually to make same permanent:

iptables-save >/etc/iptables/rules.v4

Again, this heavily depends on your host's configuration.

Finally Setting Up Mattermost

At this point the WebRTC gateway service is working properly and it's time to set up Mattermost to use it.

Open System Console and choose WebRTC (Beta) in section Integrations. From top to bottom:

Enable Mattermost WebRTC.
Provide gateway websocket URL wss://webrtc.mycompany.com:8189.
Related gateway admin URL is https://webrtc.mycompany.com:7089/admin.
Gateway admin secret literally matches secret string set in main configuration of Janus WebRTC gateway as admin_secret.
The STUN URI is stun://webrtc.mycompany.com:5349.

When your Mattermost service is running without encryption over HTTP you might need to use different URLs:

gateway websocket URL must be ws://webrtc.mycompany.com:8188.
gateway admin URL is http://webrtc.mycompany.com:7088/admin.

TURN server mustn't be set up by default, but might be required in your configuration. In that case fill in URLs and credentials as given on setting up coturn before.

After saving settings you may return to chat view. All users interested in video chatting need to enable this pre-release feature in section Extended of their Account Settings. In the end you should be able to klick on peer user's icon and start video call.

Go back