Setup a Raspberry Pi 4B to publish a webcam with hardware accelerated video encoding

The Raspberry Pi 4B has an integrated video processing unit, a so called GPU. I wanted to use the GPU to offload the video encoding processing from the CPU when I publish my webcam as a RTSP video stream from the Pi board running Ubuntu 20.04 64 bit and using ffmpeg. 

Download and install rtsp-simple-server

1. Using a browser, download and extract the open source rtsp-simple-server from https://github.com/aler9/rtsp-simple-server into a folder, e.g. /path/to/rtsp/
   
   The files rtsp-simple-server and rtsp-simple-server.yml are extracted out.
   
   
2. Open up a Terminal. At the prompt, change directory to the previously created folder /path/to/rtsp/.
   
   $ cd /path/to/rtsp/
   
   
3. Move the binary to the folder /usr/local/bin/
   
   $ sudo mv rtsp-simple-server /usr/local/bin/
   
   
4. Move the yml configuration file to the folder /usr/local/etc/
   
   $ sudo mv rtsp-simple-server.yml /usr/local/etc/
   

Configure rtsp-simple-server to publish the webcam from ffmpeg

1.  Open up a Terminal. At the prompt, change directory to the directory /usr/local/etc/.
   
   $ cd /usr/local/etc
   
   
2. Using a text editor, edit the configuration file rtsp-simple-server.yml. Create a new path name, e.g. cam under the paths key.
   
   

   paths:
     cam:
       runOnDemand: ffmpeg -hide_banner -s 1280x720 -r 25 -i /dev/video0 -b:v 2M -c:v h264_v4l2m2m -f rtsp rtsp://localhost:$RTSP_PORT/$RTSP_PATH
       runOnDemandRestart: yes
   

   
   Notes:
   -s 1280x720 is the source video resolution size from the webcam
   -r 25 is the source sample rate
   -i /dev/video0 is the source webcam video
   -b:v 2M is the output video bit rate
   -c:v h264_v4l2m2m is the video encoder to use
   
   
3. Save and close the configuration file.
   

 Create and start the rtsp-simple-server service

1.  Open up a Terminal. Type in the following to create the systemd service.
   
   

   sudo tee /etc/systemd/system/rtsp-simple-server.service >/dev/null << EOF
   [Unit]
   After=network.target
   [Service]
   ExecStart=/usr/local/bin/rtsp-simple-server /usr/local/etc/rtsp-simple-server.yml
   [Install]
   WantedBy=multi-user.target
   EOF
   

   
   
2. Enable and start the service by executing the following commands. Or reboot the system.
   
   $ sudo systemctl enable rtsp-simple-server
   $ sudo systemctl start rtsp-simple-server
   
   

Viewing the webcam stream using VLC

1. If the configuration parameters are correct, then start up VLC on a PC on the network.
   
   VLC starts up.
   
   
   
2. Select Media | Open Network Stream.
   
   The Open Media dialog box appears.
   
   
   
3. In the URL field, type in the address of the Raspberry Pi e.g. rtsp://192.168.18.5:8554/cam. Then click Play.
   
   If all the parameters are correct, then VLC should show a stream from the webcam.
   
   VLC displaying a webcam stream.
   
   
   Note: if the video stream looks wrong i.e. greenish and weird as shown above, then the ffmpeg version installed on the Raspberry Pi is an older buggy version 4.2.4 and you need to replace it by downloading the latest ffmpeg and building from source.
   

Replacing ffmpeg and build from source

1. Open up a Terminal and remove the system installed ffmpeg with the following command:
   
   $ sudo apt remove ffmpeg
   
   
2. Using a browser and follow the instructions on https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu to install the latest ffmpeg.
   
   
3. Compilation on the Raspberry Pi 4B may take a while, maybe an hour or two so be aware. After compiling successfully, you can try to use VLC and open up the webcam steam again.
   
   VLC displaying the webcam stream correctly.
   

How to setup rtsp-simple-server service for publishing from gstreamer using the vaapi encoder

The rtsp-simple-server README instructions on https://github.com/aler9/rtsp-simple-server is a little sparse on publishing as a systemd service using Gstreamer and the vaapi plugin. I spent some time trying to get the service to run properly. This post documents the steps I used on an Intel system board running Ubuntu 20.

Install GStreamer

1. Optional. If Gstreamer has not been installed, then follow the instructions on https://gstreamer.freedesktop.org/documentation/installing/on-linux.html?gi-language=c.
   
   
2. If the GStreamer VAAPI plugin has not been installed, run the following command in a Terminal:
   
   $ sudo apt install gstreamer1.0-vaapi
   
   

 Download and install rtsp-simple-server

1. Using a browser, download and extract the rtsp-simple-server binary from the github repo https://github.com/aler9/rtsp-simple-server into a folder, e.g. /path/to/rtsp/
   
   The files rtsp-simple-server and the rtsp-simple-server.yml are extracted out into the directory /path/to/rtsp/.
   
   
   
2. Open up a Terminal. At the prompt, change directory to the extracted location.
   
   $ cd /path/to/rtsp/
   
   
3. Move the binary to the folder /usr/local/bin/
   
   $ sudo mv rtsp-simple-server /usr/local/bin/
   
   
4. Move the configuration file to the folder /usr/local/etc/
   
   $ sudo mv rtsp-simple-server.yml /usr/local/etc/
   
   

Configure rtsp-simple-server to publish from gstreamer

1. Open up a Terminal. At the prompt, change directory to the directory /usr/local/etc/.
   
   $ cd /usr/local/etc/
   
   
2. Using a text editor, edit the configuration file rtsp-simple-server.yml. Create a new path name e.g. mystream under the paths key.
   
   

   paths:
       mystream:
           runOnDemand: gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,framerate=30/1,width=320,height=240' ! videoconvert ! vaapih264enc ! h264parse ! rtspclientsink location=rtsp://localhost:$RTSP_PORT/$RTSP_PATH
           runOnDemandRestart: yes
   

   
   
3. In the runOnDemand key, type in the Gstreamer pipeline to read from a video source, perform encoding and finally to publish to the rtsp server.
   
   

   gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,framerate=30/1,width=320,height=240' ! videoconvert ! vaapih264enc bitrate=200000 ! h264parse ! rtspclientsink location=rtsp://localhost:$RTSP_PORT/$RTSP_PATH
   

   Note 1: device points to the video source.
   Note 2: video/x-raw line specifies the video format, frame rate, and resolutions to use
   Note 3: videoconvert converts the color space
   Note 4: vaapih264 performs the video encoding using the Intel GPU
   Note 5: rtspclientsink publishes the video to the rtsp server
   
   
4. Save and close the configuration file.
   
   

Create and start the rtsp-simple-server service

1.  Open up a Terminal. Type in the following to create the systemd service.
   
   

   sudo tee /etc/systemd/system/rtsp-simple-server.service >/dev/null << EOF
   [Unit]
   After=network.target
   [Service]
   ExecStart=/usr/local/bin/rtsp-simple-server /usr/local/etc/rtsp-simple-server.yml
   [Install]
   WantedBy=multi-user.target
   EOF
   

   
   
   
2. To enable and start the service, run the following:
   
   $ sudo systemctl enable rtsp-simple-server
   $ sudo systemctl start rtsp-simple-server
   
   

Optional. Define LIBVA environment variables

On some boards, when using the vaapi encoding plugin, gstreamer may not run with the error message:

WARNING: erroneous pipeline: no element "vaapih264enc"

In this case, setting the environment variables LIBVA_DRIVERS_PATH and LIBVA_DRIVER_NAME for the service may solve the problem. An example of the command to create the rtsp-simple-server service with the environment variables is shown below:

sudo tee /etc/systemd/system/rtsp-simple-server.service >/dev/null << EOF
[Unit]
After=network.target
[Service]
EnvironmentName="LIBVA_DRIVERS_PATH=/usr/lib/x86_64-linux-gnu/dri/"
EnvironmentName="LIBVA_DRIVER_NAME=i965"
ExecStart=/usr/local/bin/rtsp-simple-server /usr/local/etc/rtsp-simple-server.yml
[Install]
WantedBy=multi-user.target
EOF

How to quickly publish an RTSP stream from a webcam on Ubuntu

I wanted to quickly publish an RTSP video stream from a webcam on Linux without having to work with the complexity of sources and layers in OBS Studio. I found this neat software RtspSimpleServer downloadable from https://github.com/aler9/rtsp-simple-server. 

 For my testing purposes using just the default parameters, I did the following:

Install and run RtspSimpleServer

1. Using a browser, download and extract the RtspSimpleServer binary from the github repo https://github.com/aler9/rtsp-simple-server into a folder, e.g. /path/to/rtsp/
   
   The files rtsp-simple-server and the rtsp-simple-server.yml are extracted out into a directory /path/to/rtsp/.
   
   
2. Open up a Terminal. At the prompt, type in the cd command to change to the directory of the rtsp-simple-server.
   
   $ cd /path/to/rtsp/
   
   
3. At the prompt, run the rtsp-simple-server server:
   
   $ ./rtsp-simple-server
   
   Process messages appear to show the server is running.
   
   
   
   

Publish a video stream

1. Optional. If the video for linux utils are not installed, then run the apt command to install it.
   
   $ sudo apt install v4l-utils
   
   
2. Open up a new Terminal. Type in the ffmpeg command to publish the stream from the webcam device (assuming /dev/video0).
   
   $ ffmpeg \
   -f v4l2 \
   -framerate 90 \
   -re -stream_loop -1 \
   -video_size 640x320 \
   -input_format mjpeg \
   -i /dev/video0 \
   -c copy \
   -f rtsp \
   rtsp://localhost:8554/mystream
   
   
   
   Processing messages appear.
   
   
    

Open the stream with a VLC client

1. Open up a new Terminal.
   
    
2. Run the vlc command to open the published stream.
   
   $ vlc rtsp://localhost:8554/mystream
   
   
   
   The VLC client pops up to show the RTSP stream from the webcam.