Monday, September 19, 2022

Run Quectel's Connection Manager as a systemd service on Ubuntu

I found myself in possession of a modem by Quectel (https://www.quectel.com) to hook up to a Raspberry Pi running Ubuntu 20.04. I wanted to auto run Quectel's connection manager executable quectel-CM on system start up but found limited information available online from Quectel. So I had to roll up my sleeves and make my own systemd service for that purpose.

This post outlines the steps I took:

Download and compile the quectel-CM executable

  1. Using an Internet browser, download the latest Quectel LTE, 5G Linux USB driver, e.g. Quectel_LTE5G_Linux_USB_Driver_V1.0.zip from https://www.quectel.com/download-zone.
     
  2.  Extract QConnectManager_Linux_V1.0.zip into a folder e.g. /home/ubuntu/Downloads/quectel-CM/.

  3. In a Terminal, change directory to the extracted folder.

    $ cd ~/Downloads/quectel-CM/

  4. Use the make command to compile the connection manager executable.

    $ make

    The quectel-CM executable is compiled.

Place the quectel-CM exe to the run time directory

  1. In a Terminal, type in the following commands to change directory to the extracted directory.

    $ cd /home/ubuntu/Downloads/quectel-CM/

  2. Copy the executable to /usr/local/bin/.

    $ sudo cp quectel-CM /usr/local/bin

Create a systemd service file

  1.  In the directory /etc/systemd/system, use a text editor to create a service file e.g. quectelcm.service.

    $ sudo vi /etc/systemd/system/quectelcm.service

  2. Type in the following lines, save and exit the file:
    [Unit]
    Description=Systemd service for running Quectel's Connection Manager quectel-CM executable
    
    [Service]
    ExecStart=/usr/local/bin/quectel-CM 
    Restart=always
    RestartSec=5
    
    [Install]
    WantedBy=multi-user.target
    

  3. Give the service file the following executable permission:

    $ sudo chmod 664 /etc/systemd/system/quectelcm.service

 Create the service

  1. In the Terminal, type in the following commands:

    $ sudo systemctl daemon-reload
    $ sudo systemctl enable quectelcm

    The quectelcm service is created.

 

Running and monitoring the service

  1. You can either reboot the Raspberry Pi or type in the following command to start the service in a Terminal.

    $ sudo systemctl start quectelcm

  2. To monitor the quectelcm service, use the journalctl command:

    $ journalctl -u quectelcm -f


Note: In order for quectel-CM to request and set the machine's network address, the software prerequisites net-tools and udhcpc must be installed on the Raspberry Pi.


Monday, July 11, 2022

Blender: how to make an edge flush with another edge by snapping and scaling

Making an plane edge geometry flush with another plane geometry is a typical CAD task. In Blender 3D, it is possible to do the same using simple Move and Scale commands, but with the right settings. 

The screenshot below shows 2 plane objects in the top view. This post shows the steps to adjust the bottom plane to snap nicely to the plane on the left.

Set to Snap to Vertex

  1.  In the Blender 3D Viewport menu bar, click the Snap icon and choose Vertex as shown below.




Snap the vertex of one plane to the vertex of the other plane

  1. In the Blender 3D Viewport, make sure the Edit Mode is enabled and Select Mode is Vertex.

  2. In the Blender Toolbar, select the Move icon. Then click on the bottom vertex of the plane to be moved.

  3. On the keyboard, press down CTRL and drag the bottom vertex to snap to the vertex of the other plane.


     

 Scale the edge vertices by axis to zero

  1. In the Blender 3D Viewport menu, set the Transform Pivot Point to Active Element as shown below.




     
  2. On the keyboard, press ALT-a to deselect all selection.

  3. In the Toolbar, click the Select Box icon.




  4. Click the top vertex of the plane to be scaled.

  5. Press SHIFT and click the bottom vertex.

    Note: since the Transform Pivot Point is set to Active Element, the scaling will be relative to the last selected vertex.

  6. Press S.

    The plane scales dynamically around the last selected vertex as the cursor is moved.


  7. Limit the direction of scaling to the X-axis (in this example) which is perpendicular to the matching edges. Press X.

    The plane can only scale along the X axis.


  8. Press 0.

    The planes are snapped nicely to one another.

Monday, June 27, 2022

WebApp for Singapore Weather forecasts rewritten using Flutter

I thought the old Singapore Weather Nowcast WebApp at this site https://dominoc925-pages.appspot.com/webapp/weather_sg/default.html written using the AngularJS framework and Bootstrap was a good candidate for a Flutter app. So I rolled up the sleeves and rewrote it using Flutter. Now it has Material design and this is how it looks and works.

4 day outlooks

For the Flutter version, I included additional data from the National Environment Agency (NEA)'s 4 day outlooks with temperature, wind and humidity values.


2 hour nowcasts

The 2 hour weather nowcast is now displayed in a responsive 2 pane display.


On small screens, a tab bar will appear displaying the nowcast list and map on separate tabs. 

 

24 hour forecasts

The new 24 hour forecasts are separated by 3 tabs as shown in the screenshot below. Clicking each tab will show the forecast for that time period. And like the 2 hour nowcast display, the 24 hour forecasts display is also responsive to screen sizes.

To try out the new Flutter WebApp, visit this web site: https://dominoc925-pages.appspot.com/webapp/weather_sg/default.html.


Monday, May 16, 2022

Convert Geoids from BYN to GTX format with this WebApp

This WebApp came about because I wanted to use some Geoid files for some vertical datum corrections with Proj (which uses .gtx format) but could only find Geoid files in Natural Resources Canada's .byn format.

  1. To convert Geoid BYN files to GTX, open up a browser to the following url: https://dominoc925-pages.appspot.com/webapp/byn2gtx/index.html.


    The WebApp is loaded.





     
  2. Click the Add File button.

    The File Upload dialog appears.




  3. Browse and select a .byn file e.g. EGM96.byn. Click Open.


    The Geoid file is displayed in a list and the Convert button is enabled.
    Note: the Geoid .byn is loaded locally to the browser and not transferred to some server on the Internet.



  4. Click the Convert button.

    The Geoid file is converted into .gtx format and the Process log and Save As icons are enabled.



  5. Optional. To view the process log, click the Show Process Log icon.

    The process log messages are displayed.



  6. To save the converted .gtx Geoid locally, click the Save Converted File icon.

    The Geoid file is saved.


  7. Optional. If necessary, you can use a GIS software like QGIS to display the converted .gtx file, as shown below.


     

Monday, May 2, 2022

Time Converter App for time conversion in the browser developed with Flutter

I recently learned Flutter (https://flutter.dev/) - a cross platform SDK for building native Android, IOS, Windows, Linux, Web, and MacOS applications with a single code base. For practice, I built a simple application - a Time Converter App for converting time values from various formats to other formats. The formats include GPS time, Julian date, Unix epoch, and UTC date time.

Using the Time Converter App

  1. To use the application, open up a browser and go to https://dominoc925-pages.appspot.com/webapp/timeconverterapp/.

    The app page is loaded.


  2. In the Convert from drop down field, select the source time format e.g. gps.


     

  3. In the to format drop down field, select the destination time format e.g.  utc.

  4. In the entry fields, click the clock icon to use the current system time values. Or type in the values e.g. 2207 as shown below.




  5. Click the blue Convert button.

    If the input values are valid, then the converted time value will be shown in the Output field.


Monday, March 28, 2022

Workaround for gphoto2 with certain Sony camera models problem on system restart

For some Sony camera models e.g. the Sony Alpha-A7R III, gphoto2 will not be able to connect and/or trigger an image capture after a Linux computer system restart with the camera powered on. This issue is described in more detail at the gphoto2 github repository https://github.com/gphoto/gphoto2/issues/279 but it is currently unresolved.

gphoto2 will be able to work again with the Sony camera if you physically extract out and plug in the USB cable again. But I wanted to do it programmatically. I found this great utility uhubctl at https://github.com/mvp/uhubctl that I can use with a supported smart USB hub to switch a USB port power off and on. You need to connect the Sony camera to the USB hub, and the hub to a Linux PC, Ubuntu in my case.

In this post, I describe the steps I use to setup and programmatically switch the power to the USB port of the hub connecting to the camera (using a Canon for illustration as I don't have a Sony on hand).

Install uhubctl

  1. On the Linux PC, open up a Terminal and install uhubctl.

    $ sudo apt install uhubctl

Identify the USB hub's vendor id and product id

  1. In a Terminal, type in the following command:

    $ lsusb

    A list of usb devices is shown.



  2. Note down the vendor id and product id of the smart USB hub.

    05e3:0610 is identified as the vendor id and product id of the USB hub.
     

Identify the port number of the USB hub that is connected to the camera

  1. In a Terminal, type in the following command:

    $ sudo uhubctl

    A listing of hubs and ports is displayed.


  2. First, look for the smart hub with the previously noted vendor id and product id, e.g. 05e3:0610.
     
  3. Then under the selected hub, identify the port with the Sony(Canon) camera connected.

    Port number 1 is identified.

Switch off the port to the camera

  1. In a Terminal, type in the following command to switch off the USB port of the camera:

    $ sudo uhubctl -n 05e3:0610 -p 1 -a 0

    where -n 05e3:0610 is the vendor id and product id of the USB hub,
    -p 1 is the port number of the camera
    -a 0 is to power off




Switch on the port to the camera

  1.  In a Terminal, type in the following command to switch on the port to the camera:

    $ sudo uhubctl -n 05e3:0610 -p 1 -a 1

    where -n 05e3:0610 is the vendor id and product id of the USB hub
    -p 1 is the port number of the camera
    -a 1 is to power on



Use gphoto2

From this point on, the Sony camera should be functional again from gphoto2.


Monday, March 21, 2022

Setup a Raspberry Pi 4B to publish a webcam with hardware accelerated video encoding

The Raspberry Pi 4B has an integrated video processing unit, a so called GPU. I wanted to use the GPU to offload the video encoding processing from the CPU when I publish my webcam as a RTSP video stream from the Pi board running Ubuntu 20.04 64 bit and using ffmpeg

Download and install rtsp-simple-server

  1. Using a browser, download and extract the open source rtsp-simple-server from https://github.com/aler9/rtsp-simple-server into a folder, e.g. /path/to/rtsp/

    The files rtsp-simple-server and rtsp-simple-server.yml are extracted out.

  2. Open up a Terminal. At the prompt, change directory to the previously created folder /path/to/rtsp/.

    $ cd /path/to/rtsp/

  3. Move the binary to the folder /usr/local/bin/

    $ sudo mv rtsp-simple-server /usr/local/bin/

  4. Move the yml configuration file to the folder /usr/local/etc/

    $ sudo mv rtsp-simple-server.yml /usr/local/etc/

Configure rtsp-simple-server to publish the webcam from ffmpeg

  1.  Open up a Terminal. At the prompt, change directory to the directory /usr/local/etc/.

    $ cd /usr/local/etc

  2. Using a text editor, edit the configuration file rtsp-simple-server.yml. Create a new path name, e.g. cam under the paths key.

    paths:
      cam:
        runOnDemand: ffmpeg -hide_banner -s 1280x720 -r 25 -i /dev/video0 -b:v 2M -c:v h264_v4l2m2m -f rtsp rtsp://localhost:$RTSP_PORT/$RTSP_PATH
        runOnDemandRestart: yes
    

    Notes:
    -s 1280x720 is the source video resolution size from the webcam
    -r 25 is the source sample rate
    -i /dev/video0 is the source webcam video
    -b:v 2M is the output video bit rate
    -c:v h264_v4l2m2m is the video encoder to use


  3. Save and close the configuration file.

 Create and start the rtsp-simple-server service

  1.  Open up a Terminal. Type in the following to create the systemd service.

    sudo tee /etc/systemd/system/rtsp-simple-server.service >/dev/null << EOF
    [Unit]
    After=network.target
    [Service]
    ExecStart=/usr/local/bin/rtsp-simple-server /usr/local/etc/rtsp-simple-server.yml
    [Install]
    WantedBy=multi-user.target
    EOF
    

  2. Enable and start the service by executing the following commands. Or reboot the system.

    $ sudo systemctl enable rtsp-simple-server
    $ sudo systemctl start rtsp-simple-server

Viewing the webcam stream using VLC

  1. If the configuration parameters are correct, then start up VLC on a PC on the network.

    VLC starts up.


  2. Select Media | Open Network Stream.

    The Open Media dialog box appears.


  3. In the URL field, type in the address of the Raspberry Pi e.g. rtsp://192.168.18.5:8554/cam. Then click Play.

    If all the parameters are correct, then VLC should show a stream from the webcam.

    VLC displaying a webcam stream.


    Note: if the video stream looks wrong i.e. greenish and weird as shown above, then the ffmpeg version installed on the Raspberry Pi is an older buggy version 4.2.4 and you need to replace it by downloading the latest ffmpeg and building from source.

Replacing ffmpeg and build from source

  1. Open up a Terminal and remove the system installed ffmpeg with the following command:

    $ sudo apt remove ffmpeg

  2. Using a browser and follow the instructions on https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu to install the latest ffmpeg.

  3. Compilation on the Raspberry Pi 4B may take a while, maybe an hour or two so be aware. After compiling successfully, you can try to use VLC and open up the webcam steam again.

    VLC displaying the webcam stream correctly.

Monday, January 24, 2022

How to setup rtsp-simple-server service for publishing from gstreamer using the vaapi encoder

The rtsp-simple-server README instructions on https://github.com/aler9/rtsp-simple-server is a little sparse on publishing as a systemd service using Gstreamer and the vaapi plugin. I spent some time trying to get the service to run properly. This post documents the steps I used on an Intel system board running Ubuntu 20.


Install GStreamer

  1. Optional. If Gstreamer has not been installed, then follow the instructions on https://gstreamer.freedesktop.org/documentation/installing/on-linux.html?gi-language=c.

  2. If the GStreamer VAAPI plugin has not been installed, run the following command in a Terminal:

    $ sudo apt install gstreamer1.0-vaapi

 Download and install rtsp-simple-server

  1. Using a browser, download and extract the rtsp-simple-server binary from the github repo https://github.com/aler9/rtsp-simple-server into a folder, e.g. /path/to/rtsp/

    The files rtsp-simple-server and the rtsp-simple-server.yml are extracted out into the directory /path/to/rtsp/.


  2. Open up a Terminal. At the prompt, change directory to the extracted location.

    $ cd /path/to/rtsp/

  3. Move the binary to the folder /usr/local/bin/

    $ sudo mv rtsp-simple-server /usr/local/bin/

  4. Move the configuration file to the folder /usr/local/etc/

    $ sudo mv rtsp-simple-server.yml /usr/local/etc/

Configure rtsp-simple-server to publish from gstreamer

  1. Open up a Terminal. At the prompt, change directory to the directory /usr/local/etc/.

    $ cd /usr/local/etc/

  2. Using a text editor, edit the configuration file rtsp-simple-server.yml. Create a new path name e.g. mystream under the paths key.

    paths:
        mystream:
            runOnDemand: gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,framerate=30/1,width=320,height=240' ! videoconvert ! vaapih264enc ! h264parse ! rtspclientsink location=rtsp://localhost:$RTSP_PORT/$RTSP_PATH
            runOnDemandRestart: yes
    

  3. In the runOnDemand key, type in the Gstreamer pipeline to read from a video source, perform encoding and finally to publish to the rtsp server.

    gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,framerate=30/1,width=320,height=240' ! videoconvert ! vaapih264enc bitrate=200000 ! h264parse ! rtspclientsink location=rtsp://localhost:$RTSP_PORT/$RTSP_PATH
    
    Note 1: device points to the video source.
    Note 2: video/x-raw line specifies the video format, frame rate, and resolutions to use
    Note 3: videoconvert converts the color space
    Note 4: vaapih264 performs the video encoding using the Intel GPU
    Note 5: rtspclientsink publishes the video to the rtsp server


  4. Save and close the configuration file.

Create and start the rtsp-simple-server service

  1.  Open up a Terminal. Type in the following to create the systemd service.

    sudo tee /etc/systemd/system/rtsp-simple-server.service >/dev/null << EOF
    [Unit]
    After=network.target
    [Service]
    ExecStart=/usr/local/bin/rtsp-simple-server /usr/local/etc/rtsp-simple-server.yml
    [Install]
    WantedBy=multi-user.target
    EOF
    


  2. To enable and start the service, run the following:

    $ sudo systemctl enable rtsp-simple-server
    $ sudo systemctl start rtsp-simple-server

Optional. Define LIBVA environment variables

On some boards, when using the vaapi encoding plugin, gstreamer may not run with the error message:

WARNING: erroneous pipeline: no element "vaapih264enc"

In this case, setting the environment variables LIBVA_DRIVERS_PATH and LIBVA_DRIVER_NAME for the service may solve the problem. An example of the command to create the rtsp-simple-server service with the environment variables is shown below:

sudo tee /etc/systemd/system/rtsp-simple-server.service >/dev/null << EOF
[Unit]
After=network.target
[Service]
EnvironmentName="LIBVA_DRIVERS_PATH=/usr/lib/x86_64-linux-gnu/dri/"
EnvironmentName="LIBVA_DRIVER_NAME=i965"
ExecStart=/usr/local/bin/rtsp-simple-server /usr/local/etc/rtsp-simple-server.yml
[Install]
WantedBy=multi-user.target
EOF

Monday, January 17, 2022

gstreamer command to encode videos using an Intel GPU on Ubuntu

An alternative to ffmpeg is the gstreamer library, which comes with optional plug-ins to perform video encoding using Intel GPUs. 

Gstreamer can be installed on Ubuntu by following instructions on  https://gstreamer.freedesktop.org/documentation/installing/on-linux.html?gi-language=c

Assuming gstreamer has been installed on Ubuntu, you can run the following command to save the video into an output.mp4 video file.

$ gst-launch-1.0 \
v4l2src device=/dev/video0 num-buffers=300 ! \
'video/x-raw,framerate=10/1,width=1280,height=720' ! \
videoconvert ! \
vaapih264enc ! \
h264parse ! \
filesink location=output.mp4

Note 1: device specifies the video source /dev/video0 and num-buffers specifies the number of frames to read.

Note 2: The line video/x-raw specifies the format, frame rate and resolution to read from the video source.

Note 3: vaapih264enc specifies the Intel Video Accelerated encoder to use.

Note 4: filesink location specifies the output video file.

While the gstreamer command is processing the video, in another terminal, run the command to monitor the Intel GPU.

$ sudo intel_gpu_top

As shown above, the printout in red indicates the GPU is being used.

Caution: On some Intel boards I have tested, sometimes running the gstreamer vaapih264enc plugin resulted in the following error message even though the plugin has been installed:

WARNING: erroneous pipeline: no element "vaapih264enc"

In my case, I managed to resolve that error by setting the following environment variables before running the encoding command:

$ export LIBVA_DRIVERS_PATH=/usr/lib/x86_64-linux-gnu/dri/

$ export LIBVA_DRIVER_NAME=i965