[Raspberry Pi] - Advanced Video Surveillance with ustreamer and motion/motion-UI

Introduction

https://raw.githubusercontent.com/lbr38/documentation/main/docs/images/raspberrypi/motion/cctv.jpg

Video surveillance has been one of the flagship projects for Raspberry Pi since the early days of the board. Tools like motion and dedicated operating systems like motionPie have quickly become a reference for this type of use.

However, limitations are quickly felt despite the advancements of the board. Video encoding is a CPU and RAM-intensive process, and the Raspberry Pi quickly becomes overloaded when setting up an advanced video surveillance system with multiple cameras.

Solution

The proposed solution here attempts to address the overload issue by separating tasks:

The ARM boards (Raspberry Pi or others) are responsible for capturing and streaming the video feed generated by the connected camera.
A central server receives the streams and handles the resource-intensive tasks of motion detection and encoding (motion).

The “camera boards” are placed in different locations inside or outside the monitored premises, taking necessary precautions to protect them from moisture if applicable.

The server is kept indoors, ensuring its safety, as it serves as the central point from which the user can review all events and view the live stream.

Everything is connected to the local network using wired connections. We exclude Wi-Fi cameras here as they can be easily disabled with a simple Wi-Fi jammer.

Here’s an illustration:

https://raw.githubusercontent.com/lbr38/documentation/main/docs/images/raspberrypi/motion/motion.png

Prerequisites

1 or more Raspberry Pi (or competing boards) for the “camera” part.

Their power can be low to moderate since their role is only to stream the feed without processing.

Personally, I use Orange-Pi zero LTS (4CPU, 512MB RAM, 1 Ethernet port, and 1 USB port). Their compact size allows them to be easily installed anywhere, and their POE port enables powering them with a single Ethernet cable. For the camera, I use a waterproof USB dome camera purchased from Amazon connected to the Orange-Pi’s USB port.

1 central server

Preferably a “home server,” as powerful as possible. It is advisable to avoid using an ARM board that may quickly become overwhelmed during video processing. Keep in mind that the more cameras there are, the heavier the processing will be. The server should run an OS such as Debian, CentOS…

Prepare each component:

Install the necessary OS (e.g., Raspbian or Armbian) on each ARM board and on the central server (e.g., Debian 11).
Update system packages and firmware if needed.
Configure fixed IP addresses for the boards and the server.

Configuration

For the rest of the article:

I will assume that the surveillance installation is done on a Raspberry Pi board (as it is the most common), connected to a USB camera or dome, and running a Debian-based OS (Armbian/Raspbian).
I will assume that the server is running a Debian-based OS.

Furthermore, the package manager used will be apt, and the package names installed will be specific to Debian systems (the names may vary if you decide to use a different OS).

Camera Configuration

The goal here is to set up ustreamer to stream camera feeds over http.

The advantage of ustreamer compared to the well-known mjpg-streamer is that it offers more options, is clearer and easier to use, and, most importantly, it generates more native logs than mjpg-streamer, which greatly facilitates debugging.

Repeat this section “Camera Configuration” for each Raspberry Pi connected to a USB camera.

Note: All configurations are done as root.

Ustreamer

Connect to the Raspberry Pi via ssh and install some necessary packages for compilation:

apt install git make gcc build-essential

Start the installation of ustreamer by compiling it (it’s easy), but first, you need to install some additional libraries:

# If Raspbian (Raspberry Pi OS):
apt install libevent-dev libjpeg8-dev libbsd-dev

# If using a different OS, see: https://github.com/pikvm/ustreamer#building

# Then, clone the ustreamer project:
cd /home/pi/
git clone --depth=1 https://github.com/pikvm/ustreamer

# And compile:
cd ustreamer
make

Check with lsusb if the connected USB camera is recognized by the system. In my case, with the USB dome camera, it displays:

lsusb
Bus 001 Device 008: ID 05a3:9230 ARC International Camera      # USB Camera
Bus 001 Device 009: ID 0424:7800 Standard Microsystems Corp.
Bus 001 Device 007: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub
Bus 001 Device 006: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Creating the stream start and stop scripts, the user pi will execute these scripts:

mkdir -p /home/pi/scripts/stream

Stream start script:

vim /home/pi/scripts/stream/start-stream.sh

Insert the following content:

#!/bin/bash

DATE=$(date +%Y-%m-%d)
TIME=$(date +%Hh%M)
RESOLUTION="1920x1080"
FRAMERATE="25"
USTREAMER="/home/pi/ustreamer/ustreamer"
LOG="/home/pi/scripts/stream/ustreamer.log"


function help()
{
    echo "Usage: $0 [options]"
    echo "Options:"
    echo "  --1080p"
    echo "  --720p"
    echo "  --low"
    echo "  --fps=FRAMERATE"
    echo "  --help"
}

while [ $# -ge 1 ];do
    case "$1" in
        --1080p)
            RESOLUTION="1920x1080"
        ;;
        --720p)
            RESOLUTION="1280x720"
        ;;
        --low)
            RESOLUTION="640x480"
        ;;
        --fps)
            FRAMERATE="$2"
            shift
        ;;
        --help)
            help
            exit
        ;;
        *)
    esac
    shift
done

# Cleaning log file
echo -n> "$LOG"
exec &> >(tee -a "$LOG")

echo "$DATE - $TIME - Starting stream"

"$USTREAMER" --device=/dev/video0 --slowdown --workers 2 -e 30 -K 0 -r "$RESOLUTION" -m MJPEG --host 0.0.0.0 --port 8888 --device-timeout 2 --device-error-delay 1 2>&1 &

exit

Stream stop script:

vim /home/pi/scripts/stream/stop-stream.sh

Insert the following content:

#!/bin/bash

# Search for the process ID of ustreamer
PID="$(/bin/ps -aux | /bin/grep 'ustreamer' | egrep -v 'grep|ustreamer.log' | /usr/bin/awk '{print $2}')"

if [ -z "$PID" ];then
    echo "No active process found"
    exit
fi

echo "Stopping ustreamer... "
kill "$PID" > /dev/null 2>&1
sleep 1

# Check if the process is still running
if /bin/ps -aux | /bin/grep 'ustreamer' | egrep -v 'grep|ustreamer.log';then
    echo "Process is still running, killing it"
    kill -9 "$PID"
    exit
fi

echo "OK"

exit

Adjust the permissions for what was just created:

chmod 700 /home/pi/scripts/stream/*.sh
chown -R pi:pi /home/pi/scripts

Temporarily log in as pi and start the stream to test. It is possible to specify a resolution and framerate as parameters for the start stream script. By default, the stream is launched with 1920x1080 resolution and 25 fps:

su pi
/home/pi/scripts/stream/start-stream.sh &

# Example to start the stream in 720p and 30 fps:
/home/pi/scripts/stream/start-stream.sh --720p --fps 30 &

It should display some logs on the screen.

Open http://CAMERA_IP_ADDRESS:8888 in a browser, the ustreamer homepage should be accessible, and the stream can be viewed by clicking on /stream.

Still as pi, create a cron task that will automatically start the stream after rebooting the Raspberry Pi:

crontab -e

@reboot /home/pi/scripts/start-camera.sh &

Server Configuration

The goal here is to set up motion-UI (a web interface for motion) to analyze the camera streams in the house and detect motion.

Notes:

The system used here is Debian 11.
All configurations are performed as root.

motion-UI

Overview

motion-UI is a web interface developed to manage the operation and configuration of motion more easily.

It is an open-source project available on GitHub: https://github.com/lbr38/motion-UI

The interface presents itself as very simplistic and responsive, allowing for mobile usage (Android application available here: https://github.com/lbr38/motion-UI/releases/tag/android-1.0).

It also allows the setup of email alerts in case of motion detection, and it can automatically enable or disable video surveillance based on a specified time range or the presence of “trusted” devices on the local network (e.g., smartphones).

The interface is divided into several tabs:

An tab dedicated to cameras and live stream. The cameras are then arranged in grids on the screen (at least on a PC screen), somewhat like the surveillance screens of a facility, for example.
An tab for starting and stopping the service motion and associated services (automatic startup, alerts in case of detection).
An tab listing the events that have occurred and been detected by motion, with the ability to view the images or videos captured directly from the web page.
An tab with a few graphs summarizing the recent activity of the motion service and the events that have occurred.

Installing docker

Start by installing the package repository for docker:

apt install ca-certificates curl gnupg -y

sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

echo \
"deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \
"$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Then install docker :

apt update -y
apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin -y

Installation of motion-UI

The installation should be done with a regular user (non-root).

Pull the latest available image and adapt the FQDN value to your domain name:

docker run -d --restart always --name motionui \
   -e FQDN=motionui.example.com \
   -p 8080:8080 \
   -v /etc/localtime:/etc/localtime:ro \
   -v /var/lib/docker/volumes/motionui-data:/var/lib/motionui \
   -v /var/lib/docker/volumes/motionui-captures:/var/lib/motion \
   lbr38/motionui:latest

Two persistent volumes are created on the host system:

motionui_data /var/lib/docker/volumes/motionui-data/: contains the motion-UI database.
motionui-captures /var/lib/docker/volumes/motionui-captures/: stores the images and videos captured by motion (keep this data!).

Once the installation is complete, motion-UI is accessible directly (without SSL certificate for now) at http://<SERVER_IP>:8080

Use the default credentials to authenticate:

Login: admin
Password: motionui

After logging in, you can change your password from the user profile (top right corner).

Proceed with setting up a reverse-proxy to access motion-UI with a dedicated domain name and SSL certificate.

Reverse-proxy nginx

Install nginx:

apt install nginx -y

Remove the default vhost:

rm /etc/nginx/sites-enabled/default

Then create a new vhost dedicated to motion-UI:

vim /etc/nginx/sites-available/motionui.conf

Insert the following content, adapting certain values:

The parameter <SERVER-IP> should be set to the server’s IP address.
The parameters <FQDN> should be set to the dedicated domain name for motion-UI.
The paths to the SSL certificate and associated private key (<PATH-TO-CERTIFICATE> and <PATH-TO-PRIVATE-KEY>) should be provided accordingly.

upstream motionui_docker {
    server 127.0.0.1:8080;
}

# Disable some logging
map $request_uri $loggable {
    /ajax/controller.php 0;
    default 1;
}

server {
    listen <SERVER-IP>:80;
    server_name <FQDN>;

    access_log /var/log/nginx/<FQDN>_access.log combined if=$loggable;
    error_log /var/log/nginx/<FQDN>_error.log;

    return 301 https://$server_name$request_uri;
}

server {
    listen <SERVER-IP>:443 ssl;
    server_name <FQDN>;

    # Path to SSL certificate/key files
    ssl_certificate <PATH_TO_CERTIFICATE>;
    ssl_certificate_key <PATH_TO_PRIVATE_KEY>;

    # Path to log files
    access_log /var/log/nginx/<FQDN>_ssl_access.log combined if=$loggable;
    error_log /var/log/nginx/<FQDN>_ssl_error.log;

    # Security headers
    add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload;" always;
    add_header Referrer-Policy "no-referrer" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-Download-Options "noopen" always;
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Permitted-Cross-Domain-Policies "none" always;
    add_header X-Robots-Tag "none" always;
    add_header X-XSS-Protection "1; mode=block" always;

    # Remove X-Powered-By, which is an information leak
    fastcgi_hide_header X-Powered-By;

    location / {
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_pass http://motionui_docker;
    }
}

Create a symbolic link to enable the vhost:

ln -s /etc/nginx/sites-available/motionui.conf /etc/nginx/sites-enabled/motionui.conf

Restart nginx to apply the changes:

nginx -t && systemctl restart nginx

motion-UI is now accessible at https://<FQDN>

Adding a Camera

Use the + button to add a camera.

Specify if the camera provides a video stream or just a static image that requires reloading (if yes, specify the refresh interval in seconds).
Provide a name and the URL to the camera’s video/image stream.
Choose to enable motion detection on this camera. Note that if the selected stream is a static image, a second URL pointing to a video stream needs to be specified because motion is unable to perform motion detection on a stream of static images (it is not capable of automatically reloading the image).
Specify a username/password if the stream is protected.

Once the camera is added, motion-UI automatically creates the motion configuration for this camera. Note that the created motion configuration is relatively minimalistic but sufficient to function in all cases. It is possible to modify this configuration in advanced mode and add custom parameters if needed (see Camera Configuration section).

Camera Configuration

If there is a need to modify the configuration of a camera, simply click on the Configure button.

From here, it is possible to modify the general settings of the camera (e.g., name, URL, etc.) and change the rotation of the image.

It is also possible to modify the motion configuration of the camera (motion detection).

Please note that it is recommended to avoid modifying motion parameters in advanced mode, or at least not without knowing precisely what you are doing.

For example, it is better to avoid modifying the following parameters:

The name and URL parameters (camera_name, netcam_url, netcam_userpass, and rotate) have values derived from the general camera settings. Therefore, it is necessary to modify them directly from the Global settings fields.
Parameters related to codecs (picture_type and movie_codec) should not be modified, or else you may no longer be able to view the captures directly from motion-UI.
Event parameters (on_event_start, on_event_end, on_movie_end, and on_picture_save) should not be modified, as it may result in the inability to record motion detection events and receive alerts.

Testing Event Recording

To do this from the motion-UI interface: manually start motion by clicking the Start capture button.

Then, make a movement in front of a camera to trigger an event.

If everything goes well, a new ongoing event should appear after a few seconds in the motion-UI interface.

Automatic Start and Stop of Motion

Use the Enable and configure autostart button to activate and configure automatic startup.

Two types of automatic startup and shutdown of motion can be configured:

Based on the specified time ranges for each day. The motion service will be active between the specified time range.
Based on the presence of one or more connected IP devices on the local network. If none of the configured devices are present on the local network, the motion service will start, assuming that no one is present at home. Motion-UI regularly sends a ping to determine if the device is present on the network, so make sure to configure static IP leases from the router for each device at home (smartphones).

Configure Alerts

Use the Enable and configure alerts button to enable and configure the alerts.

Configuring alerts requires two points of configuration:

An SPF record for the domain name dedicated to motion-UI.
The event recording must be working (see ‘Testing Event Recording’).

Alert Time Slots Configuration

Specify the time slots during which you want to receive alerts if motion is detected. To enable alerts for an entire day, enter 00:00 for both the start and end slots.
Enter the recipient email address(es) that will receive the alert emails. Multiple email addresses can be specified by separating them with commas.

Testing Alerts

Once the previously mentioned points have been properly configured and the motionui service is running, you can test the sending of alerts.

To do this from the motion-UI interface:

Temporarily disable motion’s autostart if enabled, to prevent it from stopping motion just in case.
Manually start motion (Start capture).

Then, make a movement in front of a camera to trigger an alert.

Security

Now that the video surveillance system is operational, it is time to secure the entire setup.

I cannot go into detail about all the security configurations to implement, but here are some basic ideas:

The camera streams should only be accessible by the server.

In other words, the access URLs to ustreamer http://CAMERA_IP_ADDRESS:8888 should only be accessible by the server.

To achieve this, establish firewall rules (such as iptables) on the Raspberry Pis to allow only the server to access them via HTTP.

The SSH configuration of the cameras should be strengthened (using key authentication, disallowing root login, etc.).

Ideally, implement firewall rules that allow only the server and possibly another local network IP (as a backup) to connect via SSH.

The server is the central entry point and should be made as secure as possible.

Start by implementing robust firewall rules to allow only certain IPs to connect via SSH from the local network.

Implement a strengthened SSH configuration (using key authentication, disallowing root login, etc.).

If you want to access it from outside (e.g., to access motion-UI), the best solution is to set up a VPN that allows access to the home network from outside (the Freebox router supports this). Another solution would be to set up port forwarding on the router, but in this case, intrusion attempts will be immediate, and the forwarded ports will be constantly scanned by internet bots.