[enh/venv]: feat(install): introduce venv-based runtime setup with proper permissions

Replace the implicit dependency on system-wide Python packages with a dedicated virtual environment and explicit runtime dependency management. Changes: - Create a project-local Python venv during installation - Install runtime dependencies explicitly via requirements-runtime.txt - Fix ModuleNotFoundError issues in isolated environments (e.g. requests) - Detect the actual user when running via sudo (SUDO_USER) - Assign correct ownership to the BOSWatch directory for venv usage - Apply setgid (2775) to the log directory so files created with sudo remain manageable by the user - Set explicit permissions for Python scripts and config files This ensures a reproducible, system-independent installation and prevents permission issues during runtime and log creation.
2026-03-30 17:44:54 +02:00 · 2025-11-21 14:59:15 +01:00 · 2025-11-21 14:59:15 +01:00 · a3f429cbf4
parent 7d4cb57a6e
commit a3f429cbf4
7 changed files with 108 additions and 44 deletions
--- a/docu/docs/develop/ModulPlugin.md
+++ b/docu/docs/develop/ModulPlugin.md
@ -79,7 +79,8 @@ Eine Auflistung der bereitgestellten Informationen findet sich im entsprechenden
 **Bitte beachten:**

 - Selbst vom Modul hinzugefügte Felder **müssen** in der Modul Dokumentation unter `Paket Modifikation` aufgeführt werden.
- Sollte ein Modul oder Plugin Felder benutzen, welche in einem anderen Modul erstellt werden, **muss** dies im Punkt `Abhänigkeiten` des jeweiligen Moduls oder Plugins dokumentiert werden.
+- Sollte ein Modul oder Plugin Felder benutzen, welche in einem anderen Modul erstellt werden, **muss** dies im Punkt `Abhängigkeiten` des jeweiligen Moduls oder Plugins dokumentiert werden.
+- Benötigt das Modul Bibliotheken, welche **nicht** standardmäßig in Python geliefert werden, **muss** dies unter `Externe Abhängigkeiten` vermerkt sein. ZUSÄTZLICH **muss** die Bibliothek in die `requirements-runtime.txt` eingetragen werden.

 ### Rückgabewert bei Modulen
 Module können Pakete beliebig verändern. Diese Änderungen werden im Router entsprechend weitergeleitet.
--- a/docu/docs/service.md
+++ b/docu/docs/service.md
@ -16,7 +16,7 @@ cd /opt/boswatch3

 Das Installationsskript `install_service.py` wird anschließend mit Root-Rechten ausgeführt:
 ```bash
-sudo python3 install_service.py
+sudo /opt/boswatch3/venv/bin/python3 install_service.py
 ```
 Es folgt ein interaktiver Ablauf, bei dem du gefragt wirst, welche YAML-Dateien installiert oder entfernt werden sollen.

@ -99,7 +99,7 @@ cd /opt/boswatch3
 After that, run the install script `install_service.py` with root permissions:

 ```bash
-sudo python3 install_service.py -l en 
+sudo /opt/boswatch3/venv/bin/python3 install_service.py -l en 
 ```

 You will be guided through an interactive selection to install or remove desired services.
--- a/docu/docs/usage.md
+++ b/docu/docs/usage.md
@ -3,8 +3,8 @@ Nach dem Neustart kannst du BOSWatch3 wie folgt starten:

 ```bash
 cd /opt/boswatch3
-sudo python3 bw_client.py -c config/client.yaml
-sudo python3 bw_server.py -c config/server.yaml
+sudo /opt/boswatch3/venv/bin/python3 bw_client.py -c client.yaml
+sudo /opt/boswatch3/venv/bin/python3 bw_server.py -c server.yaml
 ```

 ## Optional: Als Dienst einrichten
@ -17,8 +17,8 @@ After reboot, you can start BOSWatch3 as follows:

 ```bash
 cd /opt/boswatch3
-sudo python3 bw_client.py -c config/client.yaml
-sudo python3 bw_server.py -c config/server.yaml
+sudo /opt/boswatch3/venv/bin/python3 bw_client.py -c client.yaml
+sudo /opt/boswatch3/venv/bin/python3 bw_server.py -c server.yaml
 ```

 ## Optional: Setup as a Service
--- a/install.sh
+++ b/install.sh
@ -9,7 +9,7 @@
                German BOS Information Script
                     by Bastian Schroll
@file:        install.sh
-@date:        14.04.2020
+@date:        21.11.2025
@author:      Bastian Schroll, Smeti
@description: Installation File for BOSWatch3
 """
@ -70,6 +70,10 @@ if [[ $EUID -ne 0 ]]; then
   exit 1
 fi

+# check actual user and group (for right assignment venv)
+ACTUAL_USER=${SUDO_USER:-$USER}
+ACTUAL_GROUP=$(id -gn $ACTUAL_USER)
+
 echo "This may take several minutes... Don't panic!"
 echo ""
 echo "Caution, script does not install a webserver with PHP and MySQL"
@ -99,32 +103,36 @@ for (( i=1; i<=$#; i=$i+2 )); do
    esac
 done

-mkdir -p ${boswatchpath} ${boswatch_install_path}
+mkdir -p ${boswatch_install_path}
+
+# Set path for VENV
+VENV_PATH=${boswatchpath}/venv

 echo ""

 tput cup 13 15
-echo "[ 1/9] [#--------]"
+echo "[ 1/10] [#---------]"
 tput cup 15 5
 echo "-> make an apt-get update................"
 apt-get update -y > ${boswatch_install_path}/setup_log.txt 2>&1

 tput cup 13 15
-echo "[ 2/9] [##-------]"
+echo "[ 2/10] [##--------]"
 tput cup 15 5
 echo "-> download GIT and other stuff.........."
 apt-get -y install git cmake build-essential libusb-1.0 qmake6 qt6-base-dev libpulse-dev libx11-dev sox >> ${boswatch_install_path}/setup_log.txt 2>&1
 exitcodefunction $? download stuff

 tput cup 13 15
-echo "[ 3/9] [###------]"
+echo "[ 3/10] [###-------]"
 tput cup 15 5
-echo "-> download Python, Yaml and other stuff.."
-apt-get -y install python3 python3-yaml python3-pip alsa-utils>> ${boswatch_install_path}/setup_log.txt 2>&1
+echo "-> download Python, venv, and other stuff.."
+# 'python3-venv' und 'python3-pip' must be installed for venv creation and management
+apt-get -y install python3 python3-pip python3-venv alsa-utils >> ${boswatch_install_path}/setup_log.txt 2>&1
 exitcodefunction $? download python

 tput cup 13 15
-echo "[ 4/9] [####-----]"
+echo "[ 4/10] [####------]"
 tput cup 15 5
 echo "-> download rtl_fm........................."
 cd ${boswatch_install_path}
@ -134,7 +142,7 @@ git checkout 2659e2df31e592d74d6dd264a4f5ce242c6369c8
 exitcodefunction $? git-clone rtl-sdr

 tput cup 13 15
-echo "[ 5/9] [#####----]"
+echo "[ 5/10] [#####-----]"
 tput cup 15 5
 echo "-> compile rtl_fm......................"
 mkdir -p build && cd build
@ -151,7 +159,7 @@ ldconfig >> ${boswatch_install_path}/setup_log.txt 2>&1
 exitcodefunction $? ldconfig rtl-sdr

 tput cup 13 15
-echo "[ 6/9] [######---]"
+echo "[ 6/10] [######----]"
 tput cup 15 5
 echo "-> download multimon-ng................"
 cd ${boswatch_install_path}
@ -161,7 +169,7 @@ exitcodefunction $? git-clone multimonNG
 cd ${boswatch_install_path}/multimonNG/

 tput cup 13 15
-echo "[ 7/9] [#######--]"
+echo "[ 7/10] [#######---]"
 tput cup 15 5
 echo "-> compile multimon-ng................."
 mkdir -p build
@ -176,9 +184,9 @@ make install >> ${boswatch_install_path}/setup_log.txt 2>&1
 exitcodefunction $? qmakeinstall multimonNG

 tput cup 13 15
-echo "[ 8/9] [########-]"
+echo "[ 8/10] [########--]"
 tput cup 15 5
-echo "-> download BOSWatch3.................."
+echo "-> download BOSWatch3 and install dependencies..."

 case ${branch} in
  "dev") git clone -b develop https://github.com/BOSWatch/BW3-Core ${boswatchpath} >> ${boswatch_install_path}/setup_log.txt 2>&1 && \
@ -188,35 +196,69 @@ case ${branch} in
 esac

 tput cup 13 15
-echo "[9/9] [#########]"
+echo "[ 9/10] [#########-]"
+tput cup 15 5
+echo "-> create and configure Python venv........"
+# generate the venv
+python3 -m venv ${VENV_PATH} >> ${boswatch_install_path}/setup_log.txt 2>&1
+exitcodefunction $? create-venv
+
+cd ${boswatchpath}/
+# install only the necessary runtime dependencies in the virtual environment
+${VENV_PATH}/bin/pip install -r requirements-runtime.txt >> ${boswatch_install_path}/setup_log.txt 2>&1
+exitcodefunction $? pip-install-runtime
+
+tput cup 13 15
+echo "[10/10] [##########]"
 tput cup 15 5
 echo "-> configure..........................."
 cd ${boswatchpath}/
-chmod +x *
 echo $'# BOSWatch3 - blacklist the DVB drivers to avoid conflicts with the SDR driver\n blacklist dvb_usb_rtl28xxu \n blacklist rtl2830\n blacklist dvb_usb_v2\n blacklist dvb_core' >> /etc/modprobe.d/boswatch_blacklist_sdr.conf
+exitcodefunction $? configure blacklist
+
+tput cup 15 5
+echo "-> set permissions......................"
+# set ownership of boswatch directory to actual user (for venv usage)
+chown -R ${ACTUAL_USER}:${ACTUAL_GROUP} ${boswatchpath}
+exitcodefunction $? chown boswatch-directory
+# executable rights for python scripts
+chmod +x ${boswatchpath}/*.py
+exitcodefunction $? chmod python-scripts
+# Log directory with setgid bit (2775) - new files inherit group ownership
+# This ensures that log files created by root (when running with sudo) 
+# still belong to the user's group and can be deleted/modified by the user
+mkdir -p ${boswatchpath}/log
+chown ${ACTUAL_USER}:${ACTUAL_GROUP} ${boswatchpath}/log
+chmod 2775 ${boswatchpath}/log
+exitcodefunction $? chmod log-directory
+# Config files readable and writable for owner
+chmod 664 ${boswatchpath}/config/*.yaml 2>/dev/null || true

 tput cup 17 1
-tput rev # Schrift zur besseren lesbarkeit Revers
-echo "BOSWatch is now installed in ${boswatchpath}/   Installation ready!"
-tput sgr0 # Schrift wieder Normal
+tput rev # letters for better readability inverted
+echo "BOSWatch is now installed in ${boswatchpath}/ and the venv in ${VENV_PATH}/. Installation ready!"
+tput sgr0 # letters back to normal
 tput cup 19 3
 echo "Watch out: to run BOSWatch3 you have to modify the server.yaml and client.yaml!"
 echo "Do the following step to do so:"
 echo "sudo nano ${boswatchpath}/config/client.yaml   eg. server.yaml"
 echo "and modify the config as you need. This step is optional if you are upgrading an old version of BOSWatch3."
 echo "You can read the instructions on https://docs.boswatch.de/"
-tput setaf 1 # Rote Schrift
+tput setaf 1 # red letters
 echo "Please REBOOT before the first start"
-tput setaf 9 # Schrift zurücksetzen
-echo "start Boswatch3 with"
+tput setaf 9 # reset letters
+echo "start Boswatch3 with (mind activation of venv!:"
+echo "source ${VENV_PATH}/bin/activate"
 echo "sudo python3 bw_client.py -c client.yaml   and    sudo python3 bw_server.py -c server.yaml"
+echo "or directly without activation:"
+echo "sudo ${VENV_PATH}/bin/python3 bw_client.py -c client.yaml   and    sudo ${VENV_PATH}/bin/python3 bw_server.py -c server.yaml"

 tput cnorm

 # cleanup
 mkdir ${boswatchpath}/log/install -p
 mv ${boswatch_install_path}/setup_log.txt ${boswatchpath}/log/install/
-rm ${boswatch_install_path} -R
+rm -rf ${boswatch_install_path} # rf ist safer, as ${boswatch_install_path} is known

 if [ $reboot = "true" ]; then
  /sbin/reboot
--- a/install_service.py
+++ b/install_service.py
@ -10,7 +10,7 @@ r"""!
                     by Bastian Schroll

@file:        install_service.py
-@date:        15.11.2025
+@date:        21.11.2025
@author:      Claus Schichl
@description: Install Service File with argparse CLI
 """
@ -151,14 +151,12 @@ TEXT = {
 }


-# === COLORAMA AUTO-INSTALL (dual language) ===
+# === COLORAMA AUTO-INSTALL ===
 def colorama_auto_install():
    r"""
-    Auto-installs colorama if missing.
-    Note: Language detection happens before colorama is available.
+    Auto-installs colorama if missing using pip in the current venv.
    """
    # recognize language early (before colorama installation)
-    import argparse
    early_parser = argparse.ArgumentParser(add_help=False)
    early_parser.add_argument('--lang', '-l', choices=['de', 'en'], default='de')
    early_args, _ = early_parser.parse_known_args()
@ -176,7 +174,8 @@ def colorama_auto_install():

        # install Colorama
        print(txt["colorama_install"])
-        subprocess.run(["sudo", "apt", "install", "-y", "python3-colorama"], check=False)
+        python_exe = sys.executable
+        subprocess.run([python_exe, "-m", "pip", "install", "colorama"], check=False)

        # retry importing Colorama
        try:
@ -248,6 +247,7 @@ def setup_logging(verbose=False, quiet=False):
    return logger


+# === Helpers ===
 def t(key):
    r"""
    Translation helper: returns the localized string for the given key.
@ -358,12 +358,12 @@ def install_service(yaml_file, dry_run=False):
    service_path = SERVICE_DIR / service_name

    if is_server:
-        exec_line = f"/usr/bin/python3 {BW_DIR}/bw_server.py -c {yaml_file}"
+        exec_line = f"{BW_DIR}/venv/bin/python3 {BW_DIR}/bw_server.py -c {yaml_file}"
        description = "BOSWatch Server"
        after = "network-online.target"
        wants = "Wants=network-online.target"
    else:
-        exec_line = f"/usr/bin/python3 {BW_DIR}/bw_client.py -c {yaml_file}"
+        exec_line = f"{BW_DIR}/venv/bin/python3 {BW_DIR}/bw_client.py -c {yaml_file}"
        description = "BOSWatch Client"
        after = "network.target"
        wants = ""
@ -382,17 +382,15 @@ Restart=on-abort
 [Install]
 WantedBy=multi-user.target
 """
-
    logging.info(t("creating_service_file").format(yaml_file, service_name))

    if not dry_run:
        try:
-            with open(service_path, 'w', encoding='utf-8') as f:
-                f.write(service_content)
+            service_path.write_text(service_content, encoding='utf-8')
+            verify_service(service_path)
        except IOError as e:
            logging.error(t("file_write_error").format(service_path, e))
            return
-        verify_service(service_path)

    execute("systemctl daemon-reload", dry_run=dry_run)
    execute(f"systemctl enable {service_name}", dry_run=dry_run)
@ -414,6 +412,22 @@ WantedBy=multi-user.target
        logging.info(t("dryrun_status_check").format(service_name))


+# === import / install colorama ===
+colorama_available, Fore, Style = colorama_auto_install()
+
+if not colorama_available:
+    # provides dummy classes if colorama is not available (no crash)
+    class DummyStyle:
+        RESET_ALL = ""
+        BRIGHT = ""
+
+    class DummyFore:
+        RED = GREEN = YELLOW = BLUE = CYAN = MAGENTA = WHITE = RESET = ""
+
+    Fore = DummyFore()
+    Style = DummyStyle()
+
+
 def remove_service(service_name, dry_run=False):
    r"""
    Stops, disables and removes the given systemd service.
--- a/requirements-runtime.txt
+++ b/requirements-runtime.txt
@ -0,0 +1,7 @@
+# requirements-runtime.txt (dependencies needed at runtime)
+# for venv all needed dependencies
+pyyaml
+requests # Telegram
+aiohttp # HTTP, Divera
+mysql-connector-python # MySQL
+geocoder # Geocoding
--- a/requirements.txt
+++ b/requirements.txt
@ -1,6 +1,6 @@
-# for pip to install all needed
-# called with 'pip install -r requirements.txt'
-pyyaml
+# requirements.txt (complete dependencies)
+# for venv all needed dependencies
+-r requirements-runtime.txt

 # for documentation generating
 mkdocs