AndyMelton.net
andymelton

How to Set Up Ollama and Open WebUI on Linux

Purpose

This tutorial will guide you through the process of setting up Ollama and Open Web UI on a Debian-based Linux distribution (e.g., Debian or Ubuntu). Ollama will be used to acquire freely available large language models (LLMs). Open WebUI will provide a ChatGPT-like interface to interact with those models (LLMs). These instructions do not utilize Docker.

Pre-Requisites

Required

  • A Debian-based Linux distribution should already be installed.
  • Other Linux distributions do work of course, but the steps in this guide focus on Debian distributions.
  • Windows Subsystem for Linux (WSL) can be used on a Windows computer.

Recommended

  • While identifying the best GPU to use for LLM work is out-of-scope for this tutorial, the author recommends an NVIDIA GPU with at least 8 GB of VRAM. The author used the following GPUs to develop this tutorial:
    • NVIDIA Quadro RTX 4000 w/8 GB of VRAM
    • NVIDIA RTX A2000 w/8GB of VRAM
  • If you are making use of a GPU:
    • You must ensure that the latest non-free drivers are installed.
    • Use nvidia-smi to monitor GPU usage.
Steps to Install Ollama

1.) Install curl and pipx. Curl will allow us to retrieve the Ollama package. Pipx will allow us to easily install Open WebUI.

sudo apt-get install curl pipx

2.) Install Ollama using command provided on the Ollama GitHub page.

curl -fsSL https://ollama.com/install.sh | sh

3.) Pull (download) one of the freely available models from Ollama’s repositories. For the purposes of this tutorial, we will be using gemma2 (from Google) with 9 billion (9b) parameters.

ollama pull gemma2:9b

4.) Run gemma2:9b

ollama run gemma2:9b

5.) Enter a general question. Example: “Why is the sky blue?”

    • Depending on your compute power, it may take some time for the response to be generated.
    • If you are making use of a GPU and it is maxed out at 100% for several minutes, you will want to attempt to run a smaller model.
Steps to Install Open WebUI

1.) Install pipx. This will allow us to more easily install Open WebUI.

sudo apt install pipx

2.) Install Open WebUI via pipx.

pipx install open-webui

3.) Utilize the “ensurepath” command to ensure that the pipx path is in the PATH environment variable.

pipx ensurepath

4.) Restart the terminal or SSH session being used to issue commands.

5.) Start Open WebUI.

open-webui serve

6.) From a web browser, navigate to the Open WebUI interface. The interface should be located at http://YourServerIPAddress:8080

7.) Once at the Open WebUI login screen, create an account and login. The first account that you create will be the administrator account for your instance.

Steps to Ensure Ollama and Open WebUI Start During Boot
1.) Start by creating a service file. This will be used to start open-webui as a service when the system starts up.
sudo nano /etc/systemd/system/open-webui.service
2.) Add the following content to the service file, once entered, save and close the file.

Command: [Unit] Description=Open WebUI Service After=network. Target [Service] ExecStart=/home/username/.local/pipx/venvs/open-webui/bin/open-webui serve Restart=always User=username WorkingDirectory=/home/username/ Environment=”PATH=/home/username/.local/pipx/venvs/open-webui/bin:/usr/bin” [Install] WantedBy=multi-user. Target

NOTES:
  • Replace username with your username.
  • From the terminal, you can issue the command “open-webui –help” to view the list of available commands for open-webui. This is especially useful if you need to troubleshoot.
3.) Set permissions on the open-webui parent directories.
sudo chmod +x /home/username/.local/pipx/venvs/open-webui/bin/open-webui
sudo chmod -R 755 /home/username/.local/pipx/venvs/open-webui
4.) Reload internal database of unit files (i.e., reload the database that provides instructions for the starting/stopping of services).
sudo systemctl daemon-reload
5.) Enable the open-webui.service
sudo systemctl enable open-webui.service
6.) Start the open-webui.service
sudo systemctl start open-webui.service
7.) Check the status of the open-webui.service
sudo systemctl status open-webui.service
8.) Reboot the computer to ensure that the open-webui service started.
sudo reboot
9.) Once the system has restarted, ensure that you are able to access the Open WebUI interface from a remote system at http://YOURSERVERIP:8080

How to Set Up Passwordless SSH & SFTP Access Between Two Linux Servers

Purpose

These steps will guide you through the process of establishing seamless access between two servers using a public/private key pair. The goal is to enable both systems to connect seamlessly via SSH and SFTP without requiring a password. This method is used in the RADAR project to pass WAV files from the recording computer to the processing computer.

Pre-Requisites
  • You understand that private keys should be treated securely. Only transfer them to intended systems and over secure networks.
  • You have SSH and SFTP access to both servers.
  • You will be using a third computer to complete this setup.
    • You will use the third computer to copy and paste the public key into the authorized_keys file during an SSH session.
    • You will use SFTP to transfer the private key file.
System Reference

We will refer to the following systems throughout this guide:

  • Server 1
    • Hostname: RADAR01
    • Username: recorder
  • Server 2
    • Hostname: RADAR02
    • Username: processor

NOTE: The “Steps to Create the Public/Private Key Pairs,” “Steps to Configure Public Keys,” and “Test SSH & SFTP Access” sections should be completed on both servers.

Steps to Create the Public/Private Key Pairs

1.) Ensure you are in the home directory by issuing the following command:

cd ~

2.) Issue the following command to generate a public/private key file with a key size of 4096. When prompted for a passphrase, press Enter to leave it blank and continue.

ssh-keygen -t rsa -b 4096

If successful, the following public/private key files will be created in the /home/username/.ssh directory:

id_rsa
id_rsa.pub

It is recommended to change the names of these files to avoid confusion when transferring them between servers.

Server Default File Name Purpose Recommended Name Change
Server 1 id_rsa Private key RADAR01_recorder_-_id_rsa
Server 1 id_rsa.pub Public key RADAR01_recorder_-_id_rsa.pub
Server 2 id_rsa Private key RADAR02_processor_-_id_rsa
Server 2 id_rsa.pub Public key RADAR02_processor_-_id_rsa.pub

 

  • Server 1’s private key file (RADAR01_recorder_-_id_rsa) will need to be copied to server 2.
  • Server 2’s private key file (RADAR02_processor_-_id_rsa) will need to be copied to server 1.
  • Server 1’s public key contents (RADAR01_recorder_-_id_rsa.pub) will need to be copied into the authorized_keys file of both Server 1 and Server 2.
  • Server 2’s public key contents (RADAR02_processor_-_id_rsa.pub) will need to be copied into the authorized_keys file of both Server 2 and Server 1.

NOTE: When it is time to copy the contents of the public key file, you will open the public key file using nano and then copy/paste the contents between SSH sessions.

Steps to Configure Public Keys

1.) Change into the .ssh directory of the applicable user’s home directory.

cd ~/.ssh

2.) Open the authorized_keys file.

nano authorized_keys

3.) Copy/paste the public key of both Server 1 and Server 2 as new rows below any already established keys.

4.) Save and close the authorized_keys file.

Steps to Check OpenSSH Server Configuration

1.) Open the sshd_config file.

sudo nano /etc/ssh/sshd_config

2.) Ensure the following settings are enabled:

PubkeyAuthentication yes
PasswordAuthentication no # Once key-based auth is confirmed
PermitRootLogin no
AuthorizedKeysFile .ssh/authorized_keys

3.) Restart the OpenSSH service.

sudo systemctl restart ssh
Test SSH & SFTP Access

At this point you should be able to connect into Server 1 from Server 2 and vice versa.

We will test the connection using SSH’s verbose mode. That will allow us to review any errors in the authentication attempt which will help with troubleshooting.

If all goes well, you should automatically be connected to the system you wanted to connect to via SSH.

From Server 1 or Server 2, issue the following command:

ssh -vvv X.X.X.X

NOTES:

  • Replace X.X.X.X with the IP address of the remote system. If the hostname resolves correctly, you can use it instead of the IP address.
  • This command will show verbose debug information which is helpful for troubleshooting. We will only use this command for initial testing. Once testing concludes, we will switch to the normal command.

Assuming the connection was successful, you can type exit and then connect into the remote system using one of the following commands:

ssh X.X.X.X
ssh username@X.X.X.X

NOTE: Replace X.X.X.X with the IP address of the remote system. If the hostname resolves correctly, you can use it instead of the IP address.
SFTP access is configured similarly:

sftp X.X.X.X
sftp username@X.X.X.X

NOTE: Replace X.X.X.X with the IP address of the remote system. If the hostname resolves correctly, you can use it instead of the IP address.

Test SFTP Batch File Transfer

The following commands can be used to test SFTP transfers. You can place them all into a single file (e.g., sftp_batch_transfer_test.txt) and run them ran as a batch process. Ensure you replace the paths to files with actual files.

put /path/to/local/file.txt /path/to/remote/
get /path/to/remote/file.txt /path/to/local/
bye

To run the commands as batch:

sftp -b sftp_commands.txt Debian
Commands for Troubleshooting

SSH Logs

sudo journalctl -u ssh -f

SSH in Verbose Mode

ssh -vvv Debian

Confirm SSH Agent Has Loaded the Keys

ssh-add -l

SSH Keyscan

This will allow you to view the public keys of a remote server.

Confirm SSH Agent Has Loaded the Keys

ssh-keyscan X.X.X.X

NOTE: Replace X.X.X.X with the IP address of the remote system. If the hostname resolves correctly, you can use it instead of the IP address.

How to Set Up OpenAI Whisper for Locally Hosted Audio Transcription on Linux

Purpose

This guide will walk you through installing and using OpenAI’s Whisper to transcribe WAV files locally on Linux.

Pre-Requisites
Required

  • A Debian-based Linux distribution should already be installed.
    • While other Linux distributions work, this guide focuses on Debian-based distributions.
    • You can also use Windows Subsystem for Linux (WSL) on a Windows computer.
  • Prepare a WAV file (1-5 minutes in length) for transcription.
  • The following dependencies should be installed:
    • python3
      • Required for running Python packages and scripts.
    • pipx
      • This is a Python package manager that will be used for installing Whisper into an isolated environment. This will minimize interference with other Python packages.
    • ffmpeg
      • Required by whisper for processing audio files.

Recommended

  • Although selecting the best GPU for LLM workloads is beyond the scope of this tutorial, the author used the following GPUs to develop this tutorial:
    • NVIDIA Quadro RTX 4000 w/8 GB of VRAM
    • NVIDIA RTX A2000 w/8GB of VRAM
  • If you are making use of a GPU:
    • You must ensure that the latest non-free drivers are installed.
    • Use nvidia-smi to monitor GPU usage.
Steps to Install OpenAI Whisper

1.) Install openai-whisper using pipx

pipx install openai-whisper

2.) You will likely receive a message indicating the following:

NOTE: ‘/home/username/.local/bin’ is not on your PATH environment variable. These apps will not be globally accessible until your PATH is updated. Run `pipx ensurepath` to automatically add it, or manually modify your PATH in your shell’s config file (i.e. ~/.bashrc).

Issue the following command to add ‘/home/username/.local/bin’ to your PATH environment variable.

pipx ensurepath

3.) Re-launch your terminal session.

4.) Confirm successful whisper installation by initiating the help command.

whisper --help
Steps to Test Transcription Capabilities

Begin transcription of your WAV file. Ensure that your terminal session is in the same directory as your WAV file.

When using a GPU:

whisper your-audio-file.wav --model turbo --device cuda

When using CPU only:

whisper your-audio-file.wav --model turbo --device cpu
  • Replace your-audio-file.wav with the name of your WAV file.
  • You can change the model size (e.g., ‘large’) based on your system’s performance and transcription speed needs.
    • Review OpenAI’s Whisper model card on GitHub for a comparison of each option available.
    • The author finds the ‘turbo’ model performs well on 8 GB VRAM GPUs, transcribing a 55-minute WAV file in approximately 10 minutes.
  • –device cuda indicates the CUDA cores of your GPU should be utilized. You can remove it for CPU-only transcription, but be prepared for slower transcriptions.

How to Set Up a Local LAMP Stack with WordPress on Debian-Based Linux

Purpose

These steps will guide you through the process of setting up a Debian-based Linux distribution with Apache, MariaDB, PHP, phpMyAdmin, and WordPress.

Pre-Requisites
  • You understand that the purpose of this tutorial is for setting up a WordPress instance to be used in a locally hosted environment by a limited set of users. If you wish to use these steps for a production environment, you will need to perform additional security hardening.
  • You already have a computer or virtual machine prepared with a Debian-based Linux distribution.
  • You have sudo privileges.
  • You have SSH and SFTP access to the server.
    • For the purposes of this tutorial, we will be remoting into the Debian-based Linux computer.
Steps to Install Apache, MariaDB, PHP

1.) Issue the following command to update the package list.

sudo apt-get update

2.) Issue the following command to install Apache, MariaDB, and PHP. Accept the additional packages to be installed.

sudo apt-get install apache2 mariadb-server php

3.) Confirm successful installation of Apache by navigating to http://YOURIPADDRESS or http://YOURHOSTNAME

  • Note that we are using HTTP vice HTTPS.
  • If unsuccessful, confirm that you do not have a firewall blocking your access.

4.) Confirm successful installation of MariaDB by issuing the command below. Without specifying credentials, you should receive an “Access denied” message. This will suffice for now.

mariadb

5.) Confirm successful installation of php by issuing the following command to identify the version that is installed.

php -v
Steps to Configure an Apache Site

1.) Create a dedicated user account for the site.

sudo adduser websiteaccountusername

2.) Provide a password and user details (e.g., Name, Location, etc.) for the new user account.

3.) Once the new user account has been created, add the account to the www-data group.

sudo usermod -aG www-data websiteaccountusername

4.) Switch into the websiteaccountusername account to build the directory structure. Use the following command and enter the password when prompted.

su websiteaccountusername

5.) Ensure that you are in the websiteaccountusername home directory.

cd ~

6.) Issue the following command to build the necessary directories.

mkdir -p ~/htdocs/{logs,public_html}

7.) Exit out of thewebsiteaccountusername account. You should now be back in your normal account that you use to administrate the server.

exit

8.) Update the permissions of /home/websiteaccountusername/htdocs so that Apache can access them. Issue the following commands.

sudo chown -R websiteaccountusername:www-data /home/websiteaccountusername/htdocs
sudo chmod -R 755 /home/websiteaccountusername/htdocs

9.) Grant Apache the ability to traverse /home and /home/websiteaccountusername. Issue the following commands.

sudo chmod o+x /home
sudo chmod o+x /home/websiteaccountusername

10.) Navigate to the /etc/apache2/sites-available directory. Once there, we will create a new file using nano.

cd /etc/apache2/sites-available/
sudo nano yoursitenamehere.conf

11.) Populate the contents of yoursitenamehere.conf with the following.

<VirtualHost *:80>
ServerAdmin webmaster@localhost
DocumentRoot /home/websiteaccountusername/htdocs/public_html

<Directory /home/websiteaccountusername/htdocs/public_html>
Options +Indexes +FollowSymLinks
AllowOverride All
Require all granted
</Directory>

ErrorLog ${APACHE_LOG_DIR}/yoursitenamehere_error.log
CustomLog ${APACHE_LOG_DIR}/yoursitenamehere_access.log combined
</VirtualHost>

12.) To avoid any potential conflicts with the default site that Apache creates, disable it.

sudo a2dissite 000-default.com

13.) Enable the virtual host and restart the Apache service by issuing the following commands.

sudo a2ensite yoursitenamehere
sudo systemctl reload apache2

14.) Open a web browser and navigate to http://YOURSERVERIP or http://LOCALHOST to confirm access.

Steps to Secure the MariaDB Installation
1.) Issue the following command to begin the process of securing the MariaDB installation.

sudo mysql_secure_installation

2.) You will be prompted for the current root password of MariaDB. Since there is currently no password, press Enter.

3.) Perform the additional security tasks, when prompted.

Switch to unix_socket authentication? No
Change the root password? Yes
Remove anonymous users? Yes
Disallow root login remotely? Yes
Remove test database and access to it? Yes
Reload privilege tables now? Yes

4.) You can further confirm access to MariaDB by logging into the database from the console. Otherwise, we will proceed with installing phpMyAdmin for database administration. NOTE: When finished, type exit; (exit followed by a semicolon).

mariadb -u yourusernamehere -p
Steps to Create a User Account for Database Management
Using the following steps, we will create a user account using the MariaDB console that has full management permissions on the database server. We will then use this account via phpMyAdmin for any additional database administration tasks needed (e.g., creating our database for WordPress).

1.) Issue the following command to access the MariaDB console.

sudo mariadb

2.) Issue the following commands to create a new user account and grant all privileges to that user. Replace USERNAME and PASSWORD (between the single quotes) with your desired username and password.

CREATE USER 'USERNAME'@'localhost' IDENTIFIED BY 'PASSWORD';
GRANT ALL PRIVILEGES ON *.* to 'USERNAME'@'localhost' WITH GRANT OPTION;

3.) Type exit and press Enter to exit the MariaDB console.

Steps to Install phpMyAdmin

1.) Issue the following command to begin the process to install phpMyAdmin. Accept the additional packages to be installed.

sudo apt-get install phpmyadmin

2.) When prompted, select apache2 from the list of web server options.

3.) When prompted, allow the installer to configure a database for phpMyAdmin with dbconfig-common.

4.) When prompted, enter a password for the phpMyAdmin database user account.

Steps to Logon to phpMyAdmin to Create Databases

Using the following steps, we will logon to phpMyAdmin and create a database and database user account for our WordPress installation.

1.) Navigate to http://YOURIPADDRESSHERE/phpmyadmin

2.) Login using your database manager user account.

3.) Click on the User accounts button along the top.

4.) Click on Add user account.

User account overview screen of phpMyAdmin

5.) Specify username and password.

6.) Select the option for Create database with same name and grant all privileges.

7.) Click on Go.

8.) Your new database should appear in the sidebar. If it does not, click on the Home icon.

Steps to Install WordPress

The following steps will be performed from a Windows desktop. The steps are similar, if not the same on other OS’.

1.) Download the latest version of WordPress from WordPress.org (6.7.2 as of this writing) and extract the ZIP file.

2.) Rename wp-config-sample.php to wp-config.php and open it in your preferred text editor (Notepad on Windows will suffice).

3.) Populate the applicable fields with your database information (i.e., database name, user, password, and host). Since we are hosting locally, leaving ‘localhost’ is fine.

4.) Navigate to the WordPress.org Secret-Key Service to generate your secret keys (https://api.wordpress.org/secret-key/1.1/salt/ is the current URL).

WordPress Secret Keys

Once generated, replace the defaults.

5.) Connect to your server using SFTP.

6.) Navigate to /home/websiteaccountusername/htdocs/public_html

7.) Upload the contents of your local WordPress files.

8.) Open a web browser and navigate to http://YOURSERVERIP or http://LOCALHOST.

9.) You should now be presented with a WordPress installation screen. Follow the steps to complete.

How to Schedule Audio Capture via Line-in on Linux

Purpose

These steps will guide you through the process of scheduling audio capture via line-in on a Linux-based computer.

Pre-Requisites
Equipment

The following equipment and software will be used for this tutorial:

    • Raspberry Pi 3 Model A Plus Rev 1.0
    • USB Audio Adapter from Adafruit
    • Raspbian 11 (Bullseye)
    • 64 GB MicroSD Card
      • NOTES:
        • A larger MicroSD card should be used if you don’t plan to offload the audio files daily.
        • While out of scope for this tutorial, recording to an alternate storage location would be ideal to reduce writes to the MicroSD card.

Software

It is assumed that the following is already configured/installed:

    • Raspbian has already been set up on the Raspberry Pi.
    • If not, refer to Getting started with your Raspberry Pi.
    • A user account that will be used for the recording task has been created.
    • The following (non-standard) software packages have been installed:
      • arecord
      • ffmpeg
    • You have access to the Raspberry Pi desktop or can retrieve the recorded WAV files via SFTP for confirmation of playability.
    • Line in cable is connected to the USB Audio Adapter and is supplying audio to be captured.

Create Directories

From the recording user’s home directory, creating the following directory structure:

    • recordings
      • active
      • transfer
    • scripts
Steps to Test Audio Capture

We will need to identify the capabilities of the USB Audio Adapter.

1.) Connect the USB Audio Adapter to the Raspberry Pi.

2.) Enter the following command into the terminal to identify the capabilities of the USB Audio Adapter:

arecord -D hw:1,0 --dump-hw-params

You should see output like the following:

Output of: arecord -D hw:1,0 --dump-hw-params

Take note of the device name (e.g., hw:1,0).

The CHANNELS field lets us know that we only have 1 audio channel available for recording (i.e., mono only vice stereo).

3.) Issue the following command into the terminal to record a 10 second audio file using arecord.

arecord -D hw:1,0 -f S16_LE -r 44100 -c 1 -d 10 test.wav

You should see output like the following:

arecord -D hw:1,0 -f S16_LE -r 44100 -c 1 -d 10 test.wav

4.) Assuming a successful recording using arecord, issue the following command into the terminal to record a 10 second audio file using ffmpeg.

ffmpeg -f alsa -i default -ac 1 -t 10 test.wav

You should see output like the following:

ffmpeg -f alsa -i default -ac 1 -t 10 test.wav

5.) Assuming a successful recording, proceed to Creating a Bash Script for Recording.

Steps to Create a Bash Script for Recording

1.) Navigate to the /home/username/scripts directory.

2.) Create a new shell script using nano. Issue the following command:

nano record-audio.sh

3.) In the blank text editor, copy/paste the following script:

#!/bin/bash

TIMESTAMP=$(date +"%Y-%m-%d_%H-%M-%S")

/usr/bin/ffmpeg -f alsa -i plughw:CARD=Set,DEV=0 -ac 1 -t 3300 $HOME/recordings/active/audio_$TIMESTAMP.wav

mv $HOME/recordings/active/audio_$TIMESTAMP.wav $HOME/recordings/transfer

This script:

  • Defines the timestamp that will be applied to the output file.
  • Tells ffmpeg to record audio on DEV=0 for 55 minutes (3300 seconds).
    • Adjust for your device.
    • You may want to adjust this to 10 seconds for initial testing.
  • Indicates where the file should be saved and the name that should be applied to the file.
    • Files actively being recorded are placed into the active directory.
    • Completed files are moved into the transfer directory. Transfer meaning the file is completed and ready to be transferred for processing.
    • $HOME indicates the home directory of the user running the script.
    • Adjust to your project’s needs.

4.) Use Ctrl + X to close the file. When prompted, save the file.

5.) Make the script executable by issuing the following command:

chmod +x ~/scripts/record-audio.sh

6.) Attempt to run the script manually. From the scripts directory, issue the following command:

./record-audio.sh

If confirmed to run successfully, press Ctrl + C to end the recording. Proceed to Use CRON to Schedule 24×7 Recording.

Steps to Configure a CRON Job for 24x7 Recording

1.) Issue the following command to launch the CRON job editor.

crontab -e

2.) Add the following command to the bottom of the file:

0 * * * * $HOME/scripts/record-audio.sh

NOTE: If you would like to have a log generated you can use the following instead. Just note that the log file can grow substantially as time progresses. Make sure to clean it out or stop outputting a log when no longer needed.

0 * * * * $HOME/scripts/record-audio.sh >> $HOME/logs/record_audio.log 2>&1

Some examples of other recording times:

0 9 * * * To record every day at 9 AM.
*/15 * * * * To record every 15 minutes.

Additional Useful Commands

grep CRON /var/log/syslog Confirm the CRON job is running.
tail -f /var/log/syslog | grep CRON Use tail (not installed by default) to monitor the output syslog continuously.
tail -f /var/log/syslog | grep –color=always CRON Apply color to the output of tail.