Zabbix

Zabbix model

Zabbix is an open source network monitoring software comprised of several components in a Client-Server model.

Zabbix Server

A central node running Zabbix Server is responsible for managing a MySQL database and performing passive checks on its clients, called Hosts.

Overview

Zabbix Server can be installed in several ways but the two most efficient ways to set one up are through either Linux packages or through an appliance, though the appliance is not meant for production usage.

Installation from packages

This guide / reference assumes Debian 11 (Bullseye), but the concept can be applied nearly identically to any distro, so long as you adjust one or two steps due to package manager differences. Also, make sure you have a MySQL server setup beforehand. On that note, before we get ahead of ourselves, we're most likely going to use the lxc-turnkey-mysql container template within Proxmox, and so there's one single extra step before we install Zabbix:

Pre-emptive en_US UTF8 Fix

The default configuration of MySQL / MariaDB in the turnkey container is missing a specific locale file that, if missing, just completely busts Zabbix's MySQL integration beyond comprehension. Easy fix though. From your MySQL database, run dpkg-reconfigure locales and select en_US UTF8. For good measure run update-locales after.

Get Zabbix server repositorry

The first step is to add Zabbix' repository to our server. For Zabbix 6.4 on Debian 11, this looks like this:

wget https://repo.zabbix.com/zabbix/6.4/debian/pool/main/z/zabbix-release/zabbix-release_6.4-1+debian11_all.deb

dpkg -i zabbix-release_6.4-1+debian11_all.deb

apt update

Install server, frontend, and agent

The next step is to install several packages from that repository. Server is the core of Zabbix, the frontend gives us access to the web interface, and agent is installed onto the server for self-diagnostics. This can be done as a one liner.

apt install zabbix-server-mysql zabbix-frontend-php zabbix-apache-conf zabbix-sql-scripts zabbix-agent

Most of these are self-explanatory but zabbix-sql-scripts is particularly important in the next step.

Create database

Zabbix, on a broad enough scale, can be understood to be a layer on top of an SQL database, so naturally we need to create a database for it to connect to. The official instructions have a few vague points that I'm adjusting. Before we can create the database we should copy the files that zabbix-sql-scripts generated over to the server. Use SCP to copy /usr/share/zabbix-sql-scripts/mysql/server.sql.qz from Zabbix Server to the MySQL server.
Open MySQL using mysql -uroot -p
and then run the following SQL query commands:

create database zabbix character set utf8mb4 collate utf8mb4_bin;

create user 'zabbix'@'IPOFZABBIXSERVER' identified by 'DBPASSWORD'

grant all privileges on zabbix.* to 'zabbix'@'IPOFZABBIXSERVER';

set global log_bin_trust_function_creators = 1;

quit;

Then, from the database server's shell, we need to use zcat to import a pre-defined database schema into MySQL. This is more or less the last step. Please note that this command has no stdout as it's piped into the MySQL command, so it may seem like it's hanging. It takes a few minutes at the least, up to 20 on slow machines. This is normal. Just leave it open.

zcat /path/to/server.sql.qz | mysql --default-character-set = utf8mb4 -uzabbix -p zabbix

When this is done, open mysql using -uroot -p again, and set global log_bin_trust_functioin_creators = 0.
Lastly, on the Zabbix server open /etc/zabbix/zabbix_server.conf and add DBPassword=[password]
Enable and restart the server, agent, and frontend.

systemctl enable zabbix-server zabbix-agent apache2

systemctl restart zabbix-server zabbix-agent apache2

Visit the frontend at http://[ip]/zabbix/ for the last few installation steps, which are explained on-screen (essentially just giving Zabbix Server the IP of the MySQL database.)

Zabbix Proxy

The process for Zabbix Proxy is very similar, but it needs a separate database, on a separate MySQL install rather than just a different MySQL database itself - I believe this has to do with MySQL socket locking and naming conflicts. In theory, this could be hacked around a bit, using a different zabbix user, but I haven't confirmed this and it's not officially supported, and I have confirmed it simply does not function otherwise. The process is very similar, so here are some condensed instructions. Create a separate MySQL / MariaDB installation, either as a separate container or as an install on the proxy itself. Make sure to perform the UTF8 fix as described above.

create database zabbix_proxy character set utf8mb4 collate utf8mb4_bin;

create user 'zabbix'@'IPOFZABBIXPROXY' identified by 'DBPASSWORD'

grant all privileges on zabbix_proxy.* to 'zabbix'@'IPOFZABBIXPROXY';

set global log_bin_trust_function_creators = 1;

quit;

Copy /usr/share/zabbix-sql-scripts/mysql/proxy.sql from the proxy to the DB.

cat zabbix-sql-scripts/mysql/proxy.sql | mysql --default-character-set=utf8mb4 -uzabbix -p zabbix_proxy

set global log_bin_trust_function_creators = 0;

Set the DBPassword field in /etc/zabbix/zabbix_proxy.conf

systemctl restart zabbix-proxy

systemctl enable zabbix-proxy

Post-install Configuration Changes

These initial installations of Zabbix Server and Zabbix Proxy are missing a few configuration changes that render them useless. Here's a rundown of the ones that matter first:

/etc/zabbix/zabbix_server.conf

DBName=zabbix (name of the SQL database

DBUser=zabbix (name of the MySQL user Zabbix interacts through - I think this might be the source of conflicts with Proxy. Will test later.)

DBPassword=Password (as set for MySQL previously. Wonder if there's a way to have this be a hidden variable.)

StartDiscoverers can be set to anything between 0 and 1000. This is the number of subprocesses that polls the network on a regular basis (see "Zabbix Autodiscovery") for agents.

/etc/zabbix/zabbix_proxy.conf

ProxyMode=0 (0 is Active mode, 1 is Passive mode - Active is preferred)

Server=SERVERIP (IP or DNS name for the Server the Proxy is a slave to)

Hostname=zabbixProxy (This is SUPER important - the hostname is used in some communication somewhere between Server and Proxy and if this is mismatched / not known to the Server literally nothing works!)

DBHost=MySQLServerIP (defaults to localhost.)

DBName=zabbix_proxy

DBUser=zabbix

DBPassword=password

Zabbix Agent

Installing agent on a host is really straight forward. Install the repository as before...

wget https://repo.zabbix.com/zabbix/6.4/debian/pool/main/z/zabbix-release/zabbix-release_6.4-1+debian11_all.deb

dpkg -i zabbix-release_6.4-1+debian11_all.deb

apt update

apt install zabbix-agent

Important settings in /etc/zabbix/zabbix_agentd.conf:

Server=IPOFSERVERORPROXY

ServerActive=IPOFSERVERORPROXY (especially important with proxies)

hostname is the hostname of the host you're configuring agent on - must match Zabbix Server host configuration

There's really not much else you have to do for Agent.

systemctl restart zabbix-agent

systemctl enable zabbix-agent