I am looking for a new monitoring solution for my homelab. This may sound like a trivial endeavor, but it appears to be dicier and more convolted than it sounds at first blush.
I have worked with a number of them over my time as a sysadmin, however a lot of them are blissfully ignorant of the special bits of FreeBSD that I would like to also monitor. Put another way, they are linux-centric. My home network is currently 20 hosts, of which 18 are FreeBSD, and 2 are Devuan linux. That number flexes and breathes as I experiment with new software or begin or end new projects. Most of the expansion is in the form of jails and/or bhyve vms.
My current solution is Zabbix, but making changes to triggers and changing what it presents as alarms is a bit convoluted, bordering on...janky. Plus it is a bit heavy for my 20-odd hosts, making relatively heavy use of a database. My personal preference is PostgreSQL, as I have worked with it for the last 15 years at work...Though I am in no way a DBA. I've also worked with Nagios back in the day, and I recall setting it up and keeping host configs up to date and accurate was very...Fiddly.
In addition, it becomes obvious fairly quickly that there is a dearth of FreeBSD knowledge on the team. As an example, in the event of a single disk in a ZFS pool faulting, Zabbix reports "pool is offline." Which, unless you are using a single-disk pool, losing a disk does not take your pool down. (Of course, the whole "server dropping offline" when you actually lose your pool will probably get your attention more quickly than a (bogus) ZFS error. I have a dozen TrueNAS machines at work, most of which have at least 60 drives. I currently have a pool consisting of 10 6-drive RAID-Z2 VDEVs, which means I could conceivably lose up to 20 drives before losing the pool. The other thing is that I have not been able to update Zabbix 7. 7.0.16 - 7.0.19 all complain about not being able to find libxml2.so.2.
So after several days looking for a FreeBSD-friendly monitoring rig, it seems a lot of them are non-free. I also ran into many of them that are cloud-based. I am not a fan of "the clowd" or spraying my monitoring data (or anything else) into the ether. Other tools are loaded with linux-isms, for instance, having a primary installation via docker container.
Many of the ones that remain seem to be either old, or in desperate need of a refresh, or end up being too heavy (Icinga?) or too fiddly (Prometheus + Grafana). I am still looking, and will probably stand up one or two in jails and see how they perform. So at this point, looking at monit/mmonit, SyMon, possibly Munin, and Icinga, but continuing to look.