Warning: Please note that new versions of FastNetMon Advanced do not use InfluxDB. It was replaced by Clickhouse, which is more flexible and reliable. You still can use it for existing installations, but we do not recommend using it for new installations.
Introduction
FastNetMon has very solid and extensive support for exporting metrics and alerts to InfluxDB. We recommend using our installer tool for installing and enabling integration with InfluxDB. This article will cover InfluxDB integration in detail and will provide enough information for you to integrate FastNetMon with existing or third-party installations of InfluxDB.
By default, InfluxDB stores data for 7 days, but you can easily change it using this guide.
FastNetMon can export two types of data to InfluxDB:
FastNetMon does not export flows as is to InfluxDB, as this database is not suitable for storing this kind of data. In this case, we recommend using Clickhouse.
To enable InfluxDB integration (both for metrics and alerts), you need to enable InfluxDB integration:
sudo fcli set main influxdb true
Then you need to provide the IP address and port where you’ve installed InfluxDB:
sudo fcli set main influxdb_host 127.0.0.1 sudo fcli set main influxdb_port 8086
As influxdb_host, you can use IPv4, IPv6 address or even a hostname, which will be resolved by FastNetMon using the system resolver. IPv6 address should be provided in the following format without any square brackets:
sudo fcli set main influxdb_host ::1
After this, you need to set the InfluxDB database name:
sudo fcli set main influxdb_database fastnetmon
By default, FastNetMon does not enable authentication for InfluxDB, but we have support for it:
sudo fcli set main influxdb_auth true sudo fcli set main influxdb_user fastnetmon sudo fcli set main influxdb_password secure_password
To create a user in the InfluxDB site, run the influx tool and run the following command:
CREATE USER admin WITH PASSWORD 'influxsecure999' WITH ALL PRIVILEGES
By default, FastNetMon exports metrics every second to offer real-time visibility, but you can control it using this option:
sudo fcli set main influxdb_push_period 1
If you have multiple FastNetMon instances which export data to the same InfluxDB, you may consider the option to add custom tags in the following way:
sudo fcli set main influxdb_custom_tags true
Then you can add a single tag with the following name and value:
sudo fcli set main influxdb_tag_name server sudo fcli set main influxdb_tag_value fastnetmon5
As another option, you may add any number of tags this way:
sudo fcli set main influxdb_tags_table foo=bar sudo fcli set main influxdb_tags_table server=superserver5
To remove tag, use this command:
sudo fcli delete main influxdb_tags_table server
In addition to the option to export metrics directly into InfluxDB, we offer Kafka integration, which can be used for HA setups. You can read more details about it here.
To enable attack alerts export to InfluxDB, you need to use the following options:
sudo fcli set main influxdb_attack_notification true
By default, FastNetMon exports total (IPv4, IPv6), per network (IPv4, IPv6), per host (IPv4, IPv6) and per hostgroup and system counters to InfluxDB. In some cases, per-host IPv4 counters generate a very significant load on the database, and we offer the option to disable per-host traffic export this way:
sudo fcli set main influxdb_skip_host_counters true
If you want to remove existing hosts’ metrics data from InfluxDB to free up disk space, you can run influx tool:
influx -database fastnetmon
After this, run the following command to remove all data older than 1000 seconds (to remove all data, simply remove the where section). Run the following command:
delete from hosts_traffic where time > 1000s;
IPv6 per-host counters are enabled by defaul,t but you can disable them this way:
sudo fcli set main influxdb_push_host_ipv6_counters false
For flexible traffic counters, we offeran optional capability to export all flexible metrics to InfluxDB, which can be enabled this way:
sudo fcli set main influxdb_push_host_ipv4_flexible_counters true sudo fcli set main influxdb_push_host_ipv6_flexible_counters true
To debug any issues with InfluxDB export, we offerthe following system metrics:
sudo fcli show system_counters |grep influx influxdb_writes_total 9536189 influxdb_writes_failed 0
If you see a significant number of influxdb_writes_failed it may be a sign that something is wrong with InfluxDB. First, you need to check that the daemon is running:
sudo systemctl status influxdb
Example output with correctly running InfluxDB looks like following:
● influxdb.service - InfluxDB is an open-source, distributed, time series database
Loaded: loaded (/lib/systemd/system/influxdb.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2022-06-24 14:11:47 UTC; 2 months 4 days ago
Docs: https://docs.influxdata.com/influxdb/
Main PID: 20690 (influxd)
Tasks: 16 (limit: 19189)
Memory: 258.4M
CGroup: /system.slice/influxdb.service
└─20690 /usr/bin/influxd -config /etc/influxdb/influxdb.conf
In case of issues, it may provide some details about the reasons for failure. As the next step, we can recommend checking log files:
sudo journalctl -u influxdb -n 100 -f
Often InfluxDB failure is caused by a disk space shortage as it needs at least 10G or more disk space for compaction operations. You can check disk space usage in the following way:
df -h
After fixing the disk space shortage, you will need to restart the InfluxDB daemon:
sudo systemctl restart influxdb
Note: In cases of large networks, InfluxDB will need a rather long time to start, in the range of tens of minutes.
To configure disk space usage for InfluxDB, we can recommend the following articles:
- InfluxDB data retention configuration
How could I select the top K hosts by traffic in InfluxDB?
use fastnetmon select top(packets_incoming, host, 10),host from hosts_traffic where time > now() - 1h;
How could I select the top K network by traffic amount in InfluxDB?
use fastnetmon; select top(packets_incoming, network, 10),network from networks_traffic where time > now() - 1h;