Warning
Please note that new version of FastNetMon Advanced do not use InfluxDB. It was replaced by more flexible and reliable Clickhouse. You still can use it for existing installation but we do not recommend using it for new installations.
Introduction
FastNetMon has very solid and extensive support for exporting metrics and alerts to InfluxDB. We recommend using our installer tool for installing and enabling integration with InfluxDB. This article will cover InfluxDB integration in details and will provide enough information for you to integrate FastNetMon with existing or third party installation of InfluxDB.
By default, InfluxDB stores data for 7 days but you can easily change it using this guide.
FastNetMon can export two types of data to InfluxDB:
FastNetMon does not export flows as is to InfluxDB as this database is not suitable for storing this kind of data. In this case we recommend using Clickhouse.
To enable InfluxDB integration (both for metrics and alerts) you need to enable InfluxDB integration:
sudo fcli set main influxdb true
Then you need to provide IP address and port where you’ve installed InfluxDB:
sudo fcli set main influxdb_host 127.0.0.1 sudo fcli set main influxdb_port 8086
As influxdb_host you can use IPv4, IPv6 address or even hostname which will be resolved by FastNetMon using system resolver. IPv6 address should be provided in following format without any square brackets:
sudo fcli set main influxdb_host ::1
Then you need to set InfluxDB database name:
sudo fcli set main influxdb_database fastnetmon
By default, FastNetMon does not enable authentication for InfluxDB but we have support for it:
sudo fcli set main influxdb_auth true sudo fcli set main influxdb_user fastnetmon sudo fcli set main influxdb_password secure_password
To create user in InfluxDB site run influx tool and run following command:
CREATE USER admin WITH PASSWORD 'influxsecure999' WITH ALL PRIVILEGES
By default, FastNetMon exports metrics every second to offer real time visibility but you can control it using this option:
sudo fcli set main influxdb_push_period 1
If you have multiple FastNetMon instance which export data to same InfluxDB you may consider option to add custom tags this way:
sudo fcli set main influxdb_custom_tags true
Then you can add single tag with following name and value:
sudo fcli set main influxdb_tag_name server sudo fcli set main influxdb_tag_value fastnetmon5
As another option you may add any number of tags this way:
sudo fcli set main influxdb_tags_table foo=bar sudo fcli set main influxdb_tags_table server=superserver5
To remove tag use this command:
sudo fcli delete main influxdb_tags_table server
In addition to option to export metrics directly into InfluxDB we offer Kafka integration which can be used for HA setups. You can read more details about it here.
To enable attack alerts export to InfluxDB you need to use following options:
sudo fcli set main influxdb_attack_notification true
By default, FastNetMon export total (IPv4, IPv6), per network (IPv4, IPv6), per host (IPv4, IPv6) and per hostgroup and system counters to InfluxDB. In some cases per host IPv4 counters generate very significant load on database and we offer option to disable per host traffic export this way:
sudo fcli set main influxdb_skip_host_counters true
If you want to remove existing hosts metrics data from InfluxDB to free up disk space you can run influx tool:
influx -database fastnetmon
Then remove run following command to remove all data older then 1000 seconds (to remove all data just remove where section) run following command:
delete from hosts_traffic where time > 1000s;
IPv6 per host counters are enable by default but you can disable them this way:
sudo fcli set main influxdb_push_host_ipv6_counters false
For flexible traffic counters we offer optional capability to export all flexible metrics to InfluxDB which can be enabled this way:
sudo fcli set main influxdb_push_host_ipv4_flexible_counters true sudo fcli set main influxdb_push_host_ipv6_flexible_counters true
To debug any issues with InfluxDB export we offer following system metrics:
sudo fcli show system_counters |grep influx influxdb_writes_total 9536189 influxdb_writes_failed 0
If you see significant number of influxdb_writes_failed it may be a sign that something wrong with InfluxDB. First you need to check that daemon is running:
sudo systemctl status influxdb
Example output with correctly running InfluxDB looks this way:
● influxdb.service - InfluxDB is an open-source, distributed, time series database
Loaded: loaded (/lib/systemd/system/influxdb.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2022-06-24 14:11:47 UTC; 2 months 4 days ago
Docs: https://docs.influxdata.com/influxdb/
Main PID: 20690 (influxd)
Tasks: 16 (limit: 19189)
Memory: 258.4M
CGroup: /system.slice/influxdb.service
└─20690 /usr/bin/influxd -config /etc/influxdb/influxdb.conf
In case of issues it may have some details about reasons of failure. As next step we can recommend checking log files:
sudo journalctl -u influxdb -n 100 -f
Pretty often InfluxDB failure may be caused by disk space shortage as it need at least 10G or more disk space for compaction operations. You can check disk space usage this way:
df -h
After fixing disk space shortage you will need to restart InfluxDB daemon:
sudo systemctl restart influxdb
In cases of large networks InfluxDB will need quiet long time to start (in range of tens of minutes), be patient.
To configure disk space usage for InfluxDB we can recommend following articles:
- InfluxDB data retention configuration
How could I select top K hosts by traffic in InfluxDB?
use fastnetmon select top(packets_incoming, host, 10),host from hosts_traffic where time > now() - 1h;
How could I select top K network by traffic amount in InfluxDB?
use fastnetmon; select top(packets_incoming, network, 10),network from networks_traffic where time > now() - 1h;