As an alternative to InfluxDB you could store traffic metrics in ClickHouse.
In addition to our default traffic metrics storage InfluxDB we offer complete support for Clickhouse.
It has many advantages over InfluxDB:
- Unlimited scalability
- Predictable memory usage
- Clustering / redundancy support
- Ability to store not only metrics but traffic too.
We recommend using Clickhouse for metrics for any network which exceeds 50.000 active hosts.
To enable this capability you will need to install visual graphic stack. It will install and configure all daemons for you.
Please keep in mind that FastNetMon does not offer exactly same set of exported metrics for Clickhouse as for InfluxDB. Some metrics types will be added in future.
To enable metrics export to Clickhouse you need to run this command:
sudo fcli set main clickhouse_metrics true sudo fcli commit
Then you need to set database host:
sudo fcli set main clickhouse_metrics_host 127.0.0.1 sudo fcli commit
By default, we use Clickhouse on same machine but for large installations you may consider running it on another machine and in that case you will need to set external IP address.
And then set port, we use native TCP/IP port for sending data to Clickhouse:
sudo fcli set main clickhouse_metrics_port 9000 sudo fcli commit
Set database name:
sudo fcli set main clickhouse_metrics_database fastnetmon sudo fcli commit
Optionally, you may set username and password:
sudo fcli set main clickhouse_metrics_username default sudo fcli set main clickhouse_metrics_password pass sudo fcli commit
By default, FastNetMon exports traffic for all hosts to Clickhosue but you may change it using this option to save disk space:
sudo fcli set main clickhouse_metrics_per_protocol_counters false sudo fcli commit
You may even configure how often FastNetMon export traffic to Clickhouse, by default it does it every single second to provide real time graphs:
sudo fcli set main clickhouse_metrics_push_period 1 sudo fcli commit
FastNetMon does not rely on pre-created tables and it can create all required tables in Clickhouse automatically for you.
Clickhouse will create following tables for metrics export:
- total_metrics, total_metrics_ipv6
- network_metrics, network_metrics_ipv6
- host_metrics, host_metrics_ipv6
- asn_metrics_ipv4, asn_metrics_ipv6
To monitor that Clickhouse export works as expected you can check these counters:
sudo fcli show system_counters|grep click clickhouse_metrics_writes_total 2044855 clickhouse_metrics_writes_failed 0
Clickhouse may use significant amount of disk space and you can find detailed guide about ways to control disk space usage by it.
In deep cooperation with community we offer following dashboards:
- Top 10 hosts by load link
- Per network traffic link
- Total traffic link
- Traffic for specified host link
- Traffic for specified network link
Also, you could use such query in clickhouse-client tool to retrieve data:
select host, packets_incoming from fastnetmon.host_metrics where metricDate == today() AND metricDateTime >= now() - 5 order by packets_incoming desc limit 10;