As an alternative to InfluxDB you could store traffic metrics in ClickHouse. This capability is available only for FastNetMon Advanced.

In addition to our legacy / deprecated traffic metrics storage InfluxDB we offer complete support for Clickhouse.

It has many advantages over InfluxDB:

  • Unlimited scalability
  • Predictable memory usage
  • Clustering / redundancy support
  • Ability to store not only metrics but traffic too.

We recommend using Clickhouse for metrics for any network which exceeds 50.000 active hosts.

To enable this capability you will need to install visual graphic stack. It will install and configure all daemons for you.

To enable metrics export to Clickhouse you need to run this command:

sudo fcli set main clickhouse_metrics true
sudo fcli commit

Then you need to set database host:

sudo fcli set main clickhouse_metrics_host
sudo fcli commit

By default, we use Clickhouse on same machine but for large installations you may consider running it on another machine and in that case you will need to set external IP address. This field supports hostnames starting from 2.0.348. You can use both IPv4 and IPv6 addresses for clickhouse_metrics_host.

And then set port, we use native TCP/IP port for sending data to Clickhouse:

sudo fcli set main clickhouse_metrics_port 9000
sudo fcli commit

Set database name:

sudo fcli set main clickhouse_metrics_database fastnetmon
sudo fcli commit

Optionally, you may set username and password:

sudo fcli set main clickhouse_metrics_username default
sudo fcli set main clickhouse_metrics_password ""
sudo fcli commit

You may even configure how often FastNetMon export traffic to Clickhouse, by default it does it every single second to provide real time graphs:

sudo fcli set main clickhouse_metrics_push_period 1
sudo fcli commit

FastNetMon does not rely on pre-created tables and it can create all required tables in Clickhouse automatically for you.

Clickhouse will create following tables for metrics export:

  • total_metrics, total_metrics_ipv6
  • network_metrics, network_metrics_ipv6
  • host_metrics, host_metrics_ipv6
  • asn_metrics_ipv4, asn_metrics_ipv6

To monitor that Clickhouse export works as expected you can check these counters:

sudo fcli show system_counters|grep click
clickhouse_metrics_writes_total                            2044855 
clickhouse_metrics_writes_failed                           0 

Clickhouse may use significant amount of disk space and you can find detailed guide about ways to control disk space usage by it.

We offer following dashboards which will be installed by our instiller tool automatically.

Also, you could use such query in clickhouse-client tool to retrieve data:

select host, packets_incoming from fastnetmon.host_metrics where metricDate == today() AND metricDateTime >= now() - 5 order by packets_incoming desc limit 10;

If you use existing installation of Clickhouse you need to grant following permissions for Clickhouse user:


Top talkers pre-calculation

To provide lightning fast reports for top talker hosts in network FastNetMon has ability to pre-calculate hosts which generate most traffic each second and export traffic for them to Clickhouse.

This ability is enabled by default on all new installations. On old installations you can enable it this way:

sudo fcli set main clickhouse_metrics_export_top_hosts true
sudo fcli commit

After enabling this option FastNetMon will create tables:

  • host_metrics_ipv4_top
  • host_metrics_ipv6_top

These tables are very similar to tables used to storing regular per host traffic: host_metrics, host_metrics_ipv6.

In these tables FastNetMon will store top 10 hosts in each particular second. It will use 4 different approaches to calculate top talkers:

  • Top hosts by incoming bytes
  • Top hosts by incoming packets
  • Top hosts by outgoing bytes
  • Top hosts by outgoing packets

If some hosts was picked up as top by more then one approach then we will store it only once.

Clickhouse password reset

In some rare cases you may need to reset password for Clickhouse user used to access database from Grafana:

wget -Oinstaller
sudo chmod +x installer
sudo ./installer -reset_clickhouse_password

Due to Grafana limitations about inability to reload datasource configuration change you may need to run following commands to reload configuration:

sudo apt install -y sqlite3
sudo sqlite3 /var/lib/grafana/grafana.db

And then you will nee to remove cached datasource:

delete from data_source where uid = 'fastnetmonclickhouseofficialdatasourceuid';

And after that restart Grafana:

sudo systemctl restart grafana-server

After all these changes you will be able to access Clickhouse dashboards in Grafana.

24/7 Tech Support

Email Us