09.12.2017

FastNetMon Advanced Traffic Persistence

Introduction

FastNetMon has bundled ability to store all traffic inside special column-oriented database. You could store all traffic inside it and retrieve it using well-known SQL syntax.

System requirements

You should have working instance of FastNetMon before using this guide. Also, you need to install visual traffic stack because we use Grafana for traffic persistence too.

Download ASN mapping database

It’s mandatory step if you are interested in ASN tagging for your traffic. If your device populates it in Netflow v5/v9 and IPFIX stream you do not need it.

sudo fill_dictionaries

Configure FastNetMon to store traffic into traffic database

sudo fcli set main traffic_db_host 127.0.0.1
sudo fcli set main traffic_db_port 8100
sudo fcli set main traffic_db enable
sudo fcli commit

And start traffic_db daemon:

sudo service traffic_db restart

Make example queries

Start client:

clickhouse-client --host 127.0.0.1

Example query:

select * from fastnetmon.traffic;

Execute ASN query:

use fastnetmon
SELECT dstAsn, count(dstAsn) AS cnt FROM traffic GROUP BY dstAsn  ORDER BY cnt DESC  LIMIT 10

As output, you could find number of packets per destination ASN for required time slice.

Clickhouse does not have capability to remove old data automatically, you can use our script for this task.

Example dashboards

Then you could use our pre-created dashboards:

  • Destination ASN report link
  • Source ASN report link
  • Bandwidth per arbitrary ASN link
  • Bandwidth per arbitrary IPv4 prefix link
  • Bandwidth per arbitrary IPv6 prefix link
  • Detailed queries about all IPv4 traffic link
  • Detailed queries about all IPv6 traffic link