If you use IPFIX or jFlow with Juniper MX you may notice that you can see only fraction of your all traffic or you do not observe any traffic at all.
There are many reasons why it may happen but as first step we recommend checking flow export statistics using this command:
run show services accounting flow inline-jflow fpc-slot 0
Example output looks like:
Flow information
FPC Slot: 0
Flow Packets: 351249300, Flow Bytes: 442103634726
Active Flows: 392, Total Flows: 81309409
Flows Exported: 141878284, Flow Packets Exported: 139750373
Flows Inactive Timed Out: 31317878, Flows Active Timed Out: 49991139
Total Flow Insert Count: 31318270
IPv4 Flows:
IPv4 Flow Packets: 350051006, IPv4 Flow Bytes: 440445759152
IPv4 Active Flows: 391, IPv4 Total Flows: 81055216
IPv4 Flows Exported: 141405211, IPv4 Flow Packets exported: 139277471
IPv4 Flows Inactive Timed Out: 31226179, IPv4 Flows Active Timed Out: 49828646
IPv4 Flow Insert Count: 31226570
IPv6 Flows:
IPv6 Flow Packets: 1198294, IPv6 Flow Bytes: 1657875574
IPv6 Active Flows: 1, IPv6 Total Flows: 254193
IPv6 Flows Exported: 473073, IPv6 Flow Packets Exported: 472902
IPv6 Flows Inactive Timed Out: 91699, IPv6 Flows Active Timed Out: 162493
IPv6 Flow Insert Count: 91700
Another very useful command to see if we have any issues is this one:
show services accounting errors inline-jflow fpc-slot 0
Example output may look like:
Error information
FPC Slot: 0
Flow Creation Failures: 4783091798
Route Record Lookup Failures: 0, AS Lookup Failures: 0
Export Packet Failures: 0
Memory Overload: No, Memory Alloc Fail Count: 0
IPv4:
IPv4 Flow Creation Failures: 4783091798
IPv4 Route Record Lookup Failures: 0, IPv4 AS Lookup Failures: 0
IPv4 Export Packet Failures: 0
In some cases this issue may happen when you have very small IPv4 Flow Tables and router cannot store all flows. As consequence FastNetMon will see only part of your bandwidth. From Juniper router perspective you may confirm issue using this command:
Large number of errors “Flow Creation Failures” means that you experience issue caused by very small flow table size.
You can confirm current size of table that way:
start shell pfe network fpcX show jnh 0 inline-services flow-table-info
Example output:
Configured IPv4 Flow Table in Unit: 0 Configured IPv6 Flow Table in Unit: 0 Configured VPLS Flow Table in Unit: 0 Programmed IPv4 Flow Table Size : 3932160 Programmed IPv6 Flow Table Size : 1024 Programmed VPLS Flow Table Size : 1024 IPv6 Extended Attribute : 0 IPv4 Ring Buffer Size : 262144 IPv6 Ring Buffer Size : 262144 VPLS Ring Buffer Size : 262144
Juniper allows setting ipv4-flow-table-size up to 245. Each entry means 256K flows in table. But we do not recommend using maximum value as it may overload router.
Instead, we suggest using slow process to increase it: 15, 20, 30, 40, 50 until number of 90, 180, 180+ second flows disappears from FastNetMon completely.
NB! Changes mentioned in next section may cause immediate card / router reboot (prior 16.1R1 and 15.1F2). Please be very careful!
You can apply flow-table-size this way:
set chassis fpc inline-services flow-table-size ipv4-flow-table-size 30
After applying changes you may see that values in sections “Configured IPv4 Flow Table in Unit” and “Programmed IPv4 Flow Table Size” (can by calculated from “IPv4 Flow Table in Unit” using multiplication by 256k) do not match each other. In this case you have to reboot your router.