This is to announce nProbe 9.4 stable that is an incremental update of 9.2 released last fall. The goal of this maintenance release is to pave the way to pervasive embedded systems support as we now support

• OPNsense/pfSense/FreeBSD
  Soon we’ll make a separate announcement as soon as more ntop packages will be available for these platforms.
• Ubiquity EdgeRouter X
  Read this blog post for learning more about sub 100$ Ubiquity-based hardware probes.
• OpenWRT

In addition we have decided to simplify the nProbe versions that were hard to understand for most of the users. Starting with 9.4 there are only three versions:

• Pro (unchanged)
• Enterprise S: same as the former Pro with HTTP and DNS plugins, to which we have added NetFlow-Lite collector plugin.
• Enterprise M: a new version that includes all plugins.

Due to this change the former nProbe standard version has been discontinued and users freely upgraded to Pro. Individual plugins will not be available anymore but incorporated in the Enterprise editions. License prices are unchanged, but new features have been added to the software.

You can read more about transition

• nProbe version technical note
  It covers version changes and maintenance information
• nProbe product page
  It contains a detailed list of the features and limitations of the various versions.

Enjoy !

This is to announce the immediate availability of both ntopng and nProbe for OPNsense, pfSense and FreeBSD, directly supported by ntop, with nightly builds and all the features present on all other supported platforms such as Linux, Windows and MacOS. You can now

• Monitor network traffic based on nDPI.
• Encrypted traffic analysis (ETA) that enables you to have visibility of encrypted traffic and answer to questions such as: what portion of my available bandwidth is used by Netflix?
• Cyber threats analysis: ntopng con be used to effectively detect attacks, anomalies and malware including SunBurst, Qakbot and Trickbot.
• Support enterprise protocols such as NetFlow, sFlow, and SNMP, that can be combined with packet-based visibility.

Installation

You can download packages from https://packages.ntop.org/FreeBSD/ and once installed they can be administered from the GUI as shown in the picture below.

Video Guides

User’s Guides

We have created specific guides for OPNsense:

• nProbe
• ntopng

that can complement the product user’s guides.

Questions ?

You can contact the ntop team and the rest of the community by using our community support.

Enjoy !

PS. Stay tuned as soon we’ll have some exciting news to share.

We are proud to announce that a couple of talks have been accepted at FOSDEM 2021, one of the most important FOSS conferences in the world that this yar will take place online due to the pandemic. In the Network monitoring, discovery and inventory devroom we will give two presentations titled “Using nDPI for  Monitoring and Security” and  “ntopng network monitoring and discovery“.

In addition, ntop has been given a virtual stand to present its opensource-related activities. The ntop team will always be available in a chatroom and a series of guests will give lightning talks during the two-days of FOSDEM.

This is the full schedule, follow hyperlinks for instructions on how to join!

No registration is necessary, just join the session clicking on the event link.

See you soon !

FOSDEM 2021 has been an awesome event. For the first time in its history, the event was fully virtual with multiple parallel live streams.

According to the infrastructure statistics, it had about 8,000 visitors of which about 5,000 were watching the live streams.

A huge volunteering work has been necessary to make this event happen and ntop is immensely grateful to all those beautiful people who have donated their expertise and time.

We, as ntop, have had the opportunity to present a couple of talks in the Network monitoring, discovery and inventory devroom, namely “Using nDPI for Monitoring and Security” and “ntopng network monitoring and discovery“.

In addition, we have been given a virtual stand (video) and a chatroom to present our opensource-related activities. We have organized a series of activities within our virtual stand, to showcase and demonstrate use cases of our opensource software.

Below you can find all the ntop-related activities we have organized, along with links to video recordings and slides.

Event	Speakers	Video	Slides
Saturday
Welcome to the stand for ntop
ntop @ FOSDEM Ongoing developments and future directions	Luca Deri
ntopng for IoT How to Profitably Use ntopng in Smart Homes	Antonis Gotsis
Industrial Network Monitoring With ntopng A Look into Industrial Network Protocols With ntopng	Martin Scheu
Using nDPI for Monitoring and Security nDPI in practice	Luca Deri
ntopng network monitoring and discovery Network discovery with ntopng	Matteo Biscosi
Chat With the ntop Developers Meet with the Community
Installation Party Tips&Tricks for best results
Sunday
ntopng and Cybersecurity Sorting Out Real-World issues with ntopng	Giordano Zambelli
ntopng Network Monitoring and Discovery A Follow-Up with Live Demo and Examples	Matteo Biscosi
ntopng Flexible Alerts – Endpoints and Recipients How to Deliver ntopng Alerts to Downstream Recipients	Matteo Biscosi
Round Table and Discussion Meet with the Community
Installation Party Tips&Tricks for Best Results

Due to pandemic, many people are now working in a delocalised world: some work from home, others from the office. To make things even more complicated, in the past remote workers used to connect to the company network via a VPN. While this option is still possible, many resources are now available from the cloud thus making VPNs obsolete in some environments, in particular for mobile workforce that connects to the Internet by means of a cellular network.

In the past months, some people contact us to ask how they could:

• Monitor the traffic in remote networks such as a small site or home office.
• Continuously check the connection speed for every worker: namely perform periodic checks such as the active tests offered by ntopng (ping, continuous ping and speedtest) so that the company can check if the remote worker/site can profitably work having sufficient and permanent bandwidth available.
• Be able to do all this while remote users might be behind a NAT (e.g. when at home) or a CGNAT (e.g. when using a mobile network connection).

The figure below shows you how you can achieve that.

As most of you now, in addition to popular tools such as nProbe and ntopng, we develop a decentralised peer-to-peer VPN named n2n that allows you to create your own network meshed topology simply installing a small software named edge (and part of the n2n toolkit) and leveraging on a supernode (still part of n2n) that allows remote edge’s to connect even if behind a network. While people can use their private supernode, ntop offers a public supernode available at supernode.ntop.org in case you do not want to setup a supernode. Note that even in this latter case your data will be completely protected due to the end-to-end encryption provided by n2n. Using n2n you can in essence create a network overlay which crosses firewall and that allows you to reach remote network edges without paying attention to network constraints such as NAT. You can read more here about n2n configuration. Note that n2n nodes can speak directly if the NAT allows it, meaning that you might be able to do A->B directly while B->supernode->A

Once n2n is setup, you have standard network connectivity (ping, http etc.) and you can now monitor your network without setting up any VPN server. Now let’s make a simple monitoring example as in the above picture.

• Start nProbe on each satellite node as follows:
  • nprobe -i eth0 –zmq tcp://192.168.X.1:1234 (where X is the network of the remote site)
• Start ntopng at the central monitoring console as follows:
  • ntopng -i tcp://192.168.2.1:1234 -i tcp://192.168.3.1:1234 -i tcp://192.168.4.1:1234 -i “view:tcp://192.168.2.1:1234,tcp://192.168.3.1:1234,tcp://192.168.4.1:1234”

and this is the final result

Enjoy !

Telemetry protocols such as sFlow/NetFlow, SNMP or packet-based traffic analysis are the source of data for network traffic monitoring. For a long time visibility was the main issue and people were attracted by new tools such as Grafana that allowed them to put on a screen a lot of data. A big misconception in this area is that the more data you see the less you understand as networks are constantly growing in both size and complexity and adding second or a third screen a PC isn’t the perfect way to scale up in known what is happening in a network. The score is a numerical indicator (present in all ntopng versions) that when non-null it indicates that some kind of issue is present: the higher is the score the worst is the problem associated, and thus the higher is the attention that a network operator should put on this resource.

The main source of score are the flows that have three score values:

• The flow score that indicates how bad is this flow. For instance a flow with severe retransmissions has a non null flow score.
• The flow source host score: a numerical value associated with the source host. For instance if in the above flow retransmissions are only destination->client the source host score for this flow will be zero, but if the source has also retransmissions the value will be positive.
• The flow destination host score: same as before but for the flow destination host.

As flows can have multiple issues, each issue found on a flow (for instance a TCP retransmissions or an outdated TLS version detected on a flow) contribute to the score. Hence the flow/source/destination score is the sum of all individual scores found on such flow. The flow score is computed by the flow user scripts (Settings -> User Scripts) when they are executed on a flows.

The host score is computed as the sum of all flow scores where such host is either client or server, complemented with additional scores eventually found on this host. A typical example is a network scan: every individual flow part of scan might have a zero score, but as this host is doing a scan, this results on a non-zero score. The host score is used to compute the score of other elements such as the Autonomous System, a Network, an Interface score that is the sum of individual scores in an iterative process as described above.

Using the flow score you can fo to Maps -> Hosts Map to see hosts are creating (or receiving) most trouble. In the above picture if you move the mouse over the purple or salmon coloured circles, you will see the name of the host creating most problems as the circle radius is proportional to the score associated. Inside each host (please make sure that Settings -> Preferences -> TimeSeries -> Host Timeseries is set to full) you can see a timeseries of the host score that you can use to see how healthy is a host and when troubles have been detected.

The next challenge we’re working on is to implement a method that allows you to detect score changes and alert you when this happens. This to drive network analysts towards the problem without having to constantly watch hundred of micro-windows.

Enjoy !

Most people use nProbe and ntopng to collect flows using an architecture similar to the one below

where nprobe and ntopng are started as follows:

• nprobe -3 <collector port> -i none -n none —zmq "tcp://*:1234" --zmq-encryption-key <pub key>
• ntopng -i tcp://nprobe_host:1234 --zmq-encryption-key <pub key>

In this case ntopng communicates with nProbe over an encrypted channel and flows are sent in a compact binary format for maximum performance. If you do not need nProbe to cache and aggregate flows, you can also add --collector-passthrough on the nProbe side to further increase flow collection.

This setup is good as long as you have a reasonable amount of flows to collect according to the system performance. In case you have too many flows to collect (e.g. you have many routers sending nProbe, or a big router sending many flows per second) this setup does not no scale up because it does not take advantage of multicore architectures. The solution is to

• Distribute the load across multiple (instead of one) nProbe instance.
• On ntopng collect flows over multiple ZMQ interfaces and eventually aggregate them with a view interface.

In essence you need to adopt an architecture as follows

where you start nProbe/ntopng as follows:

• nprobe -i none -n none -3 2055 —zmq tcp://127.0.0.1:1234

• nprobe -i none -n none -3 2056 —zmq tcp://127.0.0.1:1235

• nprobe -i none -n none -3 2057 —zmq tcp://127.0.0.1:1236

• ntopng -i tcp://127.0.0.1:1234 -i tcp://127.0.0.1:1235 -i tcp://127.0.0.1:1236 -i view:all

In this case each nProbe instance will process flows concurrently and the same will happen on ntopng. All you need to do is to try to share the ingress load across all the nProbe instances so that each instance receives a similar amount of flows and the workload is shared across them.

Instead, in case you have a single router exporting flows towards the same nProbe instance, the above trick cannot be used so we need to load balance differently. This can be achieved as follows

where you configure nProbe this way:

• nprobe -i none -n none -3 2055 --collector-passthrough —zmq tcp://127.0.0.1:1234 —zmq tcp://127.0.0.1:1235 —zmq tcp://127.0.0.1:1236

• ntopng -i tcp://127.0.0.1:1234 -i tcp://127.0.0.1:1235 -i tcp://127.0.0.1:1236 -i view:all

In this case nProbe will load balance collected flows across multiple egress ZMQ and ntopng will collect flows concurrently as every ZMQ interface runs on a separate thread.

Please note that:

• The --collector-passthrough is optional but it can reduce the load on nProbe, that is desirable if you have a big router exporting flows towards the same instance.
• The above solution can also be applied to packet processing (nprobe -i ethX) and not just to flow collection (in this case --collector-passthroughis not necessary). Namely the more you spread the load across interfaces, the more you exploit multicore systems and your system can scale up.

Enjoy !

In this post Martin shows how he has used ntopng to detect Qakbot trojan. Many thanks for this contribution.

Introduction

I am using ntopng for network monitoring quite some time now and I was curios to see, what ntopng would alert when detecting malware.

The website malware traffic analysis is a great source for malware captured in network traffic. I decided to take a Qakbot infection with spambot activity [1]. From the pcap file name we see to expect

• Qakbot, a active [2] banking trojan [3]
• Cobalt Strike, a commercial penetration testing tool, unfortunately pirated versions of Cobalt Strike have been leaked [4]
• Spambot activity, distributing QakBot

1. Further I used a tutorial how to examine Qakbot infections with Wireshark from Unit42 [5].

For the analysis I setup a virtual machine with Ubuntu Server 20.04 LTS and installed the current ntopng nightly build [6], using the community license.

Ntopng per default is built for continuous network monitoring, but directing the interface to a pcap, it can read and process it.

Analysis

To start ntopng sudo is needed, followed by the -i for interface or pcap file:

sudo ntopng -i 2020-12-07-Qakbot-with-Cobalt-Strike-and-spambot-activity.pcap

The web gui can be reached over

http://localhost:3000 or http://<server ip>:3000

Ntopng will prompt for username and password and you will have to select a new admin password. Once logged in, you will see the interface overview, showing that the selected interface is a pcap dumb and the name of the pcap file:

A quick look at the traffic summary icons reveal lots of work ahead, and many problems are being identified:

Now rather than checking out the flow warnings or errors, I navigate to the “Host Map” from the Maps menu and select “Alerted Flows” in the dropdown filter menu. This view gives a fast overview of hosts with alerts. The hosts map shows on the x-axis hosts identified as servers with alerts and in the y-axis host identified as clients. Note: a host can a be a server and/or a client. The size of the circles is proportional with the number of alerts, i.e. the biggest circle shows the host with the most alerts. In the filter dropdown you can find other useful filters, like “Traffic Ratio” or “DNS Queries vs Replies”.

Host overview

I proceed with the biggest circle, the top left one, as the host has around 170 alerts. A mouse over reveals the IP address of 10.12.7.101, clicking on the circle opens the host information page:

Regarding security concerns, I am interested in the above market items:

1. The host is a Windows 8 client
2. The host has a security score as a client of 19’905, for a single host a very high number. Ntop uses scores as a risk-based approach to security or protocol related flaws. E.g. malicious data in a flow adds 210 to the host score, suspicious data in a flow adds 200. The idea is not to have absolute values or limits, like above 1000 is bad, below 100 all good, instead, the score can be seen as an indicator of possible problems.
3. Out of 634 flows from or to this host, 168 are with an alert. That means that approx. 1/4 of the flows have an issue. Very unusual indeed.
4. For this host 152 SMTP (or SMTPS) flows were counted. This means the user is connected to many different SMTP servers. Again, this is very unusual for a client. A average user maybe has 5 or 10 different email accounts, leading to 10+ SMTP flows, but 152 SMTP connections do not look good at all.

Now let us check the alarms generated by ntopng. By clicking on the caution sign  , the flow alerts of the host view opens:

Host flow alerts

The list I filter by score, in order to get the alerts with highest score.

Note:

We could further filter the list by “Severity”. But with this host it is not needed, basically there are only Errors, with the highest severity.

Let us have a closer look at the first error, “TLS Certificate Self-signed”. We can drill down by pressing the Magnifying glass Button:

Flow Details

Here again there is a lot of information about the flow. For the security analysis we take a closer look at the three highlighted items:

1. The extracted url of the client request, hXXp://amajai-technologies[.]world/ looks at first sight accurate, maybe the TLD .world is for a technology provider a bit off. But doesn’t it look similar to what it actually tries to copy, akamaitechnologies.com? Look-alike domains [7] are easily overseen in daily work and are a real cyber security threat. 
2. The TLS certificate is self-signed: Red flag. No reputable or professional service uses a self-signed certificate for external / internet communication. Furthermore the TLS issuer and subject fields are empty, the red flag just got bigger. For reference, how the certificate issuer information looks like:
3. Assumptions that something is odd with this flow are validated by the JA3 [8] client fingerprint. The crossed circle, , indicates that the client fingerprint is listed as malicious. Follow the JA3 to sslbl.abuse.ch for further details and yes, the client certificate is malicious:
   
   Although the blacklisted JA3 is not Qakbot, we just found out that this host is a well infected with the Gozi banking trojan.

Conclusion

Even for analysing recorded network traffic, ntopng is great help for getting an overview of what happened on the network. But the real value of ntopng is, it does all the scoring and alerting in real time. Facing above malware, I might not know immediately what exact type of malware it is, but the generated alerts are strong indicators that something with that host is not ok and it can be isolated from the network for further analysis.

Got curious? Get started and try to find the spam activity or the Qakbot flow(s) yourself!

We invite other members of our community to contact us if they want to share their work on this blog.

References

1. https://www.malware-traffic-analysis.net/2020/12/07/2020-12-07-Qakbot-with-Cobalt-Strike-and-spambot-activity.pcap.zip Note: the zip is password protected. Use infected to extract the pcap.
2. https://urlhaus.abuse.ch/browse/tag/Qakbot
3. https://malpedia.caad.fkie.fraunhofer.de/details/win.qakbot
4. https://www.zdnet.com/article/cobalt-strike-and-metasploit-accounted-for-a-quarter-of-all-malware-c-c-servers-in-2020/
5. https://unit42.paloaltonetworks.com/tutorial-qakbot-infection/
6. https://packages.ntop.org/
7. https://info.phishlabs.com/blog/what-is-a-look-alike-domain
8. https://engineering.salesforce.com/tls-fingerprinting-with-ja3-and-ja3s-247362855967

At the upcoming Sharkfest Europe 2021 we’ll talk about using Wireshark in cybersecurity. Part of the talk will focus on nDPI and Wireshark integration. Since the last release nDPI features flow risk analysis, that is basically a numerical indication of potential risks associated with a network communication ranging from ‘TLS Certificate Expired’ to more complicated ‘Suspicious DGA domain name’ and ‘SQL injection’. You can find a comprehensive list of increasingly growing risks here.

For the impatiens, this is a quick guide on how to play with this integration.

Prerequisite

Download and install nDPI, including the script ndpi.lua used by Wireshark to talk with nDPI. You can find full installation instructions at the nDPI/Wireshark page.

Usage

Open a pcap file or do live capture using the nDPI extcap plugin that enabled Wireshark to use nDPI.

You can do that by clicking on the wheel icon next to the nDPI interface and selecting a pcap or an interface name from the dialog window that will open after the click on the icon.

Once you click on start Wireshark will operate as usual, start dissecting packets. The above plugin will add a new header highlighted in red in the picture below.

For all flows, nDPI will report the application protocol. Furthermore for those flows for which a non zero risk has been identified, the flow risk is reported both in string (suitable for humans) and numeric (suitable for creating Wireshark filtering). The numerical value is a bitmap with bit set to 1 for the corresponding identified risk. In the above picture we show an example dissecting Anydesk traffic. You can find testing pcaps in the nDPI test regression tests directory.

We hope you will attend SharkFest Europe for learning more about all this, and the other extensions we have coded in nDPI and Wireshark.

Enjoy !

This is to introduce a new nProbe feature that brings IPS (Intrusion Prevention System) support via nDPI for Linux and FreeBSD (including OPNsense and pfSense). As shown in the picture below, nProbe acts as a transparent bridge (with kernel offload) for applying pass/drop/shape rules to the forwarded traffic. Our goal is to combine the power of DPI and nDPI cybersecurity features to all nProbe users.

When deployed on a firewall/gateway (including OPNsense/pfSense), nProbe can both monitor and apply policies to monitored traffic. Typical use case include (but are not limited to):

• Drop unwanted protocols (e.g. Tor).
• Block “unclean” traffic (e.g. TLS traffic with self-signed certificates, HTTP traffic with suspicious SQL injection).
• Shape protocols that take a lot of bandwidth (e.g. BitTorrent).
• Prevent specific hosts from being accessed by specific countries or region (e.g. my core servers can be accessed only from USA).
• Drop traffic from/to hosts that have been marked as malware.
• Assign fair bandwidth (Linux only) to protocols by preventing specific protocols (e.g. Netflix or YouTube) to monopolize the bandwidth.
• Block traffic from advertisement servers (similar to PiHole) or blacklisted hosts.

You can use ntopng (via the usual ZMQ interface) to monitor and report nProbe policed traffic. As you can see in the picture below dropped traffic is depicted with a strikethrough line.

IPS Policy Configuration

In order to enable the IPS mode, you need to start nProbe with a new parameter –ips-mode (e.g. –ips-mode ips_rules.txt) that specifies a configuration file where you can specify your policies. Example:

# Pool definition
{"pool":{"id":1,"name":"my pool 1","ip": [ "131.114.0.0/16"], "mac": [ ]},"policy": {"id": 1} }
{"pool":{"id":2,"name":"my pool 2","ip": [ "192.168.1.0/24"], "mac": [ ]},"policy": {"id": 2} }

# Continents: Africa / Asia-Pacific / Europe / North America / South America

# Policy definition
{"policy":{"id":0,"name":"root policy rule", "default_marker": "pass", "markers": { "countries": { "IT": "pass", "CN": "drop", "US": "pass" } } } }
{"policy":{"id":1,"root":0,"name":"my rule 1", "markers": { "categories": { "Network": 7, "Download-FileTransfer-FileSharing": 8, "DataTransfer": 8, "VPN": 8, "Video": 9, "Music": 9, "Streaming": 9, "Media": 9 }, "protocols": { "DNS": "drop" }, "countries": { "IT": "drop", "CN": "drop", "US": "pass" }, "asn" : { }, "continents" : { "Asia" : "drop"} } , "default_marker": "pass"  } }
{"policy":{"id":2,"root":0,"name":"my rule 2", "markers": { "categories": { "Video": "pass" }, "flow_risk": { "risks": [12], "marker": "drop" }, "protocols": { "DNS": "drop" }, "countries": { "IT": "pass", "US": "pass" }, "asn" : { "34984" : "drop" } }, "default_marker": "pass"  } }
#{"policy":{"id":2,"root":0,"name":"my rule 2", "markers": { "categories": { "Video": "drop" }, "protocols": { "DNS": "drop" }, "countries": { "IT": "pass", "US": "pass" } }, "default_marker": "pass"  } }

### GeoIP ###

{ "geoip": { "asn": ".GeoLite2-ASN.mmdb", "city": "GeoLite2-City.mmdb" }}

As you can see there are two main components

• Pool: list of IP and MAC addresses to which you want to apply a specific policy.
• Policy: traffic policy rule that allows you to define how to threat the traffic bound to a specific pool.

You can define multiple pools and policies for applying granular policies to selected hosts.

IPS Configuration

On Linux you need to configure NetFilter before enabling the IPS mode, whereas on FreeBSD this not necessary. On OPNsense you can enable the IPS configuration from the user interface.

You can read more about IPS configuration in the nProbe user’s guide.

IPS Performance

Below you can find the expected performance of the IPS Mode on both Linux and FreeBSD using a low-end computer or mid-range PC. As you can see the IPS performance is affected by the way the operating system enables packet bridging (see nProbe user’s guide for details).

Tests have been performed on the following conditions:

• Linux IPS mode has been tested with 4 queues using a NetFilter configuration named “kernel marker bypass”. This explains why nProbe in IPS mode is faster on a low-end box that the vanilla bridge.
• Under FreeBSD there is no kernel marker bypass mode, hence the performance decrease is severe when using nProbe as bridging is performed in user-space.

IPS Availability

The nProbe IPS mode is available free of charge (no additional license required) as part of nProbe, and it does not speak with cloud/third-party services in order to operate (i.e. your traffic is safe and not shared with anybody as other products do). Currently, the IPS mode is available in the nProbe nightly builds and it will be merge in the next stable release. The number of pools/policies depend on the nProbe version as described in the nProbe product page,

What’s Next?

This work is not yet fully completed as we’re still developing two important missing pieces that will make this work even more effective:

• Add the ability to configure nProbe IPS policies from ntopng in a simple and user-friendly way.
• [Linux] When using the kernel marker bypass mode, add the ability to produce flow information also for flows that are bridged in kernel.

Stay tuned for these news, and enjoy !

For a long time, ntop mainly focused on passive traffic analysis. As cybersecurity is becoming a main concern for many organisation and individuals, we have boosted our tools by introducing facilities for spotting threats and blocking unsafe traffic.

This month we will organise a webinar that will cover two main topics:

• How to use the nProbe IPS mode to block traffic based on nDPI and other traffic policies.
• Tutorial on using ntopng and nProbe on pfSense/OPNsense.

This event will take place Thursday May 27th at 4PM CET / 10AM EST.

You can register for this (free) webinar at this URL.

Today we had the change to talk about network visibility and cybersecurity during an event organised by the Milan Internet Exchange MIX-IT. In this talk we have presented the current state of development in this area at ntop and provided an outlook of some of the features that we’re developing and that will be released later this summer.

These are the presentation slides for those who didn’t have the change to attend the event.

Enjoy !

nDPI it is much more than a DPI library used to detect the application protocol. In the past year, nDPI has grown in terms of cybersecurity features used to detect threats and network issues leveraging on the concept of flow risk. Each nDPI-analysed flow has associated a numerical flow risk that in essence is a bitmap with a bit set to 1 whenever a risk has been detected for such flow. The list of (to date) supported flow risks are:

• HTTP suspicious user-agent
• HTTP numeric IP host contacted
• HTTP suspicious URL
• HTTP suspicious protocol header
• TLS connections not carrying HTTPS
  (e.g. a VPN over TLS)
• Suspicious DGA domain contacted
• Malformed packet
• SSH/SMB obsolete protocol
  or application version
• TLS suspicious ESNI usage
• Unsafe Protocol used
• Suspicious DNS traffic
• TLS with no SNI
• XSS (Cross Site Scripting)
• SQL Injection
• Arbitrary Code Injection/Execution
• Binary/.exe application transfer (e.g. in HTTP)
• Known protocol on non standard port
• TLS self-signed certificate
• TLS obsolete version
• TLS weak cipher
• TLS certificate expired
• TLS certificate mismatch
• DNS suspicious traffic
• HTTP suspicious content
• Risky ASN
• Risky Domain Name
• Malicious JA3 Fingerprint
• Malicious SHA1 Certificate
• Desktop of File Sharing Session
• TLS Uncommon ALPN

The flow risk bitmap can have zero (no risks), or more bits set, depending on the issues nDPI has found on the flow. For every risk, nDPI associates a severity, and for each severity a score. With “ndpiReader -h” you can display the list of know risks and their associated severity:

nDPI supported risks:
 Id Risk                                             Severity Score
  1 XSS attack                                       Severe   250
  2 SQL injection                                    Severe   250
  3 RCE injection                                    Severe   250
  4 Binary application transfer                      Severe   250
  5 Known protocol on non standard port              Low      10
  6 Self-signed Certificate                          Medium   50
  7 Obsolete TLS version (< 1.1)                     Medium   50
  8 Weak TLS cipher                                  Medium   50
  9 TLS Expired Certificate                          High     100
 10 TLS Certificate Mismatch                         High     100
 11 HTTP Suspicious User-Agent                       Medium   50
 12 HTTP Numeric IP Address                          Low      10
 13 HTTP Suspicious URL                              High     100
 14 HTTP Suspicious Header                           Medium   50
 15 TLS (probably) not carrying HTTPS                Low      10
 16 Suspicious DGA domain name                       High     100
 17 Malformed packet                                 Low      10
 18 SSH Obsolete Client Version/Cipher               Medium   50
 19 SSH Obsolete Server Version/Cipher               Medium   50
 20 SMB Insecure Version                             Medium   50
 21 TLS Suspicious ESNI Usage                        Medium   50
 22 Unsafe Protocol                                  Low      10
 23 Suspicious DNS traffic                           Medium   50
 24 SNI TLS extension was missing                    Medium   50
 25 HTTP suspicious content                          Medium   50
 26 Risky ASN                                        Medium   50
 27 Risky domain name                                Medium   50
 28 Possibly Malicious JA3 Fingerprint               Medium   50
 29 Possibly Malicious SSL Cert. SHA1 Fingerprint    Medium   50
 30 Desktop/File Sharing Session                     Low      10
 31 Uncommon TLS ALPN                                Medium   50

The flow risk score is computed as the sum of the severities for all bits set to one. Example:

TCP 192.168.149.129:43535 <-> 51.83.238.219:80 [proto: 91.252/TLS.AnyDesk][cat: RemoteAccess/12][2942 pkts/175103 bytes <-> 4001 pkts/2618640 bytes][Goodput ratio: 9/92][55.97 sec][bytes ratio: -0.875 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 19/14 7028/7028 153/126][Pkt Len c2s/s2c min/avg/max/stddev: 54/60 60/654 1514/1514 50/618][Risk: ** Known protocol on non standard port **** TLS (probably) not carrying HTTPS **** SNI TLS extension was missing **** Desktop/File Sharing Session **][Risk Score: 80][TLSv1.2][JA3C: 201999283915cc31cee6b15472ef3332][JA3S: 107030a763c7224285717ff1569a17f3][Issuer: CN=AnyNet Root CA, O=philandro Software GmbH, C=DE][Subject: C=DE, O=philandro Software GmbH, CN=AnyNet Relay][Certificate SHA-1: 9E:08:D2:58:A9:02:CD:4F:E2:4A:26:B8:48:5C:43:0B:81:29:99:E3][Firefox][Validity: 2018-11-18 02:14:23 – 2028-11-15 02:14:23][Cipher: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384][Plen Bins: 0,7,17,2,1,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,3,0,0,33,0,0,0,0,29,0,0]

The flow risk score is computed as follows:

The score value is a numerical value that represents how risky is a flow: the higher is the score, the more “risky” is a flow. Using this technique you can immediately spot flows that are more suspicious and that require more attention. This without using multivariate traffic analysis or complex algorithms. The flow score can be used a a good labelling technique for machine-learning based traffic analysis tools.

Enjoy !

In  recent ntopng versions, alerts have been significantly enriched with metadata useful to understand network and security issues. In this post, we focus on the “Attacker” and “Victim” metadata, used to enrich flow alerts and label hosts. Specifically, the client or the server of a flow is labelled as “Attacker” when it is, with high probability, the originator of one or more security issue. Similarly, when the client or the server is deemed to be under attack, it is labelled ad “Victim”. For non security-oriented use cases (e.g. severe packet drops), hosts affected/originating the issue are still highlighted with a high score value but they are not labelled as attacker/victim as these words are used only in security.

For example:

• In a DNS request for a suspicious DGA domain name, the client is considered to be an “Attacker” as it is the originator or such potentially malicious request.
• When Remote Code Execution (RCE), Cross Site Scripting (XSS) and SQL injection attempts are detected, the client is the “Attacker” and the server is the “Victim” as it is being probed/attacked by the client who is trying to exploit its vulnerabilities.
• When a binary/.exe application is downloaded over HTTP pretending it is an harmless file such as a PNG image (see Introducing nDPI Risk Analysis), the client is considered to be compromised and the server is considered to be malicious as it is sourcing suspicious files so both are labelled as “Attacker”.
• A DNS data exfiltration alert has both its client and server labelled with “Attacker” as carrying out a data exfiltration requires both the client and the server to be running exfiltration software such as iodine.

In the reminder of this post, we analyse Hancitor Infection (pcap courtesy of Malware-Traffic-Analysis) in order to demonstrate the effectiveness of ntopng with its new “Attacker” and “Victim” metadata in detecting this kind of security issues.

An Hancitor infection is basically a multi-stage chain of events that causes the targeted host to download malware files actually turning it in a malicious host in the hands of a threat actor. This post discusses the details of this kind of infections.

Let’s see what happens when we use ntopng to analyse the pcap with the Hancitor infection. You can start ntopng as

ntopng -i 2021-06-01-Hancitor-with-Cobalt-Stike-and-netping-tool.pcap -m "10.0.0.0/8"

First of all, it emerges pretty clearly from the header bar that there’s something suspicious going on, as there are thousands of flows with errors and warnings. Note that the number of alerted flows can vary according to your configuration (see in the left sidebar menu “Settings” -> “User Scripts”)

Clicking on the red “Errors” badge brings us to the flows page, filtered by flows having errors. By opening the “Status” dropdown, it becomes quite clear there is some suspicious activity as highlighted by the several suspicious DGA domain requests as well as by 2000+ of suspicious file transfers.

However, this page is not enough to understand if there is an attack going on and who is the source of the trouble. This becomes more evident when visiting the flow alerts page. Before browsing the alerts, we can set a filter to only see security-related alerts having an “Attacker”

With this filter, ntopng shows only alerts for which it has detected an attacker. Indeed, Suspicious DGA Domain alerts start popping out. The skull symbol helps in identifying the “Attacker” that, in this case, is the Hancitor-infected Windows client host.

If we keep browsing the alerts with an attacker, we will also see thousands of Suspicious File Transfer alerts. For this kind of alerts, two skulls are shown. Indeed, it is reasonable to mark as “Attacker” not only the compromised Windows host, but also the server that is distributing malicious files.

But why does file transfers for seemingly innocuous files ga.js are deemed to be suspicious? Well, because in practice, these are not Javascript files! Sometimes, they are just empty files, some other times they are binary files with unknown contents. This can be verified simply by extracting these ga.js files with Wireshark and demonstrates the effectiveness of ntopng in detecting these suspicious transfers taking place in the network. Additional suspicious files can be discovered simply by browsing these alerts. Those are requests to download Ficker Stealer and Cobal Strike.

You can identify attacker/victim in int Host Map (left sidebar menu “Maps” -> “Hosts”)

as they are the two outliers in the X/Y axis. Clicking on the bubble you can immediately jump to the host page and see what happens, as already discussed in this article.

Finally, it is worth mentioning that “Attacker” and “Victim” metadata are available also for host alerts, in order to spot hosts doing suspicious activities such as SYN scans, or high volumes of DNS or ICMP traffic. In addition to use the score in realtime as shown above, you can also monitor the score overtime in order to detect when it deviates from its expected behaviour. This is compulsory to detect “soft” behavioral changes that wouldn’t be detected with the score technique shown above, that instead is used to spot attackers with a more aggressive behaviour.

In summary, we have implemented in ntopng both “signature based” (i.e. by means of alerts) and “behavioural based” (i.e. metric analysis overtime to detect deviations from expected behaviour) anomaly detection. In a future port, we’ll elaborate more in detail how behavioural changes are detected and reported.

Enjoy !

Introduction

Recently, we have added Intrusion Prevention System (IPS) capabilities to our nProbe. Those capabilities are available starting from the latest 9.5 version, both for Linux and FreeBSD – including OPNsense and pfSense, and are available with all nProbe versions and licenses (see the product page for additional details).

On Linux, nProbe leverages the netfilter framework. In essence, the kernel send packets to nProbe via NF_QUEUE which, in turn, gives each packet a pass/drop verdict so that it can be dropped or let it continue its journey through the network. In addition, nProbe has the ability to mark packets to create policies that go beyond the simple pass/drop but can include, shaping, routing, and so on. Using markers it is also possible to offload traffic packets. In essence, once a flow packet has been marked, the kernel can be told to bridge additional flow packets directly, without sending them to nProbe. This is beneficial and allows to reach very good bridging performances. On FreeBSD, the NetFilter framework is not available and nProbe leverages netmap. For this reason, all packets need to be sent to userspace to nProbe. Hence, measured performances are reduced if compared to Linux. The nProbe product page reports typical performance figures for both Linux and FreeBSD.

For those of you who prefer a video resource, here’s the step-by-step tutorial

otherwise you can read the rest of this article for details.

Creating a Layer-7 Traffic Policer

In this post, we create a Layer-7 traffic policer on a Linux machine, put as a bump-in-the-wire between the clients we want to policy and the router that handles their traffic. A commodity 4x1Gb Intel 82580 network card installed on the machine is used to bridge the traffic between the clients and the router. nProbe IPS will operate on top of this bridge, that is created with 2 of the 4 interfaces of the network card:

• enp2s0f0 is connected to a WiFi access point where clients are connected.
• enp2s0f1 is connected to the Router that in charge of handling clients’ traffic.

Packets reaching one of the two bridged interfaces, before being bridged, are sent to nProbe via NFQUEUE that emits a verdict. If the packet verdict is drop, then the packet is dropped and not bridged. Otherwise, the packet is allowed to reach the other interface.

Let’s now see how to – quickly and easily – configure the bridge, NFQUEUE and nProbe to implement the Layer-7 traffic policer.

Configuring the Bridge and NFQUEUE

This part of the configuration prepares the system to:

• Bridge clients’ traffic
• Send clients’ traffic to nProbe for policy enforcement

To facilitate the configuration, we have prepared a shell script that takes care of the above. This script, distributed along with any packaged version of nProbe, becomes automatically available at the following path once nProbe is installed:

 /usr/share/nprobe/netfilter/scripts/policy_and_bridge_simple.sh

The script begins with a brief configuration section that allows interface names to be configured. Interface names are pretty much everything that needs to be specified. After the necessary configuration changes have been applied, the script can be run as ./policy_and_bridge_simple.sh to set up the system.

In short, the script carries out the following tasks. First, it creates a bridge using Linux bridging capabilities. Then, it instructs the kernel to send packets to nProbe, rather than bridging them automatically. This is done with a couple of iptables rules, indicating that packets entering the system from bridged interfaces must go through a NFQUEUE.

For the sake of example, iptables can be used to send packets entering the system from enp2s0f0 and enp2s0f1 to the NFQUEUE queue identified with id 0 as follows:

iptables -t mangle -A FORWARD -j NFQUEUE --queue-num 0 --queue-bypass -m physdev --physdev-in enp2s0f0
iptables -t mangle -A FORWARD -j NFQUEUE --queue-num 0 --queue-bypass -m physdev --physdev-in enp2s0f1

Configuring nProbe IPS

nProbe, to operate in IPS mode, needs a rules file. Policy rules are applied to groups of hosts (identified with IP or MAC Addresses) and give the ability to pass/drop/mark packets on the basis of their Layer-7 application protocol, Layer-7 application category, country, flow risk, and so on.

A very simple configuration file can be the following

# Pool definition
{"pool":{"id":1,"name":"Local Networks","ip": [ "172.16.12.0/24", "172.16.4.0/24", "192.168.2.0/24"], "mac": []},"policy": {"id": 0} }

# Policy definition
{"policy":{"id":0,"name":"root policy rule", "default_marker": "pass", "markers": { "categories": { }, "protocols": {"Facebook": "drop" }, "countries": { }, "asn" : { }, "continents" : { } } } }

The file basically tells nProbe to drop Facebook traffic ("protocols": {"Facebook": "drop" }) for all hosts belonging to networks "172.16.12.0/24", "172.16.4.0/24", "192.168.2.0/24". A pass verdict is emitted for all non-Facebook traffic ("default_marker": "pass").

Now that the file is ready, assuming it is saved as nprobe-ips.config, nProbe can be started as follows

$ /usr/bin/nprobe --ips-mode nprobe-ips.config -i nf:0 -n none

Relevant options have the following meaning:

• --ips-mode nprobe-ips.config tells to operate in IPS mode and specifies the rules file
• nf:0 indicates the NFQUEUE queue used by the kernel to send packets to nProbe. Note that this id 0 MUST match with the id indicated in the configuration of policy_and_bridge_simple.sh. In general, it is safe to use 0.

From this point on, traffic policies are enforced for all clients. For example, this is what happens to one of the client that tries to reach Facebook before and after Layer-7 policer activation.

Conclusion

In this post, we have seen how to quickly create a Layer-7 traffic policer on Linux using nProbe IPS. A very simple "protocols": {"Facebook": "drop" } policy has been created just for the sake of example. In practice, nProbe IPS is flexible enough to create complex policies using also Layer-7 categories, countries, autonomous systems and so on.

Moreover, by leveraging packet marks, effective kernel traffic offload and traffic shaping can be implemented. There are scripts distributed with any packaged version of nProbe available under /usr/share/nprobe/netfilter/ that can be used for reference when implementing offload and shaping.

Network interfaces natively support RX and TX directions, so tools such as ntopng can detect the traffic directions and depict this information accordingly.

In the above picture that ntopng shows in the top menubar, TX traffic is depicted in blue and RX in green. All simple.

Now suppose you need to analyse sFlow/NetFlow/IPFIX flows, and be interested to understand how much traffic leaves/enters your network. Example suppose you generate IPFIX flows on your Internet gateway: how much of this traffic is sent to the Internet and how much is received? With flow analysis we know the direction at the interface level (i.e. flow X has been received on interfaceId 5 and send on InterfaceId 3) but how do we map interfaces to global traffic directions?

In the latest dev packages you can specify this information in nProbe by specifying what is the list of your local networks. Observed traffic sent by a local host to a remote host is considered TX, whereas everything else is considered RX. Example suppose that your 192.168.1.0/24,172.16.0.0/16 network is monitored by observing flows on the Internet gateways. In order to specify directions based on IPs you can do this (flows are collected on port 2055 and sent via ZMQ to ntopng running on the same host where nProbe is active)

nprobe –collector-port 2055 -n none –zmq tcp://127.0.0.1:1234 –local-networks “192.168.1.0/24,172.16.0.0/16”

and

ntopng -i tcp://127.0.0.1:1234

This way automatically when nProne is in collector mode (–collector-port), the flow direction is set using the list of local networks. When ntopng receives the traffic, it interprets the direction according to what specified by nProbe in the flow with no other configuration on the ntopng side (e.g. –local-networks).

Note that:

• flow direction works not only for flow collection but also when generating flows sniffing packets, so you can have a comprehensive solution for both flow and packet traffic analysis.
• you can have multiple nProbe instances sending traffic to the same ntopng, and each nProbe instance compute the flow direction locally on the probe. This allows you to monitor heterogeneous/overlapping networks.

Enjoy !

Amazon Virtual Private Cloud (VPC) flow logs and in essence text-based Netflow-like logs consisting of fields that describe the traffic flow. They are often collected on disk and published to S3 buckets or CloudWatch for an AWS-centric monitoring infrastructure (extra AWS charge is necessary).

Now suppose that you want to use this information to monitor your VPC using ntop tools or turn these logs in industry standard NetFlow/IPFIX flows that can be ingested in any monitoring application unable to understand this proprietary log format. In this case you can use nProbe to process these flow logs and send them to ntopng or your favourite flow collection application, not to mention the ability to deliver them to Elastic or Kafka-based consumers. The setup for VPC logs processing is similar to what you have with flow collection, with the exception that nProbe is not receiving flows via UDP (as it happens with sFlow/NetFlow/IPFIX) but it is monitoring a directory where VPC logs are stored.

For this purpose we have extended the format for the collection port parameter to support these logs. In essence nProbe:

• Watches a directory where flows are stored.
• Uses the VPC flow logs to fill up the flow cache, similar to what happens when collecting flows via UDP.
• Exports them to collectors (e.g. ntopng in the above example) using the standard nProbe features.
• Deletes the VPC log file, after processing.

Below you can find a couple of examples where you collect flows and send them to a local ntopng instance for realtime traffic visualisation and analysis.

• Collect sFlow/NetFlow/IPFIX flows on port 2055:
  nprobe -3 2055 --zmq tcp://127.0.0.1:1234
• Collect AWS VPC flows on directory /data/vpc:
  nprobe -3 /data/vpc --zmq tcp://127.0.0.1:1234
• Run ntopng to collect flows from nProbe:
  ntopng -i tcp://127.0.0.1:1234

Using nProbe you can now transform your proprietary VPC logs into industry-standard NetFlow/IPFX flows and process them with your existing monitoring infrastructure without using proprietary formats or tools. Note that VPC logs do not include information such as DPI, so that nProbe can only guess application protocol with nDPI port/address-based heuristics. Remember that nProbe can run on AWS for full traffic visibility in case you need advanced monitoring information.

This extension is present in the current nProbe nightly build (all versions, Pro and Enterprise) and it will be integrated in the next stable release.

Enjoy!

Introduction

ntopng, in combination with nProbe, can be used to collect NetFlow. Their use for NetFlow collection is described in detail here.

In this post we measure the performance of nProbe and ntopng when used together to collect, analyze, and dump NetFlow data. The idea is to provide performance figures useful to understand the maximum rate at which NetFlow can be processed without loss of data.

Before giving the actual figures, it is worth discussing briefly the most relevant unit of measure that will be used, i.e., the number of flows per second – fps in short. NetFlow is per se a protocol designed to export communication flows. It carries flows inside UDP datagrams, and each UDP datagram carries multiple flows. For this reason, it is natural to choose the number of flows per unit of time when it comes to analyzing the performance of ntopng and nProbe when dealing with NetFlow. Indeed, we will use fps to quantify the number of flows analysed and written to disk when giving performance figures.

Knowing how much your NetFlow actually accounts for, in terms of fps, is also fundamental to make sense of the figures we are giving and to understand if ntopng and nProbe perform well enough for your network. In general, a few Mbps of NetFlow are able to carry thousands of flows per second. If you do not know the how many flows per second your NetFlow accounts for, you can consider that, as a ballpark figure, 10 Mbps of NetFlow v9 carry approximately 12,000 fps. Note that this is just a ballpark as the flow size depends on the flow template for v9 and IPFIX.

Performance Figures

Finally, it’s time to look at the figures, that are given both for two and four interfaces.

The rationale of expressing performance numbers for a variable number of interfaces is that multiple interfaces:

• Can be used to effectively balance the NetFlow across multiple cores
• Can be used to keep the traffic of multiple NetFlow sources separated

However, to give consistent numbers, results are also expressed in terms of fps per interface in the first column. Note also that results apply when using interface views too and that, when nIndex flow dump is enabled, the expected reduction in the processing rate is between 2 and 3 Kfps.

Example

To give a real example that can help you in making sense of the figures above, let’s consider a case where you want to collect NetFlow at 85 Mbps, i.e., approximately, 100 Kfps. To collect at 100 Kfps, according to the table above, we need at least 3 interfaces. Let’s set up an ntopng instance with four interfaces to say on the safe side – traffic is aggregated together using the view with -i view:all.

./ntopng -i tcp://*:5556c -i tcp://*:5557c -i tcp://*:5558c -i tcp://*:5559c -iview:all -F"nindex"

Assuming NetFlow is arriving on port 2056 and ntopng is running on host 192.168.2.225, we can configure nProbe to collect NetFlow and load-balance flows on the four interfaces above as follow

 ./nprobe -i none -n none --collector-port 2056 -b 1 -T "@NTOPNG@" --zmq tcp://192.168.2.225:5556 --zmq tcp://192.168.2.225:5557 --zmq tcp://192.168.2.225:5558 --zmq tcp://192.168.2.225:5559 --zmq-probe-mode --collector-passthrough

Final Remarks

In this post we have seen the performance figures of ntopng and nProbe when used to collect NetFlow. We have seen how to quantify the flows per second (fps) carried by NetFlow and have also determined that the combination of nProbe and ntopng is suitable for the collection of NetFlow at 100+ Kfps. Figures given are valid for the latest ntopng 4.3 (soon ntopng 5.0 stable) and nProbe 9.5 (soon 9.6 stable), that represent a significant step towards high-speed flow collection. Indeed, their performance exceeds the performance of the previous versions by at least a 15%.

You can read more about high-speed flow collection here or directly refer to the user guide.

In our previous post we have analysed the performance of the pipeline nProbe+ntopng for those who need to collect flows and analyse them, trigger alerts, create timeseries, provide a realtime monitoring console, dump them to nIndex and inform remote recipients in case of some problem is detected. This is the main difference between the ntop solution and a NetFlow collector whose main goal is to dump flows on a database with any or little flow analysis. In essence the current state of the art with 4 nProbe instances sending data to a single ntopng instance is a process rate of ~100k flows/second.

In this post we analyse the performance of nProbe alone, so that you can make up your mind if:

• You only need to collect flows and dump them on disk or on a database.
• You want to create a distributed collection architecture where you have multiple collection points and you need to know how fast you can scale.

Goal of this article is to show you the various options we can offer you in terms of flow collections and processing, and show you that maximum performance you can achieve on a single stand-alone system (no clustering or multi-node deployment) when collecting hundred of thousand flows/sec and dumping them on a open-source database able to store trillion of records and retrieve them with sub second queries.

nProbe: How to Dump Flows Into ClickHouse

One of the new features of the recent nProbe dev builds is the ability to dump flows in ClickHouse, a column-oriented database able to scale to hundred of billion or records/node using a single node and no more than 16 GB or RAM regardless of the number of records. The nice thing about ClickHouse is that you can both access this database using the native client (as nProbe does for importing data) or as a MySQL database listening on port 9004 (MySQL default port is 3306). As ClickHouse is integrated with Grafana, you can easily create dashboards with a few clicks if you want. Currently ClickHouse support is integrated in the nProbe Enterprise M and up versions (not available on Windows), and it will be integrated in the next stable nProbe release due by the end of this month. Similar to MySQL, you can enable ClickHouse in nProbe with:

 --clickhouse=<host[@port]>:<dbname>:<prefix>:<user>:<pw>

Example if you have a the same box nProbe and ClickHouse running you can start nProbe as

nprobe --clickhouse=127.0.0.1:ntop::default:helloworld

where ntop is the database name, default is the ClickHouse username with password helloworld. nProbe stores temporary data (used to batch data for ClickHouse import) in the directory specified with -P or on a temporary directory if -P is not specified.

As already said, once your data is on ClickHouse you can query it using mysql client or the native client as follows:

clickhouse-client --password xxxxxxx 
ClickHouse client version 21.7.3.14 (official build).
Connecting to localhost:9000 as user default.
Connected to ClickHouse server version 21.6.5 revision 54448.

ClickHouse server version is older than ClickHouse client. It may indicate that the server is out of date and can be upgraded.

pc-ntop :) use ntop

USE ntop

Query id: 8c8d9ea5-5396-41ee-a648-3524c3c22add

Ok.

0 rows in set. Elapsed: 0.001 sec. 

pc-ntop :) select count(*) from flows where IPV4_SRC_ADDR = IPv4StringToNum('192.168.0.1')

SELECT count(*)
FROM flows
WHERE IPV4_SRC_ADDR = IPv4StringToNum('192.168.0.1')

Query id: ce8c8375-aa35-4f94-a1ca-9a09d891902d

┌───count()─┐
│ 110894877 │
└───────────┘

1 rows in set. Elapsed: 0.035 sec. Processed 110.90 million rows, 443.58 MB (3.19 billion rows/s., 12.76 GB/s.)

pc-ntop :) Bye.

In a future ntopng version we will eventually add support for querying data directly from ntopng.

nProbe Performance

Below you can find a table with the nProbe performance we have measured on a Ubuntu 18.04 system based on Intel E3-1241 and a SSD drive where data is stored. In order to measure that maximum nProbe performance we have added a new testing CLI option named  –simulate-collection that simulates infinite flow ingress collection so that we can measure the worst case with randomly generated flows. The base command line option we have used (it needs to be slightly adapted according to the test case) is 

nprobe -b 1 –simulate-collection -t 10 -n none –clickhouse=127.0.0.1@9004:ntop::default:XXXXX -T “%IPV4_SRC_ADDR %IPV4_DST_ADDR %INPUT_SNMP %OUTPUT_SNMP %IN_PKTS %IN_BYTES %FIRST_SWITCHED %LAST_SWITCHED %L4_SRC_PORT %L4_DST_PORT %TCP_FLAGS %PROTOCOL %SRC_TOS %SRC_AS %DST_AS %EXPORTER_IPV4_ADDRESS %IPV6_SRC_ADDR” -P /tmp/ramdisk/

As you can see flows are dumped to /tmp/ramdisk/ (created with sudo mount -t tmpfs -o size=5G myramdisk /tmp/ramdisk) before are batch imported into ClickHouse that is running on the same host where nProbe runs. 

Below you can find the results of all test cases:

The above figures are per nProbe instance, so you can run multiple nProbe instances for maximising performance. In our tests with a local ClickHouse database, we can import up to 150k Fps with two nProbe instances that seems to be the maximum a single ClickHouse database can handle.

As a final note you can see, the figures above exceed the nProbe+ntopng performance of 100k Fps, but it is hard to compare them 1:1 as:

• nProbe+ClickHouse is just collecting flows and storing them on a high-performance MySQL-compatible database, as most collectors on the market do. No data analysis, alerting or traffic timeseries are created.
• nProbe+ntopng is a complete NetFlow traffic solution, with data analysis, anomaly detection, traffic behaviour analysis, timeseries etc.

So depending on what are your requirements you can choose one or the other. Note that you can use nProbe+ClickHouse+ntopng for the best of both worlds. We believe it is a great achievement to be able to collect, analyse and dump over 100k flows/sec on a single host with no need to create a cluster of database instances (necessary for instance if you want to dump flows with nProbe/ntopng into Elastic) or costly systems to both build and maintain.

Enjoy !

Collecting flows on large networks with hundred of routers can be challenging. Beside the number of flows to be collected, another key point is to be able to visualize the informations in a simple yet effective way. ntopng allows you to create up to 32 virtual flow collection interfaces that can be used to avoid merging collected flows: unfortunately they are not enough when collecting flows from 100+ routers. In the latest ntopng and nProbe dev versions (soon to become stable), we have implemented the concept of observation point, that in IPFIX is defined as a location in the network where packets can be observed.

The problem we want to solve is: how-to to cluster flows becoming from the same site regardless of the probe IP that originates them, avoid merging them with those coming from other sites, but still have the ability so see them as a whole at the interface level where flows are collected. Each nProbe instance can be configured to set a numerical value for the observationPoint Id that uniquely identifies a site. Depending on the site size, a site can have one or multiple probes. In nProbe the observationPoint is set with the -E flag as follows:

Rome

• nprobe -E 0:1234 --zmq tcp://192.168.1.100:1234 --zmq-probe-mode -i eth1
• nprobe -E 0:1234 --zmq tcp://192.168.1.100:1234 --zmq-probe-mode -i eth2
• nprobe -E 0:1234 --zmq tcp://192.168.1.100:1234 --zmq-probe-mode -i eth3

Paris

• nprobe -E 0:1235 --zmq tcp://192.168.1.100:1234 --zmq-probe-mode -i eno1

Berlin

• nprobe -E 0:1236 --zmq tcp://192.168.1.100:1234 --zmq-probe-mode -i enp2s0f0
• nprobe -E 0:1236 --zmq tcp://192.168.1.100:1234 --zmq-probe-mode -i enp2s0f1

Central  ntopng Flow Collector

• ntopng -i tcp://92.168.1.100:1234c

When flows are sent by nProbe, they are uniquely marked with the observationPoint Id that is honoured by ntopng during flow collection and reported in the web interface.

In the top menubar, ntopng lists all the known observationPoint Ids in the dropdown menu: this way a network analyst can select the observationPoint he wants to visualise while hiding flows from a different observation point. On the left sidebar Probes menu, it is possible to list all the known observationPoint Ids, set a custom name clicking on the wheel icon, and visualize traffic statistics clicking on the chart icon.

While flows are kept split, hosts, ASs, networks etc traffic is merged at the interface level regardless of the observation Point Id that has originated it. This choice allows statistics not to be duplicated when hosts from different observationPoints talk together. For instance if a host from the Rome site is talking to www.google.com, and a host from Berlin is also talking to www.google.com, ntopng will keep in memory a single www.google.com host entry accounting the total of the traffic. When flows are stored on nIndex, they both have the observationPoint Id and the probe IP address that exported it: this enables network analysts to drill down a problem (e.g. reported with a ntopng generated alert) up to the traffic origin.

In summary, using observation points it is possible to partition the traffic according to a site, while keeping consistent counters for hosts, ASs, and networks. Note that in case of networks with overlapping addresses (e.g. two hosts with address 192.168.1.1 deployed at two different locations), in order to avoid merging traffic for hosts with the same IP but physically different, it is instead advised to collect their traffic using two different ntopng virtual collection interfaces.

Enjoy!