index

The first (DNS) record captures the DNS tranaction that occurs when wget looks for the IP address of this website. From the vantage point of SRIFlow, at the boundary router, we observe the DNS Server (130.107.X.Y) performs the DNS lookup to the treas.gov server, which is located in Washington DC. The second and third (FLOW) records capture the initialization (INIT) and completion (FIN) of the TCP connection from the client (130.107.A.B) to the treas.gov webserver (166.123.208.148). See our Data Format page for details on the field names and semantics.

Software Description


System Requirements

    Unix/MacOS - SRIFlow has been compiled and tested on Linux and depends on GNU GCC 4.6.*
    or later, dev-libpcap, and the Maxmind GeoIP database (free version is fine, but the licensed version
    is great too).

    Windows - sorry, we'll release one only if there is strong interest.

    See Installation instructions for installing dev-libpcap and the Maxmind GeoIP libraries onto
    your host. It takes less than 5 minutes.

Features - SRIFlow produces three record types in its output stream:

    * Flow Events: report all phases of TCP and UDP network flows, as they happen : initialization,
       updates, completions, resets, timeouts

    * Stat Events: report network wide statistics at time intervals you specify (all the stats you need
      to spot floods, outages, and relevant anomalies

    * DNS Events: reports DNS transactions, integrates DNS into flow records, provides geolocation
       attributes, reports up to N bytes of bi-directional payload

    * SRIFlow Modes
             - produces text-based CSV and XML report format
             - operates on batch tcpdump files or live mode
             - output may be logged to stdout or pushed or pulled via TCP-sockets: multi-client listen mode
               for serving data to multiple monitors, or netcat mode for pushing data to logging services

SRIFlow vs NetFlow

Installation Instructions

The SRIFlow code depends on libpcap libGeoIP and uses GNU-make and gcc/g++ to build. There is no autoconfig script. If you need special flags to access libpcap and/or libGeoIP, edit the Makefile and add them in the first four lines (usually not necessary).

The default make target builds the SRIFlow binary. The 'all' target builds some other useful tools as well, and attempts to make the SRIFlow binary setuid root, so it can access tap devices.

See the sriflow.8 manual page for details on how to run it.

Other C++ compilers should work if they support the C++11 standard, but may require changes to the FLAGS in the Makefile.

Example Build Instruction on Ubuntu
1. SRIFlow employs Maxmind's GeoIPCity binary database to perform lookups.

OPTION ONE: If you buy GeoIPCity One can purchase a license for GeoIPCity on the Maxmind.com website and install it on to the target SRIFlow host.

note: If you would like to relocate GeoIPCity.dat, then you can use a config file line

GeoLookup = /path/to/GeoIPCity.dat

OPTION TWO: You can download GeoLiteCity for free from Maxmind. The GeoLite databases are free to download, are slightly less accurate, and are updated on the first Tuesday of each month.

To Download the public LibGeoIP database:

1. Download and install LibGeoIP from Maxmind:
% wget < http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz >
% tar -zxvf GeoIP.tar.gz
% cd GeoIP-1.4.8/
% ./configure
% make
% make check
% make install

2. Install libpcap
Ubuntu: % apt-get install libpcap-dev
or Debian: % apt-get build-essential clang libpcap-dev

3. Build SRIFlow
% make
% make install

Other Projects

What's in the Cisco NetFlow Record Format?

Summary of fields provide by NetFlow

NetFlow provides basic source and destination attributes, QoS attributes, and route link attributes. Great for diagnosing or billing data, sucks for malicious flow analysis.

NetFlow enables template extensions to capture additional information and there are version difference in NetFlow fields

NetFlow provide a best-effort audit trail of flows seen crossing the tap point boundary. NetFlow records are produced when the flow is complete.

What's in the SRIFlow Flow Record Format?

Summary of fields provide by
SRIFlow Flow Records

Flow Records provide a live flow event transition trail, not an after-the-flow audit trail

Flow Records
- Tracks both TCP and UDP flows
-   Summary key statistics of each flow
-   Reports all states of the flow:   INIT,
    UPDATE, TIMEOUT, RSTs, FIN
-   Reports DNS names when DNS
    lookup occurs before flow
-   bi-direction IP geolocation attributes
-   (Optional) provide bi-directional data
    payloads, up to N bytes

What's in the SRIFlow STAT Record Format?

Summary of fields provide by
SRIFlow Flow Records

STATS Records are produced at user-specified time intervals. Each record reports:

- network-wide TCP / UDP flow
counts and directions
- network-wide TCP / UDP data
volumes and directions
- network-wide TCP/UDP packet
volumes and directions
- active internal and external IPs
seen during the interval
- all flow pending stats: syns,
incomplete flows, resets

What's in the SRIFlow DNS Transaction Record Format?

Summary of fields provide by
SRIFlow DNS Transaction Records

DNS Transactions Records summarize the outcome of DNS queries observed by SRIFlow. Each DNS Transaction record reports

Query Source and Content
Full Response Components
Query IP Geo Attributes
Time, TTL, Type

An Example Output From SRIFlow

Here is an example network flow, using wget to perform a simple web document retrieval:

% wget https://www.fms.treas.gov/frsummary/frsummary2012.pdf

Here are the SRIFlow records produce from this flow: [our CSV DATA FORMAT]

Download

    - For Unix and MacOS

    - SRIFlow, Version 1.1
       [sriflow.distribution1.1.zip,
       sriflow.distribution1.1.tar.gz ]
       by Chris Dodd and Phil Porras

    - License :
       Apache License, Version 2.0

    - SRIFlow Man Page

    - SRIFlow CSV Field Format

Why SRIFlow?

For years the security and network visualization communities have adopted NetFlow to drive various security analyses and visual displays. However, NetFlow was written to help network operators analyze network architectures, perform billing functions, and to diagnose network failures. NetFlow is an audit trail that is truly ill-suited to drive efficient live security analyses. Yet, network flow analytics can produce extremely useful insights into your network security posture.

Instead, we designed SRIFlow specifically to drive flow-by-flow contextual security analyses and to diagnose overall network statistics that provide you much deeper ability to conduct live security analyses and to drive richer tactical visualizations of your network activity.

Enjoy!

SRI's Internet Security Group

DNS,130.107.X.Y,8.8.8.8,1390781910,0,UDP,A,fms.treas.gov,OK,166.123.208.148,fms.treas.gov,
US,Washinton, 38.8979,-77.0417

FLOW,130.107.A.B,53581,166.123.208.148,443,1390781910,1390781910,0,TCP,INIT,2,0,0,0,2,1,0,
0,0,1,0,0,2,0,1,0,0v1,,,myHost.domain.com,fms.treas.gov,US,Menlo Park,US,Washington,
37.459,-122.178,38.8979,-77.0417

FLOW,130.107.A.B,53581,166.123.208.148,443,1390781910, 1390781912,2,TCP,FIN,139,3,228,
686,142,1,270,1354,365624,271, 2,686,140,365624,270,0,0v1,=(808c01030100)c=…/=(0300800000)
f=00,=(1603010ef0020000)F…, myHost.domain.com, fms.treas.gov, US, Menlo Park,US,
Washington,37.459,-122.178,38.8979,-77.041