Getting Data Into PNDA

It’s easy to get data into PNDA by design; ingested data should be encapsulated in an Avro schema and published on a pre-defined Kafka topic or set of topics.


Note that this is a big data infrastructure-level data schema, not an application-level data schema; i.e. this is a wrapper which goes around the raw data, hence there is no normalisation fo the data required (more on that in a future post).

There are a number of data sources that we’ve already integrated with PNDA:

Data Type Data Aggregator Data Aggregator Reference PNDA Producer Plugin Reference
BGP (inc. BGP LS) OpenBMP!
BGP PMACCT (BGP listener)
Bulk Ingest PNDA Bulk Ingest Tool
Cisco XR streaming telemetry Pipeline
CollectD (CollectD supports multiple plugins as listed here Logstash
IoT sensor via HTTP Node-RED
Logstash (Logstash supports multiple plugins as listed here Logstash
NETCONF Notifications ODL
Netflow / IPFIX Logstash
Netflow / IPFIX / sFlow pmacct
Openstack Work in progress
sFlow Logstash
SNMP Metrics and Traps ODL
SNMP Traps Logstash
Syslog Logstash
Syslog (RFC3164 or RFC5424 – needed for newer IOS/IOS XR/ NX OS etc.) Logstash

If these don’t address your needs, it’s relatively simple to write a custom data producer for PNDA; see the section in the PNDA guide.  If you’ve written one you want to upstream, we’d welcome your contribution!



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s