How deep packet inspection and Simple Network Management Protocol is being replaced by network telemetry
"Do not go gentle into that good night, Old age should burn and rave at close of day; Rage, rage against the dying of the light." -- a rapturous ode to the unassailable tenacity of the human spirit by the Welsh poet Dylan Thomas written in 1914 couldn't better describe what I've seen over the past two decades in the quiet obsolescence of legacy security solutions like antivirus and network intrusion detection.
These solutions have found their final resting places at the beginnings of the 20th century to be replaced by a newer, smarter way of doing things, whether it's mobile push notifications replacing passwords, network threat analysis replacing network IDS, machine learning replacing signatures, encrypted protocols replacing unencrypted protocols, to now, telemetry data replacing deep packet inspection (DPI) and simple network management protocol (SNMP). While there still remains situations in which DPI and SNMP version 1, 2, or 3 are perfectly suited -- the fact remains that the old is quickly being replaced by the new.
R.I.P. DPI
Deep packet inspection is the analysis of packet data traversing a network from the outermost header of the datagram (Ethernet Header) to its trailer in the payload containing application data. DPI is being quickly rendered obsolete by the growing amount of north-south and east-west encrypted traffic on networks, which now accounts for over 73% of the traffic on the internet.
DPI became a thing in the initial days of unified threat management (UTM) and network IDS that combined the functionality of an intrusion detection system (IDS) and an intrusion prevention system (IPS) with a traditional stateful firewall. The combination made it possible to detect certain attacks that neither the IDS/IPS nor the stateful firewall could see on their own. Stateful firewalls, while able to see the beginning and end of a packet flow, could not catch events on their own as they would be out of bounds for a particular application. DPI-enabled devices had the ability to look at Layer 2 and beyond Layer 3 of the OSI model. In some cases, DPI could even be invoked to look through Layers 2-7 of the OSI model. This included headers and data protocol structures as well as the payload of the message.
Having said that, the very nature of DPI inspecting beyond the shallow headers of a packet all the way to its payload is rendered impotent when encryption is used in command and control (c2), if the attack is carried inside a VPN tunnel, or if a web application attack is performed over SSL. The fact of the matter is, as unencrypted protocols like HTTP, telnet, and FTP are replaced by HTTPS, SSH, and SFTP, DPI will eventually become no more than a fond memory for my generation and generations before me who were around long enough to remember it.
One man's telemetry is not the same as the other's
"One man's trash is another man's treasure," the etymology of which has been lost to time but best describes a lack of standardization on what exactly network telemetry is. Juniper defines it through the lens of its own proprietary "Junos Telemetry Interface (JTI), which uses Google's Protobuf message while Cisco recently discussing model-based telemetry powered by the YANG modeling language considers telemetry data to be a generic catch-all for NetFlow, IPFIX, and other types of flow data from existing infrastructure devices, such as routers, switches, firewalls, proxies, and endpoints.
Goodbye SNMP, it was you, not me
SNMP stands for Simple Network Management Protocol and is commonly used for monitoring availability, throughput, utilization, delay, and errors collected on devices, interfaces, links, memory, and CPU. Ask any penetration tester though, and she'll tell you that the most common finding in every penetration test is the default use of SNMP community strings (effectively, the SNMP password for read and write access) in every penetration test she has performed. SNMP has historically never been the most secure implementation, oft-times the most commonly misconfigured service where the vendor's default community strings were never changed after placed into production.
Vulnerabilities and misconfigurations aside, SNMP has serious scalability issues -- SNMP really doesn't scale in today's high density platforms. While SNMP can perform a push when sending a trap, the most common configuration and average SNMP client performs a pull every 5-30 minutes leaving any sort of real-time monitoring much to be desired. While SNMP version 3 implements much needed security from version 1, an entire article could be written on the fails of SNMP so I digress.
Enter Network Telemetry
Network telemetry is increasingly becoming the "soupe du jour" for many network and security engineers who have grown tired of SNMP MIB pulls and Cisco's command line interface (CLI) commands for monitoring. Unlike SNMP, network telemetry is a continuous, real-time streaming feed in what are called subscriptions. Further still, unlike MIBs with SNMP, model-driven telemetry (MDT) enables networking devices to precisely describe their capabilities to the outside world -- describing what kind of data they expose (e.g. interfaces statistics, configuration options, etc); the data type (string, integer, etc); any restrictions on the data (optional or required, etc); and even what kind of operations are supported on the data. The data model is like a contract, an agreement to obey instructions that conform to the model and return data according to the rules of the model to the requester.
With CLI, admins memorized show commands. With SNMP, admins requested a MIB. With MDT, admins specify the YANG model that contains the data they want. Practically speaking, that means retrieving the supported models from the router in real-time using NETCONF or fetching them from github and exploring them offline using tools like pyang.
I'm not only the president, but I'm also a client
To say that I'm a big believer in network telemetry as a more effective ground truth in network threat detection using NTA solutions or security analytics platforms, such as SAS Cybersecurity over pattern matching payloads would be a gross understatement. Afterall, I have't exactly been shy in proselytizing the death of SIEM.
I've seen the future and it's painted in Hadoop data lakes of system and network telemetry, fed by ML-powered NTA solutions, whose events are made sense of by security analytics platforms, and whose autonomous response is fanged through SOAR.
The biggest Advanced Persistent Threat (APT) attack I ever was engaged in incident response and forensics was the compromise of a large biotech company. The indicators of that compromise were the Netflows that all of a sudden spiked to near full link utilization as a result of data hoarding by the adversaries to staging servers. No IDS signatures were used against payloads for that one.
Cisco's Stealthwatch, formerly Lancope, ingests network telemetry data from existing network equipment capable of generating netflow data natively, processing that metadata through its machine learning models combined with Cisco's Identity Services Exchange (ISE) to answer the who, what, where, when, and how users and devices are using the network that it calls a "network as a sensor," converting every device on the network capable of generating Netflow, IPFIX, and other types of flow data as one massive sensor from router, switches, firewalls, and proxies.
Network behavior analytics through the monitoring of telemetry data is also paramount to monitoring encrypted north-south and east-west traffic as it doesn't require DPI -- relying solely on the metadata in the headers of the packet that Cisco fondly refers to as Encrypted Traffic Analytics.
Summary
In Summary, the first two decades of the 21st century will soon come to a close as we enter 2020 and while the greatest cybersecurity advancements were made over the last twenty years, I can't think of a more timely allegory that delivers the message "nothing lasts forever" than what's happening in Paris tonight. Right now, the world is watching the centuries-old Notre Dame cathedral, which survived the Nazis and two world wars continue to burn. But while the old technologies of yesterday give way to mankind's continued pursuit of technological advancement, the old relics where our greatest achievements at the time once stood, will eventually be reimagined, rebuilt, and made better.
Comment,
So what's your opinion? Is DPI dead, especially with 73% of the Internet traffic now encrypted? Has SNMP been replaced by model-driven telemetry (MDT)? Have you moved to ingestion of telemetry over antiquated protocols, such as SNMP or away from pattern-recognition systems such as network IDS in lieu of a NTA powered by network telemetry? Leave your comments in the section below!
Like and Reshare!
As usual, if you liked this article, please support me by clicking LIKE and share it with your own feed! This is the best possible way that you can support me and my continued research. If anyone has anything to add or comment on in this article, please feel free to share it with everyone below in the comments section! Learn more about me at my homepage at www.alissaknight.com, LinkedIn, watch my VLOGs on my YouTube channel, listen to my weekly podcast episodes, or follow me on Twitter @alissaknight.