“How do I find bad stuff on the network?”
The path to knowledge for the practice of NSM typically always begins with that question. It’s because of that question that we refer to NSM as a practice, and someone who is a paid professional in this field as a practitioner of NSM.
Scientists are often referred to as practitioners because of the evolving state of the science. As recently as the mid 1900s, medical science believed that milk was a valid treatment for ulcers. As time progressed, it was found that ulcers were caused by bacteria called helicobacter pylori and that dairy products could actually further aggravate an ulcer. Perceived facts change because although we would like to believe most sciences are exact, they simply aren’t. All scientific knowledge is based upon educated guesses utilizing the best available data at the time. As more data becomes available over time, answers to old questions change, and this redefines things that were once considered facts. This is true for Doctors as practitioners of medical science, and it is true for us as practitioners of NSM.
Unfortunately, when I started practicing NSM there weren’t a lot of reference materials available on the topic. Quite honestly, there still aren’t. Aside from the occasional blog postings of industry pioneers and a few select books, most individuals seeking to learn more about this field are left to their own devices. I feel that it is pertinent to clear up one very important misconception to eliminate potential confusion regarding my previous statement. There are menageries of books available on the topics TCP/IP, packet analysis, and various intrusion detection systems. Although the concepts presented in those texts are important facets of NSM, they don’t constitute the practice of NSM as a whole. That would be like saying a book about wrenches teaches you how to diagnose a car that won’t start.
With that in mind, my co-authors and I are incredibly excited to announce our newest project, a book entitled "Applied Network Security Monitoring". This book is dedicated to the practice of NSM. This means that rather than simply providing an overview of the tools or individuals components of NSM, we will speak to the process of NSM and how those tools and components support the practice.
This book is intended to be a training manual on how to become an NSM analyst. If you’ve never performed NSM analysis, then this book is designed to provide the baseline skills necessary to begin performing these duties. If you are already a practicing analyst, then my hope is that this book will provide a foundation that will allow you to grow your analytic technique in such a way as to make you much more effective at the job you are already doing. We’ve worked with several good analysts who were able to become great analysts because they were able to enhance their effectiveness with some of the techniques presented here.
The effective practice of NSM requires a certain level of adeptness with a variety of tools. As such, the book will discuss several of these tools as well, including the Snort, Bro, and Suricata IDS tools, the SiLK and Argus netflow analysis tool sets, as well as other tools like Snorby, Security Onion, and more.
This book focuses almost entirely on free and open source tools. This is in an effort to appeal to a larger grouping of individuals who may not have the budget to purchase commercial analytic tools such as NetWitness or Arcsight, and also to demonstrate that effective NSM can be achieved without a large budget. Ultimately, talented individuals are what make an NSM program successful. In addition, these open source tools often provide more transparency in how they interact with data, which is also incredibly beneficial to the analyst when working with data at an intimate level.
Table of Contents
Chapter 1: The Practice of Network Security Monitoring
The first chapter is devoted to defining network security monitoring and its relevance in the modern security landscape. It discusses a lot of the core terminology and assumptions that will be used and referenced throughout the book.
Part 1: Collection
Chapter 2: Driving Data Collection
The first chapter in the Collection section of ANSM provides an introduction to data collection and an overview of its importance. This chapter provides a framework for making decisions regarding what data should be collected using a risk-based approach.
Chapter 3: The Sensor Platform
This chapter introduces the most critical piece of hardware in an NSM deployment, the sensor. This includes a brief overview of the various NSM data types, and then discusses important considerations for purchasing and deploying sensors. Following, this chapter covers the placement of NSM sensors on the network, including a primer on creating network visibility maps for analyst use. This chapter also introduces Security Onion, which will be references throughout the book as our lab environment.
Chapter 4: Full Packet Capture Data
This section begins with an overview of the importance of full packet capture data. It will examine use cases that demonstrate its usefulness, and then demonstrate several methods of capturing and storing PCAP data with tool such as Netsniff-NG, Daemonlogger, and OpenFPC.
Chapter 5: Session Data
This chapter discusses the importance of session data, along with a detailed overview of Argus and the SiLK toolset for the collection and analysis of netflow data.
Chapter 6: Protocol Metadata
This chapter look at methods for generating metadata from other data sets, and the usefulness of integrating it into the NSM analytic process. This includes coverage of the packet string (PSTR) data format, as well as other tools used to create protocol metadata.
Chapter 7: Statistical Data
The final data type that will be examined is statistical data. This chapter will discuss use cases for the creation of this data type, and provide some effective methods for its creation and storage. Tools such as rwstats, treemap, and gnuplot will be used.
Part 2: Detection
Chapter 8: Indicators of Compromise
This chapter examines the importance of Indicators of Compromise (IOC), how they can be logically organized, and how they can be effectively managed for incorporation into an NSM program. This also includes a brief overview of the intelligence cycle, and threat intelligence.
Chapter 9: Target Based Detection
The first detection type that will be discussed is target based detection. This will include basic methods for detecting communication with certain hosts within the context of the previously discussed data types.
Chapter 10: Signature Based Detection with Snort
The most traditional form of intrusion detection is signature based. This chapter will provide a primer on this type of detection and discuss the usage of the Snort IDS. This will include the use of Snort, and a detailed discussion on the creation of Snort signatures. Several practical examples and case scenarios will be present in this chapter.
Chapter 11: Signature Based Detection with Suricata
This chapter will provide a primer on signature based detection with Suricata. This will include several practical examples and use cases.
Chapter 12: Anomaly Based Detection with Bro
Anomaly based identification is an area that has gotten quite a bit more attention in recent years. This chapter will cover Bro, one of the more popular anomaly based detection solutions. This will cover a detailed review of the Bro architecture, the Bro language, and several use cases.
Chapter 13: Early Warning AS&W with Canary Honeypots
Previously only used for research purposes, operational honeypots can be used as an effective means for attack sense and warning. This chapter will provide examples of how this can be done, complete with code samples and deployment case scenarios.
Part 3: Analysis
Chapter 14: Packet Analysis
The most critical skill in NSM is packet analysis. This chapter covers the analysis of packet data with Tcpdump and Wireshark. It also covers basic to advanced packet filtering.
Chapter 15: Friendly Intelligence
This chapter focuses on performing research related to friendly devices. This includes a framework for creating an asset model, and a friendly host characteristics database.
Chapter 16: Hostile Intelligence
This chapter focuses on performing research related to hostile devices. This includes strategies for performing open source intelligence (OSINT) research.
Chapter 17: Differential Diagnosis of NSM Events
This is the first chapter of the book that focuses on a diagnostic method of analysis. Using the same differential technique used by physicians, NSM analysts can be much more effective in the analysis process.
Chapter 18: Incident Morbidity and Mortality
Once again borrowing from the medical community, the concept of incident morbidity and mortality can be used to continually refine the analysis process. This chapter explains techniques for accomplishing this.
Chapter 19: Malware Analysis for NSM
This isn’t a malware analysis book by any stretch of the imagination, but this chapter focuses on methods an NSM analyst can use to determine whether or not a file is malicious.
Chris Sanders, Lead Author
Chris Sanders is an information security consultant, author, and researcher originally from Mayfield, Kentucky. That’s thirty miles southwest of a little town called Possum Trot, forty miles southeast of a hole in the wall named Monkey's Eyebrow, and just north of a bend in the road that really is named Podunk.
Chris is a Senior Security Analyst with InGuardians. He has as extensive experience supporting multiple government and military agencies, as well as several Fortune 500 companies. In multiple roles with the US Department of Defense, Chris significantly helped to further to role of the Computer Network Defense Service Provider (CNDSP) model, and helped to create several NSM and intelligence tools currently being used to defend the interests of the nation.
Chris has authored several books and articles, including the international best seller "Practical Packet Analysis" form No Starch Press, currently in its second edition. Chris currently holds several industry certifications, including the CISSP, GCIA, GPEN, GCIH, and GREM.
In 2008, Chris founded the Rural Technology Fund. The RTF is a 501(c)(3) non-profit organization designed to provide scholarship opportunities to students form rural areas pursuing careers in computer technology. The organization also promotes technology advocacy in rural areas through various support programs.
When Chris isn't buried knee-deep in packets, he enjoys watching University of Kentucky Wildcat basketball, amateur drone building, BBQing, and spending time at the beach. Chris currently resides in Charleston, South Carolina.
Liam Randall, Co-Author
Liam Randall is a principal security consultant with Cincinnati, OH based GigaCo. Originally, from Louisville, KY he worked his way through school as a sysadmin while getting his Bachelors in Computer Science at Xavier University. He first got his start in high security writing device drivers and XFS based software for Automated Teller Machines.
Presently he consults on high volume security solutions for the Fortune 500, Research and Education Networks, various branches of the armed service, and other security focused groups. As a contributor to the open source SecurityOnion distribution and the Berkeley based Bro-IDS network security package you can frequently find him speaking about cutting edge blue team tactics on the conference circuit.
A father and a husband, Liam spends his weekends fermenting wine, working in his garden, restoring gadgets, or making cheese. With a love of the outdoors he and his wife enjoy competing in triathlons, long distance swimming and enjoying their community.
Jason Smith, Co-Author
Jason Smith is an intrusion detection analyst by day and junkyard engineer by night. Originally from Bowling Green, Kentucky, Jason started his career mining large data sets and performing finite element analysis as a budding physicist. By dumb luck, his love for data mining led him to information security and network security monitoring where he took up a fascination with data manipulation and automation.
Jason has a long history of assisting state and federal agencies with hardening their defensive perimeters and currently works as an Information Security Analyst with the Commonwealth of Kentucky. As part of his development work, he has created several open source projects, several of which have become "best-practice" tools for the DISA CNDSP program.
Jason regularly spends weekends in the garage building anything from arcade cabinets to open wheel racecars. Other hobbies include home automation, firearms, monopoly, playing guitar and eating. Jason has a profound love of rural America, a passion for driving, and an unrelenting desire to learn. Jason is currently living in Frankfort, Kentucky.
The tentative release date for Applied NSM is during the third quarter of 2013.