9 comments on “Parsing Bro 2.2 Logs with Logstash

  1. Nice article. Now that Security Onion utilizes Bro 2.2 are you planning an article on getting logstash and kibana in parallel with ELSA?

  2. Hopefully this will format correctly in the comments -- an awk script for generating the match lines above from bro logs:

    #!/usr/bin/awk -f

    BEGIN { FS = "\t" }

    $1 == "#fields" {
    sub(/^#fields\t/, "", $0)
    gsub(/\t/, ">(.*?))\\t(? [ \"message\", \"(?(.*))\" ]"
    }

    Edit by Jason: Chris followed up with an additional script that removes the BEGIN statement:

    #!/usr/bin/awk -f

    $1 == "#fields" {
    sub(/^#fields\t/, "", $0)
    gsub(/\t/, ">(.*?))\\t(? [ \"message\", \"(?(.*))\" ]"
    }

    • Excellent parser, Chris! My original parser was a mess of awk and sed, but seemed to work well for me. This definitely looks much more civilized.

  3. For anyone using the config file above for bro/logstash in Security Onion, make sure you replace the log path in the config file. To change with vi, just do:

    :%s/\/opt\/bro2/\/nsm\/bro/g

    @Chris: I'm still working my way through your book, but everything that I've read so far is great.
    We've already ordered a couple of copies for our analysts to read, but they're backordered! I'm glad I grabbed my copy when I did! Hope you'll check out what we're doing with logs:

    https://quadrantsec.com/product_technology/
    https://github.com/beave/sagan

    I'm hoping that it will find its way into SO one of these days!

    I already spend so much time exploring ELSA... now you've given me another console to get lost in...

    Thanks. ;-)

  4. Is there a way to get logstash to add a hostname to network logs like netflow or IDS? It seems like that would be an easy way to combine host and network evidence, and not have to worry about DHCP changes.

Leave a Reply