2 comments on “Calculating IDS Signature Precision

  1. Great post Chris! How do you suggest dealing revisions to rules? If a rule (IDS/SIEM/whatever) is tweaked to be more accurate, using its prior version as a precision metric may skew the accuracy of the current version. Would you suggest "starting over" with this metric with each revision to the rule?

    • Thanks, Dustin! Great question. If the change does affect the signatures precision, then keeping with good statistics would dictate that you reset the precision of the rule to zero. This is a good thing though, because now you can collect new data points for the modified rule and compare the new precision statistic with the old one. This is really handy because then you can actually point to numbers and say revision 2 of the rule was 12% more precise than revision 1. Great for tracking if your improvements actually worked, and also useful if you have to justify to management why you spent so long tweaking a rule.

Leave a Reply