Planet Suricata

Suricata 3.2 was recently released:
https://suricata-ids.org/2016/12/01/suricata-3-2-available/

I've packaged it and the following package is now available:
securityonion-suricata - 3.2-1ubuntu1securityonion2

This new package should resolve the following issue:

Issue 1026: Suricata 3.2
https://github.com/Security-Onion-Solutions/security-onion/issues/1026

This package has been tested by Wes Lambert.  Thanks, Wes!

Updating
This package is now available in our stable repo.  Please see the following page for full update instructions:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Upgrade

This update will back up each of your existing suricata.yaml files to suricata.yaml.bak and migrate your HOME_NET and EXTERNAL_NET variables.  You'll then need to do the following:

• re-apply any other local customizations to your suricata.yaml file(s)
• update ruleset and restart Suricata as follows:
  sudo rule-update

Want to show your support for Security Onion?
Security Onion t-shirts are available in our CafePress store!
http://www.cafepress.com/securityonion/11820053

Training
Security Onion Solutions provides onsite, online, and on-demand training.  For more information, please see:
https://securityonionsolutions.com

Support
Need support?  Please see:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Support

Thanks!

Suricata 3.1.3 was recently released:
https://suricata-ids.org/2016/11/01/suricata-3-1-3-released/

I've packaged it and the following package is now available:
securityonion-suricata - 3.1.3-1ubuntu1securityonion2
This new package should resolve the following issue:

Issue 1014: Suricata 3.1.3
https://github.com/Security-Onion-Solutions/security-onion/issues/1014

This package has been tested by Wes Lambert.  Thanks, Wes!

Updating
This package is now available in our stable repo.  Please see the following page for full update instructions:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Upgrade

This update will back up each of your existing suricata.yaml files to suricata.yaml.bak and migrate your HOME_NET and EXTERNAL_NET variables.  You'll then need to do the following:

• re-apply any other local customizations to your suricata.yaml file(s)
• update ruleset and restart Suricata as follows:
  sudo rule-update

Security Onion t-shirts are available in our CafePress store!
http://www.cafepress.com/securityonion/11820053

Training
Security Onion Solutions provides onsite, online, and on-demand training.  For more information, please see:
https://securityonionsolutions.com

Support
Need support?  Please see:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Support

Thanks!

Suricata 3.1.2 was recently released:
https://suricata-ids.org/2016/09/07/suricata-3-1-2-released/

I've packaged it and the following package is now available:
securityonion-suricata - 3.1.2-1ubuntu1securityonion1

This new package should resolve the following issue:

Issue 994: Suricata 3.1.2
https://github.com/Security-Onion-Solutions/security-onion/issues/994

This package has been tested by Wes Lambert.  Thanks, Wes!

Updating
This package is now available in our stable repo.  Please see the following page for full update instructions:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Upgrade

This update will back up each of your existing suricata.yaml files to suricata.yaml.bak and migrate your HOME_NET and EXTERNAL_NET variables.  You'll then need to do the following:

• re-apply any other local customizations to your suricata.yaml file(s)
• update ruleset and restart Suricata as follows:
  sudo rule-update

Want to show your support for Security Onion?
Security Onion t-shirts are available in our CafePress store!
http://www.cafepress.com/securityonion/11820053

Training
Security Onion Solutions provides onsite, online, and on-demand training.  For more information, please see:
https://securityonionsolutions.com

Support
Need support?  Please see:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Support

Thanks!

I've updated the NSM scripts to wipe Suricata's stats.log when starting/restarting Suricata.  The new package is:
securityonion-nsmnow-admin-scripts - 20120724-0ubuntu0securityonion140

This new package has been tested by Wes Lambert (thanks!) and should resolve the following issue:

Issue 968: NSM: wipe stats.log when restarting Suricata
https://github.com/Security-Onion-Solutions/security-onion/issues/968

Updating
This package is now available in our stable repo.  Please see the following page for full update instructions:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Upgrade

Want to show your support for Security Onion?
Security Onion t-shirts are available in our CafePress store!
http://www.cafepress.com/securityonion/11820053

Conference
Security Onion Conference will be on Friday September 9 and registration closes in just a few weeks!
https://securityonion.net/conference

Training
Need training?  Please see:
https://securityonionsolutions.com

Support
Need support?  Please see:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Support

Thanks!

Suricata IDS/IPS/NSM is a highly scalable, modular and flexible platform. There are numerous configuration options available which empower a lot.

This blog post aims to give you an overview of the ones that have an impact on the memory consumption for Suricata and how does the suricata.yaml config settings affect the memory usage of/for Suricata and the system it resides on.

One of the always relevant questions with regards to performance tuning and production deployment is  - What is the total memory consumption of Suricata? Or to be correct - what is the total memory that Suricata will consume/use and how can that be calculated and configured more precisely?

The details of the answer are very relevant since it will most certainly affect the deployment set up. The risk of not correctly setting up the configuration can lead to a RAM starvation which in turn would force the use of swap which would most likely make your particular set up not optimal (to be frank - useless).

In this blog post we will try to walk through the relevant settings in  a suricata.yaml configuration example and come up with an equation for the total memory consumption.

For this particular set up I use:

• af-packet running mode with 16 threads configuration
• runmode: workers
• latest dev edition(git - 2.1dev (rev dcbbda5)) of Suricata at the time of this writing.
• IDS mode is used in this example
• Debian Jessie/Ubuntu LTS (the OS should not matter)

Lets dive into it...

MTU size does matter

How so?

If you look into the setting for max-pending-packets in suricata.yaml

max pending packets: 1024

that will lead to the following output into suricata.log:

.....
 (tmqh-packetpool.c:398) <Info> (PacketPoolInit) -- preallocated 1024 packets. Total memory 3321856
.....

which is  - 3244 bytes per packet per thread(pool).


The size of each packet takes in the memory is the sizeof(struct
Packet_) + DEFAULT_PACKET_SIZE. So ~1.7K plus ~1.5K or about 3.2K.
Total memory used would be:

<number_of_threads>*<(sizeof(struct Packet_) + DEFAULT_PACKET_SIZE)>*<max-pending-packets> = 16 * 65534 * 3.2K = 3.20GB.

defrag:
  memcap: 512mb
  hash-size: 65536
  trackers: 65535 # number of defragmented flows to follow
  max-frags: 65535 # number of fragments to keep (higher than trackers)
  prealloc: yes
  timeout: 60

host:
  hash-size: 4096
  prealloc: 10000
  memcap: 16777216

......

(stream-tcp.c:393) <Info> (StreamTcpInitConfig) -- stream "memcap": 15032385536

......

(stream-tcp.c:475) <Info> (StreamTcpInitConfig) -- stream.reassembly "memcap": 21474836480

...

So if we sum up all the config options that have effect on the total memory consumption by Suricata with mind of the set up referred to here  (afpacket with 16 threads) we have (in bytes or mb/gb depending how you have your yaml memcap settings):

<number_of_total_detection_threads>*<((1728)+(default_packet_size))>*<max-pending-packets>
+
<defrag.memcap>
+
< host.memcap>
+
< ippair.memcap>
+
 < flow.memcap>
+
 <number_of_threads>*<216>* <prealloc-sessions>
+
 [per af-packet interface enabled]<af-packet_number_of_threads> * <ringsize> * <((1728)+(default_packet_size))>
+
<stream.memcap>+<stream.reassembly.memcap>
+
 <app-layer.protocols.dns.global-memcap>
+
<app-layer.protocols.http.memcap>
=
Total memory that is configured and should be available to be used by Suricata


Thank you

NOTE:
As of developments in Suricata 3.0 - sizeof(struct Packet_) is 936 not 1728 bytes

Suricata 3.1.1 was recently released:
https://suricata-ids.org/2016/07/13/suricata-3-1-1-released/

I've packaged it and the following package is now available:
securityonion-suricata - 3.1.1-1ubuntu1securityonion1

This new package should resolve the following issue:

Issue 945: Suricata 3.1.1
https://github.com/Security-Onion-Solutions/security-onion/issues/945

This package has been tested by Wes Lambert.  Thanks, Wes!

Updating
This package is now available in our stable repo.  Please see the following page for full update instructions:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Upgrade

This update will back up each of your existing suricata.yaml files to suricata.yaml.bak and migrate your HOME_NET and EXTERNAL_NET variables.  You'll then need to do the following:

• re-apply any other local customizations to your suricata.yaml file(s)
• update ruleset and restart Suricata as follows:
  sudo rule-update

Want to show your support for Security Onion?
Security Onion t-shirts are available in our CafePress store!
http://www.cafepress.com/securityonion/11820053

Conference
Security Onion Conference will be on Friday September 9 and registration is open!
https://securityonion.net/conference

Training
Need training?  Please see:
https://securityonionsolutions.com

Support
Need support?  Please see:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Support

Thanks!

Suricata 3.0.1 was recently released:
https://suricata-ids.org/2016/04/04/suricata-3-0-1-released/

I've packaged Suricata 3.0.1 and the new package version is:
securityonion-suricata - 3.0.1-1ubuntu1securityonion1

This resolves the following issue:

Issue 896: Suricata 3.0.1
https://github.com/Security-Onion-Solutions/security-onion/issues/896

Wes Lambert and wingmanjt tested this package.  Thanks, guys!

Updating
This new package is now available in our stable repo.  Please see the following page for full update instructions:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Upgrade

This update will back up each of your existing suricata.yaml files to suricata.yaml.bak and migrate your HOME_NET and EXTERNAL_NET variables.  You'll then need to do the following:

• re-apply any other local customizations to your snort.conf file(s)
• update ruleset and restart Suricata as follows:
  sudo rule-update

Want to show your support for Security Onion?
Several folks have asked about Security Onion t-shirts and they are available in our CafePress store!
http://www.cafepress.com/securityonion/11820053

Training
Our next round of online classes is coming up in a few weeks:
http://blog.securityonion.net/2016/03/next-round-of-security-onion-online.html

Conference
Security Onion Conference will be on Friday September 9 and CFP is open!
http://blog.securityonion.net/2016/03/security-onion-conference-2016-cfp.html

Support
Need support?  Please see:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Support

Thanks!

We are pleased to announce Suricata 3.0.1. This release fixes many important issues in 3.0 and upgrading is highly recommended.

Highlights

– fixes for multiple stability issues
– many memory leak fixes
– Hyperscan MPM support (experimental)

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-3.0.1.tar.gz

All Changes

For a complete list of closed tickets, please see:

– 3.0.1RC1 tickets
– 3.0.1 tickets

When upgrading from 3.0, please see these notes.

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

FireEye, Intel, ANSSI, Emerging Threats / Proofpoint, Stamus Networks,
NorCert, Ntop, Lastline, AFL project, CoverityScan

Justin Viiret, Tom Decanio, Mats Klepsland, Alexander Gozman,
Aleksey Katargin Maurizio Abba, Alessandro Guido, David Diallo,
Giuseppe Longo, Chris Wakelin, Jon Zeolla, Andreas Moe,
Nicolas Thill, Travis Green, bladeswords Alfredo Cardigliano,
Rob Mosher, Andrew Brown, Andre ten Bohmer

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

November 9-11 we’ll be in Washington, DC, for our 2nd Suricata User Conference: http://suricon.net

If you need help installing, updating, validating and tuning Suricata we have a training program. Please see http://suricata-ids.org/training/

For support options also see http://suricata-ids.org/support/

We’re hoping for your feedback on a new release: Suricata 3.0.1RC1. We’ve fixed many issues in 3.0, including important stability issues and memory leaks. A final is expected within a week or so.

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-3.0.1RC1.tar.gz

New Features

– Feature #1535: Expose the certificate itself in TLS-lua
– Feature #1696: improve logged flow_id
– Feature #1700: enable “relro” and “now” in compile options for 3.0
– Feature #1734: gre: support transparent ethernet bridge decoding
– Feature #1740: Create counters for decode-events errors
– updated bundled libhtp to 0.5.19

Fixes

Many issues were fixed, including stability issues and many (potential) memory leaks.
Full list: https://redmine.openinfosecfoundation.org/versions/81

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:
FireEye, ANSSI, Emerging Threats / Proofpoint, Stamus Networks,
NorCert, Ntop, Lastline, AFL project, CoverityScan

Tom Decanio, Mats Klepsland, Alexander Gozman, Aleksey Katargin
Maurizio Abba, Alessandro Guido, David Diallo, Giuseppe Longo
Jon Zeolla, Andreas Moe, Nicolas Thill, Travis Green, bladeswords
Alfredo Cardigliano, Rob Mosher, Andre ten Bohmer

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

November 9-11 we’ll be in Washington, DC, for our 2nd Suricata User Conference: http://suricon.net

If you need help installing, updating, validating and tuning Suricata we have a training program. Please see http://suricata-ids.org/training/

For support options also see http://suricata-ids.org/support/

Yesterday I wrote about fuzzing Suricata with AFL. Today I’m going to show another way. Since early in the project, we’ve shipped a perl based fuzzer called ‘wirefuzz’. The tool is very simple. It takes a list of pcaps, changes random bits in them using Wiresharks editcap and runs them through Suricata. Early in the project Will Metcalf, who wrote the tool, found a lot of issues with it.

Since it’s random based fuzzing, the fuzzing is quite shallow. It is still a great way of stressing the decoder layers of Suricata though, as we need to be able to process all junk input correctly.

Lately we had an issue that I thought should have been found using fuzzing: #1653, and indeed, when I started fuzzing the code I found the issue within an hour. Pretty embarrassing.

Another reason to revisit is Address Sanitizer. It’s great because it’s so unforgiving. If it finds something it blows up. This is great for fuzzing. It’s recommended to use AFL with Asan as well. Wirefuzz does support a valgrind mode, but that is very slow. With Asan things are quite fast again, while doing much more thorough checking.

So I decided to spend some time on improving this tool so that I can add it to my CI set up.

Here is how to use it.

git clone https://github.com/inliniac/suricata -b dev-fuzz-v3.1
cd suricata
git clone https://github.com/OISF/libhtp -b 0.5.x
bash autogen.sh
export CFLAGS="-fsanitize=address"
./configure --disable-shared --sysconfdir=/etc
make
mkdir fuzzer

# finally run the fuzzer
qa/wirefuzz.pl -r=/home/victor/pcaps/*/* -c=suricata.yaml -e=0.02 \
    -p=src/suricata -l=fuzzer/ -S=rules/http-events.rules -N=1

What this command does is:

• run from the source dir, output into fuzzer/
• modify 2% of each pcap randomly while making sure the pcap itself stays valid (-e=0.02)
• use the rules file rules/http-events.rules exclusively (-S)
• use all the pcaps from /home/victor/pcaps/*/*
• return success if a single pass over the pcaps was done (-N=1)

One thing to keep in mind is that the script creates a copy of the pcap when randomizing it. This means that very large files may cause problems depending on your disk space.

I would encourage everyone to fuzz Suricata using your private pcap collections. Then report issues to me… pretty please?

*UPDATE 2/15*: the updated wirefuzz.pl is now part of the master branch.

Suricata 3.0 STABLE was recently released:
http://suricata-ids.org/2016/01/27/suricata-3-0-available/

I've packaged Suricata 3.0 STABLE for Security Onion 14.04 and the new package is as follows:
securityonion-suricata - 3.0stable-1ubuntu1securityonion1

This resolves the following issue:

Issue 847: Suricata 3.0
https://github.com/Security-Onion-Solutions/security-onion/issues/847

Wes Lambert tested this package.  Thanks, Wes!

Updating
These new packages are now available in our stable repo.  Please see the following page for full update instructions:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Upgrade

This update will back up each of your existing suricata.yaml files to suricata.yaml.bak and migrate your HOME_NET and EXTERNAL_NET variables.  You'll then need to do the following:

• re-apply any other local customizations to your suricata.yaml file(s)
• update ruleset and restart Suricata as follows:
  sudo rule-update

Want to show your support for Security Onion?
Several folks have asked about Security Onion t-shirts and they are available in our CafePress store!
http://www.cafepress.com/securityonion/11820053

Training
The next round of online training sessions is coming up soon:
http://blog.securityonion.net/2016/02/next-round-of-security-onion-online.html

Commercial Support
Need commercial support?  Please see:
http://securityonionsolutions.com

Feedback
If you have any questions or problems, please use our security-onion mailing list:
https://github.com/Security-Onion-Solutions/security-onion/wiki/MailingLists

Thanks!

AFL is a very powerful fuzzer, that tries to be smarter than random input generating fuzzers. It’s cool, but needs a bit more baby sitting. I’ve added some support to Suricata to assist AFL.

Here’s how to get started on fuzzing pcaps.

mkdir ~/tmp/fuzz
git clone https://github.com/inliniac/suricata -b dev-afl-v5
cd suricata
git clone https://github.com/OISF/libhtp -b 0.5.x
bash autogen.sh
export CFLAGS="-fsanitize=address"
export AFLDIR=/opt/afl-1.96b/bin/
export CC="${AFLDIR}/afl-gcc"
export CXX="${AFLDIR}/afl-g++"
./configure --disable-shared --sysconfdir=/etc --enable-afl

The configure output should show:
Compiler: /opt/afl-1.96b/bin//afl-gcc (exec name) / gcc (real)

make

# create tmp output dir for suricata
mkdir tmp/

# test the command to be fuzzed
src/suricata --runmode=single -k none -c suricata.yaml -l tmp/ \
    -S /dev/null \
    -r /opt/afl-1.96b/share/afl/testcases/others/pcap/small_capture.pcap

# start the fuzzer
export AFL_SKIP_CPUFREQ=1
/opt/afl-1.96b/bin/afl-fuzz -t 100000 -m none \
    -i /opt/afl-1.96b/share/afl/testcases/others/pcap/ -o aflout -- \
    src/suricata --runmode=single -k none -c suricata.yaml -l tmp/ \
    -S /dev/null -r @@

AFL should start running:

Couple of things to keep in mind:

• the above list assumes you have a /etc/suricata/ set up already, including a reference.config and classification.config
• don’t skip the test step or you risk that AFL will just fuzz some basic error reporting by Suricata
• the used ‘dev-afl-v5’ branch makes fuzzing faster and more reliable by disabling random, threading and a few other things
• src/suricata –build-info should show the compiler is afl
• keep your test cases small, even then runtime is going to be very long. AFL takes the input and modifies it to find as many unique code paths as possible

 

Fuzzing rules and YAMLs

For fuzzing rules and YAMLs the compilation steps are the same.

To fuzz rules, create a directory & test input:

mkdir testrules
echo 'alert http any any -> any any (content:"abc"; sid:1; rev:1;)' \
    > testrules/rules.txt

# test command
src/suricata -c suricata.yaml -l tmp/ --afl-parse-rules -T \
    -S testrules/rules.txt

# run AFL
export AFL_SKIP_CPUFREQ=1
/opt/afl-1.96b/bin/afl-fuzz -t 100000 -m none \
    -i testrules/ -o aflout -- \
    src/suricata -c suricata.yaml -l tmp/ --afl-parse-rules \
    -T -S @@

Finally, YAMLs:

mkdir testyamls/
cp suricata.yaml testyamls/

# test command
src/suricata -l tmp/ --afl-parse-rules -T -S testrules/rules.txt \
    -c testyamls/suricata.yaml

# run AFL
export AFL_SKIP_CPUFREQ=1
/opt/afl-1.96b/bin/afl-fuzz -t 100000 -m none \
    -i testyamls/ -o aflout -- \
    src/suricata -l tmp/ --afl-parse-rules \
    -T -S testrules/rules.txt -c @@

Note that the default YAML is HUGE for this purpose. It may be more efficient to use a sub set of it.

I plan to create some wrapper scripts to make things easier in the near future. Meanwhile, if you have crashes to report, please send them my way!

Today, almost 2 years after the release of Suricata 2.0, we released 3.0! This new version of Suricata improves performance, scalability, accuracy and general robustness. Next to this, it brings a lot of new features.

New features are too numerous to mention here, but I’d like to highlight a few:

• netmap support: finally a high speed capture method for our FreeBSD friends, IDS and IPS
• multi-tenancy: single instance, multiple detection configs
• JSON stats: making it much easier to graph the stats in ELK, etc
• Much improved Lua support: many more fields/protocols available, output scripts

Check the full list here in the announcement: http://suricata-ids.org/2016/01/27/suricata-3-0-available/

New release model

As explained here, this is the first release of the new release model where we’ll be trying for 3 ‘major’ releases a year. We originally hoped for a month of release candidate cycles, but due to some issues found and the holidays + travel on my end it turned into 2 months.

My goal is to optimize our testing and planning to reduce this further, as this release cycle process is effectively an implicit ‘freeze’. Take a look at the number of open pull requests to see what I mean. For the next cycle I’ll also make the freeze explicit, and announce it.

Looking forward

While doing a release is great, my mind is already busy with the next steps. We have a bunch of things coming that are exciting to me.

Performance: my detection engine rewrite work has been tested by many already, and reports are quite positive. I’ve heard reports up to 25% increase, which is a great bonus considering the work was started to clean up this messy code.

ICS/SCADA: Jason Ish is finalizing a DNP3 parser that is very full featured, with detection, logging and lua support. Other protocols are also being developed.

Documentation: we’re in the process of moving our user docs from the wiki to sphinx. This means we’ll have versioned docs, nice pdf exports, etc. It’s already 180 pages!

Plus lots of other things. Keep an eye out on our mailing lists, bug tracker or IRC channel.

We’re proud to announce Suricata 3.0. This is a major new release improving Suricata on many fronts.

Download

http://www.openinfosecfoundation.org/download/suricata-3.0.tar.gz

Features and Improvements

• improved detection options, including multi-tenancy and xbits
• performance and scalability much improved
• much improved accuracy and robustness
• Lua scripting capabilities expanded significantly
• many output improvements, including much more JSON
• NETMAP capture method support, especially interesting to FreeBSD users
• SMTP inspection and file extraction

For a full list of features added, please see:
https://redmine.openinfosecfoundation.org/versions/80

Upgrading

Upgrades from 2.0 to 3.0 should be mostly seamless. Here are some notes:

https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Upgrading_Suricata_20_to_Suricata_30

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

FireEye, ProtectWise, ANSSI, Emerging Threats /
Proofpoint, Stamus Networks, Ntop, AFL project, CoverityScan

Aaron Campbell, Aleksey Katargin, Alessandro Guido,
Alexander Gozman, Alexandre Macabies, Alfredo Cardigliano,
Andreas Moe, Anoop Saldanha, Antti Tönkyrä, Bill Meeks,
Darien Huss, David Abarbanel, David Cannings, David Diallo,
David Maciejak, Duarte Silva, Eduardo Arada, Giuseppe Longo,
Greg Siemon, Hayder Sinan, Helmut Schaa, Jason Ish,
Jeff Barber, Ken Steele, lessyv, Mark Webb-Johnson,
Mats Klepsland, Matt Carothers, Michael Rash, Nick Jones,
Pierre Chifflier, Ray Ruvinskiy, Samiux A, Schnaffon,
Stephen Donnelly, sxhlinux, Tom DeCanio, Torgeir Natvig,
Travis Green, Zachary Rasmor

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

November 9-11 we’ll be in Washington, DC, for our 2nd Suricata User Conference: http://oisfevents.net

If you need help installing, updating, validating and tuning Suricata we have a training program. Please see http://suricata-ids.org/training/

For support options also see http://suricata-ids.org/support/

We’re happy to announce Suricata 3.0RC3. RC3 fixes a few issues in RC2 that require some more testing. The plan is to release the stable quickly after the holidays, so please help us test this release!

Fixes:

• Bug #1632: Fail to download large file with browser
• Bug #1634: Fix non thread safeness of Prelude analyzer
• Bug #1640: drop log crashes
• Bug #1645: Race condition in unix manager
• Bug #1647: FlowGetKey flow-hash.c:240 segmentation fault (master)
• Bug #1650: DER parsing issue (master)

Get the release here:

http://www.openinfosecfoundation.org/download/suricata-3.0RC3.tar.gz

Known issues & missing features

In a development release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

The OISF development team is pleased to announce Suricata 2.0.11. This release fixes a number of important issues in the 2.0 series.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0.11.tar.gz

Changes

• Bug #1572: 2.0.8 FlowGetKey flow-hash.c:240 segmentation fault (icmp destination unreachable)
• Bug #1637: drop log crashes
• Bug #1639: 2.0.x: Fix non thread safeness of Prelude analyzer
• Bug #1649: DER parsing issue
• Bug #1651: HTTP body tracking using excessive memory
• Bug #1652: SMTP parsing issue (2.0.x)
• Bug #1653: DNS over TCP parsing issue (2.0.x)
• Bug #1654: TCP reassembly bug (2.0.x)

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

• Mark Webb-Johnson
• Nick Jones
• Hayder Sinan
• Samiux A

Known issues & missing features

If you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

Training & Support

Need help installing, updating, validating and tuning Suricata? We have trainings coming up. Paris in July, Barcelona in November: see http://suricata-ids.org/training/

For support options also see http://suricata-ids.org/support/

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

We’re happy to announce Suricata 3.0RC2. RC2 fixes a few issues in RC1 that require some more testing. The plan still is to release the stable within a few weeks, so please help us test this release!

Fixes:

• Bug #1551: –enable-profiling-locks broken
• Bug #1602: eve-log prefix field feature broken
• Bug #1614: app_proto key missing from EVE file events
• Bug #1615: disable modbus by default
• Bug #1616: TCP reassembly bug
• Bug #1617: DNS over TCP parsing issue
• Bug #1618: SMTP parsing issue
• Feature #1635: unified2 output: disable by default

Get the release here:

http://www.openinfosecfoundation.org/download/suricata-3.0RC2.tar.gz

Known issues & missing features

In a development release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

We’re happy to announce Suricata 3.0RC1. This release replaces 2.1beta4 as the new development release. The plan is to release the stable within a few weeks, so please help us test this release!

Lots of improvements:

• Multi-tenancy for detection
• Big email logging update by Eric Leblond
• Work on Lua and JSON output for various protocols by Mats Klepsland
• Redis output support by Eric Leblond
• JSON output for stats, rules profiling
• Colorized output on the commandline
• Support for the base64_decode and base64_data keywords by Jason Ish
• TLS and DNS lua support
• file_data support for SMTP by Giuseppe Longo
• Support wild cards in rule loading by Alexander Gozman

Packet capture got a lot of love:

• PF_RING optimizations by Alfredo Cardigliano
• Netmap updates by Aleksey Katargin
• AF_PACKET updated by Eric Leblond
• DAG fixes by Stephen Donnelly

Other than that, lots of cleanups and optimizations:

• stateful detection overhaul
• stream engine updates

Get the release here:

http://www.openinfosecfoundation.org/download/suricata-3.0RC1.tar.gz

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

• Alexander Gozman
• Mats Klepsland
• Giuseppe Longo
• Alfredo Cardigliano
• Aleksey Katargin
• Alessandro Guido
• Antti Tönkyrä
• Tom DeCanio
• Aaron Campbell
• DIALLO David
• David Cannings
• Helmut Schaa
• Jeff Barber
• Schnaffon
• Torgeir Natvig
• Zachary Rasmor
• Alexandre Macabies
• Stephen Donnelly

Known issues & missing features

In a development release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

The OISF development team is pleased to announce Suricata 2.0.10. This release fixes a number of important issues in the 2.0 series.

A number of other issues were fixed. Upgrading is highly recommended.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0.10.tar.gz

Changes

• Bug #1596: dns parser issue reported & fixed by Aaron Campbell
• Bug #1554: stored: false in files log when files were actually stored
• Feature #1581: support LINKTYPE_NULL

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

• Aaron Campbell
• Giuseppe Longo
• Greg Siemon

Known issues & missing features

If you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

Training & Support

Need help installing, updating, validating and tuning Suricata? We have trainings coming up. Paris in July, Barcelona in November: see http://suricata-ids.org/training/

For support options also see http://suricata-ids.org/support/

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

As the team is back from a very successful week in Barcelona, I’d like to take a moment on what we discussed and decided on with regards to development.

One thing no one was happy with is how the release schedules are working. Releases were meant to reasonably frequent, but the time between major releases was growing longer and longer. The 2.0 branch for example, is closing in on 2 years as the stable branch. The result is that many people are missing out on many of the improvements we’ve been doing. Currently many people using Suricata actually use a beta version, of even our git master, in production!

What we’re going to try is time based releases. Pretty much releases will be more like snapshots of the development branch. We think this can work as our dev branch is more and more stable due to our extensive QA setup.

Of course, we’ll have to make sure we’re not going to merge super intrusive changes just before a release. We’ll likely get into some pattern of merge windows and (feature) freezes, but how this will exactly play out is something we’ll figure out as we go.

We’re going to try to shoot for 3 of such releases per year.

In our redmine ticket tracker, I’ve also created a new pseudo-version ‘Soon’. Things we think should be addressed for the next release, will be added there. But we’ll retarget the tickets when they are actually implemented.

Since it’s already almost 2 years since we’ve done 2.0, we think the next release warrants a larger jump in the versioning. So we’re going to call it 3.0. The first release candidate will likely be released this week hopefully followed by a stable in December.

If you like fiddling with Suricata development, maybe you can get paid to do it.

Companies ask me regularly if I can recommend Suricata developers. I’m going to assemble a list of people who are interested in such work. If you like me to consider you in such cases, drop me an email.

If you really want me to *recommend* you, it’s important that I actually know you somewhat. So becoming a (volunteer) contributor will help a lot.

Things to mention in your email:
– interests
– github profile
– open source contributions
– social media, blog links
– availability, whether you’re a contractor or looking for a real J-O-B

Who knows, maybe something good will come from it!

Btw, if you’re not a dev but great at research, or deployments and tuning, I know some ppl that are always looking for such new hires as well!

The OISF development team is pleased to announce Suricata 2.0.9. This release fixes a number of issues in the 2.0 series.

Couple of important fixes: defrag evasion, a crash when using certain rules (mixing regular content and relative bytejumps with dce option) and better detection of TCP retransmissions with different data.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0.9.tar.gz

Changes

• Bug#1558: stream: retransmission not detected (2.0.x)
• Bug #1550: Segmentation Fault at detect-engine-content-inspection.c:438
• Bug #1564: defrag: evasion issue
• Bug #1431: stream: last_ack update issue leading to stream gaps (2.0.x)
• Bug #1483: 2.0.x backport: Leading whitespace in flowbits variable names
• Bug #1490: http_host payload validation erroring on uppercase PCRE metacharacters
• Bug #1501: 2.0.x backport: Add HUP coverage to output json-log
• Bug #1510: 2.0.x: address var parsing issue
• Bug #1513: stream_size <= and >= modifiers function as < and > (equality is not functional) (2.0.x)
• Update bundled libhtp to 0.5.18

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

• Jérémy Beaume
• Erik Hjelmvik
• Alessandro Guido
• Alexandre Macabies
• Darren Spruell
• Jay MJ
• Charles Smutz

Known issues & missing features

If you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

Training & Support

Need help installing, updating, validating and tuning Suricata? We have a training coming up in Barcelona in November: see http://suricata-ids.org/training/

For support options also see http://suricata-ids.org/support/

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

In the default suricata.yaml stream section we have:

stream:
  memcap: 32mb
  checksum-validation: no      # reject wrong csums
  async-oneside: true
  midstream: true
  inline: no                  # auto will use inline mode in IPS mode, yes or no set it statically
  reassembly:
    memcap: 64mb
    depth: 1mb                  # reassemble 1mb into a stream
    toserver-chunk-size: 2560
    toclient-chunk-size: 2560
    randomize-chunk-size: yes
    #randomize-chunk-range: 10
    #raw: yes
    #chunk-prealloc: 250
    #segments:
    #  - size: 4
    #    prealloc: 256
    #  - size: 16
    #    prealloc: 512
    #  - size: 112
    #    prealloc: 512
    #  - size: 248
    #    prealloc: 512
    #  - size: 512
    #    prealloc: 512
    #  - size: 768
    #    prealloc: 1024
    #  - size: 1448
    #    prealloc: 1024
    #  - size: 65535
    #    prealloc: 128

So what are these segment preallocations for?
Let's have a look. When Suricata exits (or kill -15 PidOfSuricata) it produces a lot of useful statistics in the suricata.log file (you can enable that from the suricata.yaml and use the "-v" switch (verbose) when starting Suricata):
The example below is for exit stats.
   

tail -20 StatsByDate/suricata-2014-06-01.log
[24344] 1/6/2014 -- 01:45:52 - (source-af-packet.c:1810) <Info> (ReceiveAFPThreadExitStats) -- (AFPacketeth314) Packets 7317661624, bytes 6132661347126
[24344] 1/6/2014 -- 01:45:52 - (stream-tcp.c:4643) <Info> (StreamTcpExitPrintStats) -- Stream TCP processed 3382528539 TCP packets
[24345] 1/6/2014 -- 01:45:52 - (source-af-packet.c:1807) <Info> (ReceiveAFPThreadExitStats) -- (AFPacketeth315) Kernel: Packets 8049357450, dropped 352658715
[24345] 1/6/2014 -- 01:45:52 - (source-af-packet.c:1810) <Info> (ReceiveAFPThreadExitStats) -- (AFPacketeth315) Packets 7696486934, bytes 6666577738944
[24345] 1/6/2014 -- 01:45:52 - (stream-tcp.c:4643) <Info> (StreamTcpExitPrintStats) -- Stream TCP processed 3357321803 TCP packets
[24346] 1/6/2014 -- 01:45:52 - (source-af-packet.c:1807) <Info> (ReceiveAFPThreadExitStats) -- (AFPacketeth316) Kernel: Packets 7573051188, dropped 292897219
[24346] 1/6/2014 -- 01:45:52 - (source-af-packet.c:1810) <Info> (ReceiveAFPThreadExitStats) -- (AFPacketeth316) Packets 7279948375, bytes 6046562324948
[24346] 1/6/2014 -- 01:45:52 - (stream-tcp.c:4643) <Info> (StreamTcpExitPrintStats) -- Stream TCP processed 3454330660 TCP packets
[24329] 1/6/2014 -- 01:45:53 - (stream-tcp-reassemble.c:502) <Info> (StreamTcpReassembleFree) -- TCP segment pool of size 4 had a peak use of 60778 segments, more than the prealloc setting of 256
[24329] 1/6/2014 -- 01:45:53 - (stream-tcp-reassemble.c:502) <Info> (StreamTcpReassembleFree) -- TCP segment pool of size 16 had a peak use of 314953 segments, more than the prealloc setting of 512
[24329] 1/6/2014 -- 01:45:53 - (stream-tcp-reassemble.c:502) <Info> (StreamTcpReassembleFree) -- TCP segment pool of size 112 had a peak use of 113739 segments, more than the prealloc setting of 512
[24329] 1/6/2014 -- 01:45:53 - (stream-tcp-reassemble.c:502) <Info> (StreamTcpReassembleFree) -- TCP segment pool of size 248 had a peak use of 17893 segments, more than the prealloc setting of 512
[24329] 1/6/2014 -- 01:45:53 - (stream-tcp-reassemble.c:502) <Info> (StreamTcpReassembleFree) -- TCP segment pool of size 512 had a peak use of 31787 segments, more than the prealloc setting of 512
[24329] 1/6/2014 -- 01:45:53 - (stream-tcp-reassemble.c:502) <Info> (StreamTcpReassembleFree) -- TCP segment pool of size 768 had a peak use of 30769 segments, more than the prealloc setting of 1024
[24329] 1/6/2014 -- 01:45:53 - (stream-tcp-reassemble.c:502) <Info> (StreamTcpReassembleFree) -- TCP segment pool of size 1448 had a peak use of 89446 segments, more than the prealloc setting of 1024
[24329] 1/6/2014 -- 01:45:53 - (stream-tcp-reassemble.c:502) <Info> (StreamTcpReassembleFree) -- TCP segment pool of size 65535 had a peak use of 81214 segments, more than the prealloc setting of 128
[24329] 1/6/2014 -- 01:45:53 - (stream.c:182) <Info> (StreamMsgQueuesDeinit) -- TCP segment chunk pool had a peak use of 20306 chunks, more than the prealloc setting of 250
[24329] 1/6/2014 -- 01:45:53 - (host.c:245) <Info> (HostPrintStats) -- host memory usage: 390144 bytes, maximum: 16777216
[24329] 1/6/2014 -- 01:45:55 - (detect.c:3890) <Info> (SigAddressCleanupStage1) -- cleaning up signature grouping structure... complete
[24329] 1/6/2014 -- 01:45:55 - (util-device.c:185) <Notice> (LiveDeviceListClean) -- Stats for 'eth3':  pkts: 124068935209, drop: 5245430626 (4.23%), invalid chksum: 0

Notice all the "TCP segment pool" messages. This is the actual tcp segment pool reassembly stats for that period of time that Suricata was running. We could adjust accordingly in the suricata.yaml (as compared to the default settings above)
   

stream:
  memcap: 14gb
  checksum-validation: no      # reject wrong csums
  midstream: false
  prealloc-sessions: 375000
  inline: no                  # auto will use inline mode in IPS mode, yes or no set it statically
  reassembly:
    memcap: 20gb
    depth: 12mb                  # reassemble 1mb into a stream
    toserver-chunk-size: 2560
    toclient-chunk-size: 2560
    randomize-chunk-size: yes
    #randomize-chunk-range: 10
    raw: yes
    chunk-prealloc: 20556

    segments:
      - size: 4
        prealloc: 61034
      - size: 16
        prealloc: 315465
      - size: 112
        prealloc: 114251
      - size: 248
        prealloc: 18405
      - size: 512
        prealloc: 30769
      - size: 768
        prealloc: 31793
      - size: 1448
        prealloc: 90470
      - size: 65535
        prealloc: 81342

   


   
The total RAM (reserved) consumption for these preallocations (from the stream.reassembly.memcap value ) would be:

4*61034 + 16*315465 + 112*114251 + 248*18405 + 512*30769 + 768*31793 + 1448*90470 + 65535*81342 

= 5524571410 bytes
= 5.14 GB of RAM

So we could preallocate the tcp segments and take the Suricata tuning even a step further and improve performance as well.

So now when you start Suricata with the "-v" switch in your suricata.log with this specific set up described above you should see something like:

...
...
[30709] 1/6/2014 -- 12:17:34 - (stream-tcp-reassemble.c:425) <Info> (StreamTcpReassemblyConfig) -- segment pool: pktsize 4, prealloc 61034
[30709] 1/6/2014 -- 12:17:34 - (stream-tcp-reassemble.c:425) <Info> (StreamTcpReassemblyConfig) -- segment pool: pktsize 16, prealloc 315465
[30709] 1/6/2014 -- 12:17:34 - (stream-tcp-reassemble.c:425) <Info> (StreamTcpReassemblyConfig) -- segment pool: pktsize 112, prealloc 114251
[30709] 1/6/2014 -- 12:17:34 - (stream-tcp-reassemble.c:425) <Info> (StreamTcpReassemblyConfig) -- segment pool: pktsize 248, prealloc 18405
[30709] 1/6/2014 -- 12:17:34 - (stream-tcp-reassemble.c:425) <Info> (StreamTcpReassemblyConfig) -- segment pool: pktsize 512, prealloc 30769
[30709] 1/6/2014 -- 12:17:34 - (stream-tcp-reassemble.c:425) <Info> (StreamTcpReassemblyConfig) -- segment pool: pktsize 768, prealloc 31793
[30709] 1/6/2014 -- 12:17:35 - (stream-tcp-reassemble.c:425) <Info> (StreamTcpReassemblyConfig) -- segment pool: pktsize 1448, prealloc 90470
[30709] 1/6/2014 -- 12:17:35 - (stream-tcp-reassemble.c:425) <Info> (StreamTcpReassemblyConfig) -- segment pool: pktsize 65535, prealloc 81342
[30709] 1/6/2014 -- 12:17:35 - (stream-tcp-reassemble.c:461) <Info> (StreamTcpReassemblyConfig) -- stream.reassembly "chunk-prealloc": 20556
...
...

NOTE:
Those 5.14 GB RAM in the example here will be preallocated (taken) from the stream.reassembly.memcap value. In other words it will not consume 5.14 GB of RAM more.

So be careful when setting up preallocation in order not to preallocate more of what you have.
In my case of 10Gbps suricata.yaml config I had:

stream:
  memcap: 14gb
  checksum-validation: no      # reject wrong csums
  midstream: false
  prealloc-sessions: 375000
  inline: no                  # auto will use inline mode in IPS mode, yes or no set it statically
  reassembly:
    memcap: 20gb
    depth: 12mb                  # reassemble 1mb into a stream

What this helps with is that it lowers CPU usage/contention for TCP segment allocation during reassembly - it is already preallocated and Suricata just uses it instead of creating it everytime it needs it. It also helps minimize the initial drops during startup.

Highly adaptable and  flexible.

OS used/tested for this tutorial - Debian Wheezy and/or Ubuntu LTS 12.0.4
With 3.2.0 and 3.5.0 kernel level respectively with Suricata 2.0dev at the moment of this writing.



This article consists of the following major 3 sections:

• Network card drivers and tuning
• Kernel specific tunning
• Suricata.yaml configuration  (file extraction specific)

Network and system  tools:

apt-get install ethtool bwm-ng iptraf htop

Network card drivers and tuning

Our card is Intel 82599EB 10-Gigabit SFI/SFP+

rmmod ixgbe
sudo modprobe ixgbe FdirPballoc=3
ifconfig eth3 up

then (we disable irqbalance and make sure it does not enable itself during reboot)

 killall irqbalance
 service irqbalance stop

 apt-get install chkconfig
 chkconfig irqbalance off

Get the Intel network driver form here (we will use them in a second) - https://downloadcenter.intel.com/default.aspx

Download to your directory of choice then unzip,compile and install:

wget http://sourceforge.net/projects/e1000/files/ixgbe%20stable/3.18.7/ixgbe-3.18.7.tar.gz
tar -zxf ixgbe-3.18.7.tar.gz      
cd /home/pevman/ixgbe-3.18.7/src      
make clean && make && make install

Set irq affinity - do not forget to change eth3  below with the name of the network interface you are using:

 cd ../scripts/
 ./set_irq_affinity  eth3

 You should see something like this:

root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ./set_irq_affinity  eth3
no rx vectors found on eth3
no tx vectors found on eth3
eth3 mask=1 for /proc/irq/101/smp_affinity
eth3 mask=2 for /proc/irq/102/smp_affinity
eth3 mask=4 for /proc/irq/103/smp_affinity
eth3 mask=8 for /proc/irq/104/smp_affinity
eth3 mask=10 for /proc/irq/105/smp_affinity
eth3 mask=20 for /proc/irq/106/smp_affinity
eth3 mask=40 for /proc/irq/107/smp_affinity
eth3 mask=80 for /proc/irq/108/smp_affinity
eth3 mask=100 for /proc/irq/109/smp_affinity
eth3 mask=200 for /proc/irq/110/smp_affinity
eth3 mask=400 for /proc/irq/111/smp_affinity
eth3 mask=800 for /proc/irq/112/smp_affinity
eth3 mask=1000 for /proc/irq/113/smp_affinity
eth3 mask=2000 for /proc/irq/114/smp_affinity
eth3 mask=4000 for /proc/irq/115/smp_affinity
eth3 mask=8000 for /proc/irq/116/smp_affinity
root@suricata:/home/pevman/ixgbe-3.18.7/scripts#

Now we have the latest drivers installed (at the time of this writing) and we have run the affinity script:

   *-network:1
       description: Ethernet interface
       product: 82599EB 10-Gigabit SFI/SFP+ Network Connection
       vendor: Intel Corporation
       physical id: 0.1
       bus info: pci@0000:04:00.1
       logical name: eth3
       version: 01
       serial: 00:e0:ed:19:e3:e1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi msix pciexpress vpd bus_master cap_list ethernet physical fibre
       configuration: autonegotiation=off broadcast=yes driver=ixgbe driverversion=3.18.7 duplex=full firmware=0x800000cb latency=0 link=yes multicast=yes port=fibre promiscuous=yes
       resources: irq:37 memory:fbc00000-fbc1ffff ioport:e000(size=32) memory:fbc40000-fbc43fff memory:fa700000-fa7fffff memory:fa600000-fa6fffff

We need to disable all offloading on the network card in order for the IDS to be able to see the traffic as it is supposed to be (without checksums,tcp-segmentation-offloading and such..) Otherwise your IDPS would not be able to see all "natural" network traffic the way it is supposed to and will not inspect it properly.

This would influence the correctness of ALL outputs including file extraction. So make sure all offloading features are OFF !!!

When you first install the drivers and card your offloading settings might look like this:

root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -k eth3
Offload parameters for eth3:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: on
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: on
root@suricata:/home/pevman/ixgbe-3.18.7/scripts#

So we disable all of them, like so (and we load balance the UDP flows for that particular network card):

ethtool -K eth3 tso off
ethtool -K eth3 gro off
ethtool -K eth3 ufo off
ethtool -K eth3 lro off
ethtool -K eth3 gso off
ethtool -K eth3 rx off
ethtool -K eth3 tx off
ethtool -K eth3 sg off
ethtool -K eth3 rxvlan off
ethtool -K eth3 txvlan off
ethtool -N eth3 rx-flow-hash udp4 sdfn
ethtool -N eth3 rx-flow-hash udp6 sdfn
ethtool -C eth3 rx-usecs 1 rx-frames 0
ethtool -C eth3 adaptive-rx off

Your output should look something like this:

root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 tso off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 gro off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 lro off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 gso off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 rx off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 tx off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 sg off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 rxvlan off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 txvlan off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -N eth3 rx-flow-hash udp4 sdfn
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -N eth3 rx-flow-hash udp6 sdfn
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -n eth3 rx-flow-hash udp6
UDP over IPV6 flows use these fields for computing Hash flow key:
IP SA
IP DA
L4 bytes 0 & 1 [TCP/UDP src port]
L4 bytes 2 & 3 [TCP/UDP dst port]

root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -n eth3 rx-flow-hash udp4
UDP over IPV4 flows use these fields for computing Hash flow key:
IP SA
IP DA
L4 bytes 0 & 1 [TCP/UDP src port]
L4 bytes 2 & 3 [TCP/UDP dst port]

root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -C eth3 rx-usecs 0 rx-frames 0
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -C eth3 adaptive-rx off

Now we doublecheck and run ethtool again to verify that the offloading is OFF:

root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -k eth3
Offload parameters for eth3:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
rx-vlan-offload: off
tx-vlan-offload: off

Ring parameters on the network card:

root@suricata:~# ethtool -g eth3
Ring parameters for eth3:
Pre-set maximums:
RX:             4096
RX Mini:        0
RX Jumbo:       0
TX:            4096

Current hardware settings:
RX:             512
RX Mini:        0
RX Jumbo:       0
TX:             512

We can increase that to the max Pre-set RX:

root@suricata:~# ethtool -G eth3 rx 4096

Then we  have a look again:

root@suricata:~# ethtool -g eth3
Ring parameters for eth3:
Pre-set maximums:
RX:             4096
RX Mini:        0
RX Jumbo:       0
TX:             4096
Current hardware settings:
RX:             4096
RX Mini:        0
RX Jumbo:       0
TX:             512

Making network changes permanent across reboots

On Ubuntu for example you can do:

root@suricata:~# crontab -e

Add the following:

# add cronjob at reboot - disbale network offload
@reboot /opt/tmp/disable-network-offload.sh

and your disable-network-offload.sh script (in this case under /opt/tmp/ ) will contain the following:

#!/bin/bash
ethtool -K eth3 tso off
ethtool -K eth3 gro off
ethtool -K eth3 ufo off
ethtool -K eth3 lro off
ethtool -K eth3 gso off
ethtool -K eth3 rx off
ethtool -K eth3 tx off
ethtool -K eth3 sg off
ethtool -K eth3 rxvlan off
ethtool -K eth3 txvlan off
ethtool -N eth3 rx-flow-hash udp4 sdfn
ethtool -N eth3 rx-flow-hash udp6 sdfn
ethtool -C eth3 rx-usecs 1 rx-frames 0
ethtool -C eth3 adaptive-rx off

with:

chmod 755 disable-network-offload.sh

To make sure you have the ixgbe module always loaded at boot time you can add "ixgbe" to the  /etc/modules file.

Kernel specific tunning

Certain adjustments in parameters in the kernel can help as well :

sysctl -w net.core.netdev_max_backlog=250000
sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.rmem_default=16777216
sysctl -w net.core.optmem_max=16777216

Making kernel changes permanent across reboots

example:

echo 'net.core.netdev_max_backlog=250000' >> /etc/sysctl.conf

reload the changes:

sysctl -p

OR for all the above adjustments:

echo 'net.core.netdev_max_backlog=250000' >> /etc/sysctl.conf
echo 'net.core.rmem_max=16777216' >> /etc/sysctl.conf
echo 'net.core.rmem_default=16777216' >> /etc/sysctl.conf
echo 'net.core.optmem_max=16777216' >> /etc/sysctl.conf
sysctl -p

Suricata.yaml configuration  (file extraction specific)

As of Suricata 1.2  - it is possible to detect and extract/store over 5000 types of files from HTTP sessions.

Specific file extraction instructions can also be found in the official page documentation.

The following libraries are needed on the system running Suricata :

apt-get install libnss3-dev libnspr4-dev

Suricata also needs to be compiled with file extraction enabled (not covered here).

In short in the suriacta.yaml, those are the sections below that can be tuned/configured and affect the file extraction and logging:
(the bigger the mem values the better on a busy link )

  - eve-log:
      enabled: yes
      type: file #file|syslog|unix_dgram|unix_stream
      filename: eve.json
      # the following are valid when type: syslog above
      #identity: "suricata"
      #facility: local5
      #level: Info ## possible levels: Emergency, Alert, Critical,
                   ## Error, Warning, Notice, Info, Debug
      types:
        - alert
        - http:
            extended: yes     # enable this for extended logging information
        - dns
        - tls:
            extended: yes     # enable this for extended logging information
        - files:
            force-magic: yes   # force logging magic on all logged files
            force-md5: yes     # force logging of md5 checksums
        #- drop
        - ssh

For file store to disk/extraction:

   - file-store:
      enabled: yes       # set to yes to enable
      log-dir: files    # directory to store the files
      force-magic: yes   # force logging magic on all stored files
      force-md5: yes     # force logging of md5 checksums
      #waldo: file.waldo # waldo file to store the file_id across runs

 stream:
  memcap: 32mb
  checksum-validation: no      # reject wrong csums
  inline: auto                  # auto will use inline mode in IPS mode, yes or no set it statically
  reassembly:
    memcap: 128mb
    depth: 1mb                  # reassemble 1mb into a stream

  
depth: 1mb , would mean that in one tcp reassembled flow, the max size of the file that can be extracted is just about 1mb.

Both stream.memcap and reassembly.memcap (if reassembly is needed) must be big enough to accommodate the whole file on the fly in traffic that needs to be extracted PLUS any other stream and reassembly tasks that the engine needs to do while inspecting the traffic on a particular link.

 app-layer:
  protocols:
....
....
     http:
      enabled: yes
      # memcap: 64mb

The default limit for mem usage for http is 64mb   , that could be increased , ex - memcap: 4GB -  since HTTP is present everywhere and a low memcap on a busy HTTP link would limit the inspection and extraction size ability.

       libhtp:

         default-config:
           personality: IDS

           # Can be specified in kb, mb, gb.  Just a number indicates
           # it's in bytes.
           request-body-limit: 3072
           response-body-limit: 3072

The default values above control how far the HTTP request and response body is tracked and also limit file inspection. This should be set to a much higher value:

        libhtp:

         default-config:
           personality: IDS

           # Can be specified in kb, mb, gb.  Just a number indicates
           # it's in bytes.
           request-body-limit: 1gb
           response-body-limit: 1gb

 or 0 (which would mean unlimited):

       libhtp:

         default-config:
           personality: IDS

           # Can be specified in kb, mb, gb.  Just a number indicates
           # it's in bytes.
           request-body-limit: 0
           response-body-limit: 0

and then of course you would need a rule loaded(example):

alert http any any -> any any (msg:"PDF file Extracted"; filemagic:"PDF document"; filestore; sid:11; rev:11;)

That's it.

This is a blogpost about getting around the following error when using Suricata with pfring:

(source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
(tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1

However in my case eth2 existed, was up and running and the pfring module was loaded. So what happened in a bit more detail below :

I experienced this after a git pull update/upgrade of Suricata (latest at the moment of this writing) and after I re compiled pfring (using the latest pfring from git (https://github.com/ntop/PF_RING.git).

My set up (linux Debian/Ubuntu like systems):

root@suricata:/var/data/log/suricata# ifconfig eth2
eth2      Link encap:Ethernet  HWaddr 00:e0:ed:19:e3:e0
          inet6 addr: fe80::2e0:edff:fe19:e3e0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2962266192 errors:0 dropped:5527381 overruns:0 frame:0
          TX packets:19 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2867936692537 (2.8 TB)  TX bytes:3345 (3.3 KB)

The pfring set up I had was configured like this below:

root@suricata:/var/data/log/suricata# modprobe pf_ring transparent_mode=0 min_num_slots=65534

A regular check reveals nothing abnormal:

root@suricata:/var/data/log/suricata# modinfo pf_ring && cat /proc/net/pf_ring/info
filename:       /lib/modules/3.14.0-031400-generic/kernel/net/pf_ring/pf_ring.ko
alias:          net-pf-27
description:    Packet capture acceleration and analysis
author:         ntop.org
license:        GPL
srcversion:     E344EB01757B55E97A93D0C
depends:     
vermagic:       3.14.0-031400-generic SMP mod_unload modversions
parm:           min_num_slots:Min number of ring slots (uint)
parm:           perfect_rules_hash_size:Perfect rules hash size (uint)
parm:           transparent_mode:(deprecated) (uint)
parm:           enable_debug:Set to 1 to enable PF_RING debug tracing into the syslog (uint)
parm:           enable_tx_capture:Set to 1 to capture outgoing packets (uint)
parm:           enable_frag_coherence:Set to 1 to handle fragments (flow coherence) in clusters (uint)
parm:           enable_ip_defrag:Set to 1 to enable IP defragmentation(only rx traffic is defragmentead) (uint)
parm:           quick_mode:Set to 1 to run at full speed but with upto one socket per interface (uint)
PF_RING Version          : 6.1.1 (dev:250a67fe1082121ac511a19ebc3fe1fc5f494bfe)
Total rings              : 0

Standard (non DNA/ZC) Options
Ring slots               : 65534
Slot version             : 16
Capture TX               : Yes [RX+TX]
IP Defragment            : No
Socket Mode              : Standard
Total plugins            : 0
Cluster Fragment Queue   : 0
Cluster Fragment Discard : 0

Suricata and pfring have been installed as explained here - on the Suricata redmine wiki.

root@suricata:~# ldd /usr/local/bin/suricata
    linux-vdso.so.1 =>  (0x00007fff419fe000)
    libhtp-0.5.17.so.1 => /usr/local/lib/libhtp-0.5.17.so.1 (0x00007f32af5a1000)
    libGeoIP.so.1 => /usr/lib/x86_64-linux-gnu/libGeoIP.so.1 (0x00007f32af372000)
    libluajit-5.1.so.2 => /usr/local/lib/libluajit-5.1.so.2 (0x00007f32af103000)
    libmagic.so.1 => /usr/lib/x86_64-linux-gnu/libmagic.so.1 (0x00007f32aeee7000)
    libcap-ng.so.0 => /usr/local/lib/libcap-ng.so.0 (0x00007f32aece2000)
    libpfring.so => /usr/local/lib/libpfring.so (0x00007f32aeaa3000)
    libpcap.so.1 => /usr/local/pfring/lib/libpcap.so.1 (0x00007f32ae80e000)
    libnet.so.1 => /usr/lib/x86_64-linux-gnu/libnet.so.1 (0x00007f32ae5f5000)
    libjansson.so.4 => /usr/lib/x86_64-linux-gnu/libjansson.so.4 (0x00007f32ae3e8000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f32ae1ca000)
    libyaml-0.so.2 => /usr/lib/x86_64-linux-gnu/libyaml-0.so.2 (0x00007f32adfaa000)
    libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f32add6b000)
    libnss3.so => /usr/lib/x86_64-linux-gnu/libnss3.so (0x00007f32ada31000)
    libnspr4.so => /usr/lib/x86_64-linux-gnu/libnspr4.so (0x00007f32ad7f4000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f32ad42e000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f32ad215000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f32acf0f000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f32acd0a000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f32acaf4000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f32af7d4000)
    libnuma.so.1 => /usr/lib/x86_64-linux-gnu/libnuma.so.1 (0x00007f32ac8e9000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f32ac6e0000)
    libnssutil3.so => /usr/lib/x86_64-linux-gnu/libnssutil3.so (0x00007f32ac4b5000)
    libplc4.so => /usr/lib/x86_64-linux-gnu/libplc4.so (0x00007f32ac2b0000)
    libplds4.so => /usr/lib/x86_64-linux-gnu/libplds4.so (0x00007f32ac0ab000)

Further more my Suricata start line was like this:

suricata --pfring-int=eth2 --pfring-cluster-id=99 --pfring-cluster-type=cluster_flow -c /etc/suricata/peter-yaml/suricata-pfring.yaml --pidfile /var/run/suricata.pid -v

Even though everything seems fine - I could  not start Suricata with pfring:

[31591] 5/8/2015 -- 17:10:31 - (tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1
[31591] 5/8/2015 -- 17:10:31 - (source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
[31592] 5/8/2015 -- 17:10:31 - (tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1
[31592] 5/8/2015 -- 17:10:31 - (source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
[31593] 5/8/2015 -- 17:10:32 - (tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1
[31593] 5/8/2015 -- 17:10:32 - (source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
[31594] 5/8/2015 -- 17:10:32 - (tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1
[31594] 5/8/2015 -- 17:10:32 - (source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
....

I was getting that error even though I reloaded the pfring module:

rmmod pr_ring
modprobe pf_ring transparent_mode=0 min_num_slots=65534

the way I usually do...

In short - this is the fix:

LD_LIBRARY_PATH=/usr/local/pfring/lib suricata --pfring-int=eth2  --pfring-cluster-id=99 --pfring-cluster-type=cluster_flow  -c /etc/suricata/peter-yaml/suricata-pfring.yaml --pidfile /var/run/suricata.pid -v

Notice the use of:

LD_LIBRARY_PATH=/usr/local/pfring/lib suricata 

More information about what is LD_LIBRARY_PATH     

To get rid of LD_LIBRARY_PATH you can create a pfring.conf file in /etc/ld.so.conf.d/ containing:

/usr/local/pfring/lib

and run

sudo ldconfig

We recently released a securityonion-suricata package for Suricata 2.0.8:
http://blog.securityonion.net/2015/05/suricata-208.html

An issue was found in the packaging:
https://groups.google.com/d/topic/security-onion/1MmmmO2XOyc/discussion

I've updated the securityonion-suricata package to resolve this issue.

The new package version is:
securityonion-suricata - 2.0.8-0ubuntu0securityonion2

Issues Resolved

Issue 742: securityonion-suricata package missing debian/install
https://github.com/Security-Onion-Solutions/security-onion/issues/742

Updating
The new package is now available in our stable repo.  Please see the following page for full update instructions:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Upgrade

Feedback
If you have any questions or problems, please use our security-onion mailing list:
https://github.com/Security-Onion-Solutions/security-onion/wiki/MailingLists

Commercial Support
Need commercial support?  Please see:
http://securityonionsolutions.com

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://github.com/Security-Onion-Solutions/security-onion/wiki/TeamMembers

Thanks!

We have updated the official Ubuntu PPA to Suricata 2.1beta4. To use this PPA read our docs here.

If you’re using this PPA, updating is as simple as:

apt-get update && apt-get upgrade

The PPA Ubuntu packages have IPS mode through NFQUEUE enabled.

The Windows MSI installer of the Suricata 2.1beta4 release is now available.

Download it here: suricata-2.1beta4-1-32bit.msi

After downloading, double click the file to launch the installer. The installer is now signed.

If you have a previous version installed, please remove that first.

We have updated the official Ubuntu PPA to Suricata 2.0.8. To use this PPA read our docs here.

To install Suricata through this PPA, enter:
sudo add-apt-repository ppa:oisf/suricata-stable
sudo apt-get update
sudo apt-get install suricata

If you’re already using this PPA, updating is as simple as:
sudo apt-get update && sudo apt-get upgrade

The PPA Ubuntu packages have IPS mode through NFQUEUE enabled.

The Windows MSI installer of the Suricata 2.0.8 release is now available.

Download it here: Suricata-2.0.8-1-32bit.msi

After downloading, double click the file to launch the installer. The installer is now signed.

If you have a previous version installed, please remove that first.

Recently (few hrs ago as of writing this blog) there was  a new feature (thanks to gozzy) introduced in Suricata IDS/IPS/NSM  - wildcard rule loading capability.

As of the moment the feature is available in our git master. If you are wondering how to get that up and running or do not have the latest Suricata from git master - here is a quick tutorial (Debian/Ubuntu):

1)

apt-get -y install libpcre3 libpcre3-dbg libpcre3-dev build-essential \
autoconf automake libtool libpcap-dev libnet1-dev libyaml-0-2 \
libyaml-dev zlib1g zlib1g-dev libmagic-dev libcap-ng-dev \
libjansson-dev pkg-config libnss3-dev libnspr4-dev git-core

2)

git clone git://phalanx.openinfosecfoundation.org/oisf.git && cd oisf/ &&  git clone https://github.com/ironbee/libhtp.git -b 0.5.x

3)

 ./autogen.sh && \
 ./configure --prefix=/usr/ --sysconfdir=/etc/ --localstatedir=/var/ \
 --enable-geoip --enable-unix-socket \
 --with-libnss-libraries=/usr/lib --with-libnss-includes=/usr/include/nss/ \
 --with-libnspr-libraries=/usr/lib --with-libnspr-includes=/usr/include/nspr \
 && make clean && make && make install-full && ldconfig

To confirm -

suricata --build-info

Now that you have latest Suricta up and running - here it is what this blog post is all about  - wildcard rule loading for Suricata IDPS. Some possible scenarios of use are loading wildcarded rules form the :

Command line

Please note the "quotes" !

suricata -c /etc/suricata/suricata.yaml  -v -i eth0 -S "/etc/suricata/rules/*.rules"

Pretty self explanatory. The command above will load all .rules files from /etc/suricata/rules/

suricata -c /etc/suricata/suricata.yaml  -v -i eth0 -S "/etc/suricata/rules/emerging*"

The command above will load all emerging* rules files from /etc/suricata/rules/

Config file

You can also set that up in the suricata.yaml config file. Here is how (please note the "quotes").

In your rules section in the suricata.yaml:

# Set the default rule path here to search for the files.
# if not set, it will look at the current working dir
default-rule-path: /etc/suricata/rules
rule-files:
 #- "*.rules"
 - "emerging*"
 #- botcc.rules
 #- ciarmy.rules
 #- compromised.rules
 #- drop.rules
 #- dshield.rules
 #- emerging-activex.rules
 #- emerging-attack_response.rules

The set up above will load all emerging* files and the rules residing in those. Then you can start Suricata anyway you would like, examples:

suricata -c /etc/suricata/suricata.yaml  -v -i eth0

suricata -c /etc/suricata/suricata.yaml  -v --af-packet=eth0

 and in suricata.log you should see all emerging* rule files being loaded:

......
[13558] 22/5/2015 -- 17:19:39 - (reputation.c:620) <Info> (SRepInit) -- IP reputation disabled
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-activex.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-attack_response.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-chat.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-current_events.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-deleted.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:420) <Warning> (ProcessSigFiles) -- [ERRCODE: SC_ERR_NO_RULES(42)] - No rules loaded from /etc/suricata/rules/emerging-deleted.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-dns.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-dos.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-exploit.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-ftp.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-games.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-icmp.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:420) <Warning> (ProcessSigFiles) -- [ERRCODE: SC_ERR_NO_RULES(42)] - No rules loaded from /etc/suricata/rules/emerging-icmp.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-icmp_info.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-imap.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-inappropriate.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-info.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-malware.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-misc.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-mobile_malware.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-netbios.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-p2p.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-policy.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-pop3.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-rpc.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-scada.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-scan.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-shellcode.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-smtp.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-snmp.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-sql.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-telnet.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-tftp.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-trojan.rules
[13558] 22/5/2015 -- 17:19:41 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-user_agents.rules
[13558] 22/5/2015 -- 17:19:41 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-voip.rules
[13558] 22/5/2015 -- 17:19:41 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-web_client.rules
[13558] 22/5/2015 -- 17:19:41 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-web_server.rules
[13558] 22/5/2015 -- 17:19:41 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-web_specific_apps.rules
[13558] 22/5/2015 -- 17:19:43 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-worm.rules
.......

You can also use it like that :

# Set the default rule path here to search for the files.
# if not set, it will look at the current working dir
default-rule-path: /etc/suricata/rules
rule-files:
 #- "*.rules"
 - "*web*"
 #- "emerging*"
 #- botcc.rules
 #- ciarmy.rules
 #- compromised.rules
 #- drop.rules

That's it.

Suricata is a very flexible and powerful multithreading  IDS/IPS/NSM.

Here is a simple tutorial (tested on Debian/Ubuntu) of how to configure multiple interfaces for af-packet mode with Suricata (af-packet mode works by default/out of the box on kernels 3.2 and above). Lets say you would like to start simple IDSing with Suricata on eth1, eth2 and eth3 on a particular machine/server.


In your suricata.yaml config (usually located in /etc/suricata/) find the af-packet section and do the following:

af-packet:
  - interface: eth2
    threads: 16
    cluster-id: 98
    cluster-type: cluster_cpu
    defrag: no
    use-mmap: yes
    ring-size: 200000
    checksum-checks: kernel
  - interface: eth1
    threads: 2
    cluster-id: 97
    cluster-type: cluster_flow
    defrag: no
    use-mmap: yes
    ring-size: 30000
  - interface: eth3
    threads: 2
    cluster-id: 96
    cluster-type: cluster_flow
    defrag: no
    use-mmap: yes
    ring-size: 20000

Of course feel free to adjust the ring-sizes (packet buffers) as you see fit for your particular set up.
NOTE:  do not forget to use a different cluster-id

so now you can start suricata like so:

suricata -c /etc/suricata/suricata.yaml -v --af-packet 

That above will start Suricata which will listen on eth2 with 16 threads with cluster_type: cluster_cpu and on eth1,eth3 with 2 threads each with cluster_type: cluster_flow. Have a look in your suricata.log file for more info.

If you would like to just test and see how it goes for eth2 only:

suricata -c /etc/suricata/suricata.yaml -v --af-packet=eth2

...easy and flexible.

Suricata 2.0.8 was recently released:
http://suricata-ids.org/2015/05/06/suricata-2-0-8-available/

I've packaged Suricata 2.0.8 and it has been tested by David Zawdie (thanks!).

The new package version is:
securityonion-suricata - 2.0.8-0ubuntu0securityonion1

Issues Resolved

Issue 725: Suricata 2.0.8
https://github.com/Security-Onion-Solutions/security-onion/issues/725

Updating
The new package is now available in our stable repo.  Please see the following page for full update instructions:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Upgrade

This update will back up each of your existing suricata.yaml files to suricata.yaml.bak and migrate the HOME_NET and EXTERNAL_NET variables.  You'll then need to do the following:

• re-apply any other local customizations to suricata.yaml
• update ruleset and restart Suricata as follows:
  sudo rule-update

Feedback
If you have any questions or problems, please use our security-onion mailing list:
https://github.com/Security-Onion-Solutions/security-onion/wiki/MailingLists

Training
Need training?  We have 3-hour online classes this week:
http://securityonionsolutions.com

Commercial Support
Need commercial support?  Please see:
http://securityonionsolutions.com

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://github.com/Security-Onion-Solutions/security-onion/wiki/TeamMembers

Thanks!

We have lots of new packages queued up for release, but we need your help testing them!

If you're not already a member of our security-onion-testing group, please join and then click the following links for testing guidelines.

ELSA rev1205
https://groups.google.com/d/topic/security-onion-testing/OHhNEapIUgE/discussion

Suricata 2.0.8
https://groups.google.com/d/topic/security-onion-testing/WKeR1RViDlc/discussion

ossec_agent
https://groups.google.com/d/topic/security-onion-testing/N5gpeSHmIlk/discussion

Sphinxsearch 2.1.9
https://groups.google.com/d/topic/security-onion-testing/VWjichsRqPw/discussion

NSM
https://groups.google.com/d/topic/security-onion-testing/-cbA8FgH7lg/discussion

Setup
https://groups.google.com/d/topic/security-onion-testing/PBY2wJH9ruo/discussion

As you test each package, please add your test results to the thread.

Thanks in advance for your time and effort!

The OISF development team is proud to announce Suricata 2.1beta4. This is the fourth beta release for the upcoming 2.1 version. It should be considered a development snapshot for the 2.1 branch.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.1beta4.tar.gz

New Features

• Feature #1448: xbits support
• Feature #336: Add support for NETMAP to Suricata
• Feature #885: smtp file_data support
• Feature #1394: Improve TCP reuse support
• Feature #1445: Suricata does not work on pfSense/FreeBSD interfaces using PPPoE
• Feature #1447: Ability to reject ICMP traffic
• Feature #1410: add alerts to EVE’s drop logs

Improvements

• Optimization #1014: app layer reassembly fast-path
• Optimization #1377: flow manager: reduce (try)locking
• Optimization #1403: autofp packet pool performance problems
• Optimization #1409: http pipeline support for stateful detection
• Bug #1314: http-events performance issues

Bugs

• Bug #1340: null ptr dereference in Suricata v2.1beta2
• Bug #1352: file list is not cleaned up
• Bug #1358: Gradual memory leak using reload (kill -USR2 $pid)
• Bug #1366: Crash if default_packet_size is below 32 bytes
• Bug #1378: stats api doesn’t call thread deinit funcs
• Bug #1384: tcp midstream window issue (master)
• Bug #1388: pcap-file hangs on systems w/o atomics support (master)
• Bug #1392: http uri parsing issue (master)
• Bug #1393: CentOS 5.11 build failures
• Bug #1398: DCERPC traffic parsing issue (master)
• Bug #1401: inverted matching on incomplete session
• Bug #1402: When re-opening files on HUP (rotation) always use the append flag.
• Bug #1417: no rules loaded – latest git – rev e250040
• Bug #1425: dead lock in de_state vs flowints/flowvars
• Bug #1426: Files prematurely truncated by detection engine even though force-md5 is enabled
• Bug #1429: stream: last_ack update issue leading to stream gaps
• Bug #1435: EVE-Log alert payload option loses data
• Bug #1441: Local timestamps in json events
• Bug #1446: Unit ID check in Modbus packet error
• Bug #1449: smtp parsing issue
• Bug #1451: Fix list-keywords regressions
• Bug #1463: modbus parsing issue

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

• Kostya Kortchinsky of the Google Security Team
• the Yahoo Pentest Team
• Giuseppe Longo
• Alexander Gozman
• Ken Steele
• Andreas Moe
• David Diallo
• David Cannings
• David Maciejak
• Pierre Chifflier
• Tom DeCanio
• Zachary Rasmor
• Aleksey Katargin
• FireEye
• ANSSI
• Emerging Threats
• AFL project
• Coverity Scan
• Travis Green
• Darien Huss
• Greg Siemon
• Alessandro Guido
• Antti Tönkyrä
• Ray Ruvinskiy
• Eduardo Arada
• Michael Rash

Known issues & missing features

In a beta release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

Training & Support

Need help installing, updating, validating and tuning Suricata? We have trainings coming up. Paris in July, Barcelona in November: see http://suricata-ids.org/training/

For support options also see http://suricata-ids.org/support/

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

The OISF development team is proud to announce Suricata 2.1beta4. This is the fourth beta release for the upcoming 2.1 version. It should be considered a development snapshot for the 2.1 branch.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.1beta4.tar.gz

New Features

• Feature #1448: xbits support
• Feature #336: Add support for NETMAP to Suricata
• Feature #885: smtp file_data support
• Feature #1394: Improve TCP reuse support
• Feature #1445: Suricata does not work on pfSense/FreeBSD interfaces using PPPoE
• Feature #1447: Ability to reject ICMP traffic
• Feature #1410: add alerts to EVE’s drop logs

Improvements

• Optimization #1014: app layer reassembly fast-path
• Optimization #1377: flow manager: reduce (try)locking
• Optimization #1403: autofp packet pool performance problems
• Optimization #1409: http pipeline support for stateful detection
• Bug #1314: http-events performance issues

Bugs

• Bug #1340: null ptr dereference in Suricata v2.1beta2
• Bug #1352: file list is not cleaned up
• Bug #1358: Gradual memory leak using reload (kill -USR2 $pid)
• Bug #1366: Crash if default_packet_size is below 32 bytes
• Bug #1378: stats api doesn’t call thread deinit funcs
• Bug #1384: tcp midstream window issue (master)
• Bug #1388: pcap-file hangs on systems w/o atomics support (master)
• Bug #1392: http uri parsing issue (master)
• Bug #1393: CentOS 5.11 build failures
• Bug #1398: DCERPC traffic parsing issue (master)
• Bug #1401: inverted matching on incomplete session
• Bug #1402: When re-opening files on HUP (rotation) always use the append flag.
• Bug #1417: no rules loaded – latest git – rev e250040
• Bug #1425: dead lock in de_state vs flowints/flowvars
• Bug #1426: Files prematurely truncated by detection engine even though force-md5 is enabled
• Bug #1429: stream: last_ack update issue leading to stream gaps
• Bug #1435: EVE-Log alert payload option loses data
• Bug #1441: Local timestamps in json events
• Bug #1446: Unit ID check in Modbus packet error
• Bug #1449: smtp parsing issue
• Bug #1451: Fix list-keywords regressions
• Bug #1463: modbus parsing issue

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

• Kostya Kortchinsky of the Google Security Team
• the Yahoo Pentest Team
• Giuseppe Longo
• Alexander Gozman
• Ken Steele
• Andreas Moe
• David Diallo
• David Cannings
• David Maciejak
• Pierre Chifflier
• Tom DeCanio
• Zachary Rasmor
• Aleksey Katargin
• FireEye
• ANSSI
• Emerging Threats
• AFL project
• Coverity Scan
• Travis Green
• Darien Huss
• Greg Siemon
• Alessandro Guido
• Antti Tönkyrä
• Ray Ruvinskiy
• Eduardo Arada
• Michael Rash

Known issues & missing features

In a beta release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

Training & Support

Need help installing, updating, validating and tuning Suricata? We have trainings coming up. Paris in July, Barcelona in November: see http://suricata-ids.org/training/

For support options also see http://suricata-ids.org/support/

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

The OISF development team is pleased to announce Suricata 2.0.8. This release fixes a number of important issues in the 2.0 series.

The most important issue is a bug in the DER parser which is used to decode SSL/TLS certificates could crash Suricata. This issue was reported by Kostya Kortchinsky of the Google Security Team and was fixed by Pierre Chifflier of ANSSI.

Those processing large numbers of (untrusted) pcap files need to update as a malformed pcap could crash Suricata. Again, credits go to Kostya Kortchinsky.

A number of other issues were fixed. Upgrading is highly recommended.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0.8.tar.gz

We have a new release key (the previous expired): http://www.openinfosecfoundation.org/download/OISF.pub (00C1B70D)

Changes

• Bug #1450: tls parsing issue
• Bug #1460: pcap parsing issue
• Bug #1461: potential deadlock
• Bug #1404: Alert-Debuglog not being rotated on SIGHUP
• Bug #1420: inverted matching on incomplete session
• Bug #1462: various issues in rule and yaml parsing

Security

The TLS/DER parsing issue has CVE-2015-0971 assigned to it.

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

• Kostya Kortchinsky of the Google Security Team
• Pierre Chifflier of ANSSI
• Sundar Jeyaraman of FireEye
• Darien Huss — Emerging Threats
• Alexander Gozman
• AFL project
• Coverity Scan

Known issues & missing features

If you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

Training & Support

Need help installing, updating, validating and tuning Suricata? We have trainings coming up. Paris in July, Barcelona in November: see http://suricata-ids.org/training/

For support options also see http://suricata-ids.org/support/

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

The OISF development team is pleased to announce Suricata 2.0.8. This release fixes a number of important issues in the 2.0 series.

The most important issue is a bug in the DER parser which is used to decode SSL/TLS certificates could crash Suricata. This issue was reported by Kostya Kortchinsky of the Google Security Team and was fixed by Pierre Chifflier of ANSSI.

Those processing large numbers of (untrusted) pcap files need to update as a malformed pcap could crash Suricata. Again, credits go to Kostya Kortchinsky.

A number of other issues were fixed. Upgrading is highly recommended.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0.8.tar.gz

We have a new release key (the previous expired): http://www.openinfosecfoundation.org/download/OISF.pub (00C1B70D)

Changes

• Bug #1450: tls parsing issue
• Bug #1460: pcap parsing issue
• Bug #1461: potential deadlock
• Bug #1404: Alert-Debuglog not being rotated on SIGHUP
• Bug #1420: inverted matching on incomplete session
• Bug #1462: various issues in rule and yaml parsing

Security

The TLS/DER parsing issue has CVE-2015-0971 assigned to it.

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

• Kostya Kortchinsky of the Google Security Team
• Pierre Chifflier of ANSSI
• Sundar Jeyaraman of FireEye
• Darien Huss — Emerging Threats
• Alexander Gozman
• AFL project
• Coverity Scan

Known issues & missing features

If you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

Training & Support

Need help installing, updating, validating and tuning Suricata? We have trainings coming up. Paris in July, Barcelona in November: see http://suricata-ids.org/training/

For support options also see http://suricata-ids.org/support/

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

There is a very useful command available to Suricata IDS/IPS/NSM :

suricata --dump-config

The command above will dump all the config parameters and their respective values that are loaded by Suricata from the config file. You can run the command in any case - it does not matter if Suricata is running or not.

There is a peculiarity however. Sometimes people would think that the command(above) would dump the currently loaded config values by Suricata.... in some case it will and in some cases it will not.

So what does it depend on?.... simple:

suricata --dump-config

will dump the config settings that are loaded (or will be loaded) by Suricata by default from
/etc/suricata/suricata.yaml

So if you are running Suricata with a config file called suricata-test.yaml (or suricata.yaml located in a different directory) - you will not see those settings...unless you specify that config file in particular:

suricata --dump-config -c /etc/suricata/suricata-test.yaml

Here is a real case example.
I run Suricata for a specific test where I had specified the defrag memcap to be 512mb :

defrag:
  memcap: 512mb
  hash-size: 65536
  trackers: 65535 # number of defragmented flows to follow
  max-frags: 65535 # number of fragments to keep (higher than trackers)
  prealloc: yes
  timeout: 60

Suricata up and running:

root@LTS-64-1:~/Work # ps aux |grep suricata
root      8109  2.3  7.6 878444 308372 pts/6   Sl+  12:45   1:02 suricata -c /etc/suricata/suricata-test.yaml --af-packet=eth0 -v
root@LTS-64-1:~/Work #

And the peculiarity that this blogpost is trying to emphasize on about :

root@LTS-64-1:~/Work # suricata --dump-config  |grep defrag.memcap
defrag.memcap = 32mb
root@LTS-64-1:~/Work # suricata --dump-config -c /etc/suricata/suricata-test.yaml |grep defrag.memcap
defrag.memcap = 512mb
root@LTS-64-1:~/Work #

suricata --dump-config dumps the settings loaded(or to be loaded) from the default location /etc/suricata/suricata.yaml if you are running suricata with a yaml config with a different name than the default or with a different location that the default - in order to get those settings - you need to specify that particular yaml location, like so:

suricata --dump-config -c /etc/local/some_test_dir/suricata/suricata-test.yaml


Thanks

related article:
http://pevma.blogspot.se/2014/02/suricata-override-config-parameters-on.html

The Open Information Security Foundation is thrilled to announce the new OISF Board of Directors:

Welcoming back returning board members: Randy Caldejon, Alexandre Dulaunoy, and Ken Steele.

And join us in welcoming new board members: Leonard Jacobs, Christian Kreibich, Eric Meyer, and Amar Rathore.

This is an important time for OISF, Suricata, and open source technologies — we are excited to be working with such an esteemed board over the next 2 years.  Thanks goes out to the OISF community for casting their votes!

We are excited to announce our first annual OISF / Suricata User Conference happening this November in Barcelona, Spain!

Join us for an exciting two days of Suricata and IDS/IPS development talks, brainstorming sessions, and amazing speakers. You can also attend a 2-day Suricata training event prior to the conference to make it a full-week of learning!

The conference is FREE and open to the public – however, we ask that people register via our events website so we can ensure to accommodate everyone in regards to space and lunches. Visit: http://www.oisfevents.net.

NOTE: the 2-day Suricata training during this week is a paid event and space will fill up – so register at https://suricata-2day-barcelona.eventbrite.com

We look forward to seeing you there! As always if you have questions, contact us at info@oisf.net.

The OISF Team

Suricata IDS/IPS/NSM also allows you to do application layer anomaly  detection.
I started talking to inliniac about protocol anomaly detection rules one day on the Suricata IRC chat room...which evolved more into a discussion resulting in us updating the rule sets with some examples of how to do that.

Below are a few examples for rules usage:

HTTP

alert tcp any any -> any ![80,8080] (msg:"SURICATA HTTP not tcp port 80, 8080"; flow:to_server; app-layer-protocol:http; sid:2271001; rev:1;)

The above rule finds http traffic that is not using dest port 80 or 8080.

alert tcp any any -> any 80 (msg:"SURICATA Port 80 but not HTTP"; flow:to_server; app-layer-protocol:!http; sid:2271002; rev:1;)

The above rule is kind of the reverse of the previous one - it will alert if the tcp traffic with destination port 80 is not http.

Here is another example

TLS

alert tcp any any -> any 443 (msg:"SURICATA Port 443 but not TLS"; flow:to_server; app-layer-protocol:!tls; sid:2271003; rev:1;)

HTTPS

Detecting HTTP traffic over HTTPS port -

alert http any any -> any 443 (msg:"SURICATA HTTP clear text on port 443"; flow:to_server; app-layer-protocol:http; sid:2271019; rev:1;)

You can find the full ruleset (open source and free to use) with examples for HTTP, HTTPS, TLS, FTP, SMTP, SSH, IMAP, SMB, DCERPC, DNS, MODBUS application layer anomaly  detection  here:

https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Protocol_Anomalies_Detection

The Open Information Security Foundation (OISF) is conducting its annual online elections to fill 7 positions on the OISF board of directors.  Board members serve a two year term, therefore, current board members along with new nominees are included on this year’s ballot.

The upcoming OISF board will consist of 10 board members in total:  7 elected directors, President of OISF, Matt Jonkman, General Manager of OISF, Kelley Misata, and Suricata’s Founder and Lead Developer, Victor Julien.

Each nominee has provided a brief summary highlighting their industry experience and their passion for OISF; please take a minute to read about each of our distinguished nominees and to cast your votes NOW!

Simply follow this link:  https://www.surveymonkey.com/s/Z2L6GXZ

Polls will close Wednesday, April 15, 2015 with the new OISF Board announced on Thursday, April 16, 2015.

Best of luck and thanks goes out to all of our nominees! Questions regarding elections can be sent to info@oisf.net.

Thank you,

The OISF Team

The Open Information Security Foundation (OISF) is conducting its annual online elections to fill 7 positions on the OISF board of directors.  Board members serve a two year term, therefore, current board members along with new nominees are included on this year's ballot.

The upcoming OISF board will consist of 10 board members in total:  7 elected directors, President of OISF, Matt Jonkman, General Manager of OISF, Kelley Misata, and Suricata's Founder and Lead Developer, Victor Julien.

Each nominee has provided a brief summary highlighting their industry experience and their passion for OISF; please take a minute to read about each of our distinguished nominees and to cast your votes NOW!

Simply follow this link:  https://www.surveymonkey.com/s/Z2L6GXZ

Polls will close Wednesday, April 15, 2015 with the new OISF Board announced on Thursday, April 16, 2015.

Best of luck and thanks goes out to all of our nominees! Questions regarding elections can be sent to info@oisf.net.

Thank you,

The OISF Team

A short time ago we announced the first edition of the Suricata Developer training. I thought it be a good time to explain what we have in mind.

First, we’re planning to make this an annual event. I’m very excited about this. It should be great fun to have a week of development related discussions. I’m sure we can all learn a great deal, and share lots of ideas. Of course the social part is a nice addition.

Also, this is a great “stick behind the door” (as we’d say in Dutch) to finally get some much needed dev docs done, including architecture overview diagrams, etc.
What will we be teaching:

1. General Suricata development basics: everything from git, how to QA, unittests, debugging, etc.
2. Architecture overview
   1. API’s
   2. threading
   3. packets, flows, detection and output
3. Extending Suricata — the beef of the training:
   1. packet decoders and detection plugins
   2. app layer protocol detection, parsing, state keeping
   3. app layer detection engine integration
   4. adding logging modules

Structure

Each day will start with lectures on each of the topics. You will get an overview of the API, learn about performance aspects, how threading comes into play, etc.

After this there will be assignments/challenges to apply the newly learned skills. This should be very interactive with lots of room for questions and discussions. We’ll be providing various assignments for multiple skill levels.

Teachers

From OISF we will have lead developer Victor Julien, core developers Eric Leblond and Jason Ish in the room to give the lectures and help answer questions. Also present to assist in general Suricata related questions: Matt Jonkman and Peter Manev.

Participants

So who is this for? We target people who what to learn how to extend Suricata. If you want to add protocols, detection options or maybe new output methods, then this is for you. Or maybe you want to be a ‘core’ developer on Suricata. Then this would be an excellent start of getting into it.

Skill-set:
– advanced C experience
– Linux Experience
– Network / Security Experience
– Basic Suricata End User Experience
=> for testing your code

If you’re not yet experienced with running Suricata, we would like to suggest attending a user training first. If you book your dev seat first hit us up for discount code on one of the training sessions. We’re planning one in Europe before summer, exact date and location is to be announced.

Money

We’ll be charging for the trainings. The revenue is used to cover the cost of the event itself (travel, hotels, etc). Whats left goes into the foundations general development budget. So by attending the training you will support Suricata’s development.

Location

This first edition is generously hosted by Napatech in Copenhagen, Denmark.
If you want to book, please do so through this Eventbrite link:
https://www.eventbrite.com/e/5-day-suricata-developer-training-in-copenhagen-denmark-open-to-the-public-tickets-15667305332
Questions and general feedback and thoughts are welcome!

The Open Information Security Foundation is preparing hold the biennial (every two years) Board of Director elections and are putting out a call for nominations. We are anticipating the next several years to be both exciting and critical for OISF and Suricata, therefore, we are looking for candidates passionate about security and open source communities willing to serve as advisors on our board of directors. Your voice has a direct impact on future of OISF and Suricata – join us! The call for nominations begins today until March 31, 2015.

Read more at the OISF website.

Suricata 2.0.7 was recently released:
http://www.openinfosecfoundation.org/index.php/component/content/article/1-latest-news/204-suricata-207-available

I've packaged Suricata 2.0.7 and it has been tested by David Zawdie (thanks!).

The new package version is:
securityonion-suricata - 2.0.7-0ubuntu0securityonion1

Issues Resolved

Issue 695: Suricata 2.0.7
https://code.google.com/p/security-onion/issues/detail?id=695

Updating
The new package is now available in our stable repo.  Please see the following page for full update instructions:
https://code.google.com/p/security-onion/wiki/Upgrade

This update will back up each of your existing suricata.yaml files to suricata.yaml.bak.  You'll then need to do the following:

• re-apply any local customizations to suricata.yaml
• update ruleset and restart Suricata as follows:
• sudo rule-update

Feedback
If you have any questions or problems, please use our security-onion mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Training
Need training?  We have 4-day classes coming up in Atlanta, Seattle, and Houston.  Please see:
http://securityonionsolutions.com

Commercial Support
Need commercial support?  Please see:
http://securityonionsolutions.com

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

Thanks!

The OISF development team is pleased to announce Suricata 2.0.7. This release fixes a number of important issues in the 2.0 series.

Two major issues. The first was brought to our attention by the Yahoo Pentest Team. It’s a parsing issue in the DCERPC parser that can happen when Suricata runs out of memory. The exact scope of the problem isn’t clear, but it could certainly lead to crashes. RCE might theoretically be possible but looks like it’s very hard.

The second issue was reported by Darien Huss of Emerging Threats. This is technically a libhtp issue, but it affects Suricata detection and logging. Certain characters in the URI could confuse the parsing of the HTTP request line, leading to possible detection bypass for ‘http_uri’ and to incomplete logging of the URI. Libhtp 0.5.17 has been released to address this and is bundled in 2.0.7.

Other than that a bunch of improvements and fixes. It should work again on CentOS 5. Midstream TCP was improved and some performance optimizations for HTTP proxy traffic were made.

Upgrading is highly recommended.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0.7.tar.gz

Changes

• Bug #1385: DCERPC traffic parsing issue
• Bug #1391: http uri parsing issue
• Bug #1383: tcp midstream window issue
• Bug #1318: A thread-sync issue in streamTCP
• Bug #1375: Regressions in list keywords option
• Bug #1387: pcap-file hangs on systems w/o atomics support
• Bug #1395: dump-counters unix socket command failure
• Optimization #1376: file list is not cleaned up

Security

The DCERPC parsing issue has CVE-2015-0928 assigned to it.

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

• The Yahoo Pentest Team
• Darien Huss — Emerging Threats
• FireEye
• Dennis Lee

Known issues & missing features

If you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

The OISF development team is pleased to announce Suricata 2.0.7. This release fixes a number of important issues in the 2.0 series.

Two major issues. The first was brought to our attention by the Yahoo Pentest Team. It’s a parsing issue in the DCERPC parser that can happen when Suricata runs out of memory. The exact scope of the problem isn’t clear, but it could certainly lead to crashes. RCE might theoretically be possible but looks like it’s very hard.

The second issue was reported by Darien Huss of Emerging Threats. This is technically a libhtp issue, but it affects Suricata detection and logging. Certain characters in the URI could confuse the parsing of the HTTP request line, leading to possible detection bypass for ‘http_uri’ and to incomplete logging of the URI. Libhtp 0.5.17 has been released to address this and is bundled in 2.0.7.

Other than that a bunch of improvements and fixes. It should work again on CentOS 5. Midstream TCP was improved and some performance optimizations for HTTP proxy traffic were made.

Upgrading is highly recommended.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0.7.tar.gz

Changes

• Bug #1385: DCERPC traffic parsing issue
• Bug #1391: http uri parsing issue
• Bug #1383: tcp midstream window issue
• Bug #1318: A thread-sync issue in streamTCP
• Bug #1375: Regressions in list keywords option
• Bug #1387: pcap-file hangs on systems w/o atomics support
• Bug #1395: dump-counters unix socket command failure
• Optimization #1376: file list is not cleaned up

Security

The DCERPC parsing issue has CVE-2015-0928 assigned to it.

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

• The Yahoo Pentest Team
• Darien Huss — Emerging Threats
• FireEye
• Dennis Lee

Known issues & missing features

If you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Setting up (configuring) the right MTU (maximum transmission unit) size when running Suricata IDS/IPS.

Sometimes you can end up in a situation as follows :

capture.kernel_packets    | AFPacketeth12              | 1143428204
decoder.pkts                    | AFPacketeth12             | 1143428143
decoder.invalid                | AFPacketeth12              | 416889536

a whole lot of  decoder.invalid. Not good. What could be the reason for that? One thing you should check right away is the MTU of the traffic that is being mirrored.

What does it mean? Well there is the MTU that you set up on the server that you run Suricata on and there is the MTU that is present in the "mirrored" traffic.

What is the difference?Why should it matter?
It matters because if not set correct  it will result in a lot of decoder.invalids (dropped by Suricata) and you will be missing on a lot of traffic inspection.
Example: if  on the sniffing interface that you run Suricata on has a MTU set as 1500  and in the traffic that you mirror you have jumbo frames (MTU 9000) - most likely your decoder.invalids will show a whole lotta love in your stats.log.

How can you adjust the MTU on the interface (NIC) ? (example)
First a have  look what is the current value:

ifconfig eth0

then adjust it

ifconfig eth0 mtu 1514

By the way - what could be the max size of the MTU (and what sizes there are in general)  -
(short answer - 9216)


This is the easy part :). There are situations where you do not know what is the MTU of the "mirrored" traffic. There is a few ways to find this  - ask the network team/guy, make a phone call or two, start manually testing and setting it on the NIC to find a middle ground ....however you can also make use of the procedure shown below (in order to get the byte size of the MTU):


On your Server/Sensor
1)
Stop Suricata.

2)
Change the MTU to 9216
(the interface that Suri is sniffing on)

example - ifconfig eth0 mtu 9216

(non boot persistent)

3)
install tcpstat - if you do not have it

apt-get install tcpstat

5)
run the following (substitute the interface name with yours - that Suri is sniffing on)

tcpstat -i eth0 -l -o "Time:%S\tn=%n\tavg=%a\tstddev=%d\tbps=%b\tMaxPacketSize=%M\n"  5

8)
Let it run for  a while - lets say 1 hr. Have a look at the decoder.invalid stats in stats.log

NOTE: Do NOT just set the MTU to 9216 directly ("just to be on the safe side"). Only set it that high if needed !!

NOTE: This example below is not using the "-l" option of tcpstat as denoted in point 5) above - look at man tcpstat for more info



(tested on Ubuntu/Debian)
That's all ....feedback welcome.

I’ve been invited by SaLUG to Lecce to give some talks during their Geek Evening. I’ve done a talk on nftables and one of suricata.

The nftables talk was about the motivation behind the change from iptables. Here are the slides: Nftables

The talk on Suricata was explaining the different feature of Suricata and was showing how I’ve used it to make a study of SSH bruteforce. Here are the slides: Suricata, Netfilter and the PRC.

Thanks a lot to Giuseppe Longo, Luca Greco and all the SaLUG team, you have been wonderful hosts!

The OISF development team is proud to announce Suricata 2.1beta3. This is the third beta release for the upcoming 2.1 version. It should be considered a development snapshot for the 2.1 branch.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.1beta3.tar.gz

New Features

• Feature #1309: Lua support for Stats output
• Feature #1310: Modbus parsing and matching

Improvements

• Optimization #1339: flow timeout optimization
• Optimization #1371: mpm optimization
• Feature #1317: Lua: Indicator for end of flow
• Feature #1333: unix-socket: allow (easier) non-root usage
• Feature #1261: Request for Additional Lua Capabilities

Bugs

• Bug #977: WARNING on empty rules file is fatal (should not be)
• Bug #1184: pfring: cppcheck warnings
• Bug #1321: Flow memuse bookkeeping error
• Bug #1327: pcre pkt/flowvar capture broken for non-relative matches (master)
• Bug #1332: cppcheck: ioctl
• Bug #1336: modbus: CID 1257762: Logically dead code (DEADCODE)
• Bug #1351: output-json: duplicate logging (2.1.x)
• Bug #1354: coredumps on quitting on OpenBSD
• Bug #1355: Bus error when reading pcap-file on OpenBSD
• Bug #1363: Suricata does not compile on OS X/Clang due to redefinition of string functions (2.1.x)
• Bug #1365: evasion issues (2.1.x)

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

• Ken Steele — Tilera/EZchip
• David Diallo
• Duarte Silva
• Giuseppe Longo
• Jason Ish
• Travis Green — Emerging Threats

Known issues & missing features

In a beta release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

The OISF development team is proud to announce Suricata 2.1beta3. This is the third beta release for the upcoming 2.1 version. It should be considered a development snapshot for the 2.1 branch.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.1beta3.tar.gz

New Features

• Feature #1309: Lua support for Stats output
• Feature #1310: Modbus parsing and matching

Improvements

• Optimization #1339: flow timeout optimization
• Optimization #1371: mpm optimization
• Feature #1317: Lua: Indicator for end of flow
• Feature #1333: unix-socket: allow (easier) non-root usage
• Feature #1261: Request for Additional Lua Capabilities

Bugs

• Bug #977: WARNING on empty rules file is fatal (should not be)
• Bug #1184: pfring: cppcheck warnings
• Bug #1321: Flow memuse bookkeeping error
• Bug #1327: pcre pkt/flowvar capture broken for non-relative matches (master)
• Bug #1332: cppcheck: ioctl
• Bug #1336: modbus: CID 1257762: Logically dead code (DEADCODE)
• Bug #1351: output-json: duplicate logging (2.1.x)
• Bug #1354: coredumps on quitting on OpenBSD
• Bug #1355: Bus error when reading pcap-file on OpenBSD
• Bug #1363: Suricata does not compile on OS X/Clang due to redefinition of string functions (2.1.x)
• Bug #1365: evasion issues (2.1.x)

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

• Ken Steele — Tilera/EZchip
• David Diallo
• Duarte Silva
• Giuseppe Longo
• Jason Ish
• Travis Green — Emerging Threats

Known issues & missing features

In a beta release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

We have updated the official Ubuntu PPA to Suricata 2.0.6. To use this PPA read our docs here.

To install Suricata through this PPA, enter:
sudo add-apt-repository ppa:oisf/suricata-stable
sudo apt-get update
sudo apt-get install suricata

If you’re already using this PPA, updating is as simple as:
sudo apt-get update && sudo apt-get upgrade

The PPA Ubuntu packages have IPS mode through NFQUEUE enabled.

The Windows MSI installer of the Suricata 2.0.6 release is now available.

Download it here: Suricata-2.0.6-1-32bit.msi

After downloading, double click the file to launch the installer. The installer is now signed.

If you have a previous version installed, please remove that first.

Suricata 2.0.6 was recently released:
http://suricata-ids.org/2015/01/15/suricata-2-0-6-available/

I've packaged Suricata 2.0.6 and it has been tested by David Zawdie (thanks!).

The new package version is:
securityonion-suricata - 2.0.6-0ubuntu0securityonion1

Issues Resolved

Issue 673: Suricata 2.0.6
https://code.google.com/p/security-onion/issues/detail?id=673

Updating
The new package is now available in our stable repo.  Please see the following page for full update instructions:
https://code.google.com/p/security-onion/wiki/Upgrade

This update will back up each of your existing suricata.yaml files to suricata.yaml.bak.  You'll then need to do the following:

• re-apply any local customizations to suricata.yaml
• update ruleset and restart Suricata as follows:
  sudo rule-update

Feedback
If you have any questions or problems, please use our security-onion mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Commercial Support
Need training and/or commercial support?  Please see:
http://securityonionsolutions.com

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

Thanks!

The OISF development team is pleased to announce Suricata 2.0.6. This release fixes a number of important issues in the 2.0 series. The most important part is the fixing of evasion issues, therefore upgrading is highly recommended!

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0.6.tar.gz

Changes

• Bug #1364: evasion issues
• Bug #1337: output-json: duplicate logging
• Bug #1325: tls detection leads to tcp stream reassembly sequence gaps (IPS)
• Bug #1192: Suricata does not compile on OS X/Clang due to redefinition of string functions
• Bug #1183: pcap: cppcheck warning

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

• Martin Küchler

Known issues & missing features

If you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

The OISF development team is pleased to announce Suricata 2.0.6. This release fixes a number of important issues in the 2.0 series. The most important part is the fixing of evasion issues, therefore upgrading is highly recommended!

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0.6.tar.gz

Changes

• Bug #1364: evasion issues
• Bug #1337: output-json: duplicate logging
• Bug #1325: tls detection leads to tcp stream reassembly sequence gaps (IPS)
• Bug #1192: Suricata does not compile on OS X/Clang due to redefinition of string functions
• Bug #1183: pcap: cppcheck warning

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

• Martin Küchler

Known issues & missing features

If you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

Thanks to the hard work of Arturo Borrero Gonzalez, Suricata has just been added to the Debian ‘backports’ repository. This allows users of Debian stable to run up to date versions of Suricata.

The ‘Backports’ repository makes the Suricata and libhtp packages from Debian Testing available to ‘stable’ users. As ‘testing’ is currently in a freeze, it may take a bit of time before 2.0.5 and libhtp 0.5.16 appear.

Anyway, here is how to use it.

Install

First add backports repo to your sources:

# echo "deb http://http.debian.net/debian wheezy-backports main" > /etc/apt/sources.list.d/backports.list
# apt-get update

As explained here http://backports.debian.org/Instructions/, this will not affect your normal packages.

To prove this, check:

# apt-get install suricata -s
Conf libhtp1 (0.2.6-2 Debian:7.7/stable [amd64])
Conf suricata (1.2.1-2 Debian:7.7/stable [amd64])

Not what we want, as that is still the old version.

To install Suricata from backports, we need to specify the repo:

# apt-get install -t wheezy-backports suricata -s
Conf libhtp1 (0.5.15-1~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Conf suricata (2.0.4-1~bpo70+1 Debian Backports:/wheezy-backports [amd64])

Let’s do it!

# apt-get install -t wheezy-backports suricata
...
Setting up suricata (2.0.4-1~bpo70+1) ...
[FAIL] suricata disabled, please adjust the configuration to your needs ... failed!
[FAIL] and then set RUN to 'yes' in /etc/default/suricata to enable it. ... failed!

Suricata 2.0.4 is now installed, but it’s not yet running.
To see what features have been compiled in, run:

# suricata --build-info
This is Suricata version 2.0.4 RELEASE

Suricata Configuration:
  AF_PACKET support:                       yes
  PF_RING support:                         no
  NFQueue support:                         yes
  NFLOG support:                           no
  IPFW support:                            no
  DAG enabled:                             no
  Napatech enabled:                        no
  Unix socket enabled:                     yes
  Detection enabled:                       yes

  libnss support:                          yes
  libnspr support:                         yes
  libjansson support:                      yes
  Prelude support:                         yes
  PCRE jit:                                yes
  LUA support:                             yes
  libluajit:                               yes
  libgeoip:                                no
  Non-bundled htp:                         yes
  Old barnyard2 support:                   no
  CUDA enabled:                            no

  Suricatasc install:                      yes

It has Luajit enabled, libjansson for the JSON output, NFQ and AF_PACKET IPS modes, NSS for MD5 checksums and unix sockets. Quite a good feature set.

Run

To get it running, we need a few more steps:

Edit /etc/default/suricata:

1. Change RUN=no to RUN=yes
2. Change LISTENMODE to “af-packet”:

Now lets start it.

# service suricata start
Starting suricata in IDS (af-packet) mode... done.

And confirm that it’s running.

# ps aux|grep suricata
root     20295  1.8  4.1 200212 42544 ?        Ssl  00:50   0:00 /usr/bin/suricata -c /etc/suricata/suricata-debian.yaml --pidfile /var/run/suricata.pid --af-packet -D

Check if we’re seeing traffic:

# tail /var/log/suricata/stats.log -f|grep capture
capture.kernel_packets    | RxAFPeth01                | 406
capture.kernel_drops      | RxAFPeth01                | 0
capture.kernel_packets    | RxAFPeth11                | 0
capture.kernel_drops      | RxAFPeth11                | 0
capture.kernel_packets    | RxAFPeth01                | 411
capture.kernel_drops      | RxAFPeth01                | 0
capture.kernel_packets    | RxAFPeth11                | 0
capture.kernel_drops      | RxAFPeth11                | 0
capture.kernel_packets    | RxAFPeth01                | 417
capture.kernel_drops      | RxAFPeth01                | 0
capture.kernel_packets    | RxAFPeth11                | 0
capture.kernel_drops      | RxAFPeth11                | 0
capture.kernel_packets    | RxAFPeth01                | 587
capture.kernel_drops      | RxAFPeth01                | 0
capture.kernel_packets    | RxAFPeth11                | 0
capture.kernel_drops      | RxAFPeth11                | 0
capture.kernel_packets    | RxAFPeth01                | 593
capture.kernel_drops      | RxAFPeth01                | 0
capture.kernel_packets    | RxAFPeth11                | 0
capture.kernel_drops      | RxAFPeth11                | 0

Logging

As the init script starts Suricata in daemon mode, we need to enable logging to file:

Edit /etc/suricata/suricata-debian.yaml and go to the “logging:” section, there change the “file” portion to look like:

  - file:
      enabled: yes
      filename: /var/log/suricata/suricata.log

Note: in the YAML indentation matters, so make sure it’s exactly right.

Rules

Oinkmaster is automatically installed, so lets use that:

First create the rules directory:

mkdir /etc/suricata/rules/

Open /etc/oinkmaster.conf in your editor and add:

url = https://rules.emergingthreats.net/open/suricata-2.0/emerging.rules.tar.gz

Then run:

# oinkmaster -C /etc/oinkmaster.conf -o /etc/suricata/rules
Loading /etc/oinkmaster.conf
Downloading file from https://rules.emergingthreats.net/open/suricata-2.0/emerging.rules.tar.gz... done.
...

Edit /etc/suricata/suricata-debian.yaml and change “default-rule-path” to:

default-rule-path: /etc/suricata/rules

Finally, restart to load the new rules:

# service suricata restart

Validate

Now that Suricata is running with rules, lets see if it works:

# wget http://www.testmyids.com
--2015-01-08 01:21:30--  http://www.testmyids.com/
Resolving www.testmyids.com (www.testmyids.com)... 82.165.177.154

This should trigger a specific rule:

# tail /var/log/suricata/fast.log 
01/08/2015-01:21:30.870346  [**] [1:2100498:7] GPL ATTACK_RESPONSE id check returned root [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 82.165.177.154:80 -> 192.168.122.181:59190

Success!

Thanks

Thanks to Arturo Borrero Gonzalez for taking on this work for us. Also many thanks for Pierre Chifflier for maintaining the Suricata and libhtp packages in Debian.