Planet Suricata

August 30, 2015

Peter Manev

Suricata IDS/IPS - TCP segment pool size preallocation


In the default suricata.yaml stream section we have:
stream:
  memcap: 32mb
  checksum-validation: no      # reject wrong csums
  async-oneside: true
  midstream: true
  inline: no                  # auto will use inline mode in IPS mode, yes or no set it statically
  reassembly:
    memcap: 64mb
    depth: 1mb                  # reassemble 1mb into a stream
    toserver-chunk-size: 2560
    toclient-chunk-size: 2560
    randomize-chunk-size: yes
    #randomize-chunk-range: 10
    #raw: yes
    #chunk-prealloc: 250
    #segments:
    #  - size: 4
    #    prealloc: 256
    #  - size: 16
    #    prealloc: 512
    #  - size: 112
    #    prealloc: 512
    #  - size: 248
    #    prealloc: 512
    #  - size: 512
    #    prealloc: 512
    #  - size: 768
    #    prealloc: 1024
    #  - size: 1448
    #    prealloc: 1024
    #  - size: 65535
    #    prealloc: 128


So what are these segment preallocations for?
Let's have a look. When Suricata exits (or kill -15 PidOfSuricata) it produces a lot of useful statistics in the suricata.log file (you can enable that from the suricata.yaml and use the "-v" switch (verbose) when starting Suricata):
The example below is for exit stats.
   
tail -20 StatsByDate/suricata-2014-06-01.log
[24344] 1/6/2014 -- 01:45:52 - (source-af-packet.c:1810) <Info> (ReceiveAFPThreadExitStats) -- (AFPacketeth314) Packets 7317661624, bytes 6132661347126
[24344] 1/6/2014 -- 01:45:52 - (stream-tcp.c:4643) <Info> (StreamTcpExitPrintStats) -- Stream TCP processed 3382528539 TCP packets
[24345] 1/6/2014 -- 01:45:52 - (source-af-packet.c:1807) <Info> (ReceiveAFPThreadExitStats) -- (AFPacketeth315) Kernel: Packets 8049357450, dropped 352658715
[24345] 1/6/2014 -- 01:45:52 - (source-af-packet.c:1810) <Info> (ReceiveAFPThreadExitStats) -- (AFPacketeth315) Packets 7696486934, bytes 6666577738944
[24345] 1/6/2014 -- 01:45:52 - (stream-tcp.c:4643) <Info> (StreamTcpExitPrintStats) -- Stream TCP processed 3357321803 TCP packets
[24346] 1/6/2014 -- 01:45:52 - (source-af-packet.c:1807) <Info> (ReceiveAFPThreadExitStats) -- (AFPacketeth316) Kernel: Packets 7573051188, dropped 292897219
[24346] 1/6/2014 -- 01:45:52 - (source-af-packet.c:1810) <Info> (ReceiveAFPThreadExitStats) -- (AFPacketeth316) Packets 7279948375, bytes 6046562324948
[24346] 1/6/2014 -- 01:45:52 - (stream-tcp.c:4643) <Info> (StreamTcpExitPrintStats) -- Stream TCP processed 3454330660 TCP packets
[24329] 1/6/2014 -- 01:45:53 - (stream-tcp-reassemble.c:502) <Info> (StreamTcpReassembleFree) -- TCP segment pool of size 4 had a peak use of 60778 segments, more than the prealloc setting of 256
[24329] 1/6/2014 -- 01:45:53 - (stream-tcp-reassemble.c:502) <Info> (StreamTcpReassembleFree) -- TCP segment pool of size 16 had a peak use of 314953 segments, more than the prealloc setting of 512
[24329] 1/6/2014 -- 01:45:53 - (stream-tcp-reassemble.c:502) <Info> (StreamTcpReassembleFree) -- TCP segment pool of size 112 had a peak use of 113739 segments, more than the prealloc setting of 512
[24329] 1/6/2014 -- 01:45:53 - (stream-tcp-reassemble.c:502) <Info> (StreamTcpReassembleFree) -- TCP segment pool of size 248 had a peak use of 17893 segments, more than the prealloc setting of 512
[24329] 1/6/2014 -- 01:45:53 - (stream-tcp-reassemble.c:502) <Info> (StreamTcpReassembleFree) -- TCP segment pool of size 512 had a peak use of 31787 segments, more than the prealloc setting of 512
[24329] 1/6/2014 -- 01:45:53 - (stream-tcp-reassemble.c:502) <Info> (StreamTcpReassembleFree) -- TCP segment pool of size 768 had a peak use of 30769 segments, more than the prealloc setting of 1024
[24329] 1/6/2014 -- 01:45:53 - (stream-tcp-reassemble.c:502) <Info> (StreamTcpReassembleFree) -- TCP segment pool of size 1448 had a peak use of 89446 segments, more than the prealloc setting of 1024
[24329] 1/6/2014 -- 01:45:53 - (stream-tcp-reassemble.c:502) <Info> (StreamTcpReassembleFree) -- TCP segment pool of size 65535 had a peak use of 81214 segments, more than the prealloc setting of 128
[24329] 1/6/2014 -- 01:45:53 - (stream.c:182) <Info> (StreamMsgQueuesDeinit) -- TCP segment chunk pool had a peak use of 20306 chunks, more than the prealloc setting of 250
[24329] 1/6/2014 -- 01:45:53 - (host.c:245) <Info> (HostPrintStats) -- host memory usage: 390144 bytes, maximum: 16777216
[24329] 1/6/2014 -- 01:45:55 - (detect.c:3890) <Info> (SigAddressCleanupStage1) -- cleaning up signature grouping structure... complete
[24329] 1/6/2014 -- 01:45:55 - (util-device.c:185) <Notice> (LiveDeviceListClean) -- Stats for 'eth3':  pkts: 124068935209, drop: 5245430626 (4.23%), invalid chksum: 0


Notice all the "TCP segment pool" messages. This is the actual tcp segment pool reassembly stats for that period of time that Suricata was running. We could adjust accordingly in the suricata.yaml (as compared to the default settings above)
   
stream:
  memcap: 14gb
  checksum-validation: no      # reject wrong csums
  midstream: false
  prealloc-sessions: 375000
  inline: no                  # auto will use inline mode in IPS mode, yes or no set it statically
  reassembly:
    memcap: 20gb
    depth: 12mb                  # reassemble 1mb into a stream
    toserver-chunk-size: 2560
    toclient-chunk-size: 2560
    randomize-chunk-size: yes
    #randomize-chunk-range: 10
    raw: yes
    chunk-prealloc: 20556
    segments:
      - size: 4
        prealloc: 61034
      - size: 16
        prealloc: 315465
      - size: 112
        prealloc: 114251
      - size: 248
        prealloc: 18405
      - size: 512
        prealloc: 30769
      - size: 768
        prealloc: 31793
      - size: 1448
        prealloc: 90470
      - size: 65535
        prealloc: 81342
   


   
The total RAM (reserved) consumption for these preallocations (from the stream.reassembly.memcap value ) would be:

4*61034 + 16*315465 + 112*114251 + 248*18405 + 512*30769 + 768*31793 + 1448*90470 + 65535*81342 
= 5524571410 bytes
= 5.14 GB of RAM

So we could preallocate the tcp segments and take the Suricata tuning even a step further and improve performance as well.

So now when you start Suricata with the "-v" switch in your suricata.log with this specific set up described above you should see something like:
...
...
[30709] 1/6/2014 -- 12:17:34 - (stream-tcp-reassemble.c:425) <Info> (StreamTcpReassemblyConfig) -- segment pool: pktsize 4, prealloc 61034
[30709] 1/6/2014 -- 12:17:34 - (stream-tcp-reassemble.c:425) <Info> (StreamTcpReassemblyConfig) -- segment pool: pktsize 16, prealloc 315465
[30709] 1/6/2014 -- 12:17:34 - (stream-tcp-reassemble.c:425) <Info> (StreamTcpReassemblyConfig) -- segment pool: pktsize 112, prealloc 114251
[30709] 1/6/2014 -- 12:17:34 - (stream-tcp-reassemble.c:425) <Info> (StreamTcpReassemblyConfig) -- segment pool: pktsize 248, prealloc 18405
[30709] 1/6/2014 -- 12:17:34 - (stream-tcp-reassemble.c:425) <Info> (StreamTcpReassemblyConfig) -- segment pool: pktsize 512, prealloc 30769
[30709] 1/6/2014 -- 12:17:34 - (stream-tcp-reassemble.c:425) <Info> (StreamTcpReassemblyConfig) -- segment pool: pktsize 768, prealloc 31793
[30709] 1/6/2014 -- 12:17:35 - (stream-tcp-reassemble.c:425) <Info> (StreamTcpReassemblyConfig) -- segment pool: pktsize 1448, prealloc 90470
[30709] 1/6/2014 -- 12:17:35 - (stream-tcp-reassemble.c:425) <Info> (StreamTcpReassemblyConfig) -- segment pool: pktsize 65535, prealloc 81342
[30709] 1/6/2014 -- 12:17:35 - (stream-tcp-reassemble.c:461) <Info> (StreamTcpReassemblyConfig) -- stream.reassembly "chunk-prealloc": 20556
...
...

NOTE:
Those 5.14 GB RAM in the example here will be preallocated (taken) from the stream.reassembly.memcap value. In other words it will not consume 5.14 GB of RAM more.

So be careful when setting up preallocation in order not to preallocate more of what you have.
In my case of 10Gbps suricata.yaml config I had:

stream:
  memcap: 14gb
  checksum-validation: no      # reject wrong csums
  midstream: false
  prealloc-sessions: 375000
  inline: no                  # auto will use inline mode in IPS mode, yes or no set it statically
  reassembly:
    memcap: 20gb
    depth: 12mb                  # reassemble 1mb into a stream


What this helps with is that it lowers CPU usage/contention for TCP segment allocation during reassembly - it is already preallocated and Suricata just uses it instead of creating it everytime it needs it. It also helps minimize the initial drops during startup.

Highly adaptable and  flexible.








by Peter Manev (noreply@blogger.com) at August 30, 2015 09:41 AM

Suricata - preparing 10Gbps network cards for IDPS and file extraction


OS used/tested for this tutorial - Debian Wheezy and/or Ubuntu LTS 12.0.4
With 3.2.0 and 3.5.0 kernel level respectively with Suricata 2.0dev at the moment of this writing.



This article consists of the following major 3 sections:
  • Network card drivers and tuning
  • Kernel specific tunning
  • Suricata.yaml configuration  (file extraction specific)

Network and system  tools:
apt-get install ethtool bwm-ng iptraf htop

Network card drivers and tuning

Our card is Intel 82599EB 10-Gigabit SFI/SFP+


rmmod ixgbe
sudo modprobe ixgbe FdirPballoc=3
ifconfig eth3 up
then (we disable irqbalance and make sure it does not enable itself during reboot)
 killall irqbalance
 service irqbalance stop

 apt-get install chkconfig
 chkconfig irqbalance off
Get the Intel network driver form here (we will use them in a second) - https://downloadcenter.intel.com/default.aspx

Download to your directory of choice then unzip,compile and install:

wget http://sourceforge.net/projects/e1000/files/ixgbe%20stable/3.18.7/ixgbe-3.18.7.tar.gz
tar -zxf ixgbe-3.18.7.tar.gz      
cd /home/pevman/ixgbe-3.18.7/src      
make clean && make && make install

Set irq affinity - do not forget to change eth3  below with the name of the network interface you are using:
 cd ../scripts/
 ./set_irq_affinity  eth3


 You should see something like this:
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ./set_irq_affinity  eth3
no rx vectors found on eth3
no tx vectors found on eth3
eth3 mask=1 for /proc/irq/101/smp_affinity
eth3 mask=2 for /proc/irq/102/smp_affinity
eth3 mask=4 for /proc/irq/103/smp_affinity
eth3 mask=8 for /proc/irq/104/smp_affinity
eth3 mask=10 for /proc/irq/105/smp_affinity
eth3 mask=20 for /proc/irq/106/smp_affinity
eth3 mask=40 for /proc/irq/107/smp_affinity
eth3 mask=80 for /proc/irq/108/smp_affinity
eth3 mask=100 for /proc/irq/109/smp_affinity
eth3 mask=200 for /proc/irq/110/smp_affinity
eth3 mask=400 for /proc/irq/111/smp_affinity
eth3 mask=800 for /proc/irq/112/smp_affinity
eth3 mask=1000 for /proc/irq/113/smp_affinity
eth3 mask=2000 for /proc/irq/114/smp_affinity
eth3 mask=4000 for /proc/irq/115/smp_affinity
eth3 mask=8000 for /proc/irq/116/smp_affinity
root@suricata:/home/pevman/ixgbe-3.18.7/scripts#
Now we have the latest drivers installed (at the time of this writing) and we have run the affinity script:
   *-network:1
       description: Ethernet interface
       product: 82599EB 10-Gigabit SFI/SFP+ Network Connection
       vendor: Intel Corporation
       physical id: 0.1
       bus info: pci@0000:04:00.1
       logical name: eth3
       version: 01
       serial: 00:e0:ed:19:e3:e1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi msix pciexpress vpd bus_master cap_list ethernet physical fibre
       configuration: autonegotiation=off broadcast=yes driver=ixgbe driverversion=3.18.7 duplex=full firmware=0x800000cb latency=0 link=yes multicast=yes port=fibre promiscuous=yes
       resources: irq:37 memory:fbc00000-fbc1ffff ioport:e000(size=32) memory:fbc40000-fbc43fff memory:fa700000-fa7fffff memory:fa600000-fa6fffff



We need to disable all offloading on the network card in order for the IDS to be able to see the traffic as it is supposed to be (without checksums,tcp-segmentation-offloading and such..) Otherwise your IDPS would not be able to see all "natural" network traffic the way it is supposed to and will not inspect it properly.

This would influence the correctness of ALL outputs including file extraction. So make sure all offloading features are OFF !!!

When you first install the drivers and card your offloading settings might look like this:
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -k eth3
Offload parameters for eth3:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: on
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: on
root@suricata:/home/pevman/ixgbe-3.18.7/scripts#

So we disable all of them, like so (and we load balance the UDP flows for that particular network card):

ethtool -K eth3 tso off
ethtool -K eth3 gro off
ethtool -K eth3 ufo off
ethtool -K eth3 lro off
ethtool -K eth3 gso off
ethtool -K eth3 rx off
ethtool -K eth3 tx off
ethtool -K eth3 sg off
ethtool -K eth3 rxvlan off
ethtool -K eth3 txvlan off
ethtool -N eth3 rx-flow-hash udp4 sdfn
ethtool -N eth3 rx-flow-hash udp6 sdfn
ethtool -C eth3 rx-usecs 1 rx-frames 0
ethtool -C eth3 adaptive-rx off

Your output should look something like this:
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 tso off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 gro off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 lro off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 gso off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 rx off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 tx off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 sg off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 rxvlan off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 txvlan off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -N eth3 rx-flow-hash udp4 sdfn
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -N eth3 rx-flow-hash udp6 sdfn
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -n eth3 rx-flow-hash udp6
UDP over IPV6 flows use these fields for computing Hash flow key:
IP SA
IP DA
L4 bytes 0 & 1 [TCP/UDP src port]
L4 bytes 2 & 3 [TCP/UDP dst port]

root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -n eth3 rx-flow-hash udp4
UDP over IPV4 flows use these fields for computing Hash flow key:
IP SA
IP DA
L4 bytes 0 & 1 [TCP/UDP src port]
L4 bytes 2 & 3 [TCP/UDP dst port]

root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -C eth3 rx-usecs 0 rx-frames 0
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -C eth3 adaptive-rx off

Now we doublecheck and run ethtool again to verify that the offloading is OFF:
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -k eth3
Offload parameters for eth3:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
rx-vlan-offload: off
tx-vlan-offload: off

Ring parameters on the network card:

root@suricata:~# ethtool -g eth3
Ring parameters for eth3:
Pre-set maximums:
RX:             4096
RX Mini:        0
RX Jumbo:       0
TX:            4096
Current hardware settings:
RX:             512
RX Mini:        0
RX Jumbo:       0
TX:             512


We can increase that to the max Pre-set RX:

root@suricata:~# ethtool -G eth3 rx 4096

Then we  have a look again:

root@suricata:~# ethtool -g eth3
Ring parameters for eth3:
Pre-set maximums:
RX:             4096
RX Mini:        0
RX Jumbo:       0
TX:             4096
Current hardware settings:
RX:             4096
RX Mini:        0
RX Jumbo:       0
TX:             512

Making network changes permanent across reboots


On Ubuntu for example you can do:
root@suricata:~# crontab -e

Add the following:
# add cronjob at reboot - disbale network offload
@reboot /opt/tmp/disable-network-offload.sh

and your disable-network-offload.sh script (in this case under /opt/tmp/ ) will contain the following:



#!/bin/bash
ethtool -K eth3 tso off
ethtool -K eth3 gro off
ethtool -K eth3 ufo off
ethtool -K eth3 lro off
ethtool -K eth3 gso off
ethtool -K eth3 rx off
ethtool -K eth3 tx off
ethtool -K eth3 sg off
ethtool -K eth3 rxvlan off
ethtool -K eth3 txvlan off
ethtool -N eth3 rx-flow-hash udp4 sdfn
ethtool -N eth3 rx-flow-hash udp6 sdfn
ethtool -C eth3 rx-usecs 1 rx-frames 0
ethtool -C eth3 adaptive-rx off


with:
chmod 755 disable-network-offload.sh
To make sure you have the ixgbe module always loaded at boot time you can add "ixgbe" to the  /etc/modules file.

Kernel specific tunning


Certain adjustments in parameters in the kernel can help as well :

sysctl -w net.core.netdev_max_backlog=250000
sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.rmem_default=16777216
sysctl -w net.core.optmem_max=16777216


Making kernel changes permanent across reboots


example:
echo 'net.core.netdev_max_backlog=250000' >> /etc/sysctl.conf

reload the changes:
sysctl -p

OR for all the above adjustments:

echo 'net.core.netdev_max_backlog=250000' >> /etc/sysctl.conf
echo 'net.core.rmem_max=16777216' >> /etc/sysctl.conf
echo 'net.core.rmem_default=16777216' >> /etc/sysctl.conf
echo 'net.core.optmem_max=16777216' >> /etc/sysctl.conf
sysctl -p


Suricata.yaml configuration  (file extraction specific)

As of Suricata 1.2  - it is possible to detect and extract/store over 5000 types of files from HTTP sessions.

Specific file extraction instructions can also be found in the official page documentation.

The following libraries are needed on the system running Suricata :
apt-get install libnss3-dev libnspr4-dev

Suricata also needs to be compiled with file extraction enabled (not covered here).

In short in the suriacta.yaml, those are the sections below that can be tuned/configured and affect the file extraction and logging:
(the bigger the mem values the better on a busy link )


  - eve-log:
      enabled: yes
      type: file #file|syslog|unix_dgram|unix_stream
      filename: eve.json
      # the following are valid when type: syslog above
      #identity: "suricata"
      #facility: local5
      #level: Info ## possible levels: Emergency, Alert, Critical,
                   ## Error, Warning, Notice, Info, Debug
      types:
        - alert
        - http:
            extended: yes     # enable this for extended logging information
        - dns
        - tls:
            extended: yes     # enable this for extended logging information
        - files:
            force-magic: yes   # force logging magic on all logged files
            force-md5: yes     # force logging of md5 checksums
        #- drop
        - ssh


For file store to disk/extraction:
   - file-store:
      enabled: yes       # set to yes to enable
      log-dir: files    # directory to store the files
      force-magic: yes   # force logging magic on all stored files
      force-md5: yes     # force logging of md5 checksums
      #waldo: file.waldo # waldo file to store the file_id across runs


 stream:
  memcap: 32mb
  checksum-validation: no      # reject wrong csums
  inline: auto                  # auto will use inline mode in IPS mode, yes or no set it statically
  reassembly:
    memcap: 128mb
    depth: 1mb                  # reassemble 1mb into a stream
  
depth: 1mb , would mean that in one tcp reassembled flow, the max size of the file that can be extracted is just about 1mb.

Both stream.memcap and reassembly.memcap (if reassembly is needed) must be big enough to accommodate the whole file on the fly in traffic that needs to be extracted PLUS any other stream and reassembly tasks that the engine needs to do while inspecting the traffic on a particular link.

 app-layer:
  protocols:
....
....
     http:
      enabled: yes
      # memcap: 64mb

The default limit for mem usage for http is 64mb   , that could be increased , ex - memcap: 4GB -  since HTTP is present everywhere and a low memcap on a busy HTTP link would limit the inspection and extraction size ability.

       libhtp:

         default-config:
           personality: IDS

           # Can be specified in kb, mb, gb.  Just a number indicates
           # it's in bytes.
           request-body-limit: 3072
           response-body-limit: 3072

The default values above control how far the HTTP request and response body is tracked and also limit file inspection. This should be set to a much higher value:

        libhtp:

         default-config:
           personality: IDS

           # Can be specified in kb, mb, gb.  Just a number indicates
           # it's in bytes.
           request-body-limit: 1gb
           response-body-limit: 1gb

 or 0 (which would mean unlimited):

       libhtp:

         default-config:
           personality: IDS

           # Can be specified in kb, mb, gb.  Just a number indicates
           # it's in bytes.
           request-body-limit: 0
           response-body-limit: 0

and then of course you would need a rule loaded(example):
alert http any any -> any any (msg:"PDF file Extracted"; filemagic:"PDF document"; filestore; sid:11; rev:11;)



That's it.
























by Peter Manev (noreply@blogger.com) at August 30, 2015 06:23 AM

August 29, 2015

Peter Manev

Failed to open ethX: pfring_open error


This is a blogpost about getting around the following error when using Suricata with pfring:

(source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
(tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1

However in my case eth2 existed, was up and running and the pfring module was loaded. So what happened in a bit more detail below :

I experienced this after a git pull update/upgrade of Suricata (latest at the moment of this writing) and after I re compiled pfring (using the latest pfring from git (https://github.com/ntop/PF_RING.git).

My set up (linux Debian/Ubuntu like systems):

root@suricata:/var/data/log/suricata# ifconfig eth2
eth2      Link encap:Ethernet  HWaddr 00:e0:ed:19:e3:e0
          inet6 addr: fe80::2e0:edff:fe19:e3e0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2962266192 errors:0 dropped:5527381 overruns:0 frame:0
          TX packets:19 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2867936692537 (2.8 TB)  TX bytes:3345 (3.3 KB)
The pfring set up I had was configured like this below:
root@suricata:/var/data/log/suricata# modprobe pf_ring transparent_mode=0 min_num_slots=65534

A regular check reveals nothing abnormal:
root@suricata:/var/data/log/suricata# modinfo pf_ring && cat /proc/net/pf_ring/info
filename:       /lib/modules/3.14.0-031400-generic/kernel/net/pf_ring/pf_ring.ko
alias:          net-pf-27
description:    Packet capture acceleration and analysis
author:         ntop.org
license:        GPL
srcversion:     E344EB01757B55E97A93D0C
depends:     
vermagic:       3.14.0-031400-generic SMP mod_unload modversions
parm:           min_num_slots:Min number of ring slots (uint)
parm:           perfect_rules_hash_size:Perfect rules hash size (uint)
parm:           transparent_mode:(deprecated) (uint)
parm:           enable_debug:Set to 1 to enable PF_RING debug tracing into the syslog (uint)
parm:           enable_tx_capture:Set to 1 to capture outgoing packets (uint)
parm:           enable_frag_coherence:Set to 1 to handle fragments (flow coherence) in clusters (uint)
parm:           enable_ip_defrag:Set to 1 to enable IP defragmentation(only rx traffic is defragmentead) (uint)
parm:           quick_mode:Set to 1 to run at full speed but with upto one socket per interface (uint)
PF_RING Version          : 6.1.1 (dev:250a67fe1082121ac511a19ebc3fe1fc5f494bfe)
Total rings              : 0

Standard (non DNA/ZC) Options
Ring slots               : 65534
Slot version             : 16
Capture TX               : Yes [RX+TX]
IP Defragment            : No
Socket Mode              : Standard
Total plugins            : 0
Cluster Fragment Queue   : 0
Cluster Fragment Discard : 0
Suricata and pfring have been installed as explained here - on the Suricata redmine wiki.
root@suricata:~# ldd /usr/local/bin/suricata
    linux-vdso.so.1 =>  (0x00007fff419fe000)
    libhtp-0.5.17.so.1 => /usr/local/lib/libhtp-0.5.17.so.1 (0x00007f32af5a1000)
    libGeoIP.so.1 => /usr/lib/x86_64-linux-gnu/libGeoIP.so.1 (0x00007f32af372000)
    libluajit-5.1.so.2 => /usr/local/lib/libluajit-5.1.so.2 (0x00007f32af103000)
    libmagic.so.1 => /usr/lib/x86_64-linux-gnu/libmagic.so.1 (0x00007f32aeee7000)
    libcap-ng.so.0 => /usr/local/lib/libcap-ng.so.0 (0x00007f32aece2000)
    libpfring.so => /usr/local/lib/libpfring.so (0x00007f32aeaa3000)
    libpcap.so.1 => /usr/local/pfring/lib/libpcap.so.1 (0x00007f32ae80e000)
    libnet.so.1 => /usr/lib/x86_64-linux-gnu/libnet.so.1 (0x00007f32ae5f5000)
    libjansson.so.4 => /usr/lib/x86_64-linux-gnu/libjansson.so.4 (0x00007f32ae3e8000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f32ae1ca000)
    libyaml-0.so.2 => /usr/lib/x86_64-linux-gnu/libyaml-0.so.2 (0x00007f32adfaa000)
    libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f32add6b000)
    libnss3.so => /usr/lib/x86_64-linux-gnu/libnss3.so (0x00007f32ada31000)
    libnspr4.so => /usr/lib/x86_64-linux-gnu/libnspr4.so (0x00007f32ad7f4000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f32ad42e000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f32ad215000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f32acf0f000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f32acd0a000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f32acaf4000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f32af7d4000)
    libnuma.so.1 => /usr/lib/x86_64-linux-gnu/libnuma.so.1 (0x00007f32ac8e9000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f32ac6e0000)
    libnssutil3.so => /usr/lib/x86_64-linux-gnu/libnssutil3.so (0x00007f32ac4b5000)
    libplc4.so => /usr/lib/x86_64-linux-gnu/libplc4.so (0x00007f32ac2b0000)
    libplds4.so => /usr/lib/x86_64-linux-gnu/libplds4.so (0x00007f32ac0ab000)


Further more my Suricata start line was like this:

suricata --pfring-int=eth2 --pfring-cluster-id=99 --pfring-cluster-type=cluster_flow -c /etc/suricata/peter-yaml/suricata-pfring.yaml --pidfile /var/run/suricata.pid -v

Even though everything seems fine - I could  not start Suricata with pfring:

[31591] 5/8/2015 -- 17:10:31 - (tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1
[31591] 5/8/2015 -- 17:10:31 - (source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
[31592] 5/8/2015 -- 17:10:31 - (tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1
[31592] 5/8/2015 -- 17:10:31 - (source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
[31593] 5/8/2015 -- 17:10:32 - (tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1
[31593] 5/8/2015 -- 17:10:32 - (source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
[31594] 5/8/2015 -- 17:10:32 - (tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1
[31594] 5/8/2015 -- 17:10:32 - (source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
....

I was getting that error even though I reloaded the pfring module:
rmmod pr_ring
modprobe pf_ring transparent_mode=0 min_num_slots=65534
the way I usually do...

In short - this is the fix:

LD_LIBRARY_PATH=/usr/local/pfring/lib suricata --pfring-int=eth2  --pfring-cluster-id=99 --pfring-cluster-type=cluster_flow  -c /etc/suricata/peter-yaml/suricata-pfring.yaml --pidfile /var/run/suricata.pid -v

Notice the use of:
LD_LIBRARY_PATH=/usr/local/pfring/lib suricata 

More information about what is LD_LIBRARY_PATH     

To get rid of LD_LIBRARY_PATH you can create a pfring.conf file in /etc/ld.so.conf.d/ containing:
/usr/local/pfring/lib
and run
sudo ldconfig




by Peter Manev (noreply@blogger.com) at August 29, 2015 02:42 AM

June 08, 2015

Security Onion

New securityonion-suricata package resolves an issue

We recently released a securityonion-suricata package for Suricata 2.0.8:
http://blog.securityonion.net/2015/05/suricata-208.html

An issue was found in the packaging:
https://groups.google.com/d/topic/security-onion/1MmmmO2XOyc/discussion

I've updated the securityonion-suricata package to resolve this issue.

The new package version is:
securityonion-suricata - 2.0.8-0ubuntu0securityonion2

Issues Resolved

Issue 742: securityonion-suricata package missing debian/install
https://github.com/Security-Onion-Solutions/security-onion/issues/742

Updating
The new package is now available in our stable repo.  Please see the following page for full update instructions:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Upgrade

Feedback
If you have any questions or problems, please use our security-onion mailing list:
https://github.com/Security-Onion-Solutions/security-onion/wiki/MailingLists

Commercial Support
Need commercial support?  Please see:
http://securityonionsolutions.com

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://github.com/Security-Onion-Solutions/security-onion/wiki/TeamMembers

Thanks!

by Doug Burks (noreply@blogger.com) at June 08, 2015 10:28 AM

May 28, 2015

suricata-ids.org

Suricata Ubuntu PPA updated to 2.1beta4

We have updated the official Ubuntu PPA to Suricata 2.1beta4. To use this PPA read our docs here.

If you’re using this PPA, updating is as simple as:

apt-get update && apt-get upgrade

The PPA Ubuntu packages have IPS mode through NFQUEUE enabled.

by fleurixx at May 28, 2015 01:02 PM

Suricata 2.1beta4 Windows Installer Available

The Windows MSI installer of the Suricata 2.1beta4 release is now available.

Download it here: suricata-2.1beta4-1-32bit.msi

After downloading, double click the file to launch the installer. The installer is now signed.

If you have a previous version installed, please remove that first.

by fleurixx at May 28, 2015 12:57 PM

Suricata Ubuntu PPA updated to 2.0.8

We have updated the official Ubuntu PPA to Suricata 2.0.8. To use this PPA read our docs here.

To install Suricata through this PPA, enter:
sudo add-apt-repository ppa:oisf/suricata-stable
sudo apt-get update
sudo apt-get install suricata

If you’re already using this PPA, updating is as simple as:
sudo apt-get update && sudo apt-get upgrade

The PPA Ubuntu packages have IPS mode through NFQUEUE enabled.

by fleurixx at May 28, 2015 12:23 PM

Suricata 2.0.8 Windows Installer Available

The Windows MSI installer of the Suricata 2.0.8 release is now available.

Download it here: Suricata-2.0.8-1-32bit.msi

After downloading, double click the file to launch the installer. The installer is now signed.

If you have a previous version installed, please remove that first.

by fleurixx at May 28, 2015 12:19 PM

May 22, 2015

Peter Manev

Suricata - wildcard rule loading


Recently (few hrs ago as of writing this blog) there was  a new feature (thanks to gozzy) introduced in Suricata IDS/IPS/NSM  - wildcard rule loading capability.

As of the moment the feature is available in our git master. If you are wondering how to get that up and running or do not have the latest Suricata from git master - here is a quick tutorial (Debian/Ubuntu):

1)
apt-get -y install libpcre3 libpcre3-dbg libpcre3-dev build-essential \
autoconf automake libtool libpcap-dev libnet1-dev libyaml-0-2 \
libyaml-dev zlib1g zlib1g-dev libmagic-dev libcap-ng-dev \
libjansson-dev pkg-config libnss3-dev libnspr4-dev git-core

2)
git clone git://phalanx.openinfosecfoundation.org/oisf.git && cd oisf/ &&  git clone https://github.com/ironbee/libhtp.git -b 0.5.x

3)
 ./autogen.sh && \
 ./configure --prefix=/usr/ --sysconfdir=/etc/ --localstatedir=/var/ \
 --enable-geoip --enable-unix-socket \
 --with-libnss-libraries=/usr/lib --with-libnss-includes=/usr/include/nss/ \
 --with-libnspr-libraries=/usr/lib --with-libnspr-includes=/usr/include/nspr \
 && make clean && make && make install-full && ldconfig

To confirm -
suricata --build-info

Now that you have latest Suricta up and running - here it is what this blog post is all about  - wildcard rule loading for Suricata IDPS. Some possible scenarios of use are loading wildcarded rules form the :

Command line


Please note the "quotes" !
suricata -c /etc/suricata/suricata.yaml  -v -i eth0 -S "/etc/suricata/rules/*.rules"

Pretty self explanatory. The command above will load all .rules files from /etc/suricata/rules/
suricata -c /etc/suricata/suricata.yaml  -v -i eth0 -S "/etc/suricata/rules/emerging*"
The command above will load all emerging* rules files from /etc/suricata/rules/

Config file


You can also set that up in the suricata.yaml config file. Here is how (please note the "quotes").

In your rules section in the suricata.yaml:

# Set the default rule path here to search for the files.
# if not set, it will look at the current working dir
default-rule-path: /etc/suricata/rules
rule-files:
 #- "*.rules"
 - "emerging*"
 #- botcc.rules
 #- ciarmy.rules
 #- compromised.rules
 #- drop.rules
 #- dshield.rules
 #- emerging-activex.rules
 #- emerging-attack_response.rules
The set up above will load all emerging* files and the rules residing in those. Then you can start Suricata anyway you would like, examples:

suricata -c /etc/suricata/suricata.yaml  -v -i eth0
suricata -c /etc/suricata/suricata.yaml  -v --af-packet=eth0

 and in suricata.log you should see all emerging* rule files being loaded:

......
[13558] 22/5/2015 -- 17:19:39 - (reputation.c:620) <Info> (SRepInit) -- IP reputation disabled
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-activex.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-attack_response.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-chat.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-current_events.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-deleted.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:420) <Warning> (ProcessSigFiles) -- [ERRCODE: SC_ERR_NO_RULES(42)] - No rules loaded from /etc/suricata/rules/emerging-deleted.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-dns.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-dos.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-exploit.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-ftp.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-games.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-icmp.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:420) <Warning> (ProcessSigFiles) -- [ERRCODE: SC_ERR_NO_RULES(42)] - No rules loaded from /etc/suricata/rules/emerging-icmp.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-icmp_info.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-imap.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-inappropriate.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-info.rules
[13558] 22/5/2015 -- 17:19:39 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-malware.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-misc.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-mobile_malware.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-netbios.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-p2p.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-policy.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-pop3.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-rpc.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-scada.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-scan.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-shellcode.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-smtp.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-snmp.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-sql.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-telnet.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-tftp.rules
[13558] 22/5/2015 -- 17:19:40 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-trojan.rules
[13558] 22/5/2015 -- 17:19:41 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-user_agents.rules
[13558] 22/5/2015 -- 17:19:41 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-voip.rules
[13558] 22/5/2015 -- 17:19:41 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-web_client.rules
[13558] 22/5/2015 -- 17:19:41 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-web_server.rules
[13558] 22/5/2015 -- 17:19:41 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-web_specific_apps.rules
[13558] 22/5/2015 -- 17:19:43 - (detect.c:410) <Info> (ProcessSigFiles) -- Loading rule file: /etc/suricata/rules/emerging-worm.rules
.......

You can also use it like that :

# Set the default rule path here to search for the files.
# if not set, it will look at the current working dir
default-rule-path: /etc/suricata/rules
rule-files:
 #- "*.rules"
 - "*web*"
 #- "emerging*"
 #- botcc.rules
 #- ciarmy.rules
 #- compromised.rules
 #- drop.rules


That's it.


by Peter Manev (noreply@blogger.com) at May 22, 2015 09:38 AM

May 21, 2015

Peter Manev

Suricata - multiple interface configuration with af-packet



Suricata is a very flexible and powerful multithreading  IDS/IPS/NSM.

Here is a simple tutorial (tested on Debian/Ubuntu) of how to configure multiple interfaces for af-packet mode with Suricata (af-packet mode works by default/out of the box on kernels 3.2 and above). Lets say you would like to start simple IDSing with Suricata on eth1, eth2 and eth3 on a particular machine/server.


In your suricata.yaml config (usually located in /etc/suricata/) find the af-packet section and do the following:


af-packet:
  - interface: eth2
    threads: 16
    cluster-id: 98
    cluster-type: cluster_cpu
    defrag: no
    use-mmap: yes
    ring-size: 200000
    checksum-checks: kernel
  - interface: eth1
    threads: 2
    cluster-id: 97
    cluster-type: cluster_flow
    defrag: no
    use-mmap: yes
    ring-size: 30000
  - interface: eth3
    threads: 2
    cluster-id: 96
    cluster-type: cluster_flow
    defrag: no
    use-mmap: yes
    ring-size: 20000
Of course feel free to adjust the ring-sizes (packet buffers) as you see fit for your particular set up.
NOTE:  do not forget to use a different cluster-id

so now you can start suricata like so:

suricata -c /etc/suricata/suricata.yaml -v --af-packet 

That above will start Suricata which will listen on eth2 with 16 threads with cluster_type: cluster_cpu and on eth1,eth3 with 2 threads each with cluster_type: cluster_flow. Have a look in your suricata.log file for more info.

If you would like to just test and see how it goes for eth2 only:
suricata -c /etc/suricata/suricata.yaml -v --af-packet=eth2

...easy and flexible.







by Peter Manev (noreply@blogger.com) at May 21, 2015 11:51 AM

May 18, 2015

Security Onion

Suricata 2.0.8

Suricata 2.0.8 was recently released:
http://suricata-ids.org/2015/05/06/suricata-2-0-8-available/

I've packaged Suricata 2.0.8 and it has been tested by David Zawdie (thanks!).

The new package version is:
securityonion-suricata - 2.0.8-0ubuntu0securityonion1

Issues Resolved

Issue 725: Suricata 2.0.8
https://github.com/Security-Onion-Solutions/security-onion/issues/725

Updating
The new package is now available in our stable repo.  Please see the following page for full update instructions:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Upgrade

This update will back up each of your existing suricata.yaml files to suricata.yaml.bak and migrate the HOME_NET and EXTERNAL_NET variables.  You'll then need to do the following:


  • re-apply any other local customizations to suricata.yaml
  • update ruleset and restart Suricata as follows:
    sudo rule-update


Feedback
If you have any questions or problems, please use our security-onion mailing list:
https://github.com/Security-Onion-Solutions/security-onion/wiki/MailingLists

Training
Need training?  We have 3-hour online classes this week:
http://securityonionsolutions.com

Commercial Support
Need commercial support?  Please see:
http://securityonionsolutions.com

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://github.com/Security-Onion-Solutions/security-onion/wiki/TeamMembers

Thanks!

by Doug Burks (noreply@blogger.com) at May 18, 2015 12:20 PM

May 09, 2015

Security Onion

Testers Needed!

We have lots of new packages queued up for release, but we need your help testing them!

If you're not already a member of our security-onion-testing group, please join and then click the following links for testing guidelines.

ELSA rev1205
https://groups.google.com/d/topic/security-onion-testing/OHhNEapIUgE/discussion

Suricata 2.0.8
https://groups.google.com/d/topic/security-onion-testing/WKeR1RViDlc/discussion

ossec_agent
https://groups.google.com/d/topic/security-onion-testing/N5gpeSHmIlk/discussion

Sphinxsearch 2.1.9
https://groups.google.com/d/topic/security-onion-testing/VWjichsRqPw/discussion

NSM
https://groups.google.com/d/topic/security-onion-testing/-cbA8FgH7lg/discussion

Setup
https://groups.google.com/d/topic/security-onion-testing/PBY2wJH9ruo/discussion

As you test each package, please add your test results to the thread.

Thanks in advance for your time and effort!

by Doug Burks (noreply@blogger.com) at May 09, 2015 10:04 AM

May 08, 2015

Open Information Security Foundation

Suricata 2.1beta4 Available!

The OISF development team is proud to announce Suricata 2.1beta4. This is the fourth beta release for the upcoming 2.1 version. It should be considered a development snapshot for the 2.1 branch.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.1beta4.tar.gz

New Features

  • Feature #1448: xbits support
  • Feature #336: Add support for NETMAP to Suricata
  • Feature #885: smtp file_data support
  • Feature #1394: Improve TCP reuse support
  • Feature #1445: Suricata does not work on pfSense/FreeBSD interfaces using PPPoE
  • Feature #1447: Ability to reject ICMP traffic
  • Feature #1410: add alerts to EVE’s drop logs

Improvements

  • Optimization #1014: app layer reassembly fast-path
  • Optimization #1377: flow manager: reduce (try)locking
  • Optimization #1403: autofp packet pool performance problems
  • Optimization #1409: http pipeline support for stateful detection
  • Bug #1314: http-events performance issues

Bugs

  • Bug #1340: null ptr dereference in Suricata v2.1beta2
  • Bug #1352: file list is not cleaned up
  • Bug #1358: Gradual memory leak using reload (kill -USR2 $pid)
  • Bug #1366: Crash if default_packet_size is below 32 bytes
  • Bug #1378: stats api doesn’t call thread deinit funcs
  • Bug #1384: tcp midstream window issue (master)
  • Bug #1388: pcap-file hangs on systems w/o atomics support (master)
  • Bug #1392: http uri parsing issue (master)
  • Bug #1393: CentOS 5.11 build failures
  • Bug #1398: DCERPC traffic parsing issue (master)
  • Bug #1401: inverted matching on incomplete session
  • Bug #1402: When re-opening files on HUP (rotation) always use the append flag.
  • Bug #1417: no rules loaded – latest git – rev e250040
  • Bug #1425: dead lock in de_state vs flowints/flowvars
  • Bug #1426: Files prematurely truncated by detection engine even though force-md5 is enabled
  • Bug #1429: stream: last_ack update issue leading to stream gaps
  • Bug #1435: EVE-Log alert payload option loses data
  • Bug #1441: Local timestamps in json events
  • Bug #1446: Unit ID check in Modbus packet error
  • Bug #1449: smtp parsing issue
  • Bug #1451: Fix list-keywords regressions
  • Bug #1463: modbus parsing issue

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

  • Kostya Kortchinsky of the Google Security Team
  • the Yahoo Pentest Team
  • Giuseppe Longo
  • Alexander Gozman
  • Ken Steele
  • Andreas Moe
  • David Diallo
  • David Cannings
  • David Maciejak
  • Pierre Chifflier
  • Tom DeCanio
  • Zachary Rasmor
  • Aleksey Katargin
  • FireEye
  • ANSSI
  • Emerging Threats
  • AFL project
  • Coverity Scan
  • Travis Green
  • Darien Huss
  • Greg Siemon
  • Alessandro Guido
  • Antti Tönkyrä
  • Ray Ruvinskiy
  • Eduardo Arada
  • Michael Rash

Known issues & missing features

In a beta release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

Training & Support

Need help installing, updating, validating and tuning Suricata? We have trainings coming up. Paris in July, Barcelona in November: see http://suricata-ids.org/training/

For support options also see http://suricata-ids.org/support/

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

by Victor Julien (postmaster@inliniac.net) at May 08, 2015 03:25 PM

suricata-ids.org

Suricata 2.1beta4 Available!

Photo by Eric Leblond

The OISF development team is proud to announce Suricata 2.1beta4. This is the fourth beta release for the upcoming 2.1 version. It should be considered a development snapshot for the 2.1 branch.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.1beta4.tar.gz

New Features

  • Feature #1448: xbits support
  • Feature #336: Add support for NETMAP to Suricata
  • Feature #885: smtp file_data support
  • Feature #1394: Improve TCP reuse support
  • Feature #1445: Suricata does not work on pfSense/FreeBSD interfaces using PPPoE
  • Feature #1447: Ability to reject ICMP traffic
  • Feature #1410: add alerts to EVE’s drop logs

Improvements

  • Optimization #1014: app layer reassembly fast-path
  • Optimization #1377: flow manager: reduce (try)locking
  • Optimization #1403: autofp packet pool performance problems
  • Optimization #1409: http pipeline support for stateful detection
  • Bug #1314: http-events performance issues

Bugs

  • Bug #1340: null ptr dereference in Suricata v2.1beta2
  • Bug #1352: file list is not cleaned up
  • Bug #1358: Gradual memory leak using reload (kill -USR2 $pid)
  • Bug #1366: Crash if default_packet_size is below 32 bytes
  • Bug #1378: stats api doesn’t call thread deinit funcs
  • Bug #1384: tcp midstream window issue (master)
  • Bug #1388: pcap-file hangs on systems w/o atomics support (master)
  • Bug #1392: http uri parsing issue (master)
  • Bug #1393: CentOS 5.11 build failures
  • Bug #1398: DCERPC traffic parsing issue (master)
  • Bug #1401: inverted matching on incomplete session
  • Bug #1402: When re-opening files on HUP (rotation) always use the append flag.
  • Bug #1417: no rules loaded – latest git – rev e250040
  • Bug #1425: dead lock in de_state vs flowints/flowvars
  • Bug #1426: Files prematurely truncated by detection engine even though force-md5 is enabled
  • Bug #1429: stream: last_ack update issue leading to stream gaps
  • Bug #1435: EVE-Log alert payload option loses data
  • Bug #1441: Local timestamps in json events
  • Bug #1446: Unit ID check in Modbus packet error
  • Bug #1449: smtp parsing issue
  • Bug #1451: Fix list-keywords regressions
  • Bug #1463: modbus parsing issue

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

  • Kostya Kortchinsky of the Google Security Team
  • the Yahoo Pentest Team
  • Giuseppe Longo
  • Alexander Gozman
  • Ken Steele
  • Andreas Moe
  • David Diallo
  • David Cannings
  • David Maciejak
  • Pierre Chifflier
  • Tom DeCanio
  • Zachary Rasmor
  • Aleksey Katargin
  • FireEye
  • ANSSI
  • Emerging Threats
  • AFL project
  • Coverity Scan
  • Travis Green
  • Darien Huss
  • Greg Siemon
  • Alessandro Guido
  • Antti Tönkyrä
  • Ray Ruvinskiy
  • Eduardo Arada
  • Michael Rash

Known issues & missing features

In a beta release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

Training & Support

Need help installing, updating, validating and tuning Suricata? We have trainings coming up. Paris in July, Barcelona in November: see http://suricata-ids.org/training/

For support options also see http://suricata-ids.org/support/

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

by inliniac at May 08, 2015 02:52 PM

May 06, 2015

Open Information Security Foundation

Suricata 2.0.8 Available!

The OISF development team is pleased to announce Suricata 2.0.8. This release fixes a number of important issues in the 2.0 series.

The most important issue is a bug in the DER parser which is used to decode SSL/TLS certificates could crash Suricata. This issue was reported by Kostya Kortchinsky of the Google Security Team and was fixed by Pierre Chifflier of ANSSI.

Those processing large numbers of (untrusted) pcap files need to update as a malformed pcap could crash Suricata. Again, credits go to Kostya Kortchinsky.

A number of other issues were fixed. Upgrading is highly recommended.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0.8.tar.gz

We have a new release key (the previous expired): http://www.openinfosecfoundation.org/download/OISF.pub (00C1B70D)

Changes

  • Bug #1450: tls parsing issue
  • Bug #1460: pcap parsing issue
  • Bug #1461: potential deadlock
  • Bug #1404: Alert-Debuglog not being rotated on SIGHUP
  • Bug #1420: inverted matching on incomplete session
  • Bug #1462: various issues in rule and yaml parsing

Security

The TLS/DER parsing issue has CVE-2015-0971 assigned to it.

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

  • Kostya Kortchinsky of the Google Security Team
  • Pierre Chifflier of ANSSI
  • Sundar Jeyaraman of FireEye
  • Darien Huss — Emerging Threats
  • Alexander Gozman
  • AFL project
  • Coverity Scan

Known issues & missing features

If you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

Training & Support

Need help installing, updating, validating and tuning Suricata? We have trainings coming up. Paris in July, Barcelona in November: see http://suricata-ids.org/training/

For support options also see http://suricata-ids.org/support/

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

by Victor Julien (postmaster@inliniac.net) at May 06, 2015 05:00 PM

suricata-ids.org

Suricata 2.0.8 Available!

Photo by Eric Leblond

The OISF development team is pleased to announce Suricata 2.0.8. This release fixes a number of important issues in the 2.0 series.

The most important issue is a bug in the DER parser which is used to decode SSL/TLS certificates could crash Suricata. This issue was reported by Kostya Kortchinsky of the Google Security Team and was fixed by Pierre Chifflier of ANSSI.

Those processing large numbers of (untrusted) pcap files need to update as a malformed pcap could crash Suricata. Again, credits go to Kostya Kortchinsky.

A number of other issues were fixed. Upgrading is highly recommended.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0.8.tar.gz

We have a new release key (the previous expired): http://www.openinfosecfoundation.org/download/OISF.pub (00C1B70D)

Changes

  • Bug #1450: tls parsing issue
  • Bug #1460: pcap parsing issue
  • Bug #1461: potential deadlock
  • Bug #1404: Alert-Debuglog not being rotated on SIGHUP
  • Bug #1420: inverted matching on incomplete session
  • Bug #1462: various issues in rule and yaml parsing

Security

The TLS/DER parsing issue has CVE-2015-0971 assigned to it.

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

  • Kostya Kortchinsky of the Google Security Team
  • Pierre Chifflier of ANSSI
  • Sundar Jeyaraman of FireEye
  • Darien Huss — Emerging Threats
  • Alexander Gozman
  • AFL project
  • Coverity Scan

Known issues & missing features

If you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

Training & Support

Need help installing, updating, validating and tuning Suricata? We have trainings coming up. Paris in July, Barcelona in November: see http://suricata-ids.org/training/

For support options also see http://suricata-ids.org/support/

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

by inliniac at May 06, 2015 04:56 PM

April 25, 2015

Peter Manev

Suricata - check loaded yaml config settings with --dump-config



There is a very useful command available to Suricata IDS/IPS/NSM :
suricata --dump-config

The command above will dump all the config parameters and their respective values that are loaded by Suricata from the config file. You can run the command in any case - it does not matter if Suricata is running or not.

There is a peculiarity however. Sometimes people would think that the command(above) would dump the currently loaded config values by Suricata.... in some case it will and in some cases it will not.

So what does it depend on?.... simple:
suricata --dump-config

will dump the config settings that are loaded (or will be loaded) by Suricata by default from
/etc/suricata/suricata.yaml

So if you are running Suricata with a config file called suricata-test.yaml (or suricata.yaml located in a different directory) - you will not see those settings...unless you specify that config file in particular:
suricata --dump-config -c /etc/suricata/suricata-test.yaml
Here is a real case example.
I run Suricata for a specific test where I had specified the defrag memcap to be 512mb :
defrag:
  memcap: 512mb
  hash-size: 65536
  trackers: 65535 # number of defragmented flows to follow
  max-frags: 65535 # number of fragments to keep (higher than trackers)
  prealloc: yes
  timeout: 60

Suricata up and running:
root@LTS-64-1:~/Work # ps aux |grep suricata
root      8109  2.3  7.6 878444 308372 pts/6   Sl+  12:45   1:02 suricata -c /etc/suricata/suricata-test.yaml --af-packet=eth0 -v
root@LTS-64-1:~/Work #

And the peculiarity that this blogpost is trying to emphasize on about :
root@LTS-64-1:~/Work # suricata --dump-config  |grep defrag.memcap
defrag.memcap = 32mb
root@LTS-64-1:~/Work # suricata --dump-config -c /etc/suricata/suricata-test.yaml |grep defrag.memcap
defrag.memcap = 512mb
root@LTS-64-1:~/Work #



suricata --dump-config dumps the settings loaded(or to be loaded) from the default location /etc/suricata/suricata.yaml if you are running suricata with a yaml config with a different name than the default or with a different location that the default - in order to get those settings - you need to specify that particular yaml location, like so:

suricata --dump-config -c /etc/local/some_test_dir/suricata/suricata-test.yaml


Thanks

related article:
http://pevma.blogspot.se/2014/02/suricata-override-config-parameters-on.html


by Peter Manev (noreply@blogger.com) at April 25, 2015 01:46 AM

April 21, 2015

suricata-ids.org

New OISF Board Announced!

The Open Information Security Foundation is thrilled to announce the new OISF Board of Directors:

Welcoming back returning board members: Randy Caldejon, Alexandre Dulaunoy, and Ken Steele.

And join us in welcoming new board members: Leonard Jacobs, Christian Kreibich, Eric Meyer, and Amar Rathore.

This is an important time for OISF, Suricata, and open source technologies — we are excited to be working with such an esteemed board over the next 2 years.  Thanks goes out to the OISF community for casting their votes!

by kmisata at April 21, 2015 02:18 PM

April 20, 2015

suricata-ids.org

Announcing the first Suricata User Conference in Barcelona

oisf-barcelonaWe are excited to announce our first annual OISF / Suricata User Conference happening this November in Barcelona, Spain!

Join us for an exciting two days of Suricata and IDS/IPS development talks, brainstorming sessions, and amazing speakers. You can also attend a 2-day Suricata training event prior to the conference to make it a full-week of learning!

The conference is FREE and open to the public – however, we ask that people register via our events website so we can ensure to accommodate everyone in regards to space and lunches. Visit: http://www.oisfevents.net.

NOTE: the 2-day Suricata training during this week is a paid event and space will fill up – so register at https://suricata-2day-barcelona.eventbrite.com

We look forward to seeing you there! As always if you have questions, contact us at info@oisf.net.

The OISF Team

by inliniac at April 20, 2015 08:36 AM

April 06, 2015

Peter Manev

Suricata IDPS - Application layer anomalies protocol detection




Suricata IDS/IPS/NSM also allows you to do application layer anomaly  detection.
I started talking to inliniac about protocol anomaly detection rules one day on the Suricata IRC chat room...which evolved more into a discussion resulting in us updating the rule sets with some examples of how to do that.

Below are a few examples for rules usage:

HTTP

alert tcp any any -> any ![80,8080] (msg:"SURICATA HTTP not tcp port 80, 8080"; flow:to_server; app-layer-protocol:http; sid:2271001; rev:1;)
The above rule finds http traffic that is not using dest port 80 or 8080.


alert tcp any any -> any 80 (msg:"SURICATA Port 80 but not HTTP"; flow:to_server; app-layer-protocol:!http; sid:2271002; rev:1;)
The above rule is kind of the reverse of the previous one - it will alert if the tcp traffic with destination port 80 is not http.

Here is another example

TLS

alert tcp any any -> any 443 (msg:"SURICATA Port 443 but not TLS"; flow:to_server; app-layer-protocol:!tls; sid:2271003; rev:1;)

HTTPS

Detecting HTTP traffic over HTTPS port -

alert http any any -> any 443 (msg:"SURICATA HTTP clear text on port 443"; flow:to_server; app-layer-protocol:http; sid:2271019; rev:1;)

You can find the full ruleset (open source and free to use) with examples for HTTP, HTTPS, TLS, FTP, SMTP, SSH, IMAP, SMB, DCERPC, DNS, MODBUS application layer anomaly  detection  here:

https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Protocol_Anomalies_Detection







by Peter Manev (noreply@blogger.com) at April 06, 2015 05:01 AM

April 02, 2015

suricata-ids.org

2015 OISF Board Elections have started

OISF_tmThe Open Information Security Foundation (OISF) is conducting its annual online elections to fill 7 positions on the OISF board of directors.  Board members serve a two year term, therefore, current board members along with new nominees are included on this year’s ballot.

The upcoming OISF board will consist of 10 board members in total:  7 elected directors, President of OISF, Matt Jonkman, General Manager of OISF, Kelley Misata, and Suricata’s Founder and Lead Developer, Victor Julien.

Each nominee has provided a brief summary highlighting their industry experience and their passion for OISF; please take a minute to read about each of our distinguished nominees and to cast your votes NOW!

Simply follow this link:  https://www.surveymonkey.com/s/Z2L6GXZ

Polls will close Wednesday, April 15, 2015 with the new OISF Board announced on Thursday, April 16, 2015.

Best of luck and thanks goes out to all of our nominees! Questions regarding elections can be sent to info@oisf.net.

Thank you,

The OISF Team

by inliniac at April 02, 2015 10:29 AM

Open Information Security Foundation

2015 OISF Board Elections have started

The Open Information Security Foundation (OISF) is conducting its annual online elections to fill 7 positions on the OISF board of directors.  Board members serve a two year term, therefore, current board members along with new nominees are included on this year's ballot.

The upcoming OISF board will consist of 10 board members in total:  7 elected directors, President of OISF, Matt Jonkman, General Manager of OISF, Kelley Misata, and Suricata's Founder and Lead Developer, Victor Julien.

Each nominee has provided a brief summary highlighting their industry experience and their passion for OISF; please take a minute to read about each of our distinguished nominees and to cast your votes NOW!

Simply follow this link:  https://www.surveymonkey.com/s/Z2L6GXZ

Polls will close Wednesday, April 15, 2015 with the new OISF Board announced on Thursday, April 16, 2015.

Best of luck and thanks goes out to all of our nominees! Questions regarding elections can be sent to info@oisf.net.

Thank you,

The OISF Team

by Victor Julien (postmaster@inliniac.net) at April 02, 2015 10:18 AM

March 26, 2015

suricata-ids.org

5-Day Suricata Developer Training

A short time ago we announced the first edition of the Suricata Developer training. I thought it be a good time to explain what we have in mind.

First, we’re planning to make this an annual event. I’m very excited about this. It should be great fun to have a week of development related discussions. I’m sure we can all learn a great deal, and share lots of ideas. Of course the social part is a nice addition.

Also, this is a great “stick behind the door” (as we’d say in Dutch) to finally get some much needed dev docs done, including architecture overview diagrams, etc.
What will we be teaching:

  1. General Suricata development basics: everything from git, how to QA, unittests, debugging, etc.
  2. Architecture overview
    1. API’s
    2. threading
    3. packets, flows, detection and output
  3. Extending Suricata — the beef of the training:
    1. packet decoders and detection plugins
    2. app layer protocol detection, parsing, state keeping
    3. app layer detection engine integration
    4. adding logging modules

Structure

Each day will start with lectures on each of the topics. You will get an overview of the API, learn about performance aspects, how threading comes into play, etc.

After this there will be assignments/challenges to apply the newly learned skills. This should be very interactive with lots of room for questions and discussions. We’ll be providing various assignments for multiple skill levels.

Teachers

From OISF we will have lead developer Victor Julien, core developers Eric Leblond and Jason Ish in the room to give the lectures and help answer questions. Also present to assist in general Suricata related questions: Matt Jonkman and Peter Manev.

Participants

So who is this for? We target people who what to learn how to extend Suricata. If you want to add protocols, detection options or maybe new output methods, then this is for you. Or maybe you want to be a ‘core’ developer on Suricata. Then this would be an excellent start of getting into it.

Skill-set:
– advanced C experience
– Linux Experience
– Network / Security Experience
– Basic Suricata End User Experience
=> for testing your code

If you’re not yet experienced with running Suricata, we would like to suggest attending a user training first. If you book your dev seat first hit us up for discount code on one of the training sessions. We’re planning one in Europe before summer, exact date and location is to be announced.

Money

We’ll be charging for the trainings. The revenue is used to cover the cost of the event itself (travel, hotels, etc). Whats left goes into the foundations general development budget. So by attending the training you will support Suricata’s development.

Location

This first edition is generously hosted by Napatech in Copenhagen, Denmark.
If you want to book, please do so through this Eventbrite link:
https://www.eventbrite.com/e/5-day-suricata-developer-training-in-copenhagen-denmark-open-to-the-public-tickets-15667305332
Questions and general feedback and thoughts are welcome!

by inliniac at March 26, 2015 05:38 PM

March 25, 2015

suricata-ids.org

Call for Nominations – OISF Board of Directors

OISF_tmThe Open Information Security Foundation is preparing hold the biennial (every two years) Board of Director elections and are putting out a call for nominations. We are anticipating the next several years to be both exciting and critical for OISF and Suricata, therefore, we are looking for candidates passionate about security and open source communities willing to serve as advisors on our board of directors. Your voice has a direct impact on future of OISF and Suricata – join us! The call for nominations begins today until March 31, 2015.

Read more at the OISF website.

by inliniac at March 25, 2015 06:31 PM

March 16, 2015

Open Information Security Foundation

Call for Nominations - OISF Board of Directors

The Open Information Security Foundation is preparing hold the biennial (every two years) Board of Director elections and are putting out a call for nominations.  We are anticipating the next several years to be both exciting and critical for OISF and Suricata, therefore, we are looking for candidates passionate about security and open source communities willing to serve as advisors on our board of directors.  Your voice has a direct impact on future of OISF and Suricata - join us! The call for nominations begins today until March 31, 2015.

Online elections will begin April 1, 2015.

Please consider joining our Board of Directors or nominating someone else who would be a great asset.  To help you decided, below are some answer to some common questions:

1. As an OISF board member what will I be asked to do?
Meetings: The OISF Board of Directors meet quarterly to review foundation activities, upcoming events, financial status and strategic objectives.  Meetings are held via conference call and pre-scheduled to respect the busy schedules of our board members.  Additionally, we host annual OISF User Conferences in locations around the world with our objectives of building Suricata's development roadmap, showing appreciation for OISF's consortium members, and growing the community.  We would hope that board members make every effort to attend this important event.  Our 2015 OISF User Conference is currently being planned for early November in Barcelona, Spain.

Advocacy: Board members will be asked to actively promote OISF, Suricata, and our events throughout the year.  OISF and Suricata exist because of the commitment of our community and we look to our board members to actively help us grow our presence in the world.

Expert Advice: Board members are expected to actively provide expertise, advice and professional connections necessary to help OISF make great strides both technologically and growing the community.

2. How large is the OISF board?
The current OISF board currently consisted of 6 members from the community - led by Matt Jonkman, Kelley Misata, and Victor Julien of OISF.  We will be expanding our board to 7 members serving for a 2 year term.

3. What is in it for me if I become an OISF board member?
As a board member you will have the opportunity to steer an innovative and cutting edge open source technology, to be an integral part of the decision making process for OISF and have a beneficiary priority status in all OISF and Suricata related public or private events.  Board members will be publicly acknowledged in OISF or Suricata related events and added to the OISF website spotlighting their professional bios.  Depending on OISF's financial capacity we are hoping to offer board members partial travel reimbursement to attend the annual OISF User Conferences - this is not guaranteed, but something we are hoping to be able to offer our board members.

4. I'm interested in nominating myself or someone I know - how do I do it?
It's simple - submit your name, name of your employer and a brief statement outlining your experience and reasons for running to be on the OISF board to info@oisf.net by 5 pm EST Tuesday, March 31, 2015.  Please note, the information provided in the nomination will be included on the PUBLIC election ballots so please be brief.

Elections will begin Wednesday, April 1st and conclude on Wednesday, April 15th.  The OISF Board Members will then be announced on Thursday, April 16th.

If you have any questions please do not hesitate to reach out to us directly at info@oisf.net OR reply to list to start a conversation with the community about this process.

Thank you,
The OISF Team

by Victor Julien (postmaster@inliniac.net) at March 16, 2015 03:09 PM

March 02, 2015

Security Onion

Suricata 2.0.7

Suricata 2.0.7 was recently released:
http://www.openinfosecfoundation.org/index.php/component/content/article/1-latest-news/204-suricata-207-available

I've packaged Suricata 2.0.7 and it has been tested by David Zawdie (thanks!).

The new package version is:
securityonion-suricata - 2.0.7-0ubuntu0securityonion1

Issues Resolved

Issue 695: Suricata 2.0.7
https://code.google.com/p/security-onion/issues/detail?id=695

Updating
The new package is now available in our stable repo.  Please see the following page for full update instructions:
https://code.google.com/p/security-onion/wiki/Upgrade

This update will back up each of your existing suricata.yaml files to suricata.yaml.bak.  You'll then need to do the following:

  • re-apply any local customizations to suricata.yaml
  • update ruleset and restart Suricata as follows:
  • sudo rule-update

Feedback
If you have any questions or problems, please use our security-onion mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Training
Need training?  We have 4-day classes coming up in Atlanta, Seattle, and Houston.  Please see:
http://securityonionsolutions.com

Commercial Support
Need commercial support?  Please see:
http://securityonionsolutions.com

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

Thanks!

by Doug Burks (noreply@blogger.com) at March 02, 2015 02:53 PM

February 25, 2015

Open Information Security Foundation

Suricata 2.0.7 Available!

The OISF development team is pleased to announce Suricata 2.0.7. This release fixes a number of important issues in the 2.0 series.

Two major issues. The first was brought to our attention by the Yahoo Pentest Team. It’s a parsing issue in the DCERPC parser that can happen when Suricata runs out of memory. The exact scope of the problem isn’t clear, but it could certainly lead to crashes. RCE might theoretically be possible but looks like it’s very hard.

The second issue was reported by Darien Huss of Emerging Threats. This is technically a libhtp issue, but it affects Suricata detection and logging. Certain characters in the URI could confuse the parsing of the HTTP request line, leading to possible detection bypass for ‘http_uri’ and to incomplete logging of the URI. Libhtp 0.5.17 has been released to address this and is bundled in 2.0.7.

Other than that a bunch of improvements and fixes. It should work again on CentOS 5. Midstream TCP was improved and some performance optimizations for HTTP proxy traffic were made.

Upgrading is highly recommended.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0.7.tar.gz

Changes

  • Bug #1385: DCERPC traffic parsing issue
  • Bug #1391: http uri parsing issue
  • Bug #1383: tcp midstream window issue
  • Bug #1318: A thread-sync issue in streamTCP
  • Bug #1375: Regressions in list keywords option
  • Bug #1387: pcap-file hangs on systems w/o atomics support
  • Bug #1395: dump-counters unix socket command failure
  • Optimization #1376: file list is not cleaned up

Security

The DCERPC parsing issue has CVE-2015-0928 assigned to it.

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

  • The Yahoo Pentest Team
  • Darien Huss — Emerging Threats
  • FireEye
  • Dennis Lee

Known issues & missing features

If you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

by Victor Julien (postmaster@inliniac.net) at February 25, 2015 03:47 PM

suricata-ids.org

Suricata 2.0.7 Available!

Photo by Eric Leblond

The OISF development team is pleased to announce Suricata 2.0.7. This release fixes a number of important issues in the 2.0 series.

Two major issues. The first was brought to our attention by the Yahoo Pentest Team. It’s a parsing issue in the DCERPC parser that can happen when Suricata runs out of memory. The exact scope of the problem isn’t clear, but it could certainly lead to crashes. RCE might theoretically be possible but looks like it’s very hard.

The second issue was reported by Darien Huss of Emerging Threats. This is technically a libhtp issue, but it affects Suricata detection and logging. Certain characters in the URI could confuse the parsing of the HTTP request line, leading to possible detection bypass for ‘http_uri’ and to incomplete logging of the URI. Libhtp 0.5.17 has been released to address this and is bundled in 2.0.7.

Other than that a bunch of improvements and fixes. It should work again on CentOS 5. Midstream TCP was improved and some performance optimizations for HTTP proxy traffic were made.

Upgrading is highly recommended.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0.7.tar.gz

Changes

  • Bug #1385: DCERPC traffic parsing issue
  • Bug #1391: http uri parsing issue
  • Bug #1383: tcp midstream window issue
  • Bug #1318: A thread-sync issue in streamTCP
  • Bug #1375: Regressions in list keywords option
  • Bug #1387: pcap-file hangs on systems w/o atomics support
  • Bug #1395: dump-counters unix socket command failure
  • Optimization #1376: file list is not cleaned up

Security

The DCERPC parsing issue has CVE-2015-0928 assigned to it.

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

  • The Yahoo Pentest Team
  • Darien Huss — Emerging Threats
  • FireEye
  • Dennis Lee

Known issues & missing features

If you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

by inliniac at February 25, 2015 03:46 PM

February 21, 2015

Peter Manev

Chasing MTUs


Setting up (configuring) the right MTU (maximum transmission unit) size when running Suricata IDS/IPS.

Sometimes you can end up in a situation as follows :


capture.kernel_packets    | AFPacketeth12              | 1143428204
decoder.pkts                    | AFPacketeth12             | 1143428143
decoder.invalid                | AFPacketeth12              | 416889536

a whole lot of  decoder.invalid. Not good. What could be the reason for that? One thing you should check right away is the MTU of the traffic that is being mirrored.

What does it mean? Well there is the MTU that you set up on the server that you run Suricata on and there is the MTU that is present in the "mirrored" traffic.

What is the difference?Why should it matter?
It matters because if not set correct  it will result in a lot of decoder.invalids (dropped by Suricata) and you will be missing on a lot of traffic inspection.
Example: if  on the sniffing interface that you run Suricata on has a MTU set as 1500  and in the traffic that you mirror you have jumbo frames (MTU 9000) - most likely your decoder.invalids will show a whole lotta love in your stats.log.

How can you adjust the MTU on the interface (NIC) ? (example)
First a have  look what is the current value:
ifconfig eth0
then adjust it
ifconfig eth0 mtu 1514

By the way - what could be the max size of the MTU (and what sizes there are in general)  -
(short answer - 9216)


This is the easy part :). There are situations where you do not know what is the MTU of the "mirrored" traffic. There is a few ways to find this  - ask the network team/guy, make a phone call or two, start manually testing and setting it on the NIC to find a middle ground ....however you can also make use of the procedure shown below (in order to get the byte size of the MTU):


On your Server/Sensor
1)
Stop Suricata.

2)
Change the MTU to 9216
(the interface that Suri is sniffing on)

example - ifconfig eth0 mtu 9216
(non boot persistent)

3)
install tcpstat - if you do not have it
apt-get install tcpstat

5)
run the following (substitute the interface name with yours - that Suri is sniffing on)
tcpstat -i eth0 -l -o "Time:%S\tn=%n\tavg=%a\tstddev=%d\tbps=%b\tMaxPacketSize=%M\n"  5
6)
Give it a minute or two
If there are Jumbo frames you should see that in the output (something like) -
"MaxPacketSize=9000", if not you should see whatever the max size is.

7)
Adjust your interface MTU accordingly  - the one that Suri is sniffing
on. -> Start Suri

8)
Let it run for  a while - lets say 1 hr. Have a look at the decoder.invalid stats in stats.log

NOTE: Do NOT just set the MTU to 9216 directly ("just to be on the safe side"). Only set it that high if needed !!

NOTE: This example below is not using the "-l" option of tcpstat as denoted in point 5) above - look at man tcpstat for more info



(tested on Ubuntu/Debian)
That's all ....feedback welcome.









by Peter Manev (noreply@blogger.com) at February 21, 2015 06:34 AM

February 18, 2015

Eric Leblond

Slides of my talks at Lecce

I’ve been invited by SaLUG to Lecce to give some talks during their Geek Evening. I’ve done a talk on nftables and one of suricata.

Lecce by night

Lecce by night

The nftables talk was about the motivation behind the change from iptables. Here are the slides: Nftables

The talk on Suricata was explaining the different feature of Suricata and was showing how I’ve used it to make a study of SSH bruteforce. Here are the slides: Suricata, Netfilter and the PRC.

Thanks a lot to Giuseppe Longo, Luca Greco and all the SaLUG team, you have been wonderful hosts!

by Regit at February 18, 2015 10:44 AM

January 29, 2015

Open Information Security Foundation

Suricata 2.1beta3 Available!

The OISF development team is proud to announce Suricata 2.1beta3. This is the third beta release for the upcoming 2.1 version. It should be considered a development snapshot for the 2.1 branch.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.1beta3.tar.gz

New Features

  • Feature #1309: Lua support for Stats output
  • Feature #1310: Modbus parsing and matching

Improvements

  • Optimization #1339: flow timeout optimization
  • Optimization #1371: mpm optimization
  • Feature #1317: Lua: Indicator for end of flow
  • Feature #1333: unix-socket: allow (easier) non-root usage
  • Feature #1261: Request for Additional Lua Capabilities

Bugs

  • Bug #977: WARNING on empty rules file is fatal (should not be)
  • Bug #1184: pfring: cppcheck warnings
  • Bug #1321: Flow memuse bookkeeping error
  • Bug #1327: pcre pkt/flowvar capture broken for non-relative matches (master)
  • Bug #1332: cppcheck: ioctl
  • Bug #1336: modbus: CID 1257762: Logically dead code (DEADCODE)
  • Bug #1351: output-json: duplicate logging (2.1.x)
  • Bug #1354: coredumps on quitting on OpenBSD
  • Bug #1355: Bus error when reading pcap-file on OpenBSD
  • Bug #1363: Suricata does not compile on OS X/Clang due to redefinition of string functions (2.1.x)
  • Bug #1365: evasion issues (2.1.x)

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

  • Ken Steele — Tilera/EZchip
  • David Diallo
  • Duarte Silva
  • Giuseppe Longo
  • Jason Ish
  • Travis Green — Emerging Threats

Known issues & missing features

In a beta release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

by Victor Julien (postmaster@inliniac.net) at January 29, 2015 04:38 PM

suricata-ids.org

Suricata 2.1beta3 Available!

Photo by Eric Leblond

The OISF development team is proud to announce Suricata 2.1beta3. This is the third beta release for the upcoming 2.1 version. It should be considered a development snapshot for the 2.1 branch.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.1beta3.tar.gz

New Features

  • Feature #1309: Lua support for Stats output
  • Feature #1310: Modbus parsing and matching

Improvements

  • Optimization #1339: flow timeout optimization
  • Optimization #1371: mpm optimization
  • Feature #1317: Lua: Indicator for end of flow
  • Feature #1333: unix-socket: allow (easier) non-root usage
  • Feature #1261: Request for Additional Lua Capabilities

Bugs

  • Bug #977: WARNING on empty rules file is fatal (should not be)
  • Bug #1184: pfring: cppcheck warnings
  • Bug #1321: Flow memuse bookkeeping error
  • Bug #1327: pcre pkt/flowvar capture broken for non-relative matches (master)
  • Bug #1332: cppcheck: ioctl
  • Bug #1336: modbus: CID 1257762: Logically dead code (DEADCODE)
  • Bug #1351: output-json: duplicate logging (2.1.x)
  • Bug #1354: coredumps on quitting on OpenBSD
  • Bug #1355: Bus error when reading pcap-file on OpenBSD
  • Bug #1363: Suricata does not compile on OS X/Clang due to redefinition of string functions (2.1.x)
  • Bug #1365: evasion issues (2.1.x)

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

  • Ken Steele — Tilera/EZchip
  • David Diallo
  • Duarte Silva
  • Giuseppe Longo
  • Jason Ish
  • Travis Green — Emerging Threats

Known issues & missing features

In a beta release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

by inliniac at January 29, 2015 04:33 PM

January 21, 2015

suricata-ids.org

Suricata Ubuntu PPA updated to 2.0.6

We have updated the official Ubuntu PPA to Suricata 2.0.6. To use this PPA read our docs here.

To install Suricata through this PPA, enter:
sudo add-apt-repository ppa:oisf/suricata-stable
sudo apt-get update
sudo apt-get install suricata

If you’re already using this PPA, updating is as simple as:
sudo apt-get update && sudo apt-get upgrade

The PPA Ubuntu packages have IPS mode through NFQUEUE enabled.

by fleurixx at January 21, 2015 12:06 PM

Suricata 2.0.6 Windows Installer Available

The Windows MSI installer of the Suricata 2.0.6 release is now available.

Download it here: Suricata-2.0.6-1-32bit.msi

After downloading, double click the file to launch the installer. The installer is now signed.

If you have a previous version installed, please remove that first.

by fleurixx at January 21, 2015 12:03 PM

January 20, 2015

Security Onion

Suricata 2.0.6

Suricata 2.0.6 was recently released:
http://suricata-ids.org/2015/01/15/suricata-2-0-6-available/

I've packaged Suricata 2.0.6 and it has been tested by David Zawdie (thanks!).

The new package version is:
securityonion-suricata - 2.0.6-0ubuntu0securityonion1

Issues Resolved

Issue 673: Suricata 2.0.6
https://code.google.com/p/security-onion/issues/detail?id=673

Updating
The new package is now available in our stable repo.  Please see the following page for full update instructions:
https://code.google.com/p/security-onion/wiki/Upgrade

This update will back up each of your existing suricata.yaml files to suricata.yaml.bak.  You'll then need to do the following:

  • re-apply any local customizations to suricata.yaml
  • update ruleset and restart Suricata as follows:
    sudo rule-update


Feedback
If you have any questions or problems, please use our security-onion mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Commercial Support
Need training and/or commercial support?  Please see:
http://securityonionsolutions.com

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

Thanks!

by Doug Burks (noreply@blogger.com) at January 20, 2015 08:55 AM

January 16, 2015

suricata-ids.org

Suricata 2.0.6 Available!

Photo by Eric Leblond

The OISF development team is pleased to announce Suricata 2.0.6. This release fixes a number of important issues in the 2.0 series. The most important part is the fixing of evasion issues, therefore upgrading is highly recommended!

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0.6.tar.gz

Changes

  • Bug #1364: evasion issues
  • Bug #1337: output-json: duplicate logging
  • Bug #1325: tls detection leads to tcp stream reassembly sequence gaps (IPS)
  • Bug #1192: Suricata does not compile on OS X/Clang due to redefinition of string functions
  • Bug #1183: pcap: cppcheck warning

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

  • Martin Küchler

Known issues & missing features

If you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

by inliniac at January 16, 2015 03:55 PM

January 15, 2015

Open Information Security Foundation

Suricata 2.0.6 Available!

The OISF development team is pleased to announce Suricata 2.0.6. This release fixes a number of important issues in the 2.0 series. The most important part is the fixing of evasion issues, therefore upgrading is highly recommended!

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0.6.tar.gz

Changes

  • Bug #1364: evasion issues
  • Bug #1337: output-json: duplicate logging
  • Bug #1325: tls detection leads to tcp stream reassembly sequence gaps (IPS)
  • Bug #1192: Suricata does not compile on OS X/Clang due to redefinition of string functions
  • Bug #1183: pcap: cppcheck warning

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

  • Martin Küchler

Known issues & missing features

If you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

by Victor Julien (postmaster@inliniac.net) at January 15, 2015 09:44 AM

January 08, 2015

Victor Julien

Suricata has been added to Debian Backports

Thanks to the hard work of Arturo Borrero Gonzalez, Suricata has just been added to the openlogo-100Debian ‘backports’ repository. This allows users of Debian stable to run up to date versions of Suricata.

The ‘Backports’ repository makes the Suricata and libhtp packages from Debian Testing available to ‘stable’ users. As ‘testing’ is currently in a freeze, it may take a bit of time before 2.0.5 and libhtp 0.5.16 appear.

Anyway, here is how to use it.

Install

First add backports repo to your sources:

# echo "deb http://http.debian.net/debian wheezy-backports main" > /etc/apt/sources.list.d/backports.list
# apt-get update

As explained here http://backports.debian.org/Instructions/, this will not affect your normal packages.

To prove this, check:

# apt-get install suricata -s
Conf libhtp1 (0.2.6-2 Debian:7.7/stable [amd64])
Conf suricata (1.2.1-2 Debian:7.7/stable [amd64])

Not what we want, as that is still the old version.

To install Suricata from backports, we need to specify the repo:

# apt-get install -t wheezy-backports suricata -s
Conf libhtp1 (0.5.15-1~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Conf suricata (2.0.4-1~bpo70+1 Debian Backports:/wheezy-backports [amd64])

Let’s do it!

# apt-get install -t wheezy-backports suricata
...
Setting up suricata (2.0.4-1~bpo70+1) ...
[FAIL] suricata disabled, please adjust the configuration to your needs ... failed!
[FAIL] and then set RUN to 'yes' in /etc/default/suricata to enable it. ... failed!

Suricata 2.0.4 is now installed, but it’s not yet running.
To see what features have been compiled in, run:

# suricata --build-info
This is Suricata version 2.0.4 RELEASE

Suricata Configuration:
  AF_PACKET support:                       yes
  PF_RING support:                         no
  NFQueue support:                         yes
  NFLOG support:                           no
  IPFW support:                            no
  DAG enabled:                             no
  Napatech enabled:                        no
  Unix socket enabled:                     yes
  Detection enabled:                       yes

  libnss support:                          yes
  libnspr support:                         yes
  libjansson support:                      yes
  Prelude support:                         yes
  PCRE jit:                                yes
  LUA support:                             yes
  libluajit:                               yes
  libgeoip:                                no
  Non-bundled htp:                         yes
  Old barnyard2 support:                   no
  CUDA enabled:                            no

  Suricatasc install:                      yes

It has Luajit enabled, libjansson for the JSON output, NFQ and AF_PACKET IPS modes, NSS for MD5 checksums and unix sockets. Quite a good feature set.

Run

To get it running, we need a few more steps:

Edit /etc/default/suricata:

1. Change RUN=no to RUN=yes
2. Change LISTENMODE to “af-packet”:

Now lets start it.

# service suricata start
Starting suricata in IDS (af-packet) mode... done.

And confirm that it’s running.

# ps aux|grep suricata
root     20295  1.8  4.1 200212 42544 ?        Ssl  00:50   0:00 /usr/bin/suricata -c /etc/suricata/suricata-debian.yaml --pidfile /var/run/suricata.pid --af-packet -D

Check if we’re seeing traffic:

# tail /var/log/suricata/stats.log -f|grep capture
capture.kernel_packets    | RxAFPeth01                | 406
capture.kernel_drops      | RxAFPeth01                | 0
capture.kernel_packets    | RxAFPeth11                | 0
capture.kernel_drops      | RxAFPeth11                | 0
capture.kernel_packets    | RxAFPeth01                | 411
capture.kernel_drops      | RxAFPeth01                | 0
capture.kernel_packets    | RxAFPeth11                | 0
capture.kernel_drops      | RxAFPeth11                | 0
capture.kernel_packets    | RxAFPeth01                | 417
capture.kernel_drops      | RxAFPeth01                | 0
capture.kernel_packets    | RxAFPeth11                | 0
capture.kernel_drops      | RxAFPeth11                | 0
capture.kernel_packets    | RxAFPeth01                | 587
capture.kernel_drops      | RxAFPeth01                | 0
capture.kernel_packets    | RxAFPeth11                | 0
capture.kernel_drops      | RxAFPeth11                | 0
capture.kernel_packets    | RxAFPeth01                | 593
capture.kernel_drops      | RxAFPeth01                | 0
capture.kernel_packets    | RxAFPeth11                | 0
capture.kernel_drops      | RxAFPeth11                | 0

Logging

As the init script starts Suricata in daemon mode, we need to enable logging to file:

Edit /etc/suricata/suricata-debian.yaml and go to the “logging:” section, there change the “file” portion to look like:

  - file:
      enabled: yes
      filename: /var/log/suricata/suricata.log

Note: in the YAML indentation matters, so make sure it’s exactly right.

Rules

Oinkmaster is automatically installed, so lets use that:

First create the rules directory:

mkdir /etc/suricata/rules/

Open /etc/oinkmaster.conf in your editor and add:

url = https://rules.emergingthreats.net/open/suricata-2.0/emerging.rules.tar.gz

Then run:

# oinkmaster -C /etc/oinkmaster.conf -o /etc/suricata/rules
Loading /etc/oinkmaster.conf
Downloading file from https://rules.emergingthreats.net/open/suricata-2.0/emerging.rules.tar.gz... done.
...

Edit /etc/suricata/suricata-debian.yaml and change “default-rule-path” to:

default-rule-path: /etc/suricata/rules

Finally, restart to load the new rules:

# service suricata restart

Validate

Now that Suricata is running with rules, lets see if it works:

# wget http://www.testmyids.com
--2015-01-08 01:21:30--  http://www.testmyids.com/
Resolving www.testmyids.com (www.testmyids.com)... 82.165.177.154

This should trigger a specific rule:

# tail /var/log/suricata/fast.log 
01/08/2015-01:21:30.870346  [**] [1:2100498:7] GPL ATTACK_RESPONSE id check returned root [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 82.165.177.154:80 -> 192.168.122.181:59190

Success! :)

Thanks

Thanks to Arturo Borrero Gonzalez for taking on this work for us. Also many thanks for Pierre Chifflier for maintaining the Suricata and libhtp packages in Debian.


by inliniac at January 08, 2015 12:34 AM

January 05, 2015

Security Onion

Suricata 2.0.5

Suricata 2.0.5 was recently released:
http://www.openinfosecfoundation.org/index.php/component/content/article/1-latest-news/201-suricata-205-available

I've packaged Suricata 2.0.5 and it has been tested by David Zawdie (thanks!).

The new package version is:
securityonion-suricata - 2.0.5-0ubuntu0securityonion1

Issues Resolved

Issue 655: Suricata 2.0.5
https://code.google.com/p/security-onion/issues/detail?id=655

Updating
The new package is now available in our stable repo.  Please see the following page for full update instructions:
https://code.google.com/p/security-onion/wiki/Upgrade

This update will back up each of your existing suricata.yaml files to suricata.yaml.bak.  You'll then need to do the following:

  • re-apply any local customizations to suricata.yaml
  • update ruleset and restart Suricata as follows:

sudo rule-update

Feedback
If you have any questions or problems, please use our security-onion mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Commercial Support
Need training and/or commercial support?  Please see:
http://securityonionsolutions.com

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

Thanks!

by Doug Burks (noreply@blogger.com) at January 05, 2015 10:15 AM

December 24, 2014

suricata-ids.org

Suricata Ubuntu PPA updated to 2.0.5

We have updated the official Ubuntu PPA to Suricata 2.0.5. To use this PPA read our docs here.

To install Suricata through this PPA, enter:
sudo add-apt-repository ppa:oisf/suricata-stable
sudo apt-get update
sudo apt-get install suricata

If you’re already using this PPA, updating is as simple as:
sudo apt-get update && sudo apt-get upgrade

The PPA Ubuntu packages have IPS mode through NFQUEUE enabled.

by fleurixx at December 24, 2014 07:02 PM

December 23, 2014

Victor Julien

Profiling Suricata with JEMALLOC

JEMALLOC is a memory allocation library: http://www.canonware.com/jemalloc/

It offers many interesting things for a tool like Suricata. Ken Steele of EZchip (formerly Tilera) made me aware of it. In Ken’s testing it helps performance.

Install

wget http://www.canonware.com/download/jemalloc/jemalloc-3.6.0.tar.bz2
tar xvfj jemalloc-3.6.0.tar.bz2
cd jemalloc-3.6.0
./configure --prefix=/opt/jemalloc/
make
sudo make install

Then use it by preloading it:

LD_PRELOAD=/opt/jemalloc/lib/libjemalloc.so ./src/suricata -c suricata.yaml -l tmp/ -r ~/sync/pcap/sandnet.pcap -S emerging-all.rules -v

I haven’t benchmarked this, but if you’re running a high performance setup it may certainly be worth a shot.

Profiling

The library comes with many interesting profiling and debugging features.

make clean
./configure --prefix=/opt/jemalloc-prof/ --enable-prof
make
sudo make install

Start Suricata like this:

LD_PRELOAD=/opt/jemalloc-prof/lib/libjemalloc.so ./src/suricata -c suricata.yaml -l tmp/ -r ~/sync/pcap/sandnet.pcap -S emerging-all.rules -v

Now we don’t see any change as we need to tell jemalloc what we want.

Exit stats

Dumps a lot of stats to the screen.

MALLOC_CONF=stats_print:true LD_PRELOAD=/opt/jemalloc-prof/lib/libjemalloc.so ./src/suricata -c suricata.yaml -l tmp/ -r ~/sync/pcap/sandnet.pcap -S emerging-all.rules -v

Memory leak checks

MALLOC_CONF=prof_leak:true,lg_prof_sample:0 LD_PRELOAD=/opt/jemalloc-prof/lib/libjemalloc.so ./src/suricata -c suricata.yaml -l tmp/ -r ~/sync/pcap/sandnet.pcap -S emerging-all.rules -v
[... suricata output ...]
<jemalloc>: Leak summary: 2011400 bytes, 4523 objects, 645 contexts
<jemalloc>: Run pprof on "jeprof.22760.0.f.heap" for leak detail

Then use the pprof tool that comes with jemalloc to inspect the dumped stats.

$ /opt/jemalloc-prof/bin/pprof --show_bytes ./src/suricata jeprof.22760.0.f.heap
Using local file ./src/suricata.
Using local file jeprof.22760.0.f.heap.
Welcome to pprof!  For help, type 'help'.
(pprof) top
Total: 2011400 B
1050112  52.2%  52.2%  1050112  52.2% PacketGetFromAlloc
600064  29.8%  82.0%   600064  29.8% SCProfilePacketStart
103936   5.2%  87.2%   103936   5.2% SCACCreateDeltaTable
65536   3.3%  90.5%    66192   3.3% pcap_fopen_offline
35520   1.8%  92.2%    35520   1.8% ConfNodeNew
26688   1.3%  93.6%    26688   1.3% __GI___strdup
20480   1.0%  94.6%    20480   1.0% MemBufferCreateNew
20480   1.0%  95.6%    20480   1.0% _TmSlotSetFuncAppend
14304   0.7%  96.3%    14304   0.7% pcre_compile2
14064   0.7%  97.0%    25736   1.3% SCPerfRegisterQualifiedCounter

So it seems we don’t properly clean up our packet pools yet.

Create a PDF of this info:

$ /opt/jemalloc-prof/bin/pprof --show_bytes --pdf ./src/suricata jeprof.22760.0.f.heap > jemalloc.pdf

Dumping stats during runtime

Dump stats after every 16MiB of allocations (lg_prof_interval:24, means every 2^24 bytes, so 16MiB)

I’ve done this in a separate directory since it dumps many files.

$ mkdir jemalloc-profile
$ cd jemalloc-profile/
$ MALLOC_CONF="prof:true,prof_prefix:victor.out,lg_prof_interval:24" LD_PRELOAD=/opt/jemalloc-prof/lib/libjemalloc.so ../src/suricata -c ../suricata.yaml -l ../tmp/ -r ~/sync/pcap/sandnet.pcap -S ../emerging-all.rules -v

Then you should see new *.heap files appear, many during startup. But after some time it should slow down.

You can then visualize the diff between two dumps:

$ /opt/jemalloc-prof/bin/pprof --show_bytes --pdf ../src/suricata --base victor.out.24159.150.i150.heap victor.out.24159.200.i200.heap > jemalloc.pdf

This creates a PDF of the 200th dump taking the 150th dump as a baseline. As we dump every ~16MiB, this covers about 50 * 16 = 800MiB worth of allocations.

Further reading

http://www.canonware.com/jemalloc/
https://github.com/jemalloc/jemalloc/wiki
https://github.com/jemalloc/jemalloc/wiki/Use-Case%3A-Heap-Profiling

Many thanks to Ken Steele for pointing me to the lib and providing me with some good examples.


by inliniac at December 23, 2014 03:50 PM

December 20, 2014

Victor Julien

Crossing the Streams in Suricata

At it’s core, Suricata is a packet processor. It reads packets and pushes them through a configurable pipeline. The 2nd most important processing unit in Suricata is the flow. In Suricata we use the term flow for the bidirectional flows of packets with the same 5 tuple (proto, src ip, dst ip, sp, dp. Vlans can be added as well). In fact, much of Suricata’s threading effort revolves around the flow. In the 2 main runmodes, autofp and workers, flow based load balancing makes sure that a all packets of a single flow always go through the same threading pipeline. In workers this means one single thread, in autofp 2: the capture thread and a stream/detect/output thread.

Flows are the central unit for out ‘app layer’ parsing. Protocol parsers like HTTP don’t even have access to the original packet. It all runs on top of the stream engine, which tracks TCP flows in … our flow structure.

Another place where the flow is crucial is in many of the rules. Rules extensively use the concept of ‘flowbits’. This allows one rule to ‘flag’ a flow, and then another to check this flag. In Emerging Threats many hundreds of rules use this logic.

Ever since we started Suricata, we’ve been talking about what some called ‘global flowbits’. A bit of a strange and contradictory name, but pretty much rule writers wanted the logic of flowbits, but then applied to other units as well. So a few weeks ago I (finally) decided to check if I could quickly implement ‘hostbits’. As Suricata already has a scalable ‘host table’, it was easy add the storage of ‘bits’ there. In a few hours I had the basics working and made it public: see this pull request.

Although I got some nice feedback, I was mostly interested in what the ET folks would think, since they would be the main consumers. While presenting the work I also mentioned the xbits ideas by Michael Rash and the response was “wow, do we have ip_pair tracking now?”. Ehh, no, just ip/host based… “Ah well, I guess that is nice too”. Not exactly the response I hoped for :)

IP pair tracking is not something Suricata already did. But as the need was clear I decided to have a look at it. Turned out it was quite simple to do. The IPPair tracker is much like the Host tracking. It’s only done on demand, which sets it apart from the Flow tracking which is done unconditionally. In this case only the new keyword is making use of the IP Pair storage.

So, what I have implemented is pretty much ‘xbits’. It supports tracking by ‘ip_src’, ‘ip_dst’ and ‘ip_pair’. It uses the syntax as suggested by Michael Rash:

xbits:<set|unset|isset|isnotset|toggle>,<bitname>,\
      track <ip_src|ip_dst|ip_pair>,expire <seconds>

It’s only lightly tested, so I would appreciate testing feedback!

You’ll find the code here in PR 1275 at github. This should normally end up in Suricata 2.1, which will come out early next year.


by inliniac at December 20, 2014 11:37 PM

December 12, 2014

Open Information Security Foundation

Suricata 2.0.5 Available!

The OISF development team is pleased to announce Suricata 2.0.5. This release fixes a number of important issues in the 2.0 series.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0.5.tar.gz

Changes

  • Bug #1190: http_header keyword not matching when SYN|ACK and ACK missing
  • Bug #1246: EVE output Unix domain socket not working
  • Bug #1272: Segfault in libhtp 0.5.15
  • Bug #1298: Filestore keyword parsing issue
  • Bug #1303: improve stream ‘bad window update’ detection
  • Bug #1304: improve stream handling of bad SACK values
  • Bug #1305: fix tcp session reuse for ssh/ssl sessions
  • Bug #1307: byte_extract, within combination not working
  • Bug #1326: pcre pkt/flowvar capture broken for non-relative matches
  • Bug #1329: Invalid rule being processed and loaded
  • Bug #1330: Flow memuse bookkeeping error (2.0.x)

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

  • Jason Ish — Endace/Emulex
  • Ken Steele — Tilera
  • lessyv
  • Tom DeCanio — FireEye
  • Andreas Herz
  • Matt Carothers
  • Duane Howard
  • Edward Fjellskål
  • Giuseppe Longo

Known issues & missing features

If you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

by Victor Julien (postmaster@inliniac.net) at December 12, 2014 11:22 AM

suricata-ids.org

Suricata 2.0.5 Available!

Photo by Eric Leblond

The OISF development team is pleased to announce Suricata 2.0.5. This release fixes a number of important issues in the 2.0 series.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.0.5.tar.gz

Changes

  • Bug #1190: http_header keyword not matching when SYN|ACK and ACK missing
  • Bug #1246: EVE output Unix domain socket not working
  • Bug #1272: Segfault in libhtp 0.5.15
  • Bug #1298: Filestore keyword parsing issue
  • Bug #1303: improve stream ‘bad window update’ detection
  • Bug #1304: improve stream handling of bad SACK values
  • Bug #1305: fix tcp session reuse for ssh/ssl sessions
  • Bug #1307: byte_extract, within combination not working
  • Bug #1326: pcre pkt/flowvar capture broken for non-relative matches
  • Bug #1329: Invalid rule being processed and loaded
  • Bug #1330: Flow memuse bookkeeping error (2.0.x)

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

  • Jason Ish — Endace/Emulex
  • Ken Steele — Tilera
  • lessyv
  • Tom DeCanio — FireEye
  • Andreas Herz
  • Matt Carothers
  • Duane Howard
  • Edward Fjellskål
  • Giuseppe Longo

Known issues & missing features

If you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

by inliniac at December 12, 2014 11:21 AM

December 07, 2014

Peter Manev

Suricatasc unix socket interaction for Suricata IDS/IPS



Suricatasc is a unix socket interaction  script that is automatically installed when one compiles/installs Suricata IDS/IPS. An in depth description, prerequisites and how to documentation is located here - https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Interacting_via_Unix_Socket

However  lets look at a quick usage example - that can come very handy in certain situations.

Once you have unix socket command enabled in suricata.yaml :

unix-command:
    enabled: yes
    #filename: custom.socket # use this to specify an alternate file

the traditional way to use the script would be type suricatasc and hit Enter (on the machine running Suricata):




However you can also use it directly as a command line parameter for example :
root@suricata:~# suricatasc -c version

like so:


NOTE:
You need to quote commands involving interfaces:
root@debian64:~# suricatasc -c "iface-stat eth0"



Very handy when you want quick interaction and info from the currently running Suricata IDS/IPS.




by Peter Manev (noreply@blogger.com) at December 07, 2014 11:34 AM

Suricata - disable detection mode


There is a trick for using Suricata IDS/IPS that has come in handy - in my experience that I thought  would share and might be useful to others.

My HW was CPU 1x E5-2680 with 64 GB RAM ,  NIC a  82599EB 10-Gigabit SFI/SFP+ and mirroring about 9,5Gbps:





So I wanted to get a better understanding of what is the max log output from Suricata in that particular environment without putting detection into consideration. Just pure event logging and profiling - DNS,HTTP,TLS,SSH, File transactions, SMTP all of these.

Suricata offers just that - with a really low need for HW (having in mind we are looking at 9,5Gbps). What i did was compile suricata with the "--disable-detection" switch. What it does it simply disables detection(alerts) in Suricata  - however every other logging/parsing capability is preserved ( DNS,HTTP,TLS,SSH, File transactions, SMTP). So i downloaded a fresh copy:

git clone git://phalanx.openinfosecfoundation.org/oisf.git && cd oisf/ &&  git clone https://github.com/ironbee/libhtp.git -b 0.5.x

then

./autogen.sh && ./configure --disable-detection   &&  make clean && make && make install &&  ldconfig

enabled all JSON outputs in the eve-log section in suricata.yaml. I confirmed that detection was disabled:


and started Suricata.

Careful what you asked for :) - I was getting 10-15K logs per second :

 root@suricata:/var/log/suricata# tail -f  eve.json |perl -e 'while (<>) {$l++;if (time > $e) {$e=time;print "$l\n";$l=0}}'
1
6531
13860
10704
10877
10389
10664
10205
9996
14798
15427
14223

And the HW(CPU/RAM/HDD) usage was not much at all:


As you can see - 30-40% CPU with a third of RAM  usage.

 I used this command to count logs per second  -
tail -f  eve.json |perl -e 'while (<>) {$l++;if (time > $e) {$e=time;print "$l\n";$l=0}}'
more about similar commands and counting Suricata logs you could find here:


So this can came in handy in a few scenarious:
  • you can  actually do a  "profiling" run on that particular set up in order to size up a SIEM specification 
  • and/or you can size up your prod IDS/IPS deployment needs
  • you can also just feed all those logs info to an existing log analysis system

Thanks

by Peter Manev (noreply@blogger.com) at December 07, 2014 04:53 AM

December 05, 2014

suricata-ids.org

Suricata Ubuntu PPA updated to 2.1beta2

We have updated the official Ubuntu PPA to Suricata 2.1beta2. To use this PPA read our docs here.

If you’re using this PPA, updating is as simple as:

apt-get update && apt-get upgrade

The PPA Ubuntu packages have IPS mode through NFQUEUE enabled.

by fleurixx at December 05, 2014 01:40 PM

Suricata 2.1beta2 Windows Installer Available

The Windows MSI installer of the Suricata 2.1beta2 release is now available.

Download it here: suricata-2.1beta2-1-32bit.msi

After downloading, double click the file to launch the installer. The installer is now signed.

If you have a previous version installed, please remove that first.

by fleurixx at December 05, 2014 01:37 PM

November 11, 2014

Victor Julien

SMTP file extraction in Suricata

In 2.1beta2 the long awaited SMTP file extraction support for Suricata finally appeared. It has been a long development cycle. Originally started by BAE Systems, it was picked up by Tom Decanio of FireEye Forensics Group (formerly nPulse Technologies) followed by a last round of changes from my side. But it’s here now.

It contains:

  • a MIME decoder
  • updates to the SMTP parser to use the MIME decoder for extracting files
  • SMTP JSON log, integrated with EVE
  • SMTP message URL extraction and logging

As it uses the Suricata file handling API, it shares almost everything with the existing file handling for HTTP. The rule keyword work and the various logs work automatically with SMTP as well.

Trying it out

To enable the file extraction, make sure that the MIME decoder is enabled:

app-layer:
  protocols:
    smtp:
      enabled: yes
      # Configure SMTP-MIME Decoder
      mime:
        # Decode MIME messages from SMTP transactions
        # (may be resource intensive)
        # This field supercedes all others because it turns the entire
        # process on or off
        decode-mime: yes

        # Decode MIME entity bodies (ie. base64, quoted-printable, etc.)
        decode-base64: yes
        decode-quoted-printable: yes

        # Maximum bytes per header data value stored in the data structure
        # (default is 2000)
        header-value-depth: 2000

        # Extract URLs and save in state data structure
        extract-urls: yes

Like with HTTP, SMTP depends on the stream engine working correctly. So this page applies https://redmine.openinfosecfoundation.org/projects/suricata/wiki/File_Extraction, although of course the HTTP specific settings are irrelevant to SMTP.

Troubleshooting (SMTP) file extraction issues should always start here: https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Self_Help_Diagrams#File-Extraction-and-Logging-Issues

Logging

Enabling the SMTP logging is simple, just add ‘smtp’ to the list of types in your EVE config, like so:

  # Extensible Event Format (nicknamed EVE) event log in JSON format
  - eve-log:
      enabled: yes
      filetype: regular #regular|syslog|unix_dgram|unix_stream
      filename: eve.json
      # the following are valid when type: syslog above
      #identity: "suricata"
      #facility: local5
      #level: Info ## possible levels: Emergency, Alert, Critical,
                   ## Error, Warning, Notice, Info, Debug
      types:
        - alert:
            # payload: yes           # enable dumping payload in Base64
            # payload-printable: yes # enable dumping payload in printable (lossy) format
            # packet: yes            # enable dumping of packet (without stream segments)
            # http: yes              # enable dumping of http fields
        - http:
            extended: yes     # enable this for extended logging information
            # custom allows additional http fields to be included in eve-log
            # the example below adds three additional fields when uncommented
            #custom: [Accept-Encoding, Accept-Language, Authorization]
        - dns
        - tls:
            extended: yes     # enable this for extended logging information
        - files:
            force-magic: no   # force logging magic on all logged files
            force-md5: no     # force logging of md5 checksums
        #- drop
        - smtp
        - ssh
        # bi-directional flows
        #- flow
        # uni-directional flows
        #- newflow

URLs

As a bonus, the MIME decoder also extracts URL’s from the SMTP message body (not attachments) and logs them in the SMTP log. This should make it easy to post process them. Currently only ‘HTTP’ URLS are extracted, starting with ‘http://‘. So HTTPS/FTP or URLs that don’t have the protocol prefix aren’t logged.

Testing

Naturally, if you’re using SMTP over TLS or have STARTTLS enabled, as you should at least on public networks, none of this will work.

Please help us test this feature!


by inliniac at November 11, 2014 11:15 AM

November 06, 2014

Open Information Security Foundation

Suricata 2.1beta2 Available!

The OISF development team is proud to announce Suricata 2.1beta2. This is the second beta release for the upcoming 2.1 version. It should be considered a development snapshot for the 2.1 branch.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.1beta2.tar.gz

New Features

  • Feature #549: Extract file attachments from emails
  • Feature #1312: Lua output support
  • Feature #899: MPLS over Ethernet support
  • Feature #383: Stream logging

Improvements

  • Feature #1263: Lua: Access to Stream Payloads
  • Feature #1264: Lua: access to TCP quad / Flow Tuple
  • Feature #707: ip reputation files – network range inclusion availability (cidr)

Bugs

  • Bug #1048: PF_RING/DNA config – suricata.yaml
  • Bug #1230: byte_extract, within combination not working
  • Bug #1257: Flow switch is missing from the eve-log section in suricata.yaml
  • Bug #1259: AF_PACKET IPS is broken in 2.1beta1
  • Bug #1260: flow logging at shutdown broken
  • Bug #1279: BUG: NULL pointer dereference when suricata was debug mode.
  • Bug #1280: BUG: IPv6 address vars issue
  • Bug #1285: Lua – http.request_line not working (2.1)
  • Bug #1287: Lua Output has dependency on eve-log:http
  • Bug #1288: Filestore keyword in wrong place will cause entire rule not to trigger
  • Bug #1294: Configure doesn’t use –with-libpcap-libraries when testing PF_RING library
  • Bug #1301: suricata yaml – PF_RING load balance per hash option
  • Bug #1308: http_header keyword not matching when SYN|ACK and ACK missing (master)
  • Bug #1311: EVE output Unix domain socket not working (2.1)

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

  • Tom Decanio — FireEye
  • Ken Steele — Tilera
  • Giuseppe Longo — Emerging Threats & Ntop
  • David Abarbanel — BAE Systems
  • Jason Ish — Endace/Emulex
  • Mats Klepsland
  • Duarte Silva
  • Bill Meeks
  • Anoop Saldanha
  • lessyv

Known issues & missing features

In a beta release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

by Victor Julien (postmaster@inliniac.net) at November 06, 2014 10:28 AM

suricata-ids.org

Suricata 2.1beta2 Available!

Photo by Eric Leblond

The OISF development team is proud to announce Suricata 2.1beta2. This is the second beta release for the upcoming 2.1 version. It should be considered a development snapshot for the 2.1 branch.

Download

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-2.1beta2.tar.gz

New Features

  • Feature #549: Extract file attachments from emails
  • Feature #1312: Lua output support
  • Feature #899: MPLS over Ethernet support
  • Feature #383: Stream logging

Improvements

  • Feature #1263: Lua: Access to Stream Payloads
  • Feature #1264: Lua: access to TCP quad / Flow Tuple
  • Feature #707: ip reputation files – network range inclusion availability (cidr)

Bugs

  • Bug #1048: PF_RING/DNA config – suricata.yaml
  • Bug #1230: byte_extract, within combination not working
  • Bug #1257: Flow switch is missing from the eve-log section in suricata.yaml
  • Bug #1259: AF_PACKET IPS is broken in 2.1beta1
  • Bug #1260: flow logging at shutdown broken
  • Bug #1279: BUG: NULL pointer dereference when suricata was debug mode.
  • Bug #1280: BUG: IPv6 address vars issue
  • Bug #1285: Lua – http.request_line not working (2.1)
  • Bug #1287: Lua Output has dependency on eve-log:http
  • Bug #1288: Filestore keyword in wrong place will cause entire rule not to trigger
  • Bug #1294: Configure doesn’t use –with-libpcap-libraries when testing PF_RING library
  • Bug #1301: suricata yaml – PF_RING load balance per hash option
  • Bug #1308: http_header keyword not matching when SYN|ACK and ACK missing (master)
  • Bug #1311: EVE output Unix domain socket not working (2.1)

Special thanks

We’d like to thank the following people and corporations for their contributions and feedback:

  • Tom Decanio — FireEye
  • Ken Steele — Tilera
  • Giuseppe Longo — Emerging Threats & Ntop
  • David Abarbanel — BAE Systems
  • Jason Ish — Endace/Emulex
  • Mats Klepsland
  • Duarte Silva
  • Bill Meeks
  • Anoop Saldanha
  • lessyv

Known issues & missing features

In a beta release like this things may not be as polished yet. So please handle with care. That said, if you encounter issues, please let us know! As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.  See issues for an up to date list and to report new issues. See Known_issues for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF, its supporting vendors and the community.

by inliniac at November 06, 2014 10:25 AM

October 29, 2014

suricata-ids.org

Suricata Ubuntu PPA updated to 2.0.4

We have updated the official Ubuntu PPA to Suricata 2.0.4. To use this PPA read our docs here.

To install Suricata through this PPA, enter:
sudo add-apt-repository ppa:oisf/suricata-stable
sudo apt-get update
sudo apt-get install suricata

If you’re already using this PPA, updating is as simple as:
sudo apt-get update && sudo apt-get upgrade

The PPA Ubuntu packages have IPS mode through NFQUEUE enabled.

by fleurixx at October 29, 2014 02:42 PM

Suricata 2.0.4 Windows Installer Available

The Windows MSI installer of the Suricata 2.0.4 release is now available.

Download it here: Suricata-2.0.4-1-32bit.msi

After downloading, double click the file to launch the installer. The installer is now signed.

If you have a previous version installed, please remove that first.

by fleurixx at October 29, 2014 02:35 PM

October 26, 2014

Peter Manev

Suricata ids/ips - dropping privileges



This tutorial is intended for Linux (Debian/Ubuntu).

Install the prerequisite packages in order to compile Suricata. I add/enable some optional features so in my case I usually do:
apt-get -y install libpcre3 libpcre3-dbg libpcre3-dev \
build-essential autoconf automake libtool libpcap-dev libnet1-dev \
libyaml-0-2 libyaml-dev zlib1g zlib1g-dev make flex bison \
libmagic-dev

For Eve (all JSON output):
apt-get install libjansson-dev libjansson4
For MD5 support(file extraction):
apt-get install libnss3-dev libnspr4-dev
For GeoIP:
apt-get install libgeoip1 libgeoip-dev
For nfqueue(ips mode):
apt-get install libnetfilter-queue-dev libnetfilter-queue1 libnfnetlink-dev libnfnetlink0
For the dropping privileges part you can simply do:
apt-get install libcap-ng0 libcap-ng-dev

OR get the latest libcap-ng version form here:
http://people.redhat.com/sgrubb/libcap-ng/
like so:

wget http://people.redhat.com/sgrubb/libcap-ng/libcap-ng-0.7.4.tar.gz
tar -zxf libcap-ng-0.7.4.tar.gz
cd libcap-ng-0.7.4
./configure && make && make install
cd ..

Let's fetch and compile Suricata:
wget http://www.openinfosecfoundation.org/download/suricata-2.0.4.tar.gz
tar -xzf suricata-2.0.4.tar.gz 
cd suricata-2.0.4
 One liner... one of my favorite:

./configure --prefix=/usr/ --sysconfdir=/etc/ --localstatedir=/var/ --disable-gccmarch-native \
--enable-geoip --with-libnss-libraries=/usr/lib --with-libnss-includes=/usr/include/nss/ \
--enable-nfqueue \
--with-libcap_ng-libraries=/usr/local/lib --with-libcap_ng-includes=/usr/local/include \
--with-libnspr-libraries=/usr/lib --with-libnspr-includes=/usr/include/nspr && \
make clean && make && make install-full && ldconfig


Above we enable some other features like :
(
you can do like this
root@IDS:~/suricata-2.0.4# ./configure --help
to see what each option is for
)

but this line -
--with-libcap_ng-libraries=/usr/local/lib --with-libcap_ng-includes=/usr/local/include
is the one you need to compile and enable dropping privileges with Suricata.


Then you can run Suri like so
/usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid --af-packet -D -v --user=logstash

Make sure the log directory has the right permissions to allow the user "logstash" to write to it.
After you start Suricata  - you should see something similar:
root@IDS:~# ls -lh /var/log/suricata/
total 77M
drwxr-xr-x 2 logstash logstash 4.0K Oct 15 13:06 certs
drwxr-xr-x 2 logstash logstash 4.0K Oct 15 13:06 core
-rw-r----- 1 logstash logstash  18M Oct 26 10:48 eve.json
-rw-r----- 1 logstash logstash 806K Oct 26 10:48 fast.log
drwxr-xr-x 2 logstash logstash 4.0K Oct 15 13:06 files
drwxr-xr-x 2 logstash logstash 4.0K Oct 26 06:26 StatsByDate
-rw-r--r-- 1 root     root      58M Oct 26 10:48 stats.log
-rw-r--r-- 1 root     root     1.1K Oct 26 09:15 suricata-start.log
root@IDS:~#
Notice the user logstash ownership.

root@IDS:~# ps aux |grep suricata
logstash  2189 11.0 10.6 420448 219972 ?       Ssl  09:15  13:04 /usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid --af-packet -D -v --user=logstash
root@IDS:~#
Now you have the user logstash running (not as root) Suricata IDS/IPS.




by Peter Manev (noreply@blogger.com) at October 26, 2014 09:38 AM

October 03, 2014

suricata-ids.org

Get Trained January 26 and 27 in San Jose, CA!

Join us for this dynamic, hands-on, 2-day Suricata training event! Developers and security professionals will walk-away with not only a greater proficiency in Suricata’s core technology; but will have the unique opportunity to bring questions, challenges, and new ideas directly to Suricata’s development team.

This training session will take place on January 26 and 27 at the Tilera HQ in San Jose, CA. It will be given by Suricata expert Peter Manev, and OISF president and Emerging Threats CTO Matt Jonkman.

Some of topics that will be covered over the course of the 2-days include:

  • Compiling, Installing, and Configuring Suricata
  • Performance Factors, Rules and Rulesets
  • Capture Methods and Performance
  • Event / Data Outputs and Capture Hardware
  • Troubleshooting Common Problems
  • Advanced Tuning
  • Integration with Other Tools

You can register through eventbrite here. More info on the Suricata Training Program can be found here.

This event is generously hosted by our long time supporters: Tilera.

tilera_logo_pms361_plain

We hope to see you there!

by inliniac at October 03, 2014 09:10 AM

September 30, 2014

Security Onion

Suricata 2.0.4

Suricata 2.0.4 was recently released:
http://www.openinfosecfoundation.org/index.php/component/content/article/1-latest-news/198-suricata-204-available

I've packaged Suricata 2.0.4 and it has been tested by David Zawdie (thanks!).

The new package version is:
securityonion-suricata - 2.0.4-0ubuntu0securityonion1

Issues Resolved

Issue 600: Suricata 2.0.4
https://code.google.com/p/security-onion/issues/detail?id=600

Updating
The new packages are now available in our stable repo.  Please see the following page for full update instructions:
https://code.google.com/p/security-onion/wiki/Upgrade

This update will back up each of your existing suricata.yaml files to suricata.yaml.bak.  You'll then need to do the following:

  • re-apply any local customizations to suricata.yaml
  • update ruleset and restart Suricata as follows:
    sudo rule-update


Screenshots

Update Process
sudo rule-update

rule-update restarts Suricata

Feedback
If you have any questions or problems, please use our security-onion mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Training
Only 16 seats left for the 3-day Security Onion class in Richmond VA!
https://security-onion-class-20141020.eventbrite.com/

Commercial Support
Need commercial support?  Please see:
http://securityonionsolutions.com

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

We especially need help in answering support questions on the mailing list:
http://groups.google.com/group/security-onion

We also need help testing new packages:
http://groups.google.com/group/security-onion-testing

Thanks!

by Doug Burks (noreply@blogger.com) at September 30, 2014 07:54 AM

September 29, 2014

Victor Julien

Suricata Training Tour

After a lot of preparations, it’s finally going to happen: official Suricata trainings!

In the next couple of months I’ll be doing at least 3 sessions: a home match (Amsterdam), a workshop in Luxembourg and a session at DeepSec. Next to this, we’re planning various US based sessions on the East coast and West coast.

I’m really looking forward to doing these sessions. Other than the official content, there will be plenty of room for questions and discussions.

Hope to see you soon! :)


by inliniac at September 29, 2014 09:25 AM

suricata-ids.org

Get Trained at DeepSec in Vienna

DeepSecLogoJoin us for this dynamic, hands-on, 2-day training session. Developers and security professionals will walk-away with not only a greater proficiency in Suricata’s core technology; but will have the unique opportunity to bring questions, challenges, and new ideas directly to Suricata’s lead developers.

This training session will take place on November 18 and 19 at the DeepSec conference in Vienna . It will be given by Suricata lead developer Victor Julien, OISF president and Emerging Threats CTO Matt Jonkman, Suricata developer Eric Leblond and Suricata expert Peter Manev.

Some of topics that will be covered at this course include:

  • Compiling, Installing, and Configuring Suricata
  • Performance Factors, Rules and Rulesets
  • Capture Methods and Performance
  • Event / Data Outputs and Capture Hardware
  • Troubleshooting Common Problems
  • Integration with Other Tools

You can register at the DeepSec conference registration page here.

More info on the Suricata Training Program can be found here.

We hope to see you there!

by inliniac at September 29, 2014 09:16 AM

Open Information Security Foundation

Announcing the Suricata Training Program

The OISF team is proud to announce the start of the Suricata training program. In this program, we’ll be delivering 1 and 2 day user trainings for Suricata.

Some of topics that will be covered over the course of the 2-days include:

  • Compiling, Installing, and Configuring Suricata
  • Performance Factors, Rules and Rulesets
  • Capture Methods and Performance
  • Event / Data Outputs and Capture Hardware
  • Troubleshooting Common Problems
  • Advanced Tuning
  • Integration with Other Tools


This dynamic, hands-on, 2 day Suricata training will be delivered by the OISF development and support team.  So apart of the great content on how to install, use and troubleshoot Suricata, you will also have the great opportunity to talk in-depth about Suricata with it’s creators.

Proceeds of the trainings go straight into supporting Suricata’s development, so not only will you learn a great deal, you’ll actually be supporting Suricata’s development by taking this training.


We’re kicking off with 3 training sessions in Europe in the last quarter of 2014. For early 2015, we’re planning to do a number of US trainings. Keep an eye on this space for updates. Also, dedicated on-site training options are available.




Amsterdam, October 13 and 14: 2 day training

This training session will take place on October 13 and 14 in down town Amsterdam. It will be given by Suricata lead developer Victor Julien, and OISF president and Emerging Threats CTO Matt Jonkman. Also in the room: master rule writer William Metcalf.

You can register through eventbrite here: https://www.eventbrite.com/e/suricata-training-event-tickets-13264631871

This event is generously hosted by our friends from Intelworks.

Luxembourg, October 20: 1 day workshop

This workshop will take place on October 20 in the conference hotel of the excellent Hack.lu conference. It will be given by Suricata lead developer Victor Julien, Suricata developer Eric Leblond and Suricata expert Peter Manev.

This event is generously hosted by our friends from Hack.lu. You can register through eventbrite here:
https://www.eventbrite.com/e/suricata-workshop-hacklu-tickets-13329929177

A registration / ticket for the Hack.lu conference is NOT required for this event. Of course, we do highly recommend the conference!


DeepSec - Vienna, November 18 and 19: 2 day training event

This training session will take place on November 18 and 19 at the DeepSec conference. It will be given by Victor Julien, Eric Leblond, Peter Manev and Matt Jonkman.

The event is part of the DeepSec conference, so registrations/bookings go through: https://deepsec.net/register.html

See also http://blog.deepsec.net/?p=1893


Trainings are tracked on their own page here: http://suricata-ids.org/training/. For questions or more info, please contact us at oisf-team@openinfosecfoundation.org!

by Victor Julien (postmaster@inliniac.net) at September 29, 2014 08:43 AM

September 25, 2014

suricata-ids.org

Get Trained at Hack.lu in Luxembourg

Join us for this dynamic, hands-on, full day  Suricata workshop! Developers and security professionals will walk-away with not only a greater proficiency in Suricata’s core technology; but will have the unique opportunity to bring questions, challenges, and new ideas directly to Suricata’s lead developers.

This workshop will take place on October 20 in the conference hotel of the excellent Hack.lu conference. It will be given by Suricata lead developer Victor Julien, Suricata developer Eric Leblond and Suricata expert Peter Manev.

Some of topics that will be covered at this course include:

  • Compiling, Installing, and Configuring Suricata
  • Performance Factors, Rules and Rulesets
  • Capture Methods and Performance
  • Event / Data Outputs and Capture Hardware
  • Troubleshooting Common Problems
  • Integration with Other Tools

You can register through eventbrite here: https://www.eventbrite.com/e/suricata-workshop-hacklu-tickets-13329929177240pxlogohacklu2014.

More info on the Suricata Training Program can be found here.

This event is generously hosted by our friends from Hack.lu.

A registration / ticket for the Hack.lu conference is NOT required for this event. Of course, we do highly recommend the conference!

We hope to see you there!

by inliniac at September 25, 2014 03:45 PM