IntelMQ — Automate Incident Handling Process

IntelMQ is a solution for CERTs for collecting and processing security feeds, pastebins, tweets and log files using a message queuing protocol. It’s a community driven initiative called IHAP (Incident Handling Automation Project) which was conceptually designed by European CERTs during several InfoSec events. Its main goal is to give to incident responders an easy way to collect & process threat intelligence thus improving the incident handling processes of CERTs.

Automate Incident Handling Process

[adsense size='1']

IntelMQ’s design was influenced by AbuseHelper, however it was re-written from scratch and aims at:

• Reduce the complexity of system administration
• Reduce the complexity of writing new bots for new data feeds
• Reduce the probability of events lost in all process with persistence functionality (even system crash)
• Use and improve the existing Data Harmonization Ontology
• Use JSON format for all messages
• Integration of the existing tools (AbuseHelper, CIF)
• Provide easy way to store data into Log Collectors like ElasticSearch, Splunk
• Provide easy way to create your own black-lists
• Provide easy communication with other systems via HTTP RESTFUL API

It follows the following basic meta-guidelines:

• Don’t break simplicity – KISS
• Keep it open source – forever
• Strive for perfection while keeping a deadline
  • Reduce complexity/avoid feature bloat
  • Embrace unit testing
  • Code readability: test with unexperienced programmers
• Communicate clearly

System Overview

In the intelmq/lib/ directory you can find:

• Bots: Defines base structure for bots and handling of startup, stop, messages etc.
• Cache: For some expert bots it does make sense to cache external lookup results. Redis is used here.
• Harmonization: For defined types, checks and sanitation methods are implemented.
• Message: Defines Events and Reports classes, uses harmonization to check validity of keys and values according to config.
• Pipeline: Writes messages to message queues. Implemented for productions use is only Redis. A python-only solution is used by testing. A solution using ZMQ is in development.
• Test: Base class for bot tests with predefined test and assert methods.
• Utils: Utility functions used by system components.

Source && Download