building collector plugins admin guide

building collector plugins admin guide building collector plugins admin guide building collector plugins admin guide building collector plugins admin guide building collector plugins admin guide building collector plugins admin guide building collector plugins admin guide building collector plugins admin guide building collector plugins admin guide building collector plugins admin guide building collector plugins admin guide building collector plugins admin guide building collector plugins admin guide building collector plugins admin guide building collector plugins admin guide building collector plugins admin guide

Building Collector Plugins Admin Guide Copyright © Alienvault 2010 All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and publisher. Any trademarks referenced herein are the property of their respective holders. Building Collector Plugins - Admin Guide Page 2 Copyright © Alienvault 2010 Table of Content 1 Overview 4 1.1 OSSIM Agent Role 4 1.1.1 Event Collection 4 1.1.2 Event Normalization 4 1.2 OSSIM Server Role 6 1.2.1 Event Enrichment 6 1.2.2 Policies and Actions 7 1.3 The Configuration Workflow 8 2 Configuring Detector Plugins 10 2.1 Rsyslog 10 2.1.1 Configuration File 10 2.1.2 Listener Configuration 10 2.1.3 Filters 10 2.2 OSSIM Agent Configuration 11 2.2.1 Configuration File 11 2.2.2 Parameters 11 2.3 Detector Plugin Configuration 13 2.3.1 Configuration Files 13 2.3.2 Common Event Types 13 2.3.3 Parameters 13 2.3.4 Using Local (Plugin) Variables 15 2.3.5 Using Global (Agent) Variables 15 2.4 Aliases 16 2.4.1 Path 16 2.4.2 Predefined Regular Expressions 16 2.5 Functions 16 2.5.1 Path 16 2.5.2 Conversions 16 2.5.3 Application Specific Translations 17 2.5.4 User Defined Translations 17 2.6 Event Fields 18 2.7 Rules 19 2.7.1 Evaluation Order 19 Building Collector Plugins - Admin Guide Page 3 Copyright © Alienvault 2010 2.7.2 Structure 19 2.8 Loading Plugins 21 2.8.1 Priority and Reliability values 21 2.8.2 SQL Statement 21 2.9 Plugin Activation 22 2.9.1 Activate the Plugin on the Server Side 22 2.9.2 Activate the Plugin on the Agent Side 22 3 Log files 22 4 Debugging 22 5 Appendix 23 5.1 Regular Expressions 23 5.2 Configuration Example 25 5.2.1 Scenario 25 5.2.2 Write a script to monitor the “last” status 25 5.2.3 Log sample 25 5.2.4 Collect the logs in a new log file 25 5.2.5 Restart “rsyslog” 26 5.2.6 Check whether the new entries are written in the new log file 26 5.2.7 Create a plugin file 26 5.2.8 Register the Plugin with the OSSIM Agent 29 5.2.9 Register the Plugin with the OSSIM Server 30 5.2.10 Check whether the plugin was successfully registered 31 5.2.11 Restart the OSSIM Server 31 5.2.12 Restart the OSSIM Agent 31 5.2.13 Check whether Events and Alarms are received 32 Building Collector Plugins - Admin Guide Page 4 Copyright © Alienvault 2010 1 Overview 1.1 OSSIM Agent Role 1.1.1 Event Collection The collection process involves extracting the data logs from the source systems (Security, OS, RDBMS, etc.) and allows first steps for event log filtering. At this stage can be decided what is going to be read by the OSSIM Agent and what is going to be discarded before having an impact on the system performance. Before starting to write a plugin some actions to reduce the amount of events could be considered: - Manage the log level settings at the application and managed device level - Fix the problem that is generating events - Use Pcap filters to ignore certain hosts or networks (Snort, Tcpdump ) - In deployments with a big amount of analysed data, filtering at the application level should be done whenever possible o Log Files Good practice is to use one log file per plugin in order to increase performance. Having just one generic log file, all the plugins would have to read the same extensive content in order to catch the few relevant entries. Using rsyslog it is possible to filter the collected logs based on the syslog tags. 1.1.2 Event Normalization In the normalization stage a series of rules or functions applies to the data extracted from the source system in order to transform it in a common OSSIM format. o Raw Event The raw event might be a generic syslog message, an application log, an SNMP trap, the result of an SNMP or SQL Query or some other kind of information in a more or less structured form that is appended to a log file. Example: dmz01:/var/log/auth.log: May 30 13:15:52 dmz01 sshd[12980]: Accepted password for root from 192.168.178.20 port 4445 ssh2 Building Collector Plugins - Admin Guide Page 5 Copyright © Alienvault 2010 o Normalized Event There is a certain set of fields which are required in order to ensure a consistent evaluation and correlation of the events by the OSSIM server. These fields can be populated with information from the log message or statically through the plug-in. Example: ossim-sensor:/var/log/ossim/agent.log: 2010-05-30 13:15:49,441 Output [INFO]: event type="detector" date="1275239752" sensor="192.168.178.201" interface="eth0" plugin_id="4003" plugin_sid="7" src_ip="192.168.178.20" src_port="4445" dst_ip="192.168.178.200" dst_port="22" username="root" log="May 30 13:15:52 dmz01 sshd[12980]: Accepted password for root from 192.168.178.20 port 4445 ssh2" fdate="2010-05-30 13:15:52" tzone="0" Building Collector Plugins - Admin Guide Page 6 Copyright © Alienvault 2010 1.2 OSSIM Server Role 1.2.1 Event Enrichment The OSSIM server enriches the received normalized event with the metadata stored in the OSSIM Database. o Enriched Event The OSSIM Server enriches the event with the Priority and Reliability values, which are specific to the event type (plugin_id) and subtype (plugin_sid), as well as with the Asset Value which is specific to the Source (asset_src) and the Destination (asset_dst) hosts. Example: ossim:/var/log/ossim/server.log: 2010-05-30 06:48:41 OSSIM-Message: Event received: event id="0" alarm="0" type="detector" fdate="2010-05-30 13:15:52" date="1275239752" tzone="0" plugin_id="4003" plugin_sid="7" src_ip="192.168.178.20" src_port="4445" dst_ip="192.168.178.200" dst_port="22" sensor="192.168.178.201" interface="eth0" protocol="TCP" asset_src="2" asset_dst="2" log="May 30 13:15:52 dmz01 sshd[12980]: Accepted password for root from 192.168.178.20 port 4445 ssh2" username="root" o Priority The priority is related to threats and it reflects the importance of a specific attack, having nothing to do with a specific host or environment. It only measures the relative importance of the attack itself. Range: 0 - 5 Default value: 1 Example: A Unix server running Samba gets attacked by the Sasser worm . Apart from the fact that the attack won’t have an impact on the given environment, it has the potential to exploit a big security hole and for that reason the priority is considered as being high. o Reliability Classical risk-assessment would refer it as "probability ". Since it's quite difficult to determine how probable it is for a network to be exposed to certain vulnerabilities, the IDS related “reliability” approach was considered more appropriate. Range: 0 - 10 Default value : 1. Example: If a host connects to 5 different hosts in the same subnet using port 445, could be a normal behavior, unreliable for IDS purposes. If connecting to 15 hosts would be Building Collector Plugins - Admin Guide Page 7 Copyright © Alienvault 2010 suspicious, with 500 connections to different hosts in less than an hour the attack would get more and more reliable. o Asset Value It is assigned to both the Source and the Destination Hosts and represents the importance the host has to the enterprise. Range: 0 - 5 Default value: 1 (also used for hosts not being defined in the asset database) Example: A database server can have an asset value of 5, a development test server an asset value of 2 and an unknown host in the Internet causing a portscan event would just have an asset value of 1. o Alarm Based on the Event Priority (0-5), Event Reliability (0-10) and the Asset Value (0-5), a Risk Value (0-10) is calculated and for values equal or greater than 1 Alerts are generated. The Risk is calculated based on the following formula: Risk = (Priority * Reliability * Asset) / 25 1.2.2 Policies and Actions Policies are defined in order to define what has to be done with the events as they reach the OSSIM Server: Correlation (i.e. checked against the correlation directives) Forwarding (i.e. one copy is sent to the forensic storage) Actions (i.e. send an email) Discard - the last filter possibility before saving the event in the database, although it is recommended to filter the events as close to the source as possible. Policies can make decisions on which events are going to be filtered based on: Source and Destination Assets (Hosts, Networks, ANY ) Ports Plugin Group Time Range Building Collector Plugins - Admin Guide Page 8 Copyright © Alienvault 2010 1.3 The Configuration Workflow o Collect a Log Sample First thing to start with is checking which log messages the application generates and eventually identify sets of logs having a similar structure. Those logs having a similar structure will be where possible covered by a single collector rule. o Create a Plugin File Best is to copy one existing file and modify its content to match the new application. Should a plugin exist for a similar application, it is recommended to copy such a file, as there is a good chance that rules have a similar content and are grouped in a similar way - a generic HTTP-Proxy log will always contain a URL, a generic Firewall log will contain a Source IP Address and Source Port as well as a Destination IP Address and Destination Port. Some user defined fields might be defined for a specific application and the correlation at the server level can be simplified if similar applications use the same user defined fields. o Define a Generic Rule This is the last Rule to evaluate, which catches all the events that cannot be grouped under specific rules. o Define Specific Rules The Specific rules are defined for specific error conditions or categories of events. There might also be that one single rule is used to generate different types or subtypes of events. o Discard Noise Events that are considered noise can be discarded by OSSIM by excluding certain event subtypes (Plugin_SIDs) in the plugin file, by the way the regular expressions are defined or by using policies. However, the best way to discard events is by filtering them on the monitored device or at syslog level on the host running the OSSIM Agent. o Review the Evaluation Order The rules are evaluated alphabetically, which means that all it counts is the name of a rule and not the position in the plug-in file. The Generic Rule might even be on the first position if the name is properly chosen. Having rules alphabetically placed after the Generic Rule will have as effect that the corresponding logs will be evaluated as generic events instead of having the proper event type and subtype assigned. o Register the Plugin with the OSSIM Agent In order to have a Plugin activated and sending events to the OSSIM server, the path to the plugin file has to be specified in the Agent configuration file. Building Collector Plugins - Admin Guide Page 9 Copyright © Alienvault 2010 o Register the Plugin with the OSSIM Server This is required in order to let the server know which events should be expected and which priority and reliability values the events should get assigned. o Activate the Plugin on the Server Side Restart the OSSIM Server process. o Activate the Plugin on the Agent Side Restart the OSSIM Agent process. o Testing Using the logger command sample logs can be replayed in order to test the operation of the OSSIM Agent or Server. Building Collector Plugins - Admin Guide Page 10 Copyright © Alienvault 2010 2 Configuring Detector Plugins 2.1 Rsyslog Rsyslog is the Syslog implementation shipped with OSSIM and allows configuring filtering and forwarding in a really easy way compared to the classical syslog daemon. Syslog is also the common method to send and receive logs. Before starting with the plugin configuration it is recommended to check whether the subset of logs the plugin will normalize are saved in an individual file and whether noise can be filtered before reaching the plugin rules. 2.1.1 Configuration File /etc/rsyslog.conf 2.1.2 Listener Configuration $ModLoad imudp $UDPServerRun 514 $ModLoad imtcp $InputTCPServerRun 514 2.1.3 Filters Forward certain events to a local file if $msg contains 'error' then /var/log/error if $syslogfacility-text == 'local0' and $msg startswith 'DEVNAME' and ($msg contains 'error1' or $msg contains 'error0') then /var/log/somelog Stop processing some events if $msg contains 'error' then ~ Regex in Rsyslog http://www.rsyslog.com/user-regex.php [...]... ĨŽƌ ĞĂĐŚ ƉůƵŐŝŶ͗ - WĂŐĞ ϮϬ hƐŝŶŐ ƚŚĞ ŽƉƚŝŽŶ ĞdžĐůƵĚĞͺƐŝĚƐ DŽĚŝĨLJŝŶŐ ƚŚĞ ƌĞŐƵůĂƌ ĞdžƉƌĞƐƐŝŽŶƐ ƚŽ ĂǀŽŝĚ ŵĂƚĐŚŝŶŐ ĐĞƌƚĂŝŶ ĞǀĞŶƚƐ ŽƉLJƌŝŐŚƚ Ξ ůŝĞŶǀĂƵůƚ ϮϬϭϬ ƵŝůĚŝŶŐ ŽůůĞĐƚŽƌ WůƵŐŝŶƐ Ͳ ĚŵŝŶ 'ƵŝĚĞ 2.8 Loading Plugins 2.8.1 Priority and Reliability values &Žƌ ĞĂĐŚ WůƵŐŝŶͺ/ͬWůƵŐŝŶͺ^/ ƉĂŝƌ ƚŚĞ WƌŝŽƌŝƚLJ ĂŶĚ ZĞůŝĂďŝůŝƚLJ ǀĂůƵĞƐ ǁŝůů ŚĂǀĞ ƚŽ ďĞ ĚĞĨŝŶĞĚ ǁŚŝůĞ ƌĞŐŝƐƚĞƌŝŶŐ ƚŚĞ ƉůƵŐŝŶ ǁŝƚŚ ƚŚĞ K^^/D ^ĞƌǀĞƌ͘ 2.8.2 SQL