grok makes logstash dies when trying to parse an invalid string

Description

logstash / grok shouldnt die when parsing invalid strings.

This is the dump after the crash:

thread watchdog timeout {:thread=>#<Thread:0x11c7f72d run>, :backtrace=>["file:/iad/common/apps/logstash/logstash-1.1.9-monolithic.jar!/logstash/filterworker.rb:38:in `backtrace'", "file:/iad/common/apps/logstash/logstash-1.1.9-monolithic.jar!/logstash/filterworker.rb:38:in `run'", "file:/iad/common/apps/logstash/logstash-1.1.9-monolithic.jar!/logstash/agent.rb:738:in `each'", "file:/iad/common/apps/logstash/logstash-1.1.9-monolithic.jar!/logstash/agent.rb:738:in `run_filter'", "file:/iad/common/apps/logstash/logstash-1.1.9-monolithic.jar!/logstash/agent.rb:452:in `run_with_config'"], :thread_watchdog=>2013-04-17 13:23:55 0200, :age=>10.544, :cutoff=>10, :state=>{:event=>#<LogStash::Event:0x6edb35f8 @data={"@source"=>"file://dev1/iad/finn/devrecruitment/logs/apache/access_log-dev.finn.no-recruitment", "@tags"=>[], "@fields"=>{}, "@timestamp"=>"2013-04-17T11:23:54.882Z", "@source_host"=>"dev1", "@source_path"=>"/iad/finn/devrecruitment/logs/apache/access_log-dev.finn.no-recruitment", "@message"=>"home.schibsted.no - - [16/Apr/2013:18:01:50 +0200] \"GET /recruitment/health/smoketest?callback=jQuery19106504244991728602_1366128144067&=1366128144068 HTTP/1.1\" 200 75 \"http://192.168.41.45:3000/smoketest.html\" \"Mozilhome.schibsted.no - - [16/Apr/2013:18:02:12 +0200] \"GET /recruitment/health/smoketest?callback=jQuery19100761695952632111_1366128123412&=1366128123413 HTTP/1.1\" 200 75 \"http://localhost:3000/smoketest.html\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:20.0) Gecko/20100101 Firefox/20.0\" \"UW12BFBbIzEAABeu1fIAAASC\"", "@type"=>"apache-access"}, @cancelled=false>, :filter=>#<LogStash::Filters::Grok:0x4e8aa935 @remove_tag=[], @singles=false, @named_captures_only=true, @pattern=["%{COMBINEDAPACHELOG} \\\"%{GREEDYDATA:unique_id}\\\"", "%{COMBINEDAPACHELOG}", "%{COMMONAPACHELOG}"], @add_tag=[], @drop_if_match=false, @tags=[], @type="apache-access", @keep_empty_captures=false, @params={"type"=>"apache-access", "patterns_dir"=>["/iad/local/etc/logstash.d/grok_patterns"], "pattern"=>["%{COMBINEDAPACHELOG} \\\"%{GREEDYDATA:unique_id}\\\"", "%{COMBINEDAPACHELOG}", "%{COMMONAPACHELOG}"], "tags"=>[], "exclude_tags"=>[], "add_tag"=>[], "remove_tag"=>[], "add_field"=>{}, "match"=>{"@message"=>["%{COMBINEDAPACHELOG} \\\"%{GREEDYDATA:unique_id}\\\"", "%{COMBINEDAPACHELOG}", "%{COMMONAPACHELOG}"]}, "drop_if_match"=>false, "break_on_match"=>true, "named_captures_only"=>true, "keep_empty_captures"=>false, "singles"=>false}, @logger=#<LogStash::Logger:0x470ae2bf @target=#<IO:fd 2>, @subscriber_lock=#<Mutex:0x5da0ff10>, @data={}, @metrics=#<Cabin::Metrics:0x74c9a375 @channel=#<Cabin::Channel:0x620a9239 @subscriber_lock=#<Mutex:0x24060e78>, @metrics=#<Cabin::Metrics:0x99ffac2 @channel=#<Cabin::Channel:0x620a9239 ...>, @metrics={}, @metrics_lock=#<Mutex:0x6607db7d>>, @data={}, @subscribers={}, @level=:info>, @metrics={}, @metrics_lock=#<Mutex:0xddb1fe0>>, @subscribers={2030=>#<Cabin::Outputs::IO:0x4bc0bec5 @io=#<IO:fd 2>, @lock=#<Mutex:0x4e511a6e>>}, @level=:debug>, @add_field={}, @patterns={"@message"=>#<Grok:ile:0x3792e652 @patterns={"NETSCREENSESSIONLOG"=>"%{SYSLOGTIMESTAMP:date} %{IPORHOST:device} %{IPORHOST}: NetScreen device_id=%{WORD:device_id}%{DATA}: start_time=%{QUOTEDSTRING:start_time} duration=%{INT:duration} policy_id=%{INTolicy_id} service=%{DATA:service} proto=%{INTroto} src zone=%{WORD:src_zone} dst zone=%{WORD:dst_zone} action=%{WORD:action} sent=%{INT:sent} rcvd=%{INT:rcvd} src=%{IPORHOST:src_ip} dst=%{IPORHOST:dst_ip} src_port=%{INT:src_port} dst_port=%{INT:dst_port} src-xlated ip=%{IPORHOST:src_xlated_ip} port=%{INT:src_xlated_port} dst-xlated ip=%{IPORHOST:dst_xlated_ip} port=%{INT:dst_xlated_port} session_id=%{INT:session_id} reason=%{GREEDYDATA:reason}", "HAPROXYTIME"=>"(?

[0-9])", "HAPROXYDATE"=>"%{MONTHDAY:haproxy_monthday}/%{MONTH:haproxy_month}/%{YEAR:haproxy_year}:%{HAPROXYTIME:haproxy_time}.%{INT:haproxy_milliseconds}", "HAPROXYCAPTUREDREQUESTHEADERS"=>"%{DATA:captured_request_headers}", "HAPROXYCAPTUREDRESPONSEHEADERS"=>"%{DATA:captured_response_headers}", "HAPROXYHTTP"=>"%{SYSLOGTIMESTAMP:syslog_timestamp} %{IPORHOST:syslog_server} %{SYSLOGPROG}: %{IP:client_ip}:%{INT:client_port} \\[%{HAPROXYDATE:accept_date}
] %{NOTSPACE:frontend_name} %{NOTSPACE:backend_name}/%{NOTSPACE:server_name} %{INT:time_request}/%{INT:time_queue}/%{INT:time_backend_connect}/%{INT:time_backend_response}/%{NOTSPACE:time_duration} %{INT:http_status_code} %{NOTSPACE:bytes_read} %{DATA:captured_request_cookie} %{DATA:captured_response_cookie} %{NOTSPACE:termination_state} %{INT:actconn}/%{INT:feconn}/%{INT:beconn}/%{INT:srvconn}/%{NOTSPACE:retries} %{INT:srv_queue}/%{INT:backend_queue} (\\{%{HAPROXYCAPTUREDREQUESTHEADERS}
})?( )?(\\{%{HAPROXYCAPTUREDRESPONSEHEADERS}
})?( )?\"%{WORD:http_verb} %{URIPATHPARAM:http_request}( HTTP/%{NUMBER:http_version}\")?", "HAPROXYTCP"=>"%{SYSLOGTIMESTAMP:syslog_timestamp} %{IPORHOST:syslog_server} %{SYSLOGPROG}: %{IP:client_ip}:%{INT:client_port} \\[%{HAPROXYDATE:accept_date}
] %{NOTSPACE:frontend_name} %{NOTSPACE:backend_name}/%{NOTSPACE:server_name} %{INT:time_queue}/%{INT:time_backend_connect}/%{NOTSPACE:time_duration} %{NOTSPACE:bytes_read} %{NOTSPACE:termination_state} %{INT:actconn}/%{INT:feconn}/%{INT:beconn}/%{INT:srvconn}/%{NOTSPACE:retries} %{INT:srv_queue}/%{INT:backend_queue}", "JAVACLASS"=>"(?:[a-zA-Z0-9-]
.)[A-Za-z0-9$]", "JAVAFILE"=>"(?:[A-Za-z0-9_.-])", "JAVASTACKTRACEPART"=>"at %{JAVACLASS:class}\\.%{WORD:method}\\(%{JAVAFILE:file}:%{NUMBER:line}
)", "SYSLOGBASE2"=>"(?:%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp8601}) (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}:", "SYSLOGPAMSESSION"=>"%{SYSLOGBASE} (?=%{GREEDYDATA:message})%{WORDam_module}\\(%{DATAam_caller}
): session %{WORDam_session_state} for user %{USERNAME:username}(?: by %{GREEDYDATAam_by})?", "CRON_ACTION"=>"[A-Z ]", "CRONLOG"=>"%{SYSLOGBASE} \\(%{USER:user}
) %{CRON_ACTION:action} \\(%{DATA:message}
)", "SYSLOGLINE"=>"%{SYSLOGBASE2} %{GREEDYDATA:message}", "NAGIOSTIME"=>"\\[%{NUMBER:nagios_epoch}
]", "NAGIOS_TYPE_CURRENT_SERVICE_STATE"=>"CURRENT SERVICE STATE", "NAGIOS_TYPE_CURRENT_HOST_STATE"=>"CURRENT HOST STATE", "NAGIOS_TYPE_SERVICE_NOTIFICATION"=>"SERVICE NOTIFICATION", "NAGIOS_TYPE_HOST_NOTIFICATION"=>"HOST NOTIFICATION", "NAGIOS_TYPE_SERVICE_ALERT"=>"SERVICE ALERT", "NAGIOS_TYPE_HOST_ALERT"=>"HOST ALERT", "NAGIOS_TYPE_SERVICE_FLAPPING_ALERT"=>"SERVICE FLAPPING ALERT", "NAGIOS_TYPE_HOST_FLAPPING_ALERT"=>"HOST FLAPPING ALERT", "NAGIOS_TYPE_SERVICE_DOWNTIME_ALERT"=>"SERVICE DOWNTIME ALERT", "NAGIOS_TYPE_HOST_DOWNTIME_ALERT"=>"HOST DOWNTIME ALERT", "NAGIOS_TYPE_PASSIVE_SERVICE_CHECK"=>"PASSIVE SERVICE CHECK", "NAGIOS_TYPE_PASSIVE_HOST_CHECK"=>"PASSIVE HOST CHECK", "NAGIOS_TYPE_SERVICE_EVENT_HANDLER"=>"SERVICE EVENT HANDLER", "NAGIOS_TYPE_HOST_EVENT_HANDLER"=>"HOST EVENT HANDLER", "NAGIOS_TYPE_EXTERNAL_COMMAND"=>"EXTERNAL COMMAND", "NAGIOS_TYPE_TIMEPERIOD_TRANSITION"=>"TIMEPERIOD TRANSITION", "NAGIOS_EC_DISABLE_SVC_CHECK"=>"DISABLE_SVC_CHECK", "NAGIOS_EC_ENABLE_SVC_CHECK"=>"ENABLE_SVC_CHECK", "NAGIOS_EC_DISABLE_HOST_CHECK"=>"DISABLE_HOST_CHECK", "NAGIOS_EC_ENABLE_HOST_CHECK"=>"ENABLE_HOST_CHECK", "NAGIOS_EC_PROCESS_SERVICE_CHECK_RESULT"=>"PROCESS_SERVICE_CHECK_RESULT", "NAGIOS_EC_PROCESS_HOST_CHECK_RESULT"=>"PROCESS_HOST_CHECK_RESULT", "NAGIOS_EC_SCHEDULE_SERVICE_DOWNTIME"=>"SCHEDULE_SERVICE_DOWNTIME", "NAGIOS_EC_SCHEDULE_HOST_DOWNTIME"=>"SCHEDULE_HOST_DOWNTIME", "NAGIOS_WARNING"=>"Warning:%{SPACE}%{GREEDYDATA:nagios_message}", "NAGIOS_CURRENT_SERVICE_STATE"=>"%{NAGIOS_TYPE_CURRENT_SERVICE_STATE:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{DATA:nagios_statetype};%{DATA:nagios_statecode};%{GREEDYDATA:nagios_message}", "NAGIOS_CURRENT_HOST_STATE"=>"%{NAGIOS_TYPE_CURRENT_HOST_STATE:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_state};%{DATA:nagios_statetype};%{DATA:nagios_statecode};%{GREEDYDATA:nagios_message}", "NAGIOS_SERVICE_NOTIFICATION"=>"%{NAGIOS_TYPE_SERVICE_NOTIFICATION:nagios_type}: %{DATA:nagios_notifyname};%{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{DATA:nagios_contact};%{GREEDYDATA:nagios_message}", "NAGIOS_HOST_NOTIFICATION"=>"%{NAGIOS_TYPE_HOST_NOTIFICATION}: %{DATA:nagios_notifyname};%{DATA:nagios_hostname};%{DATA:nagios_state};%{DATA:nagios_contact};%{GREEDYDATA:nagios_message}", "NAGIOS_SERVICE_ALERT"=>"%{NAGIOS_TYPE_SERVICE_ALERT:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{DATA:nagios_statelevel};%{NUMBER:nagios_attempt};%{GREEDYDATA:nagios_message}", "NAGIOS_HOST_ALERT"=>"%{NAGIOS_TYPE_HOST_ALERT:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_state};%{DATA:nagios_statelevel};%{NUMBER:nagios_attempt};%{GREEDYDATA:nagios_message}", "NAGIOS_SERVICE_FLAPPING_ALERT"=>"%{NAGIOS_TYPE_SERVICE_FLAPPING_ALERT:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{GREEDYDATA:nagios_message}", "NAGIOS_HOST_FLAPPING_ALERT"=>"%{NAGIOS_TYPE_HOST_FLAPPING_ALERT:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_state};%{GREEDYDATA:nagios_message}", "NAGIOS_SERVICE_DOWNTIME_ALERT"=>"%{NAGIOS_TYPE_SERVICE_DOWNTIME_ALERT:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{GREEDYDATA:nagios_comment}", "NAGIOS_HOST_DOWNTIME_ALERT"=>"%{NAGIOS_TYPE_HOST_DOWNTIME_ALERT:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_state};%{GREEDYDATA:nagios_comment}", "NAGIOS_PASSIVE_SERVICE_CHECK"=>"%{NAGIOS_TYPE_PASSIVE_SERVICE_CHECK:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{GREEDYDATA:nagios_comment}", "NAGIOS_PASSIVE_HOST_CHECK"=>"%{NAGIOS_TYPE_PASSIVE_HOST_CHECK:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_state};%{GREEDYDATA:nagios_comment}", "NAGIOS_SERVICE_EVENT_HANDLER"=>"%{NAGIOS_TYPE_SERVICE_EVENT_HANDLER:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{DATA:nagios_statelevel};%{DATA:nagios_event_handler_name}", "NAGIOS_HOST_EVENT_HANDLER"=>"%{NAGIOS_TYPE_HOST_EVENT_HANDLER:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_state};%{DATA:nagios_statelevel};%{DATA:nagios_event_handler_name}", "NAGIOS_TIMEPERIOD_TRANSITION"=>"%{NAGIOS_TYPE_TIMEPERIOD_TRANSITION:nagios_type}: %{DATA:nagios_service};%{DATA:nagios_unknown1};%{DATA:nagios_unknown2};", "NAGIOS_EC_LINE_DISABLE_SVC_CHECK"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_DISABLE_SVC_CHECK:nagios_command};%{DATA:nagios_hostname};%{DATA:nagios_service}", "NAGIOS_EC_LINE_DISABLE_HOST_CHECK"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_DISABLE_HOST_CHECK:nagios_command};%{DATA:nagios_hostname}", "NAGIOS_EC_LINE_ENABLE_SVC_CHECK"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_ENABLE_SVC_CHECK:nagios_command};%{DATA:nagios_hostname};%{DATA:nagios_service}", "NAGIOS_EC_LINE_ENABLE_HOST_CHECK"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_ENABLE_HOST_CHECK:nagios_command};%{DATA:nagios_hostname}", "NAGIOS_EC_LINE_PROCESS_SERVICE_CHECK_RESULT"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_PROCESS_SERVICE_CHECK_RESULT:nagios_command};%{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{GREEDYDATA:nagios_check_result}", "NAGIOS_EC_LINE_PROCESS_HOST_CHECK_RESULT"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_PROCESS_HOST_CHECK_RESULT:nagios_command};%{DATA:nagios_hostname};%{DATA:nagios_state};%{GREEDYDATA:nagios_check_result}", "NAGIOS_EC_LINE_SCHEDULE_HOST_DOWNTIME"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_SCHEDULE_HOST_DOWNTIME:nagios_command};%{DATA:nagios_hostname};%{NUMBER:nagios_start_time};%{NUMBER:nagios_end_time};%{NUMBER:nagios_fixed};%{NUMBER:nagios_trigger_id};%{NUMBER:nagios_duration};%{DATA:author};%{DATA:comment}", "NAGIOSLOGLINE"=>"%{NAGIOSTIME} (?:%{NAGIOS_WARNING}|%{NAGIOS_CURRENT_SERVICE_STATE}|%{NAGIOS_CURRENT_HOST_STATE}|%{NAGIOS_SERVICE_NOTIFICATION}|%{NAGIOS_HOST_NOTIFICATION}|%{NAGIOS_SERVICE_ALERT}|%{NAGIOS_HOST_ALERT}|%{NAGIOS_SERVICE_FLAPPING_ALERT}|%{NAGIOS_HOST_FLAPPING_ALERT}|%{NAGIOS_SERVICE_DOWNTIME_ALERT}|%{NAGIOS_HOST_DOWNTIME_ALERT}|%{NAGIOS_PASSIVE_SERVICE_CHECK}|%{NAGIOS_PASSIVE_HOST_CHECK}|%{NAGIOS_SERVICE_EVENT_HANDLER}|%{NAGIOS_HOST_EVENT_HANDLER}|%{NAGIOS_TIMEPERIOD_TRANSITION}|%{NAGIOS_EC_LINE_DISABLE_SVC_CHECK}|%{NAGIOS_EC_LINE_ENABLE_SVC_CHECK}|%{NAGIOS_EC_LINE_DISABLE_HOST_CHECK|%{NAGIOS_EC_LINE_ENABLE_HOST_CHECK}|%{NAGIOS_EC_LINE_PROCESS_HOST_CHECK_RESULT}|%{NAGIOS_EC_LINE_PROCESS_SERVICE_CHECK_RESULT}|%{NAGIOS_EC_LINE_SCHEDULE_HOST_DOWNTIME})", "RUBY_LOGLEVEL"=>"(?EBUG|FATAL|ERROR|WARN|INFO)", "RUBY_LOGGER"=>"[DFEWI],
[%{TIMESTAMP_ISO8601:timestamp} #%{POSINTid}
] %{RUBY_LOGLEVEL:loglevel} -- %{DATArogname}: %{GREEDYDATA:message}", "USERNAME"=>"[a-zA-Z0-9_-]", "USER"=>"%{USERNAME}", "INT"=>"(?:[-]?(?:[0-9]))", "BASE10NUM"=>"(?<

[0-9A-Fa-f])(?:[-]?(?:0x)?(?:[0-9A-Fa-f]))", "BASE16FLOAT"=>"\\b(?<![0-9A-Fa-f.])(?:[-]?(?:0x)?(??:[0-9A-Fa-f](?:
.[0-9A-Fa-f])?)|(?:\\.[0-9A-Fa-f])))
b", "POSINT"=>"\\b(?:[1-9][0-9]*)
b", "NONNEGINT"=>"
b(?:[0-9])
b", "WORD"=>"\\b\\w+
b", "NOTSPACE"=>"
S+", "SPACE"=>"
s*", "DATA"=>".?", "GREEDYDATA"=>".", "QUOTEDSTRING"=>"(?>(?<

[0-9])(??:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2}))(?

$@:.,]

\\\\.)*)

", "LINUXTTY"=>"(?>/dev/pts/%{NONNEGINT})", "BSDTTY"=>"(?>/dev/tty[pq][a-z0-9])", "TTY"=>"(?:%{BSDTTY}|%{LINUXTTY})", "WINPATH"=>"(?>[A-Za-z]:|\\\\)(?:\\\\

*)", "URIPROTO"=>"[A-Za-z](
[A-Za-z+])?", "URIHOST"=>"%{IPORHOST}(?::%{POSINTort})?", "URIPATH"=>"(?:/[A-Za-z0-9$.!*'(){},~:;=#%_
])", "URIPARAM"=>"
?[A-Za-z0-9$.!'|(){},_#%&/=:;_?\\_\\[
_]]_{", "URIPATHPARAM"=>"%{URIPATH}(?:%{URIPARAM})?", "URI"=>"%{URIPROTO}://(?:%{USER}(?::}

_{)?@)?(?:%{URIHOST})?(?:%{URIPATHPARAM})?", "MONTH"=>"\\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)}
_{b", "MONTHNUM"=>"(?:0?[1-9]|1[0-2])", "MONTHDAY"=>"(?}_{?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])", "DAY"=>"(?:Mon(?:day)?|Tue(?:sday)?|Wed(?:nesday)?|Thu(?:rsday)?|Fri(?:day)?|Sat(?:urday)?|Sun(?:day)?)", "YEAR"=>"(?>\\d}
_{d){1,2}", "HOUR"=>"(?:2[0123]|[01][0-9])", "MINUTE"=>"(?:[0-5][0-9])", "SECOND"=>"(?}_{?:[0-5][0-9]|60)(?:[.,][0-9]+)?)", "TIME"=>"(?}

_{[0-9])", "DATE_US"=>"%{MONTHNUM}}_/-_%{MONTHDAY}_/-_{%{YEAR}", "DATE_EU"=>"%{YEAR}[./}_{]%{MONTHNUM}[./}_{]%{MONTHDAY}", "ISO8601_TIMEZONE"=>"(?:Z|[+}_{]%{HOUR}(?::?%{MINUTE}))", "ISO8601_SECOND"=>"(?:%{SECOND}|60)", "TIMESTAMP_ISO8601"=>"%{YEAR}}_%{MONTHNUM}_{%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE}?", "DATE"=>"%{DATE_US}|%{DATE_EU}", "DATESTAMP"=>"%{DATE}[- ]%{TIME}", "TZ"=>"(?:[PMCE][SD]T)", "DATESTAMP_RFC822"=>"%{DAY} %{MONTH} %{MONTHDAY} %{YEAR} %{TIME} %{TZ}", "DATESTAMP_OTHER"=>"%{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{TZ} %{YEAR}", "SYSLOGTIMESTAMP"=>"%{MONTH} +%{MONTHDAY} %{TIME}", "PROG"=>"(?:[}
_w._/%_{]+)", "SYSLOGPROG"=>"%{PROG}_{rogram}(?:\\[%{POSINT}_id}
_{])?", "SYSLOGHOST"=>"%{IPORHOST}", "SYSLOGFACILITY"=>"<%{NONNEGINT:facility}.%{NONNEGINT}_{riority}>", "HTTPDATE"=>"%{MONTHDAY}/%{MONTH}/%{YEAR}:%{TIME} %{INT}", "QS"=>"%{QUOTEDSTRING}", "SYSLOGBASE"=>"%{SYSLOGTIMESTAMP:timestamp} (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}:", "COMBINEDAPACHELOG"=>"%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \\[%{HTTPDATE:timestamp}}
_{] \"(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|}_{)\" %{NUMBER:response} (?:%{NUMBER:bytes}|}_{) %{QS:referrer} %{QS:agent}", "LOGLEVEL"=>"([T|t]race|TRACE|[D|d]ebug|DEBUG|[N|n]otice|NOTICE|[I|i]nfo|INFO|[W|w]arn?(?:ing)?|WARN?(?:ING)?|[E|e]rr?(?:or)?|ERR?(?:OR)?|[C|c]rit?(?:ical)?|CRIT?(?:ICAL)?|[F|f]atal|FATAL|[S|s]evere|SEVERE)"}, @pattern_files=["/iad/local/etc/logstash.d/grok_patterns/haproxy", "/iad/local/etc/logstash.d/grok_patterns/up-grok", "/iad/local/etc/logstash.d/grok_patterns/ruby", "/iad/local/etc/logstash.d/grok_patterns/nagios", "/iad/local/etc/logstash.d/grok_patterns/firewalls", "/iad/local/etc/logstash.d/grok_patterns/linux-syslog", "/iad/local/etc/logstash.d/grok_patterns/java", "/iad/local/etc/logstash.d/grok_patterns/grok-patterns", "/iad/local/etc/logstash.d/grok_patterns/finn"], @logger=#<Cabin::Channel:0x6cfce051 @subscriber_lock=#<Mutex:0x55519d9b>, @metrics=#<Cabin::Metrics:0x614256df @channel=#<Cabin::Channel:0x6cfce051 ...>, @metrics={}, @metrics_lock=#<Mutex:0x4a557c5b>>, @data={}, @subscribers={4096=>#<Cabin::Outputs::StdlibLogger:0x7c537c91 @logger=#<Logger:0x4d17535b @logdev=#<Logger::LogDevice:0x3661a2e7 @shift_age=nil, @filename=nil, @dev=#<IO:fd 1>, @mutex=#<Logger::LogDevice::LogDeviceMutex:0x440e0795 @mon_count=0, @mon_mutex=#<Mutex:0x1037d830>, @mon_owner=nil>, @shift_size=nil>, @formatter=nil, @progname=nil, @default_formatter=#<Logger::Formatter:0x42523fa1 @datetime_format=nil>, @level=0>>}, @level=:warn>, @groks=[#<Grok:0x5b27a891 @regexp=/(?<a0>(?<a1>(?}_{?<a2>\b(?:[0-9A-Za-z][0-9A-Za-z}_{]{0,62})(?:\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))}_{(?:\.(?:[0-9A-Za-z]{0,63}))(\.?|\b))|(?<a3>(?<}

_{[0-9])))) (?<a4>(?<a5>[a-zA-Z0-9_}_{]+)) (?<a6>(?<a7>[a-zA-Z0-9_}_{])) [(?<a8>(?<a9>(?}_{?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]))\/(?<a10>\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\b)\/(?<a11>[0-9])}_?<a12>(?

_{[0-9])) (?<a16>(?:[-]?(?:[0-9]))))] "(?<a17>\b\w+\b) (?<a18>(?<a19>(?:\/[A-Za-z0-9$.+!}_'(),:#%-]))(??<a20>?[A-Za-z0-9$.!'(),~#%&\/=:;]*))?) HTTP\/(?<a21>(??<a22>(?<![0-9.+])(?>[-]?(??:[0-9](?:\.[0-9])?)|(?:\.[0-9]))))))" (?<a23>(??<a24>(?<

[0-9.-])(?>[]?(??:[0-9](?:\.[0-9])?)|(?:\.[0-9]+))))))|) "(??<a27>(?<a28>[A-Za-z](+[A-Za-z])?):\/\/(??<a29>(?<a30>[a-zA-Z0-9_-]))(?::

)?@)?(??<a31>(?<a32>(??<a33>\b(?:[0-9A-Za-z][0-9A-Za-z-]{0,62})(?:\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))(?:\.(?:[0-9A-Za-z]{0,63}))(\.?|\b))|(?<a34>(?<

[0-9]))))(?:?<a35>\b(?:[0-9])\b))?))?(??<a36>(?<a37>(?:\/[A-Za-z0-9$.

'(),~#%&\/=:;_-]))?))?)|)" (?<a39>(?<a40>(??<

<[0-9])%{HOUR:haproxy_hour}:%{MINUTE:haproxy_minute}(?::%{SECOND:haproxy_second})(?![0-9])", "HAPROXYDATE"=>"%{MONTHDAY:haproxy_monthday}/%{MONTH:haproxy_month}/%{YEAR:haproxy_year}:%{HAPROXYTIME:haproxy_time}.%{INT:haproxy_milliseconds}", "HAPROXYCAPTUREDREQUESTHEADERS"=>"%{DATA:captured_request_headers}", "HAPROXYCAPTUREDRESPONSEHEADERS"=>"%{DATA:captured_response_headers}", "HAPROXYHTTP"=>"%{SYSLOGTIMESTAMP:syslog_timestamp} %{IPORHOST:syslog_server} %{SYSLOGPROG}: %{IP:client_ip}:%{INT:client_port} \\[%{HAPROXYDATE:accept_date}
] %{NOTSPACE:frontend_name} %{NOTSPACE:backend_name}/%{NOTSPACE:server_name} %{INT:time_request}/%{INT:time_queue}/%{INT:time_backend_connect}/%{INT:time_backend_response}/%{NOTSPACE:time_duration} %{INT:http_status_code} %{NOTSPACE:bytes_read} %{DATA:captured_request_cookie} %{DATA:captured_response_cookie} %{NOTSPACE:termination_state} %{INT:actconn}/%{INT:feconn}/%{INT:beconn}/%{INT:srvconn}/%{NOTSPACE:retries} %{INT:srv_queue}/%{INT:backend_queue} \\{%{HAPROXYCAPTUREDREQUESTHEADERS}
} \\{%{HAPROXYCAPTUREDRESPONSEHEADERS}
} \"%{WORD:http_verb} %{URIPATHPARAM:http_request} HTTP/%{NUMBER:http_version}\"", "HAPROXYTCP"=>"%{SYSLOGTIMESTAMP:syslog_timestamp} %{IPORHOST:syslog_server} %{SYSLOGPROG}: %{IP:client_ip}:%{INT:client_port} \\[%{HAPROXYDATE:accept_date}
] %{NOTSPACE:frontend_name} %{NOTSPACE:backend_name}/%{NOTSPACE:server_name} %{INT:time_queue}/%{INT:time_backend_connect}/%{NOTSPACE:time_duration} %{NOTSPACE:bytes_read} %{NOTSPACE:termination_state} %{INT:actconn}/%{INT:feconn}/%{INT:beconn}/%{INT:srvconn}/%{NOTSPACE:retries} %{INT:srv_queue}/%{INT:backend_queue}", "JAVACLASS"=>"(?:[a-zA-Z0-9]
.)[A-Za-z0-9]", "JAVAFILE"=>"(?:[A-Za-z0-9_.-])", "JAVASTACKTRACEPART"=>"at %{JAVACLASS:class}\\.%{WORD:method}\\(%{JAVAFILE:file}:%{NUMBER:line}
)", "SYSLOGBASE2"=>"(?:%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp8601}) (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}:", "SYSLOGPAMSESSION"=>"%{SYSLOGBASE} (?=%{GREEDYDATA:message})%{WORDam_module}\\(%{DATAam_caller}
): session %{WORDam_session_state} for user %{USERNAME:username}(?: by %{GREEDYDATAam_by})?", "CRON_ACTION"=>"[A-Z ]", "CRONLOG"=>"%{SYSLOGBASE} \\(%{USER:user}
) %{CRON_ACTION:action} \\(%{DATA:message}
)", "SYSLOGLINE"=>"%{SYSLOGBASE2} %{GREEDYDATA:message}", "NAGIOSTIME"=>"\\[%{NUMBER:nagios_epoch}
]", "NAGIOS_TYPE_CURRENT_SERVICE_STATE"=>"CURRENT SERVICE STATE", "NAGIOS_TYPE_CURRENT_HOST_STATE"=>"CURRENT HOST STATE", "NAGIOS_TYPE_SERVICE_NOTIFICATION"=>"SERVICE NOTIFICATION", "NAGIOS_TYPE_HOST_NOTIFICATION"=>"HOST NOTIFICATION", "NAGIOS_TYPE_SERVICE_ALERT"=>"SERVICE ALERT", "NAGIOS_TYPE_HOST_ALERT"=>"HOST ALERT", "NAGIOS_TYPE_SERVICE_FLAPPING_ALERT"=>"SERVICE FLAPPING ALERT", "NAGIOS_TYPE_HOST_FLAPPING_ALERT"=>"HOST FLAPPING ALERT", "NAGIOS_TYPE_SERVICE_DOWNTIME_ALERT"=>"SERVICE DOWNTIME ALERT", "NAGIOS_TYPE_HOST_DOWNTIME_ALERT"=>"HOST DOWNTIME ALERT", "NAGIOS_TYPE_PASSIVE_SERVICE_CHECK"=>"PASSIVE SERVICE CHECK", "NAGIOS_TYPE_PASSIVE_HOST_CHECK"=>"PASSIVE HOST CHECK", "NAGIOS_TYPE_SERVICE_EVENT_HANDLER"=>"SERVICE EVENT HANDLER", "NAGIOS_TYPE_HOST_EVENT_HANDLER"=>"HOST EVENT HANDLER", "NAGIOS_TYPE_EXTERNAL_COMMAND"=>"EXTERNAL COMMAND", "NAGIOS_TYPE_TIMEPERIOD_TRANSITION"=>"TIMEPERIOD TRANSITION", "NAGIOS_EC_DISABLE_SVC_CHECK"=>"DISABLE_SVC_CHECK", "NAGIOS_EC_ENABLE_SVC_CHECK"=>"ENABLE_SVC_CHECK", "NAGIOS_EC_DISABLE_HOST_CHECK"=>"DISABLE_HOST_CHECK", "NAGIOS_EC_ENABLE_HOST_CHECK"=>"ENABLE_HOST_CHECK", "NAGIOS_EC_PROCESS_SERVICE_CHECK_RESULT"=>"PROCESS_SERVICE_CHECK_RESULT", "NAGIOS_EC_PROCESS_HOST_CHECK_RESULT"=>"PROCESS_HOST_CHECK_RESULT", "NAGIOS_EC_SCHEDULE_SERVICE_DOWNTIME"=>"SCHEDULE_SERVICE_DOWNTIME", "NAGIOS_EC_SCHEDULE_HOST_DOWNTIME"=>"SCHEDULE_HOST_DOWNTIME", "NAGIOS_WARNING"=>"Warning:%{SPACE}%{GREEDYDATA:nagios_message}", "NAGIOS_CURRENT_SERVICE_STATE"=>"%{NAGIOS_TYPE_CURRENT_SERVICE_STATE:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{DATA:nagios_statetype};%{DATA:nagios_statecode};%{GREEDYDATA:nagios_message}", "NAGIOS_CURRENT_HOST_STATE"=>"%{NAGIOS_TYPE_CURRENT_HOST_STATE:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_state};%{DATA:nagios_statetype};%{DATA:nagios_statecode};%{GREEDYDATA:nagios_message}", "NAGIOS_SERVICE_NOTIFICATION"=>"%{NAGIOS_TYPE_SERVICE_NOTIFICATION:nagios_type}: %{DATA:nagios_notifyname};%{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{DATA:nagios_contact};%{GREEDYDATA:nagios_message}", "NAGIOS_HOST_NOTIFICATION"=>"%{NAGIOS_TYPE_HOST_NOTIFICATION}: %{DATA:nagios_notifyname};%{DATA:nagios_hostname};%{DATA:nagios_state};%{DATA:nagios_contact};%{GREEDYDATA:nagios_message}", "NAGIOS_SERVICE_ALERT"=>"%{NAGIOS_TYPE_SERVICE_ALERT:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{DATA:nagios_statelevel};%{NUMBER:nagios_attempt};%{GREEDYDATA:nagios_message}", "NAGIOS_HOST_ALERT"=>"%{NAGIOS_TYPE_HOST_ALERT:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_state};%{DATA:nagios_statelevel};%{NUMBER:nagios_attempt};%{GREEDYDATA:nagios_message}", "NAGIOS_SERVICE_FLAPPING_ALERT"=>"%{NAGIOS_TYPE_SERVICE_FLAPPING_ALERT:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{GREEDYDATA:nagios_message}", "NAGIOS_HOST_FLAPPING_ALERT"=>"%{NAGIOS_TYPE_HOST_FLAPPING_ALERT:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_state};%{GREEDYDATA:nagios_message}", "NAGIOS_SERVICE_DOWNTIME_ALERT"=>"%{NAGIOS_TYPE_SERVICE_DOWNTIME_ALERT:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{GREEDYDATA:nagios_comment}", "NAGIOS_HOST_DOWNTIME_ALERT"=>"%{NAGIOS_TYPE_HOST_DOWNTIME_ALERT:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_state};%{GREEDYDATA:nagios_comment}", "NAGIOS_PASSIVE_SERVICE_CHECK"=>"%{NAGIOS_TYPE_PASSIVE_SERVICE_CHECK:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{GREEDYDATA:nagios_comment}", "NAGIOS_PASSIVE_HOST_CHECK"=>"%{NAGIOS_TYPE_PASSIVE_HOST_CHECK:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_state};%{GREEDYDATA:nagios_comment}", "NAGIOS_SERVICE_EVENT_HANDLER"=>"%{NAGIOS_TYPE_SERVICE_EVENT_HANDLER:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{DATA:nagios_statelevel};%{DATA:nagios_event_handler_name}", "NAGIOS_HOST_EVENT_HANDLER"=>"%{NAGIOS_TYPE_HOST_EVENT_HANDLER:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_state};%{DATA:nagios_statelevel};%{DATA:nagios_event_handler_name}", "NAGIOS_TIMEPERIOD_TRANSITION"=>"%{NAGIOS_TYPE_TIMEPERIOD_TRANSITION:nagios_type}: %{DATA:nagios_service};%{DATA:nagios_unknown1};%{DATA:nagios_unknown2};", "NAGIOS_EC_LINE_DISABLE_SVC_CHECK"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_DISABLE_SVC_CHECK:nagios_command};%{DATA:nagios_hostname};%{DATA:nagios_service}", "NAGIOS_EC_LINE_DISABLE_HOST_CHECK"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_DISABLE_HOST_CHECK:nagios_command};%{DATA:nagios_hostname}", "NAGIOS_EC_LINE_ENABLE_SVC_CHECK"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_ENABLE_SVC_CHECK:nagios_command};%{DATA:nagios_hostname};%{DATA:nagios_service}", "NAGIOS_EC_LINE_ENABLE_HOST_CHECK"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_ENABLE_HOST_CHECK:nagios_command};%{DATA:nagios_hostname}", "NAGIOS_EC_LINE_PROCESS_SERVICE_CHECK_RESULT"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_PROCESS_SERVICE_CHECK_RESULT:nagios_command};%{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{GREEDYDATA:nagios_check_result}", "NAGIOS_EC_LINE_PROCESS_HOST_CHECK_RESULT"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_PROCESS_HOST_CHECK_RESULT:nagios_command};%{DATA:nagios_hostname};%{DATA:nagios_state};%{GREEDYDATA:nagios_check_result}", "NAGIOS_EC_LINE_SCHEDULE_HOST_DOWNTIME"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_SCHEDULE_HOST_DOWNTIME:nagios_command};%{DATA:nagios_hostname};%{NUMBER:nagios_start_time};%{NUMBER:nagios_end_time};%{NUMBER:nagios_fixed};%{NUMBER:nagios_trigger_id};%{NUMBER:nagios_duration};%{DATA:author};%{DATA:comment}", "NAGIOSLOGLINE"=>"%{NAGIOSTIME} (?:%{NAGIOS_WARNING}|%{NAGIOS_CURRENT_SERVICE_STATE}|%{NAGIOS_CURRENT_HOST_STATE}|%{NAGIOS_SERVICE_NOTIFICATION}|%{NAGIOS_HOST_NOTIFICATION}|%{NAGIOS_SERVICE_ALERT}|%{NAGIOS_HOST_ALERT}|%{NAGIOS_SERVICE_FLAPPING_ALERT}|%{NAGIOS_HOST_FLAPPING_ALERT}|%{NAGIOS_SERVICE_DOWNTIME_ALERT}|%{NAGIOS_HOST_DOWNTIME_ALERT}|%{NAGIOS_PASSIVE_SERVICE_CHECK}|%{NAGIOS_PASSIVE_HOST_CHECK}|%{NAGIOS_SERVICE_EVENT_HANDLER}|%{NAGIOS_HOST_EVENT_HANDLER}|%{NAGIOS_TIMEPERIOD_TRANSITION}|%{NAGIOS_EC_LINE_DISABLE_SVC_CHECK}|%{NAGIOS_EC_LINE_ENABLE_SVC_CHECK}|%{NAGIOS_EC_LINE_DISABLE_HOST_CHECK|%{NAGIOS_EC_LINE_ENABLE_HOST_CHECK}|%{NAGIOS_EC_LINE_PROCESS_HOST_CHECK_RESULT}|%{NAGIOS_EC_LINE_PROCESS_SERVICE_CHECK_RESULT}|%{NAGIOS_EC_LINE_SCHEDULE_HOST_DOWNTIME})", "RUBY_LOGLEVEL"=>"(?EBUG|FATAL|ERROR|WARN|INFO)", "RUBY_LOGGER"=>"[DFEWI],
[%{TIMESTAMP_ISO8601} #{POSINTid}
] *%{RUBY_LOGLEVEL} -- %{DATArogname}: %{DATA:message}", "USERNAME"=>"[a-zA-Z0-9_-]", "USER"=>"%{USERNAME}", "INT"=>"(?:[-]?(?:[0-9]))", "BASE10NUM"=>"(?<

[0-9A-Fa-f])(?:[-]?(?:0x)?(?:[0-9A-Fa-f]))", "BASE16FLOAT"=>"\\b(?<

\\\\)(?:\"(?:\\\\.|

)*\"|(?:'(?:\\\\.|

)')|(?:`(?:\\\\.|

+)`)))", "UUID"=>"[A-Fa-f0-9]{8}(?:[A-Fa-f0-9]{4}){3}[A-Fa-f0-9]{12}", "MAC"=>"(?:%{CISCOMAC}|%{WINDOWSMAC}|%{COMMONMAC})", "CISCOMAC"=>"(??:[A-Fa-f0-9]{4}
.){2}[A-Fa-f0-9]{4})", "WINDOWSMAC"=>"(??:[A-Fa-f0-9]{2}){5}[A-Fa-f0-9]{2})", "COMMONMAC"=>"(??:[A-Fa-f0-9]{2}{5}[A-Fa-f0-9]{2})", "IP"=>"(?<

[0-9])", "HOSTNAME"=>"
b(?:[0-9A-Za-z][0-9A-Za-z]{0,62})(?:\\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))(?:\\.(?:[0-9A-Za-z]{0,63}))(\\.?|
b)", "HOST"=>"%{HOSTNAME}", "IPORHOST"=>"(?:%{HOSTNAME}|%{IP})", "HOSTPORT"=>"(?:%{IPORHOST=~/
./}:%{POSINT})", "PATH"=>"(?:%{UNIXPATH}|%{WINPATH})", "UNIXPATH"=>"(?:/(?:[\\w_%!$@:.,-]+|\\\\.))", "LINUXTTY"=>"(?:/dev/pts/%{POSINT})", "BSDTTY"=>"(?:/dev/tty[pq][a-z0-9])", "TTY"=>"(?:%{BSDTTY}|%{LINUXTTY})", "WINPATH"=>"(?:[A-Za-z]:|\\\\)(?:\\\\

)", "URIPROTO"=>"[A-Za-z](
[A-Za-z])?", "URIHOST"=>"%{IPORHOST}(?::%{POSINTort})?", "URIPATH"=>"(?:/[A-Za-z0-9$.!'(),_{:#%_-]}_{)", "URIPARAM"=>"}
_{?[A-Za-z0-9$.!}_'(),#%&/=:;]", "URIPATHPARAM"=>"%{URIPATH}(?:%{URIPARAM})?", "URI"=>"%{URIPROTO}://(?:%{USER}(?::

)?@)?(?:%{URIHOST})?(?:%{URIPATHPARAM})?", "MONTH"=>"\\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)
b", "MONTHNUM"=>"(?:0?[1-9]|1[0-2])", "MONTHDAY"=>"(??:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])", "DAY"=>"(?:Mon(?:day)?|Tue(?:sday)?|Wed(?:nesday)?|Thu(?:rsday)?|Fri(?:day)?|Sat(?:urday)?|Sun(?:day)?)", "YEAR"=>"[0-9]", "HOUR"=>"(?:2[0123]|[01][0-9])", "MINUTE"=>"(?:[0-5][0-9])", "SECOND"=>"(??:[0-5][0-9]|60)(?:[.,][0-9])?)", "TIME"=>"(?

[0-9])", "DATE_US"=>"%{MONTHNUM}/-%{MONTHDAY}/-%{YEAR}", "DATE_EU"=>"%{YEAR}[./]%{MONTHNUM}[./]%{MONTHDAY}", "ISO8601_TIMEZONE"=>"(?:Z|[+]%{HOUR}(?::?%{MINUTE}))", "ISO8601_SECOND"=>"(?:%{SECOND}|60)", "TIMESTAMP_ISO8601"=>"%{YEAR}%{MONTHNUM}%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE}?", "DATE"=>"%{DATE_US}|%{DATE_EU}", "DATESTAMP"=>"%{DATE}[- ]%{TIME}", "TZ"=>"(?:[PMCE][SD]T)", "DATESTAMP_RFC822"=>"%{DAY} %{MONTH} %{MONTHDAY} %{YEAR} %{TIME} %{TZ}", "DATESTAMP_OTHER"=>"%{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{TZ} %{YEAR}", "SYSLOGTIMESTAMP"=>"%{MONTH} +%{MONTHDAY} %{TIME}", "PROG"=>"(?:[
w./]+)", "SYSLOGPROG"=>"%{PROGrogram}(?:\\[%{POSINTid}
])?", "SYSLOGHOST"=>"%{IPORHOST}", "SYSLOGFACILITY"=>"<%{POSINT:facility}.%{POSINTriority}>", "HTTPDATE"=>"%{MONTHDAY}/%{MONTH}/%{YEAR}:%{TIME} %{INT:ZONE}", "QS"=>"%{QUOTEDSTRING}", "SYSLOGBASE"=>"%{SYSLOGTIMESTAMP:timestamp} (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}:", "COMBINEDAPACHELOG"=>"%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \\[%{HTTPDATE:timestamp}
] \"%{WORD:verb} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}\" %{NUMBER:response} (?:%{NUMBER:bytes}|) \"(?:%{URI:referrer}|)\" %{QS:agent}", "LOGLEVEL"=>"(?EBUG|FATAL|ERROR|WARN|INFO|debug|fatal|error|warn|info)", "HTTPERRDATE"=>"%{DAY} %{SYSLOGTIMESTAMP} %{YEAR}", "COMMONAPACHELOG"=>"%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \\[%{HTTPDATE:timestamp}
] \\\"%{WORD:verb} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}\\\" %{NUMBER:response} (?:%{NUMBER:bytes}|)", "TIMESTAMP_IAD"=>"(?:
[%{DATE} %{TIME}
]|%{TIME}|)"}, @pattern="%{COMBINEDAPACHELOG} \\\"%{GREEDYDATA:unique_id}\\\"", @expanded_pattern="(?<a0>(?<a1>(??<a2>\\b(?:[0-9A-Za-z][0-9A-Za-z-]{0,62})(?:\\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))(?:\\.(?:[0-9A-Za-z]{0,63}))(\\.?|
b))|(?<a3>(?<

[0-9])) (?<a16>(?:[-]?(?:[0-9]))))
] \"(?<a17>\\b\\w+
b) (?<a18>(?<a19>(?:/[A-Za-z0-9$.+!'(),_{:#%_-]}_))(?_?<a20>
_{?[A-Za-z0-9$.!}_'(),#%&/=:;]*))?) HTTP/(?<a21>(??<a22>(?<![0-9.+])(?>[-]?(??:[0-9](?:\\.[0-9])?)|(?:
.[0-9]))))))\" (?<a23>(??<a24>(?<

[0-9.-])(?>[]?(??:[0-9](?:
.[0-9])?)|(?:
.[0-9]+))))))|) \"(??<a27>(?<a28>[A-Za-z](
[A-Za-z+]+)?)://(??<a29>(?<a30>[a-zA-Z0-9]+))(?::

*)?@)?(??<a31>(?<a32>(??<a33>
b(?:[0-9A-Za-z][0-9A-Za-z]{0,62})(?:\\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))(?:\\.(?:[0-9A-Za-z]{0,63}))(\\.?|\\b))|(?<a34>(?<

[0-9]))))(?:?<a35>\\b(?:[0-9])
b))?))?(??<a36>(?<a37>(?:/[A-Za-z0-9$.!'(),_{:#%_-]}_))(?_?<a38>
_{?[A-Za-z0-9$.!}_'(),#%&/=:;]*))?))?)|)\" (?<a39>(?<a40>(??<

[0-9])(??:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2}))(?![0-9])))) (?<a4>(?<a5>[a-zA-Z0-9]+)) (?<a6>(?<a7>[a-zA-Z0-9_])) [(?<a8>(?<a9>(??:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]))\/(?<a10>\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\b)\/(?<a11>[0-9])?<a12>(?

[0-9])) (?<a16>(?:[-]?(?:[0-9]))))] "(?<a17>\b\w+\b) (?<a18>(?<a19>(?:\/[A-Za-z0-9$.!'(),~:#%_-]))(??<a20>?[A-Za-z0-9$.

[0-9.])(?>[+]?(??:[0-9](?:\.[0-9])?)|(?:\.[0-9]))))))" (?<a23>(??<a24>(?<![0-9.])(?>[+]?(??:[0-9](?:\.[0-9])?)|(?:\.[0-9])))))) (??<a25>(??<a26>(?<![0-9.])(?>[+]?(??:[0-9](?:\.[0-9])?)|(?:\.[0-9]))))))|-) "(??<a27>(?<a28>[A-Za-z](+[A-Za-z+])?):\/\/(??<a29>(?<a30>[a-zA-Z0-9_-]))(?::

)?@)?(??<a31>(?<a32>(??<a33>\b(?:[0-9A-Za-z][0-9A-Za-z-]{0,62})(?:\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))(?:\.(?:[0-9A-Za-z]{0,63}))(\.?|\b))|(?<a34>(?<

[0-9]))))(?:?<a35>\b(?:[0-9])\b))?))?(??<a36>(?<a37>(?:\/[A-Za-z0-9$.

'(),~#%&\/=:;_-]))?))?)|)" (?<a39>(?<a40>(??<

[0-9A-Fa-f])(?:[-]?(?:0x)?(?:[0-9A-Fa-f]))", "BASE16FLOAT"=>"\\b(?<

\\\\)(?:\"(?:\\\\.|

)*\"|(?:'(?:\\\\.|

)')|(?:`(?:\\\\.|

[0-9])", "DATE_US"=>"%{MONTHNUM}/-%{MONTHDAY}/-%{YEAR}", "DATE_EU"=>"%{YEAR}[./]%{MONTHNUM}[./]%{MONTHDAY}", "ISO8601_TIMEZONE"=>"(?:Z|[+]%{HOUR}(?::?%{MINUTE}))", "ISO8601_SECOND"=>"(?:%{SECOND}|60)", "TIMESTAMP_ISO8601"=>"%{YEAR}%{MONTHNUM}%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE}?", "DATE"=>"%{DATE_US}|%{DATE_EU}", "DATESTAMP"=>"%{DATE}[- ]%{TIME}", "TZ"=>"(?:[PMCE][SD]T)", "DATESTAMP_RFC822"=>"%{DAY} %{MONTH} %{MONTHDAY} %{YEAR} %{TIME} %{TZ}", "DATESTAMP_OTHER"=>"%{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{TZ} %{YEAR}", "SYSLOGTIMESTAMP"=>"%{MONTH} +%{MONTHDAY} %{TIME}", "PROG"=>"(?:[
w./]+)", "SYSLOGPROG"=>"%{PROGrogram}(?:\\[%{POSINTid}
])?", "SYSLOGHOST"=>"%{IPORHOST}", "SYSLOGFACILITY"=>"<%{POSINT:facility}.%{POSINTriority}>", "HTTPDATE"=>"%{MONTHDAY}/%{MONTH}/%{YEAR}:%{TIME} %{INT:ZONE}", "QS"=>"%{QUOTEDSTRING}", "SYSLOGBASE"=>"%{SYSLOGTIMESTAMP:timestamp} (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}:", "COMBINEDAPACHELOG"=>"%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \\[%{HTTPDATE:timestamp}
] \"%{WORD:verb} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}\" %{NUMBER:response} (?:%{NUMBER:bytes}|) \"(?:%{URI:referrer}|)\" %{QS:agent}", "LOGLEVEL"=>"(?EBUG|FATAL|ERROR|WARN|INFO|debug|fatal|error|warn|info)", "HTTPERRDATE"=>"%{DAY} %{SYSLOGTIMESTAMP} %{YEAR}", "COMMONAPACHELOG"=>"%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \\[%{HTTPDATE:timestamp}
] \\\"%{WORD:verb} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}\\\" %{NUMBER:response} (?:%{NUMBER:bytes}|)", "TIMESTAMP_IAD"=>"(?:
[%{DATE} %{TIME}
]|%{TIME}|)"}, @pattern="%{COMBINEDAPACHELOG}", @expanded_pattern="(?<a0>(?<a1>(??<a2>\\b(?:[0-9A-Za-z][0-9A-Za-z-]{0,62})(?:\\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))(?:\\.(?:[0-9A-Za-z]{0,63}))(\\.?|
b))|(?<a3>(?<

[0-9.-])(?>[]?(??:[0-9](?:
.[0-9])?)|(?:
.[0-9]+))))))|) \"(??<a27>(?<a28>[A-Za-z](
[A-Za-z+]+)?)://(??<a29>(?<a30>[a-zA-Z0-9]+))(?::

*)?@)?(??<a31>(?<a32>(??<a33>
b(?:[0-9A-Za-z][0-9A-Za-z]{0,62})(?:\\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))(?:\\.(?:[0-9A-Za-z]{0,63}))(\\.?|\\b))|(?<a34>(?<

[0-9]))))(?:?<a35>\\b(?:[0-9])
b))?))?(??<a36>(?<a37>(?:/[A-Za-z0-9$.!'(),_{:#%_-]}_))(?_?<a38>
_{?[A-Za-z0-9$.!}_'(),#%&/=:;]*))?))?)|)\" (?<a39>(?<a40>(??<

[0-9])) (?<a16>(?:[-]?(?:[0-9]))))] \"(?<a17>\b\w+\b) (?<a18>(?<a19>(?:\/[A-Za-z0-9$.!'(),~:#%_-]))(??<a20>?[A-Za-z0-9$.

[0-9.])(?>[+]?(??:[0-9](?:\.[0-9])?)|(?:\.[0-9]))))))\" (?<a23>(??<a24>(?<![0-9.])(?>[+]?(??:[0-9](?:\.[0-9])?)|(?:\.[0-9])))))) (??<a25>(??<a26>(?<![0-9.])(?>[+]?(??:[0-9](?:\.[0-9])?)|(?:\.[0-9]))))))|))/, @patterns={"NETSCREENSESSIONLOG"=>"%{SYSLOGDATE:date} %{IPORHOST:device} %{IPORHOST}: NetScreen device_id=%{WORD:device_id}%{DATA}: start_time=%{QUOTEDSTRING:start_time} duration=%{INT:duration} policy_id=%{INTolicy_id} service=%{DATA:service} proto=%{INTroto} src zone=%{WORD:src_zone} dst zone=%{WORD:dst_zone} action=%{WORD:action} sent=%{INT:sent} rcvd=%{INT:rcvd} src=%{IPORHOST:src_ip} dst=%{IPORHOST:dst_ip} src_port=%{INT:src_port} dst_port=%{INT:dst_port} src-xlated ip=%{IPORHOST:src_xlated_ip} port=%{INT:src_xlated_port} dst-xlated ip=%{IPORHOST:dst_xlated_ip} port=%{INT:dst_xlated_port} session_id=%{INT:session_id} reason=%{GREEDYDATA:reason}", "HAPROXYTIME"=>"(?

[0-9])", "HAPROXYDATE"=>"%{MONTHDAY:haproxy_monthday}/%{MONTH:haproxy_month}/%{YEAR:haproxy_year}:%{HAPROXYTIME:haproxy_time}.%{INT:haproxy_milliseconds}", "HAPROXYCAPTUREDREQUESTHEADERS"=>"%{DATA:captured_request_headers}", "HAPROXYCAPTUREDRESPONSEHEADERS"=>"%{DATA:captured_response_headers}", "HAPROXYHTTP"=>"%{SYSLOGTIMESTAMP:syslog_timestamp} %{IPORHOST:syslog_server} %{SYSLOGPROG}: %{IP:client_ip}:%{INT:client_port} \\[%{HAPROXYDATE:accept_date}
] %{NOTSPACE:frontend_name} %{NOTSPACE:backend_name}/%{NOTSPACE:server_name} %{INT:time_request}/%{INT:time_queue}/%{INT:time_backend_connect}/%{INT:time_backend_response}/%{NOTSPACE:time_duration} %{INT:http_status_code} %{NOTSPACE:bytes_read} %{DATA:captured_request_cookie} %{DATA:captured_response_cookie} %{NOTSPACE:termination_state} %{INT:actconn}/%{INT:feconn}/%{INT:beconn}/%{INT:srvconn}/%{NOTSPACE:retries} %{INT:srv_queue}/%{INT:backend_queue} \\{%{HAPROXYCAPTUREDREQUESTHEADERS}
} \\{%{HAPROXYCAPTUREDRESPONSEHEADERS}
} \"%{WORD:http_verb} %{URIPATHPARAM:http_request} HTTP/%{NUMBER:http_version}\"", "HAPROXYTCP"=>"%{SYSLOGTIMESTAMP:syslog_timestamp} %{IPORHOST:syslog_server} %{SYSLOGPROG}: %{IP:client_ip}:%{INT:client_port} \\[%{HAPROXYDATE:accept_date}
] %{NOTSPACE:frontend_name} %{NOTSPACE:backend_name}/%{NOTSPACE:server_name} %{INT:time_queue}/%{INT:time_backend_connect}/%{NOTSPACE:time_duration} %{NOTSPACE:bytes_read} %{NOTSPACE:termination_state} %{INT:actconn}/%{INT:feconn}/%{INT:beconn}/%{INT:srvconn}/%{NOTSPACE:retries} %{INT:srv_queue}/%{INT:backend_queue}", "JAVACLASS"=>"(?:[a-zA-Z0-9]
.)[A-Za-z0-9]", "JAVAFILE"=>"(?:[A-Za-z0-9_.-])", "JAVASTACKTRACEPART"=>"at %{JAVACLASS:class}\\.%{WORD:method}\\(%{JAVAFILE:file}:%{NUMBER:line}
)", "SYSLOGBASE2"=>"(?:%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp8601}) (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}:", "SYSLOGPAMSESSION"=>"%{SYSLOGBASE} (?=%{GREEDYDATA:message})%{WORDam_module}\\(%{DATAam_caller}
): session %{WORDam_session_state} for user %{USERNAME:username}(?: by %{GREEDYDATAam_by})?", "CRON_ACTION"=>"[A-Z ]", "CRONLOG"=>"%{SYSLOGBASE} \\(%{USER:user}
) %{CRON_ACTION:action} \\(%{DATA:message}
)", "SYSLOGLINE"=>"%{SYSLOGBASE2} %{GREEDYDATA:message}", "NAGIOSTIME"=>"\\[%{NUMBER:nagios_epoch}
]", "NAGIOS_TYPE_CURRENT_SERVICE_STATE"=>"CURRENT SERVICE STATE", "NAGIOS_TYPE_CURRENT_HOST_STATE"=>"CURRENT HOST STATE", "NAGIOS_TYPE_SERVICE_NOTIFICATION"=>"SERVICE NOTIFICATION", "NAGIOS_TYPE_HOST_NOTIFICATION"=>"HOST NOTIFICATION", "NAGIOS_TYPE_SERVICE_ALERT"=>"SERVICE ALERT", "NAGIOS_TYPE_HOST_ALERT"=>"HOST ALERT", "NAGIOS_TYPE_SERVICE_FLAPPING_ALERT"=>"SERVICE FLAPPING ALERT", "NAGIOS_TYPE_HOST_FLAPPING_ALERT"=>"HOST FLAPPING ALERT", "NAGIOS_TYPE_SERVICE_DOWNTIME_ALERT"=>"SERVICE DOWNTIME ALERT", "NAGIOS_TYPE_HOST_DOWNTIME_ALERT"=>"HOST DOWNTIME ALERT", "NAGIOS_TYPE_PASSIVE_SERVICE_CHECK"=>"PASSIVE SERVICE CHECK", "NAGIOS_TYPE_PASSIVE_HOST_CHECK"=>"PASSIVE HOST CHECK", "NAGIOS_TYPE_SERVICE_EVENT_HANDLER"=>"SERVICE EVENT HANDLER", "NAGIOS_TYPE_HOST_EVENT_HANDLER"=>"HOST EVENT HANDLER", "NAGIOS_TYPE_EXTERNAL_COMMAND"=>"EXTERNAL COMMAND", "NAGIOS_TYPE_TIMEPERIOD_TRANSITION"=>"TIMEPERIOD TRANSITION", "NAGIOS_EC_DISABLE_SVC_CHECK"=>"DISABLE_SVC_CHECK", "NAGIOS_EC_ENABLE_SVC_CHECK"=>"ENABLE_SVC_CHECK", "NAGIOS_EC_DISABLE_HOST_CHECK"=>"DISABLE_HOST_CHECK", "NAGIOS_EC_ENABLE_HOST_CHECK"=>"ENABLE_HOST_CHECK", "NAGIOS_EC_PROCESS_SERVICE_CHECK_RESULT"=>"PROCESS_SERVICE_CHECK_RESULT", "NAGIOS_EC_PROCESS_HOST_CHECK_RESULT"=>"PROCESS_HOST_CHECK_RESULT", "NAGIOS_EC_SCHEDULE_SERVICE_DOWNTIME"=>"SCHEDULE_SERVICE_DOWNTIME", "NAGIOS_EC_SCHEDULE_HOST_DOWNTIME"=>"SCHEDULE_HOST_DOWNTIME", "NAGIOS_WARNING"=>"Warning:%{SPACE}%{GREEDYDATA:nagios_message}", "NAGIOS_CURRENT_SERVICE_STATE"=>"%{NAGIOS_TYPE_CURRENT_SERVICE_STATE:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{DATA:nagios_statetype};%{DATA:nagios_statecode};%{GREEDYDATA:nagios_message}", "NAGIOS_CURRENT_HOST_STATE"=>"%{NAGIOS_TYPE_CURRENT_HOST_STATE:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_state};%{DATA:nagios_statetype};%{DATA:nagios_statecode};%{GREEDYDATA:nagios_message}", "NAGIOS_SERVICE_NOTIFICATION"=>"%{NAGIOS_TYPE_SERVICE_NOTIFICATION:nagios_type}: %{DATA:nagios_notifyname};%{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{DATA:nagios_contact};%{GREEDYDATA:nagios_message}", "NAGIOS_HOST_NOTIFICATION"=>"%{NAGIOS_TYPE_HOST_NOTIFICATION}: %{DATA:nagios_notifyname};%{DATA:nagios_hostname};%{DATA:nagios_state};%{DATA:nagios_contact};%{GREEDYDATA:nagios_message}", "NAGIOS_SERVICE_ALERT"=>"%{NAGIOS_TYPE_SERVICE_ALERT:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{DATA:nagios_statelevel};%{NUMBER:nagios_attempt};%{GREEDYDATA:nagios_message}", "NAGIOS_HOST_ALERT"=>"%{NAGIOS_TYPE_HOST_ALERT:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_state};%{DATA:nagios_statelevel};%{NUMBER:nagios_attempt};%{GREEDYDATA:nagios_message}", "NAGIOS_SERVICE_FLAPPING_ALERT"=>"%{NAGIOS_TYPE_SERVICE_FLAPPING_ALERT:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{GREEDYDATA:nagios_message}", "NAGIOS_HOST_FLAPPING_ALERT"=>"%{NAGIOS_TYPE_HOST_FLAPPING_ALERT:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_state};%{GREEDYDATA:nagios_message}", "NAGIOS_SERVICE_DOWNTIME_ALERT"=>"%{NAGIOS_TYPE_SERVICE_DOWNTIME_ALERT:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{GREEDYDATA:nagios_comment}", "NAGIOS_HOST_DOWNTIME_ALERT"=>"%{NAGIOS_TYPE_HOST_DOWNTIME_ALERT:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_state};%{GREEDYDATA:nagios_comment}", "NAGIOS_PASSIVE_SERVICE_CHECK"=>"%{NAGIOS_TYPE_PASSIVE_SERVICE_CHECK:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{GREEDYDATA:nagios_comment}", "NAGIOS_PASSIVE_HOST_CHECK"=>"%{NAGIOS_TYPE_PASSIVE_HOST_CHECK:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_state};%{GREEDYDATA:nagios_comment}", "NAGIOS_SERVICE_EVENT_HANDLER"=>"%{NAGIOS_TYPE_SERVICE_EVENT_HANDLER:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{DATA:nagios_statelevel};%{DATA:nagios_event_handler_name}", "NAGIOS_HOST_EVENT_HANDLER"=>"%{NAGIOS_TYPE_HOST_EVENT_HANDLER:nagios_type}: %{DATA:nagios_hostname};%{DATA:nagios_state};%{DATA:nagios_statelevel};%{DATA:nagios_event_handler_name}", "NAGIOS_TIMEPERIOD_TRANSITION"=>"%{NAGIOS_TYPE_TIMEPERIOD_TRANSITION:nagios_type}: %{DATA:nagios_service};%{DATA:nagios_unknown1};%{DATA:nagios_unknown2};", "NAGIOS_EC_LINE_DISABLE_SVC_CHECK"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_DISABLE_SVC_CHECK:nagios_command};%{DATA:nagios_hostname};%{DATA:nagios_service}", "NAGIOS_EC_LINE_DISABLE_HOST_CHECK"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_DISABLE_HOST_CHECK:nagios_command};%{DATA:nagios_hostname}", "NAGIOS_EC_LINE_ENABLE_SVC_CHECK"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_ENABLE_SVC_CHECK:nagios_command};%{DATA:nagios_hostname};%{DATA:nagios_service}", "NAGIOS_EC_LINE_ENABLE_HOST_CHECK"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_ENABLE_HOST_CHECK:nagios_command};%{DATA:nagios_hostname}", "NAGIOS_EC_LINE_PROCESS_SERVICE_CHECK_RESULT"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_PROCESS_SERVICE_CHECK_RESULT:nagios_command};%{DATA:nagios_hostname};%{DATA:nagios_service};%{DATA:nagios_state};%{GREEDYDATA:nagios_check_result}", "NAGIOS_EC_LINE_PROCESS_HOST_CHECK_RESULT"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_PROCESS_HOST_CHECK_RESULT:nagios_command};%{DATA:nagios_hostname};%{DATA:nagios_state};%{GREEDYDATA:nagios_check_result}", "NAGIOS_EC_LINE_SCHEDULE_HOST_DOWNTIME"=>"%{NAGIOS_TYPE_EXTERNAL_COMMAND:nagios_type}: %{NAGIOS_EC_SCHEDULE_HOST_DOWNTIME:nagios_command};%{DATA:nagios_hostname};%{NUMBER:nagios_start_time};%{NUMBER:nagios_end_time};%{NUMBER:nagios_fixed};%{NUMBER:nagios_trigger_id};%{NUMBER:nagios_duration};%{DATA:author};%{DATA:comment}", "NAGIOSLOGLINE"=>"%{NAGIOSTIME} (?:%{NAGIOS_WARNING}|%{NAGIOS_CURRENT_SERVICE_STATE}|%{NAGIOS_CURRENT_HOST_STATE}|%{NAGIOS_SERVICE_NOTIFICATION}|%{NAGIOS_HOST_NOTIFICATION}|%{NAGIOS_SERVICE_ALERT}|%{NAGIOS_HOST_ALERT}|%{NAGIOS_SERVICE_FLAPPING_ALERT}|%{NAGIOS_HOST_FLAPPING_ALERT}|%{NAGIOS_SERVICE_DOWNTIME_ALERT}|%{NAGIOS_HOST_DOWNTIME_ALERT}|%{NAGIOS_PASSIVE_SERVICE_CHECK}|%{NAGIOS_PASSIVE_HOST_CHECK}|%{NAGIOS_SERVICE_EVENT_HANDLER}|%{NAGIOS_HOST_EVENT_HANDLER}|%{NAGIOS_TIMEPERIOD_TRANSITION}|%{NAGIOS_EC_LINE_DISABLE_SVC_CHECK}|%{NAGIOS_EC_LINE_ENABLE_SVC_CHECK}|%{NAGIOS_EC_LINE_DISABLE_HOST_CHECK|%{NAGIOS_EC_LINE_ENABLE_HOST_CHECK}|%{NAGIOS_EC_LINE_PROCESS_HOST_CHECK_RESULT}|%{NAGIOS_EC_LINE_PROCESS_SERVICE_CHECK_RESULT}|%{NAGIOS_EC_LINE_SCHEDULE_HOST_DOWNTIME})", "RUBY_LOGLEVEL"=>"(?EBUG|FATAL|ERROR|WARN|INFO)", "RUBY_LOGGER"=>"[DFEWI],
[%{TIMESTAMP_ISO8601} #{POSINTid}
] %{RUBY_LOGLEVEL} -- %{DATArogname}: %{DATA:message}", "USERNAME"=>"[a-zA-Z0-9_]", "USER"=>"%{USERNAME}", "INT"=>"(?:[]?(?:[0-9]))", "BASE10NUM"=>"(?<![0-9.])(?>[+]?(??:[0-9](?:
.[0-9])?)|(?:
.[0-9])))", "NUMBER"=>"(?:%{BASE10NUM})", "BASE16NUM"=>"(?<![0-9A-Fa-f])(?:[]?(?:0x)?(?:[0-9A-Fa-f]))", "BASE16FLOAT"=>"
b(?<![0-9A-Fa-f.])(?:[]?(?:0x)?(??:[0-9A-Fa-f]+(?:
.[0-9A-Fa-f])?)|(?:\\.[0-9A-Fa-f])))
b", "POSINT"=>"
b(?:[0-9])
b", "NONNEGINT"=>"\\b(?:[0-9])
b", "WORD"=>"\\b
w
b", "NOTSPACE"=>"
S+", "SPACE"=>"
s*", "DATA"=>".?", "GREEDYDATA"=>".", "QUOTEDSTRING"=>"(??<

[0-9])(??:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2}))(?

$@:.,]

\\\\.)*)

", "LINUXTTY"=>"(?:/dev/pts/%{POSINT})", "BSDTTY"=>"(?:/dev/tty[pq][a-z0-9])", "TTY"=>"(?:%{BSDTTY}|%{LINUXTTY})", "WINPATH"=>"(?:[A-Za-z]:|\\\\)(?:\\\\

*)", "URIPROTO"=>"[A-Za-z](
[A-Za-z+])?", "URIHOST"=>"%{IPORHOST}(?::%{POSINTort})?", "URIPATH"=>"(?:/[A-Za-z0-9$.!*'(),~:#%_])", "URIPARAM"=>"
?[A-Za-z0-9$.!'(),_#%&/=:;_]_{", "URIPATHPARAM"=>"%{URIPATH}(?:%{URIPARAM})?", "URI"=>"%{URIPROTO}://(?:%{USER}(?::}

_{)?@)?(?:%{URIHOST})?(?:%{URIPATHPARAM})?", "MONTH"=>"\\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)}
_{b", "MONTHNUM"=>"(?:0?[1-9]|1[0-2])", "MONTHDAY"=>"(?}_{?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])", "DAY"=>"(?:Mon(?:day)?|Tue(?:sday)?|Wed(?:nesday)?|Thu(?:rsday)?|Fri(?:day)?|Sat(?:urday)?|Sun(?:day)?)", "YEAR"=>"[0-9]", "HOUR"=>"(?:2[0123]|[01][0-9])", "MINUTE"=>"(?:[0-5][0-9])", "SECOND"=>"(?}_{?:[0-5][0-9]|60)(?:[.,][0-9])?)", "TIME"=>"(?}

_{[0-9])))) (?<a4>(?<a5>[a-zA-Z0-9_}_{]+)) (?<a6>(?<a7>[a-zA-Z0-9_}_{])) \\[(?<a8>(?<a9>(?}_{?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]))/(?<a10>\\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)}
_{b)/(?<a11>[0-9])}_?<a12>(?

_{[0-9])) (?<a16>(?:[-]?(?:[0-9]))))}
_{] \\\"(?<a17>\\b\\w+}
_{b) (?<a18>(?<a19>(?:/[A-Za-z0-9$.+!}_'(),:#%-]))(??<a20>
?[A-Za-z0-9$.!'(),~#%&/=:;]*))?) HTTP/(?<a21>(??<a22>(?<![0-9.+])(?>[-]?(??:[0-9](?:\\.[0-9])?)|(?:
.[0-9]))))))\\\" (?<a23>(??<a24>(?<

[0-9.-])(?>[]?(??:[0-9](?:
.[0-9])?)|(?:
.[0-9]+))))))|))", @capture_map={"a0"=>"COMMONAPACHELOG", "a1"=>"IPORHOST:clientip", "a2"=>"HOSTNAME", "a3"=>"IP", "a4"=>"USER:ident", "a5"=>"USERNAME", "a6"=>"USER:auth", "a7"=>"USERNAME", "a8"=>"HTTPDATE:timestamp", "a9"=>"MONTHDAY", "a10"=>"MONTH", "a11"=>"YEAR", "a12"=>"TIME", "a13"=>"HOUR", "a14"=>"MINUTE", "a15"=>"SECOND", "a16"=>"INT:ZONE", "a17"=>"WORD:verb", "a18"=>"URIPATHPARAM:request", "a19"=>"URIPATH", "a20"=>"URIPARAM", "a21"=>"NUMBER:httpversion", "a22"=>"BASE10NUM", "a23"=>"NUMBER:response", "a24"=>"BASE10NUM", "a25"=>"NUMBER:bytes", "a26"=>"BASE10NUM"}, @logger=#<Cabin::Channel:0x6cfce051 @subscriber_lock=#<Mutex:0x55519d9b>, @metrics=#<Cabin::Metrics:0x614256df @channel=#<Cabin::Channel:0x6cfce051 ...>, @metrics={}, @metrics_lock=#<Mutex:0x4a557c5b>>, @data={}, @subscribers={4096=>#<Cabin::Outputs::StdlibLogger:0x7c537c91 @logger=#<Logger:0x4d17535b @logdev=#<Logger::LogDevice:0x3661a2e7 @shift_age=nil, @filename=nil, @dev=#<IO:fd 1>, @mutex=#<Logger::LogDevice::LogDeviceMutex:0x440e0795 @mon_count=0, @mon_mutex=#<Mutex:0x1037d830>, @mon_owner=nil>, @shift_size=nil>, @formatter=nil, @progname=nil, @default_formatter=#<Logger::Formatter:0x42523fa1 @datetime_format=nil>, @level=0>>}, @level=:warn>>]>}, @threadsafe=true, @patternfiles=["file:/iad/common/apps/logstash/logstash-1.1.9-monolithic.jar!/patterns/firewalls", "file:/iad/common/apps/logstash/logstash-1.1.9-monolithic.jar!/patterns/haproxy", "file:/iad/common/apps/logstash/logstash-1.1.9-monolithic.jar!/patterns/java", "file:/iad/common/apps/logstash/logstash-1.1.9-monolithic.jar!/patterns/linux-syslog", "file:/iad/common/apps/logstash/logstash-1.1.9-monolithic.jar!/patterns/nagios", "file:/iad/common/apps/logstash/logstash-1.1.9-monolithic.jar!/patterns/ruby", "file:/iad/common/apps/logstash/logstash-1.1.9-monolithic.jar!/patterns/grok-patterns", "/iad/local/etc/logstash.d/grok_patterns/haproxy", "/iad/local/etc/logstash.d/grok_patterns/up-grok", "/iad/local/etc/logstash.d/grok_patterns/ruby", "/iad/local/etc/logstash.d/grok_patterns/nagios", "/iad/local/etc/logstash.d/grok_patterns/firewalls", "/iad/local/etc/logstash.d/grok_patterns/linux-syslog", "/iad/local/etc/logstash.d/grok_patterns/java", "/iad/local/etc/logstash.d/grok_patterns/grok-patterns", "/iad/local/etc/logstash.d/grok_patterns/finn"], @patterns_dir=["file:/iad/common/apps/logstash/logstash-1.1.9-monolithic.jar!/logstash/filters/../../patterns/*", "/iad/local/etc/logstash.d/grok_patterns"], @config={"type"=>"apache-access", "patterns_dir"=>["/iad/local/etc/logstash.d/grok_patterns"], "pattern"=>["%{COMBINEDAPACHELOG} \\\"%{GREEDYDATA:unique_id}\\\"", "%{COMBINEDAPACHELOG}", "%{COMMONAPACHELOG}"], "tags"=>[], "exclude_tags"=>[], "add_tag"=>[], "remove_tag"=>[], "add_field"=>{}, "match"=>{"@message"=>["%{COMBINEDAPACHELOG} \\\"%{GREEDYDATA:unique_id}\\\"", "%{COMBINEDAPACHELOG}", "%{COMMONAPACHELOG}"]}, "drop_if_match"=>false, "break_on_match"=>true, "named_captures_only"=>true, "keep_empty_captures"=>false, "singles"=>false}, @break_on_match=true, @match={"@message"=>["%{COMBINEDAPACHELOG} \\\"%{GREEDYDATA:unique_id}\\\"", "%{COMBINEDAPACHELOG}", "%{COMMONAPACHELOG}"]}, @exclude_tags=[]>}, :level=>:fatal, :file=>"/iad/common/apps/logstash/logstash-1.1.9-monolithic.jar!/logstash/threadwatchdog.rb", :line=>"24", :method=>"watch"}

Gliffy Diagrams

Activity

Show:

Jason Kendall March 11, 2014 at 3:33 PM

Thanks - closing.

R. Arruda March 11, 2014 at 3:14 PM

looks like its resolved at least in 1.4 snapshot. probably 1.3.3 also.

R. Arruda March 11, 2014 at 3:12 PM
Edited

Reproducing it with 1.4, you can see that it no longer crashes with 1.4

So can close this ticket.

$ cat <<'EOF' > /tmp/zb
domain.name.no - - [16/Apr/2013:18:01:50 +0200] "GET /recruitment/health/smoketest?callback=jQuery19106504244991728602_1366128144067&=1366128144068 HTTP/1.1" 200 75 "http://192.168.41.45:3000/smoketest.html" "Mozildomain.name.no - - [16/Apr/2013:18:02:12 +0200] "GET /recruitment/health/smoketest?callback=jQuery19100761695952632111_1366128123412&=1366128123413 HTTP/1.1" 200 75 "http://localhost:3000/smoketest.html" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:20.0) Gecko/20100101 Firefox/20.0" "UW12BFBbIzEAABeu1fIAAASC"
EOF

cat /tmp/zb | /iad/common/apps/jdk1.7/bin/java -jar ~/rarruda/logstash-1.4.0.dev-modified-flatjar.jar agent -e 'input { stdin { } } filter { grok { pattern => "%{COMBINEDAPACHELOG} \"%{GREEDYDATA:unique_id}\"" } } output { stdout { codec => rubydebug { } } }'
You are using a deprecated config setting "pattern" set in grok. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. You should use this instead: match => { "message" => "your pattern here" } If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=>"pattern", lugin=><LogStash::Filters::Grok --->, :level=>:warn}
{
"message" => "domain.name.no - - [16/Apr/2013:18:01:50 +0200] \"GET /recruitment/health/smoketest?callback=jQuery19106504244991728602_1366128144067&=1366128144068 HTTP/1.1\" 200 75 \"http://192.168.41.45:3000/smoketest.html\" \"Mozildomain.name.no - - [16/Apr/2013:18:02:12 +0200] \"GET /recruitment/health/smoketest?callback=jQuery19100761695952632111_1366128123412&=1366128123413 HTTP/1.1\" 200 75 \"http://localhost:3000/smoketest.html\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:20.0) Gecko/20100101 Firefox/20.0\" \"UW12BFBbIzEAABeu1fIAAASC\"",
"@version" => "1",
"@timestamp" => "2014-03-11T15:10:41.857Z",
"host" => "myhost.no",
"clientip" => "domain.name.no",
"ident" => "-",
"auth" => "-",
"timestamp" => "16/Apr/2013:18:01:50 +0200",
"rawrequest" => "GET /recruitment/health/smoketest?callback=jQuery19106504244991728602_1366128144067&=1366128144068 HTTP/1.1\" 200 75 \"http://192.168.41.45:3000/smoketest.html\" \"Mozildomain.name.no - - [16/Apr/2013:18:02:12 +0200] \"GET /recruitment/health/smoketest?callback=jQuery19100761695952632111_1366128123412&=1366128123413 HTTP/1.1",
"response" => "200",
"bytes" => "75",
"referrer" => "\"http://localhost:3000/smoketest.html\"",
"agent" => "\"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:20.0) Gecko/20100101 Firefox/20.0\"",
"unique_id" => "UW12BFBbIzEAABeu1fIAAASC"
}

Jason Kendall March 5, 2014 at 4:07 PM

Is this still an issue in 1.3.3? You should be getting a _grokfailure tag added now.

Fixed

Details

Assignee

Logstash Developers

Reporter

R. Arruda

Affects versions

1.1.9

Created April 17, 2013 at 11:31 AM

Updated March 11, 2014 at 3:33 PM

Resolved March 11, 2014 at 3:14 PM

Configure