[ossec-list] Re: OSSEC & Logstash

[mangasof . manga]

Hi JP1, you found a pattern for archive.log file?

Em quarta-feira, 18 de fevereiro de 2015 17:12:45 UTC-3, jp1...@gmail.com 
escreveu:
>
> So, this works OK for me on alerts.log - does anyone have a logstash conf 
> that works on the archives.log if you have ossec saving all logs to that?
>
> On Saturday, March 8, 2014 at 5:02:35 PM UTC-5, Joshua Garnett wrote:
>>
>> All,
>>
>> I'll probably write a blog post on this, but I wanted to share some work 
>> I've done today.  
>> http://vichargrave.com/ossec-log-management-with-elasticsearch/ shows 
>> how to use OSSEC's syslog output to route messages to Elasticsearch.  The 
>> problem with this method is it uses UDP.  Even when sending packets to a 
>> local process UDP by definition is unreliable.  Garbage collections and 
>> other system events can cause packets to be lost.  I've found it tends to 
>> cap out at around 1,500 messages per minute. 
>>
>> To address this issue I've put together a logstash config that will read 
>> the alerts from /var/ossec/logs/alerts/alerts.log.  On top of solving the 
>> reliability issue, it also fixes issues with multi-lines being lost, and 
>> adds geoip lookups for the src_ip.  I tested it against approximately 1GB 
>> of alerts (3M events).
>>
>> input {
>>   file {
>> type => "ossec"
>> path => "/var/ossec/logs/alerts/alerts.log"
>> sincedb_path => "/opt/logstash/"
>> codec => multiline {
>>   pattern => "^\*\*"
>>   negate => true
>>   what => "previous"
>> }
>>   }
>> }
>>
>> filter {
>>   if [type] == "ossec" {
>> # Parse the header of the alert
>> grok {
>>   # Matches  2014 Mar 08 00:57:49 (some.server.com) 10.1.2.3->ossec
>>   # (?m) fixes issues with multi-lines see 
>> https://logstash.jira.com/browse/LOGSTASH-509
>>   match => ["message", "(?m)\*\* Alert 
>> %{DATA:timestamp_seconds}:%{SPACE}%{WORD}?%{SPACE}\- 
>> %{DATA:ossec_group}\n%{YEAR} %{SYSLOGTIMESTAMP:syslog_timestamp} 
>> \(%{DATA:reporting_host}\) 
>> %{IP:reporting_ip}\-\>%{DATA:reporting_source}\nRule: 
>> %{NONNEGINT:rule_number} \(level %{NONNEGINT:severity}\) \-\> 
>> '%{DATA:signature}'\n%{GREEDYDATA:remaining_message}"]
>>   
>>   # Matches  2014 Mar 08 00:00:00 ossec-server01->/var/log/auth.log
>>   match => ["message", "(?m)\*\* Alert 
>> %{DATA:timestamp_seconds}:%{SPACE}%{WORD}?%{SPACE}\- 
>> %{DATA:ossec_group}\n%{YEAR} %{SYSLOGTIMESTAMP:syslog_timestamp} 
>> %{DATA:reporting_host}\-\>%{DATA:reporting_source}\nRule: 
>> %{NONNEGINT:rule_number} \(level %{NONNEGINT:severity}\) \-\> 
>> '%{DATA:signature}'\n%{GREEDYDATA:remaining_message}"]
>> }
>>
>> # Attempt to parse additional data from the alert
>> grok {
>>   match => ["remaining_message", "(?m)(Src IP: 
>> %{IP:src_ip}%{SPACE})?(Src Port: %{NONNEGINT:src_port}%{SPACE})?(Dst IP: 
>> %{IP:dst_ip}%{SPACE})?(Dst Port: %{NONNEGINT:dst_port}%{SPACE})?(User: 
>> %{USER:acct}%{SPACE})?%{GREEDYDATA:real_message}"]
>> }
>>
>> geoip {
>>   source => "src_ip"
>> }
>>
>> mutate {
>>   convert  => [ "severity", "integer"]
>>   replace  => [ "@message", "%{real_message}" ]
>>   replace  => [ "@fields.hostname", "%{reporting_host}"]
>>   add_field=> [ "@fields.product", "ossec"]
>>   add_field=> [ "raw_message", "%{message}"]
>>   add_field=> [ "ossec_server", "%{host}"]
>>   remove_field => [ "type", "syslog_program", "syslog_timestamp", 
>> "reporting_host", "message", "timestamp_seconds", "real_message", 
>> "remaining_message", "path", "host", "tags"]
>> }
>>   }
>> }
>>
>> output {
>>elasticsearch {
>>  host => "10.0.0.1"
>>  cluster => "mycluster"
>>}
>> }
>>
>> Here are a few examples of the output this generates.
>>
>> {
>>"@timestamp":"2014-03-08T20:34:08.847Z",
>>"@version":"1",
>>"ossec_group":"syslog,sshd,invalid_login,authentication_failed,",
>>"reporting_ip":"10.1.2.3",
>>"reporting_source":"/var/log/auth.log",
>>"rule_number":"5710",
>>"severity":5,
>>"signature":"Attempt to login using a non-existent user",
>>"src_ip":"112.65.211.164",
>>"geoip":{
>>   "ip":"112.65.211.164",
>>   "country_code2":"CN",
>>   "country_code3":"CHN",
>>   "country_name":"China",
>>   "continent_code":"AS",
>>   "region_name":"23",
>>   "city_name":"Shanghai",
>>   "latitude":31.0456007,
>>   "longitude":121.3997,
>>   "timezone":"Asia/Shanghai",
>>   "real_region_name":"Shanghai",
>>   "location":[
>>  121.3997,
>>  31.0456007
>>   ]
>>},
>>"@message":"Mar  8 01:00:59 someserver sshd[22874]: Invalid user 
>> oracle from 112.65.211.164\n",
>>"@fields.hostname":"someserver.somedomain.com",
>>"@fields.product":"ossec",
>>"raw_message":"** Alert 1394240459.2305861: - 
>> syslog,sshd,invalid_login,authentication_failed,\n2014 Mar 08 01:00:59 (
>> someserver.somedomain.com) 

[ossec-list] Re: OSSEC & Logstash

[mangasof . manga]

Hi JP1, you found a pattern for archive.log file?

Em quarta-feira, 18 de fevereiro de 2015 17:12:45 UTC-3, jp1...@gmail.com 
escreveu:
>
> So, this works OK for me on alerts.log - does anyone have a logstash conf 
> that works on the archives.log if you have ossec saving all logs to that?
>
> On Saturday, March 8, 2014 at 5:02:35 PM UTC-5, Joshua Garnett wrote:
>>
>> All,
>>
>> I'll probably write a blog post on this, but I wanted to share some work 
>> I've done today.  
>> http://vichargrave.com/ossec-log-management-with-elasticsearch/ shows 
>> how to use OSSEC's syslog output to route messages to Elasticsearch.  The 
>> problem with this method is it uses UDP.  Even when sending packets to a 
>> local process UDP by definition is unreliable.  Garbage collections and 
>> other system events can cause packets to be lost.  I've found it tends to 
>> cap out at around 1,500 messages per minute. 
>>
>> To address this issue I've put together a logstash config that will read 
>> the alerts from /var/ossec/logs/alerts/alerts.log.  On top of solving the 
>> reliability issue, it also fixes issues with multi-lines being lost, and 
>> adds geoip lookups for the src_ip.  I tested it against approximately 1GB 
>> of alerts (3M events).
>>
>> input {
>>   file {
>> type => "ossec"
>> path => "/var/ossec/logs/alerts/alerts.log"
>> sincedb_path => "/opt/logstash/"
>> codec => multiline {
>>   pattern => "^\*\*"
>>   negate => true
>>   what => "previous"
>> }
>>   }
>> }
>>
>> filter {
>>   if [type] == "ossec" {
>> # Parse the header of the alert
>> grok {
>>   # Matches  2014 Mar 08 00:57:49 (some.server.com) 10.1.2.3->ossec
>>   # (?m) fixes issues with multi-lines see 
>> https://logstash.jira.com/browse/LOGSTASH-509
>>   match => ["message", "(?m)\*\* Alert 
>> %{DATA:timestamp_seconds}:%{SPACE}%{WORD}?%{SPACE}\- 
>> %{DATA:ossec_group}\n%{YEAR} %{SYSLOGTIMESTAMP:syslog_timestamp} 
>> \(%{DATA:reporting_host}\) 
>> %{IP:reporting_ip}\-\>%{DATA:reporting_source}\nRule: 
>> %{NONNEGINT:rule_number} \(level %{NONNEGINT:severity}\) \-\> 
>> '%{DATA:signature}'\n%{GREEDYDATA:remaining_message}"]
>>   
>>   # Matches  2014 Mar 08 00:00:00 ossec-server01->/var/log/auth.log
>>   match => ["message", "(?m)\*\* Alert 
>> %{DATA:timestamp_seconds}:%{SPACE}%{WORD}?%{SPACE}\- 
>> %{DATA:ossec_group}\n%{YEAR} %{SYSLOGTIMESTAMP:syslog_timestamp} 
>> %{DATA:reporting_host}\-\>%{DATA:reporting_source}\nRule: 
>> %{NONNEGINT:rule_number} \(level %{NONNEGINT:severity}\) \-\> 
>> '%{DATA:signature}'\n%{GREEDYDATA:remaining_message}"]
>> }
>>
>> # Attempt to parse additional data from the alert
>> grok {
>>   match => ["remaining_message", "(?m)(Src IP: 
>> %{IP:src_ip}%{SPACE})?(Src Port: %{NONNEGINT:src_port}%{SPACE})?(Dst IP: 
>> %{IP:dst_ip}%{SPACE})?(Dst Port: %{NONNEGINT:dst_port}%{SPACE})?(User: 
>> %{USER:acct}%{SPACE})?%{GREEDYDATA:real_message}"]
>> }
>>
>> geoip {
>>   source => "src_ip"
>> }
>>
>> mutate {
>>   convert  => [ "severity", "integer"]
>>   replace  => [ "@message", "%{real_message}" ]
>>   replace  => [ "@fields.hostname", "%{reporting_host}"]
>>   add_field=> [ "@fields.product", "ossec"]
>>   add_field=> [ "raw_message", "%{message}"]
>>   add_field=> [ "ossec_server", "%{host}"]
>>   remove_field => [ "type", "syslog_program", "syslog_timestamp", 
>> "reporting_host", "message", "timestamp_seconds", "real_message", 
>> "remaining_message", "path", "host", "tags"]
>> }
>>   }
>> }
>>
>> output {
>>elasticsearch {
>>  host => "10.0.0.1"
>>  cluster => "mycluster"
>>}
>> }
>>
>> Here are a few examples of the output this generates.
>>
>> {
>>"@timestamp":"2014-03-08T20:34:08.847Z",
>>"@version":"1",
>>"ossec_group":"syslog,sshd,invalid_login,authentication_failed,",
>>"reporting_ip":"10.1.2.3",
>>"reporting_source":"/var/log/auth.log",
>>"rule_number":"5710",
>>"severity":5,
>>"signature":"Attempt to login using a non-existent user",
>>"src_ip":"112.65.211.164",
>>"geoip":{
>>   "ip":"112.65.211.164",
>>   "country_code2":"CN",
>>   "country_code3":"CHN",
>>   "country_name":"China",
>>   "continent_code":"AS",
>>   "region_name":"23",
>>   "city_name":"Shanghai",
>>   "latitude":31.0456007,
>>   "longitude":121.3997,
>>   "timezone":"Asia/Shanghai",
>>   "real_region_name":"Shanghai",
>>   "location":[
>>  121.3997,
>>  31.0456007
>>   ]
>>},
>>"@message":"Mar  8 01:00:59 someserver sshd[22874]: Invalid user 
>> oracle from 112.65.211.164\n",
>>"@fields.hostname":"someserver.somedomain.com",
>>"@fields.product":"ossec",
>>"raw_message":"** Alert 1394240459.2305861: - 
>> syslog,sshd,invalid_login,authentication_failed,\n2014 Mar 08 01:00:59 (
>> someserver.somedomain.com) 

[ossec-list] Can I build the OSSEC server without the three GeoIP packages?

[Shawn Wiley]

Is there a way to the OSSEC server without the three GeoIP packages or at 
least force the packages to not be used? I'd like to install the least 
amount of additional packages to my web server as possible.


Thanks,

Shawn

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"ossec-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ossec-list+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ossec-list] OSSEC log analysis vs sending logs directly to OSSIM

[Eponymous -]

Thanks for your response Santiago!

So the target system is actually a pfSense router (FreeBSD 10.3 based) and 
the main problem I have is that the logs are not in plaintext format - they 
use a "clog" format instead which OSSEC can't read. The only workaround at 
the moment is to run a local Syslog server on the router and log everything 
to localhost to get the logs in plaintext - I hate the idea of this really 
:)

However, I do have the option of sending the logs via Syslog to the OSSIM 
server directly but this bypasses OSSEC. 

Your point about encryption and authentication is a good one but this won't 
be an issue for me as the link between the OSSIM server and OSSEC client is 
a physically separate, cabled interface used only for that purpose. I also 
don't need e-mail notifications or active response.

That being said, do I still lose something by *not *sending the logs to the 
OSSEC client first? In particular you mentioned: "detecting possible 
security issues, misconfigurations, errors". Are you saying that OSSIM is 
unable to give me the same functionality when sending the logs from the 
client directly to the server via Syslog?

Is is still worth the effort setting up the local Syslog workaround I 
mentioned above to be able to have the OSSEC client parse the local logs?

I appreciate your continued help.



On Wednesday, September 21, 2016 at 11:58:24 PM UTC+1, Santiago Bassett 
wrote:
>
> Hi,
>
> I would advice to use OSSEC agents to collect system logs data, since you 
> already have it there doing FIM and anomalies detection anyway. Also 
> communications are authenticated and encrypted (as opposed to default 
> Syslog). 
>
> Other advantage is that you pre-process them through OSSEC decoders and 
> rules (before it gets to OSSIM 
> correlation engine), detecting possible security issues, 
> misconfigurations, errors, As well you can trigger automatic emails and 
> use active responses (if you need them).
>
> On the other hand, I don't see a lot of value in processing Snort logs 
> through OSSEC (unless you want to use active-responses or use CDBs for 
> white/black listing). I would advice to send them directly to OSSIM and 
> enable snort-syslog plugin (unless you decide to use embedded Suricata).
>
> I hope that helps,
>
> Santiago.
>
> On Wed, Sep 21, 2016 at 2:13 PM, Eponymous -  > wrote:
>
>> Hi,
>>
>> I'm new to OSSEC and also OSSIM and I've just set up a very simple 
>> topology.
>>
>> I've got OSSIM on one machine and a single FreeBSD based machine running 
>> OSSEC and Snort. I've added the agent in the Agents tab and I can see it 
>> connects fine.
>>
>> I see OSSIM and OSSEC working together to schedule and run rootkit checks 
>> and syschecks, but I also know that OSSEC can parse the system logs and 
>> Snort logs looking for security issues. Currently, the OSSEC configuration 
>> is not set up to look at logs and other than manually editing the 
>> agent.conf I can't see any way to enable this functionality from OSSIM (I'm 
>> using the agent.conf deployment feature).
>>
>> My question is:
>>
>> Should the OSSEC agent be parsing the system logs and Snort logs and then 
>> send relevant data to the OSSIM server or should I set it up to send my 
>> logs directly to the OSSIM server using Syslog, bypassing the OSSEC agent 
>> all together?
>>
>> In each case what are the advantages and disadvantages? 
>>
>> In my setup it would be the most simple for the OSSEC agent to handle 
>> rootkit checking and syschecking only, with the system logs and Snort logs 
>> being sent directly to the  OSSIM server using Syslog.
>>
>> Thanks in advance.
>>
>> -- 
>>
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "ossec-list" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to ossec-list+...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"ossec-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ossec-list+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ossec-list] Re: How to change the OSSEC installation directory in windows

[Eero Volotinen]

How about modifying the installation package?

Eero

2016-09-22 12:56 GMT+03:00 Victor Fernandez :

> Hi,
>
> when you run the OSSEC installer for Windows, you can choose the location
> where OSSEC will be installed. This shouldn't be a problem.
>
> Since OSSEC registers a background service on Windows, you should first
> install OSSEC into another partition and then create the C:\ drive image.
>
> Hope it helps.
> Best regards.
>
> Victor.
>
>
>
> On Thursday, September 22, 2016 at 10:13:30 AM UTC+2, vikas wrote:
>>
>> Hello all,
>>
>> We have a group of servers where the C:/ drive gets re-imaged daily with
>> a standard image. Since its going to be same image that all the servers
>> use, not sure how to make OSSEC part of that image and avoid agent-server
>> registration issues. So we wanted to install it on a different drive to
>> avoid the complications, but couldn't find an option to specify custom path
>> for installation. Is it possible?
>>
>> Thank you for your help!
>>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "ossec-list" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ossec-list+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"ossec-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ossec-list+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ossec-list] Re: How to change the OSSEC installation directory in windows

[Victor Fernandez]

Hi,

when you run the OSSEC installer for Windows, you can choose the location 
where OSSEC will be installed. This shouldn't be a problem.

Since OSSEC registers a background service on Windows, you should first 
install OSSEC into another partition and then create the C:\ drive image.

Hope it helps.
Best regards.

Victor.


On Thursday, September 22, 2016 at 10:13:30 AM UTC+2, vikas wrote:
>
> Hello all,
>
> We have a group of servers where the C:/ drive gets re-imaged daily with a 
> standard image. Since its going to be same image that all the servers use, 
> not sure how to make OSSEC part of that image and avoid agent-server 
> registration issues. So we wanted to install it on a different drive to 
> avoid the complications, but couldn't find an option to specify custom path 
> for installation. Is it possible? 
>
> Thank you for your help!
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"ossec-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ossec-list+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ossec-list] Re: Querying Kibana for specific event types

[Jesus Linares]

Hi,

Review *alerts.json* in order to know if you have the decoder name and the 
event id extracted in fields. Also, check out your logstash mapping. If the 
fields are not extracted in alerts.json, you can not filter by them in 
kibana.

I did the query in Wazuh and it works, so I recommend you to try it. This 
is the documentation 
.

Regards.

On Wednesday, September 21, 2016 at 3:55:17 PM UTC+2, namobud...@gmail.com 
wrote:
>
> I tried this and it didn't work, I think because decoder.name doesn't 
> exist in the logstash index. Instead of id, I have _id which is not a 
> number but a character string.
>
> On Tuesday, September 20, 2016 at 3:56:44 AM UTC-4, Jesus Linares wrote:
>>
>> Hi,
>>
>> in order to filter by an event ID of Windows, just use this query in the 
>> search bar of kibana:
>> decoder.name:"windows" AND id:"4625"
>>
>> In this case, you are filtering events with id 4625:
>> 2016 Sep 20 07:50:17 WinEvtLog: Security: AUDIT_FAILURE(*4625*): 
>> Microsoft-Windows-Security-Auditing: (no user): no domain: WIN-: An 
>> account failed to log on...
>>
>> I assume you are sending the file *alerts.json* to elasticsearch.
>>
>> Regards.
>>
>> On Monday, September 19, 2016 at 10:11:37 PM UTC+2, namobud...@gmail.com 
>> wrote:
>>>
>>> Based on this storm center article:
>>>
>>> https://isc.sans.edu/forums/diary/Windows+Events+log+for+IRForensics+Part+1/21493/
>>>
>>> I'm trying to figure out how to query Kibana for specific event ID 
>>> numbers from the dashboard search area the article mentions. Is there a 
>>> definitive guide for searching OSSEC with Kibana.
>>>
>>>
>>>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"ossec-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ossec-list+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ossec-list] How to change the OSSEC installation directory in windows

[vikas]

Hello all,

We have a group of servers where the C:/ drive gets re-imaged daily with a 
standard image. Since its going to be same image that all the servers use, 
not sure how to make OSSEC part of that image and avoid agent-server 
registration issues. So we wanted to install it on a different drive to 
avoid the complications, but couldn't find an option to specify custom path 
for installation. Is it possible? 

Thank you for your help!

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"ossec-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ossec-list+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.