Hi Victor,
Huge thanks for the detail, this would explain exactly why we're seeing
this; our OSSEC managers are likely overloaded.
It would be very helpful to include the agentid in the logfile to
understand / track where this is occurring and the number of unique agents
that are impacted, perhaps something like:
From: src/error_messages/error_messages.h
#define SEC_ERROR "%s(1217): ERROR: Error creating encrypted message
for: '%s')."
Then inside: src/remoted/sendmsg.c
msg_size = CreateSecMSG(&keys, msg, crypt_msg, agentid);
if (msg_size == 0) {
merror(SEC_ERROR, ARGV0, agentid);
return (-1);
}
The clustered nature of this issue leads me to suspect it's repeating this
error in the logfiles multiple times for a connection attempt across only
one or two agents.
Again, many thanks for the detailed response.
Graeme
On Thursday, July 28, 2016 at 5:33:29 PM UTC-7, Victor Fernandez wrote:
>
> Hi Graeme.
>
> According to the log, I think the problem occurs when the manager tries to
> send the merged.mg to an agent that has not sent the keep-alive in the
> last 20 minutes. This may happen if a lot of agents get connected, or send
> the keep-alive at the same time.
>
> So, if many agents send a keep-alive, the manager takes more than 20
> minutes to send the merged.mg to an agent, and that agent hasn't sent the
> keep-alive again, this problem occurs.
>
> I did some math: the manager sleeps one second every time it sends 27 KB.
> With a 150 KB merged.mg, OSSEC takes 20 minutes to send the complete file
> to about 216 agents.
>
> The 20-minutes check appears on src/remoted/sendmsg.c:
>
> /* If we don't have the agent id, ignore it */
> if (keys.keyentries[agentid]->rcvd < (time(0) - (2 * NOTIFY_TIME))) {
> return (-1);
> }
>
> NOTIFY_TIME is 600 (10 minutes) by default. Nevertheless OSSEC labels an
> agent as disconnected when it hasn't send the keep-alive in the last 30:30
> minutes, as we can see at src/shared/read-agents.c:
>
> if (file_status.st_mtime > (time(0) - (3 * NOTIFY_TIME + 30))) {
> return (GA_STATUS_ACTIVE);
> }
>
> Because of this, I think that this may be an issue.
>
> I think that a good approach would be to check that there aren't alerts
> about disconnected agents that connected recently.
>
> Kind regards.
>
>
> On Thursday, July 28, 2016 at 9:43:32 AM UTC-7, Graeme Stewart wrote:
>>
>> Seeing a lot of errors in the logfiles like this:
>>
>> 2016/07/28 16:41:48 ossec-remoted: ERROR: Unable to send file 'merged.mg'
>> to agent.
>> 2016/07/28 16:41:50 ossec-remoted(1217): ERROR: Error creating encrypted
>> message.
>> 2016/07/28 16:41:50 ossec-remoted: ERROR: Unable to send file 'merged.mg'
>> to agent.
>> 2016/07/28 16:41:50 ossec-remoted(1217): ERROR: Error creating encrypted
>> message.
>> 2016/07/28 16:41:50 ossec-remoted: ERROR: Unable to send file 'merged.mg'
>> to agent.
>> 2016/07/28 16:41:52 ossec-remoted(1217): ERROR: Error creating encrypted
>> message.
>> 2016/07/28 16:41:52 ossec-remoted: ERROR: Unable to send file 'merged.mg'
>> to agent.
>> 2016/07/28 16:41:52 ossec-remoted(1217): ERROR: Error creating encrypted
>> message.
>> 2016/07/28 16:41:52 ossec-remoted: ERROR: Unable to send file 'merged.mg'
>> to agent.
>> 2016/07/28 16:41:54 ossec-remoted(1217): ERROR: Error creating encrypted
>> message.
>> 2016/07/28 16:41:54 ossec-remoted: ERROR: Unable to send file 'merged.mg'
>> to agent.
>> 2016/07/28 16:41:56 ossec-remoted(1217): ERROR: Error creating encrypted
>> message.
>>
>> Any guidance on troubleshooting? Search hasn't turned up much other than
>> delete merged.mg and restart (which we've tried to no success)...
>>
>
--
---
You received this message because you are subscribed to the Google Groups
"ossec-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to ossec-list+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
