Discussion is needed because after some brainstorming, I would like to
suggest to supress the date format and to auto transform well known fields
like timereported or time generated into ISODATE when om plugin is mongodb
when a template is used.
We could try to also support string date conver
Hi Rainer,
Take your time. I use this new code in a big prod environment to see if it
is stable.
For the moment, all is working like a charm.. ( RHEL 6.5 with v7-sable and
mongodb modifs )
Alain
Rainer Gerhards wrote
> I'll review ASAP, but needed to finally work a bit on libstdlog,else thi
Pull request is rady.
Big job of cleaning.
I have also modify configure.ac, Makefile.am to conditionally use the needed
flags to compile.
Some #ifdef to compile the new code only if needed.
the syslogTime2time_t function is declared as extern in the template.h to
suppress warnings ( I hate warn
Hi Rainer,
take a look at
https://github.com/eczema/rsyslog/compare/rsyslog:master...master
This shows the diffs...
Alain
--
View this message in context:
http://rsyslog-users.1305293.n2.nabble.com/About-DateFormat-in-a-template-for-ommongodb-tp7584362p7584445.html
Sent from the rsyslog-use
Rainer,
As promised the hack is done but i need to clean my work to know how to
include mongo.h with the good directive in the makefile for template.c and
also a way to use datetime object in template.c
Alain
--
View this message in context:
http://rsyslog-users.1305293.n2.nabble.com/Abo
Well I have a primary working version of rsyslog with a real support for
mongodb ISODATE
I need help because the object library timedate is not accessible from
template.c
I had to add static inline function syslogTime2time_t.
I will try to use a fork on github monday with the diff to discus
--
in ommongodb.c:
nothing to modify !
;-)
--
I'm wrong ommondbsql must detect that the int64 is a date to use the
bson_append_date
I wrote to quickly...
Alain
--
View this message in context:
http://rsyslog-users.1305293.n2.nabble.com/About-DateFormat-in-a-templ
added to remove this part from
template.c and ommongodb.c
Alain
Rainer Gerhards wrote
> On Thu, Feb 13, 2014 at 4:38 PM, ecze <
> eczema@
> > wrote:
>
>> It sems that a little hack is necessary:
>>
>> 1. add a new properety in template ( ex: omgoda
It sems that a little hack is necessary:
1. add a new properety in template ( ex: omgodate )
2. hack template.c to support this
3. hack ommongodb.c to support bson_append_date()
Alain
I'll try to make this...
--
View this message in context:
http://rsyslog-users.1305293.n2.nabble.com/Abou
Hi all,
Which DateFormat should I use as option for the field timereported ?
All my test failed when I try to search on this field in the loganalyzer.
-> property(name="timegenerated" dateFormat="rfc3339" ) <- This doesn't work
result in mongodb:
"timereported" : "2014-02-13T15:57:21+01:00",
Indeed this patch resolve the issue...
diff --git a/plugins/ommongodb/ommongodb.c b/plugins/ommongodb/ommongodb.c
index ecfd251..446d6c3 100644
--- a/plugins/ommongodb/ommongodb.c
+++ b/plugins/ommongodb/ommongodb.c
@@ -81,7 +81,7 @@ static struct cnfparamdescr actpdescr[] = {
{ "collectio
I made some comments to fix it.
Hope it helps...
Alain
--
View this message in context:
http://rsyslog-users.1305293.n2.nabble.com/Bugzilla-Bug-513-ommongodb-required-parameter-tp7584350.html
Sent from the rsyslog-users mailing list archive at Nabble.com.
_
Hi,
Just a little remark,
On the new site ( dev.rsyslog.com) ,no links inline are visible. When I saw
that url links exists, I had to try every zone on the page to discover if
others links exists
example:
If you upgrade from v7, read the rsyslog v8 compatibility notes. if you
upgrade from v
I applause for the great enhancement !
Indeed, the structure is more clear and docs more 'findable'...
Alain
--
View this message in context:
http://rsyslog-users.1305293.n2.nabble.com/rsyslog-com-site-changes-tp7584195p7584341.html
Sent from the rsyslog-users mailing list archive at Nabb
Is it a dedicated ML for LogAnalyzer ?
I have a couple of question regarding mongodb processing...
After our last good fix to inject logs into mongodb with a well working
template, the interface shows me all
information. But, now, it is just impossible to do a search for some fields
(msg, severit
Nice.
Works as expected:
db.log.findOne()
{
"_id" : ObjectId("52fb4470bf8fbc58c1ceb7f6"),
"hostname" : "10.253.112.4",
"timereported" : "Feb 12 10:52:48",
"timegenerated" : "Feb 12 10:52:48",
"msg" : " DHCPOFFER on 10.14.169.52 to 7c:c3:a1:24:a1:35
(iP
Well,
The current ompgsql module works like a charm.. ( latest v7-stable) So your
latest commit is OK.
I will use github for the ommongodb issue ;-)
Alain
--
View this message in context:
http://rsyslog-users.1305293.n2.nabble.com/rsyslog-7-4-9-ompgsql-libpq-threads-problem-tp7584204p758
Great, I test it in a few minutes...
Other Issue fixed in my previous post ( omongodb ) before a new release
should be a good idea...
Alain
--
View this message in context:
http://rsyslog-users.1305293.n2.nabble.com/rsyslog-7-4-9-ompgsql-libpq-threads-problem-tp7584204p7584315.html
Sent fro
Well,
Bug found...
I'll send a patch ASAP..
Alain
--
View this message in context:
http://rsyslog-users.1305293.n2.nabble.com/ommongodb-storing-problem-V7-4-9-tp7584289p7584314.html
Sent from the rsyslog-users mailing list archive at Nabble.com.
__
What do you mean when you say "write to a file" ?
In my first post I wrote that the same template write with omfile all
entries perfectly.
So this template works well when output is a file.
When datas are proccessed by ommongo it seems that some value len are fixed
and strings are not terminated wi
I'll search into tplToJSON function from template.c.
This code probably misunderstand how to get a property value length
Alain
--
View this message in context:
http://rsyslog-users.1305293.n2.nabble.com/ommongodb-storing-problem-V7-4-9-tp7584289p7584290.html
Sent from the rsyslog-user
When I use a template to insert entries into mongodb:
template(name="BSON" type="list") {
constant(value="{\"sys\" : \"")
property(name="hostname")
constant(value="\", \"time\" : \"")
property(name="timereported")
constant(value="'\", \"time_rcvd\" : \"")
Ok, the patch works perfectly.
Is there anyone in charge to check, comment and commit it ?
I think this is quite critical because the *PGConn point to an invalid
address after the fork in the current versions ( 5/7/8 )
Alain
ecze wrote
> Another patch ( not yet tested ) without useless
Another patch ( not yet tested ) without useless PG connect..
-patch--
diff --git a/plugins/ompgsql/ompgsql.c b/plugins/ompgsql/ompgsql.c
index 11f346f..e236234 100644
--- a/plugins/ompgsql/ompgsql.c
+++ b/plugins/ompgsql/ompgsql.c
Little remark
To be honest, I don't see any advantage to init the PG connection during the
instance creation.
We could just suppress the call to initPgSQL and keep in this patch the test
part in the BeginTransaction to init the connection if the handle is NULL.
Alain
ecze wrote
>
Purpose of a patch for Stable Release 7.4.9 :
diff -Nru ompgsql.c /new/ompgsql.c
--- ompgsql.c 2014-01-21 13:14:21.0 +0100
+++ /new/ompgsql.c2014-02-05 12:01:51.557026475 +0100
@@ -262,8 +262,13 @@
BEGINbeginTransaction
CODESTARTbeginTransaction
+if ( pData->f_hpgsql == NULL
A simple patch resolve all my problems:
No more stopping listening , no more log lost.. imtcp, imudp works nice and
ompgsql works better.
in BEGINparseSelectorAct:
if (iPgSQLPropErr) {
errmsg.LogError(0, RS_RET_INVALID_PARAMS, "Trouble with
PgSQL connection properties. -
I will try a patch tomorrow:
340 CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0,
OMSR_RQD_TPL_OPT_SQL, (uchar*) " StdPgSQLFmt"));
341
342 /* If we detect invalid properties, we disable logging,
343 * because right properties are vital at this place.
344
SELinux not enabled at all
Found.. I think
Rsyslog open connection before forking. This is a very bad idea as said in
Postgresql libpq doc:
http://www.postgresql.org/docs/8.4/static/libpq-connect.html
After the fork, all those threads have lost the handler to the connection
database.
Alain
begin is not present if the DB is
reconnected...
Am I right ?
Alain
ecze wrote
> Some info about the ompgsql threads:
>
> in tryExec(uchar *pszCmd, instanceData *pData:
>
> modified code:
>
> if(execState != PGRES_COMMAND_OK &&
resultStatus(pgRet)),PQerrorMessage(pData->f_hpgsql));
bHadError = 1;
}
result:
postgres query execution failed: PGRES_FATAL_ERROR , could not receive data
from server: Transport endpoint is not connected
So after the first connection, the connection is closed for an unkown
reason...
Hi,
I open a new thread for ompgsql.
After three days of testing, I finally point that if rsyslogd is forked (
running background ) , each thread of ompgsql will fail to connect at first
time.
All connection are opened , closed and reopened
But this mechanism seems to block rsyslogd... othe
As the original Report when rsyslog is lanched with -n, everything is running
just fine
What's the difference for threads ?
Alain
ecze wrote
> Well not sure that the queue is involved...
>
> After some seconds ( less than 2 minutes )
> netstat -naptu | sed -e 's
Well not sure that the queue is involved...
After some seconds ( less than 2 minutes )
netstat -naptu | sed -e 's/127.0.0.1/localhost/g' -e
's/\([0-9]*\.[0-9]*\.[0-9]*\.[0-9]*\)/adrshided/g' | grep rsyslog
tcp0 0 localhost:41864 localhost:5432
ESTABLISHED 28
On the PostgresQL Side, no error at all.
The Debug shows only correct transactions. ( pg_log , debug set to max = 5
).
So , I'm still blocked. Why rsyslog :
1. stop to listen imtcp for new connection
2. stop to listen imudp 514
Valgrind doesn't show any leaks...
Here my config:
.
.
.
# Provid
Well after a 'sniffing' party ;-), I see that ompgsql is connecting to
postgres correctly but close the connection directly (without any tcp error
: the client [ompgsql] sent the fin packet ).
In the ompgsql code, it seems that initPgSQL is in charge.
Why the module close the connection ?
Why I
Indeed, the main problem seems to be a problem with ompgsql.
Without trying to write to pgsql, all is just fine.
The postgresql log shows:
LOG: unexpected EOF on client connection
and in the DEBUG:
4594.280515317:7f32d3bfe700: postgres query execution failed:
PGRES_FATAL_ERROR
4594.280565204:7
Well I'll try to give more informations Monday.
Regards,
Alain
--
View this message in context:
http://rsyslog-users.1305293.n2.nabble.com/Big-issues-with-Rsyslog-listening-on-port-514-with-IMRELP-tp7579826p7584156.html
Sent from the rsyslog-users mailing list archive at Nabble.com.
_
Hi all,
Same issue here in our environment.
RHEL 6.5 latest rsyslog V7 stable from adiscon
rsyslog, listen to port udp 514 some seconds and stops...
imtcp 514: same issue...
I see with netstat a 0.0.0.0:514 but after 3-4 incoming streams, the
daemon stop to listen and no more connections are
39 matches
Mail list logo
