thanks for the followup. i had come to a similar conclusion. 10# in a 5# sack is gonna leave some stuff on the floor... :-)
On 10/25/17, 12:09, "David Lang" <da...@lang.hm> wrote: >you can increase N so that messages you receive/read from file are intact >within >rsyslog, but then you need to figure out what to do with them before you >send >them > >you can write them to a local 'oversized' file and send some subset of >the log >with a tag to say that this isn't the full message. > >you can try to figure a sane way to split them into 1/n messages and send >them >as multiple messages (proabably through omprog and that program generates >new >messages, but you can also do this in rsyslog with multiple templates and >sending the 'same' message through the different templates to the same >destination) > >there is no generic answer to the question of how do I put 10k of data >into a 1k >message without loosing anything :-) > >David Lang > > On Wed, 25 Oct 2017, Randall Diffenderfer via rsyslog wrote: > >> Date: Wed, 25 Oct 2017 18:48:52 +0000 >> From: Randall Diffenderfer via rsyslog <rsyslog@lists.adiscon.com> >> To: Rainer Gerhards <rgerha...@hq.adiscon.com>, >> rsyslog-users <rsyslog@lists.adiscon.com> >> Cc: Randall Diffenderfer <rdiffender...@proofpoint.com> >> Subject: Re: [rsyslog] handling oversized messages >> >> i suppose i deserved thatŠ :-) >> >> however, i have to interoperate with other folks who can't/won't >>increase N Š >> >> so, i am pegged at their "N" in order to see no data corruption, as the >>messages are structured, in that they are json, and busting them up will >>be a bit problematicŠ >> >> From: Rainer Gerhards >><rgerha...@hq.adiscon.com<mailto:rgerha...@hq.adiscon.com>> >> Date: Wednesday, October 25, 2017 at 11:33 >> To: rsyslog-users >><rsyslog@lists.adiscon.com<mailto:rsyslog@lists.adiscon.com>> >> Cc: Randall Diffenderfer >><rdiffender...@proofpoint.com<mailto:rdiffender...@proofpoint.com>> >> Subject: Re: [rsyslog] handling oversized messages >> >> It may sound dumb, but: increase n! That's why this setting exists. >> >> Rainer >> >> Sent from phone, thus brief. >> >> Am 25.10.2017 19:48 schrieb "Randall Diffenderfer via rsyslog" >><rsyslog@lists.adiscon.com<mailto:rsyslog@lists.adiscon.com>>: >> >> given the global setting of "maxmessagesize=N", what is my recourse if >>i >> need to process a message > N in imfile? >> >> in other i/o modules? it appears the message is truncated at ~N, and >>not >> split (which is what i thought i had seen in the past...) >> >> _______________________________________________ >> rsyslog mailing list >> >>https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mai >>lman_listinfo_rsyslog&d=DwID-g&c=Vxt5e0Osvvt2gflwSlsJ5DmPGcPvTRKLJyp031rX >>jhg&r=9sXs16LkSD1MzhoJxgyl7R_tait1dEXqvyD3NCT1wEA&m=Tsk9ggiaNh8XQwMDW_NJS >>gtFP5Gg9BJUwk_tROvEBGk&s=RZyuuUrNE1HDXRaBvHQSIeQQqjFf6UaevYcum2vnk6Q&e=<h >>ttps://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mail >>man_listinfo_rsyslog&d=DwMFaQ&c=Vxt5e0Osvvt2gflwSlsJ5DmPGcPvTRKLJyp031rXj >>hg&r=9sXs16LkSD1MzhoJxgyl7R_tait1dEXqvyD3NCT1wEA&m=0d7WauHwvHvrFWs0bcAWNN >>xOtvaCSn7-izMA1oDTedw&s=Y2aZ4XynKrB4wgg_Y21Tuh5VisfzrJe4cTVcQkFfBow&e=> >> >>https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_profe >>ssional-2Dservices_&d=DwID-g&c=Vxt5e0Osvvt2gflwSlsJ5DmPGcPvTRKLJyp031rXjh >>g&r=9sXs16LkSD1MzhoJxgyl7R_tait1dEXqvyD3NCT1wEA&m=Tsk9ggiaNh8XQwMDW_NJSgt >>FP5Gg9BJUwk_tROvEBGk&s=GHF7DIY7y7puq6Xueai6-Fe7xPtjstJAzq7Ohp1TXt0&e=<htt >>ps://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_professi >>onal-2Dservices_&d=DwMFaQ&c=Vxt5e0Osvvt2gflwSlsJ5DmPGcPvTRKLJyp031rXjhg&r >>=9sXs16LkSD1MzhoJxgyl7R_tait1dEXqvyD3NCT1wEA&m=0d7WauHwvHvrFWs0bcAWNNxOtv >>aCSn7-izMA1oDTedw&s=iK9krYQbWQ66DQXmXGuvorYG28Ioqr9mIroDehsBFLM&e=> >> What's up with rsyslog? Follow >>https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhard >>s&d=DwID-g&c=Vxt5e0Osvvt2gflwSlsJ5DmPGcPvTRKLJyp031rXjhg&r=9sXs16LkSD1Mzh >>oJxgyl7R_tait1dEXqvyD3NCT1wEA&m=Tsk9ggiaNh8XQwMDW_NJSgtFP5Gg9BJUwk_tROvEB >>Gk&s=99ftjFUt5naxsc_8bAM_9TFPSrM6B8EpUR9vvnhyBkc&e=<https://urldefense.pr >>oofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwMFaQ&c=Vxt5e0Os >>vvt2gflwSlsJ5DmPGcPvTRKLJyp031rXjhg&r=9sXs16LkSD1MzhoJxgyl7R_tait1dEXqvyD >>3NCT1wEA&m=0d7WauHwvHvrFWs0bcAWNNxOtvaCSn7-izMA1oDTedw&s=0XCzqR0TiMEj5d53 >>jKXtPEmzmNe0C3xpAV8fiGXFyq4&e=> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>if you DON'T LIKE THAT. >> _______________________________________________ >> rsyslog mailing list >> >>https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mai >>lman_listinfo_rsyslog&d=DwID-g&c=Vxt5e0Osvvt2gflwSlsJ5DmPGcPvTRKLJyp031rX >>jhg&r=9sXs16LkSD1MzhoJxgyl7R_tait1dEXqvyD3NCT1wEA&m=Tsk9ggiaNh8XQwMDW_NJS >>gtFP5Gg9BJUwk_tROvEBGk&s=RZyuuUrNE1HDXRaBvHQSIeQQqjFf6UaevYcum2vnk6Q&e= >> >>https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_profe >>ssional-2Dservices_&d=DwID-g&c=Vxt5e0Osvvt2gflwSlsJ5DmPGcPvTRKLJyp031rXjh >>g&r=9sXs16LkSD1MzhoJxgyl7R_tait1dEXqvyD3NCT1wEA&m=Tsk9ggiaNh8XQwMDW_NJSgt >>FP5Gg9BJUwk_tROvEBGk&s=GHF7DIY7y7puq6Xueai6-Fe7xPtjstJAzq7Ohp1TXt0&e= >> What's up with rsyslog? Follow >>https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhard >>s&d=DwID-g&c=Vxt5e0Osvvt2gflwSlsJ5DmPGcPvTRKLJyp031rXjhg&r=9sXs16LkSD1Mzh >>oJxgyl7R_tait1dEXqvyD3NCT1wEA&m=Tsk9ggiaNh8XQwMDW_NJSgtFP5Gg9BJUwk_tROvEB >>Gk&s=99ftjFUt5naxsc_8bAM_9TFPSrM6B8EpUR9vvnhyBkc&e= >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
