On 06/19/2017 10:48 AM, Peter Portante wrote:
Hi Andre,
This is a hard-coded Docker size. For background see:
* https://bugzilla.redhat.com/show_bug.cgi?id=1422008, "[RFE] Fluentd
handling of long log lines (> 16KB) split by Docker and indexed into
several ES documents"
* And the reason for the original 16 KB limit:
https://bugzilla.redhat.com/show_bug.cgi?id=1335951, "heavy logging
leads to Docker daemon OOM-ing"
Not only that, but a PR to give the ability to docker to configure the
size limit [1] was rejected due to $reasons . . .
The processor that reads the json-file documents for sending to
graylog needs to be endowed with the smarts to handle reconstruction
of those log lines, most likley, obviously with some other upper bound
(as a container is not required to emit newlines in stdout or stderr.
Regards,
-peter
[1] For the 90% of cases where this problem could have been easily
solved by simply bumping the limit. Of course there will be times when
no appropriate limit can be found. Of course a client that consumes
docker logs must be able to handle partial messages, by reconstructing
them in the logging collector, or further on down the chain.
On Mon, Jun 19, 2017 at 11:43 AM, Andre Esser
<andre.es...@voidbridge.com> wrote:
We use Graylog for log visualisation. However that's not the culprit it
turns out. Log entries in the pod's log file are already split into chunks
of 16KB like this:
{"log":"The quick brown[...]jumps ov","stream":"stdout",\
"time":"2017-06-19T15:27:33.130524954Z"}
{"log":"er the lazy dog.\n","stream":"stdout",\
"time":"2017-06-19T15:27:33.130636562Z"}
So, to cut a long story short, is there any way to increase the size limit
before a log record gets split into two JSON records?
On 2017-06-19 16:21, Peter Portante wrote:
Who setup Graylog for openshift?
-peter
On Mon, Jun 19, 2017 at 11:18 AM, Andre Esser
<andre.es...@voidbridge.com> wrote:
I meant the limit in Graylog. Although I just noticed that it is actually
16384 (16KB). The line split after 2048 characters only applies on the
web
UI.
Is this a Graylog limitation and can it be extended?
On 2017-06-19 14:21, Jessica Forrester wrote:
Are you asking about logs in the web console, the `oc logs` command, or
in
Kibana?
On Mon, Jun 19, 2017 at 8:29 AM, Andre Esser <andre.es...@voidbridge.com
<mailto:andre.es...@voidbridge.com>> wrote:
Hi,
In Origin 1.4.1 all log records longer than 2048 characters are
split over two lines (longer than 4096 characters over three lines
and so on).
Is there any way to increase this limit?
Thanks,
Andre
_______________________________________________
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users
_______________________________________________
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users
_______________________________________________
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users
