Even more...assuming that a full queue will trigger full delays...when it would be this delay disabled again?...Cause in our scenario, seems the process is stuck forever, until a rsyslog restart.


On 03/10/17 10:52, mosto...@gmail.com wrote:

Full delay mark is set t0 50% of highwatermark, without apparent/documented reason

https://github.com/rsyslog/rsyslog/blob/a6ae18ee4cb4530e4142a2e98aed404dd3b03346/runtime/queue.c#L2373


On 03/10/17 10:50, Rainer Gerhards wrote:
What exactly do you mean?

Rainer

Sent from phone, thus brief.

Am 03.10.2017 10:27 schrieb "mostolog--- via rsyslog" <rsyslog@lists.adiscon.com <mailto:rsyslog@lists.adiscon.com>>:

    Hmm...wondering if theres is a bug here as:

    main_queue(
        queue.filename="main"
        queue.maxdiskspace="4G"
        queue.saveonshutdown="on"
        queue.lowwatermark="30000"
        queue.highwatermark="60000"
        queue.size="100000"
        queue.type="LinkedList"
        queue.fulldelaymark="90000"
        queue.lightdelaymark="75000"
        #queue.discardmark="9750"
        #queue.discardseverity="8"
    )

    results in:

    8544.970496200:main thread    : main Q: queue.c: params: type 1,
    enq-only 0, disk assisted 1, spoolDir '/data', maxFileSz
    16777216, maxQSize 100000, lqsize 0, pqsize 0, child 0, full
    delay 30000, light delay 75000, deq batch size 1024, high wtrmrk
    60000, low wtrmrk 30000, discardmrk 98000, max wrkr 1, min msgs
    f. wrkr 100000

    8544.970615922:main thread    : main Q[DA]: queue.c: params: type
    2, enq-only 0, disk assisted 0, spoolDir '/data', maxFileSz
    16777216, maxQSize 0, lqsize 0, pqsize 0, child 1, full delay -1,
    light delay -1, deq batch size 8, high wtrmrk 0, low wtrmrk 1,
    discardmrk 0, max wrkr 1, min msgs f. wrkr 0




    On 02/10/17 11:00, mosto...@gmail.com <mailto:mosto...@gmail.com>
    wrote:

        Hi

        Some time ago we reported issues with RELP /hanging/ and not
        forwarding messages until restart.

        Last week we were able to replicate and still figuring out
        *how rsyslog delay system works*.

        According to documentation,

            /queue.fulldelaymark//
            //Number of messages when the queue should block delayable
            messages. Messages are NO LONGER PROCESSED until the
        queue has
            sufficient space again. If a message is delayable depends
        on the
            input. For example, messages received via imtcp are delayable
            (because TCP can push back), but those received via imudp
        are not
            (as UDP does not permit a push back). The intent behind this
            setting is to leave some space in an almost-full queue for
            non-delayable messages, which would be lost if the queue
        runs out
            of space. Please note that if you use a DA queue, setting the
            fulldelaymark BELOW the highwatermark makes the queue never
            activate disk mode for delayable inputs. So this is
        probably not
            what you want//.

            queue.lightdelaymark number./

        It's our understanding that

          * TCP ack's would be delayed _/a bit/_ if
        queue.size=lightdelaymark.
          * TCP ack's would be nack's if queue.size=fulldelaymark//.
            //

        *Is that correct? Does****it behaves as low/highwatermark
        instead?* (full delay when size=fulldelaymark, stops delaying
        when size is back to lightdelaymark)


        The following configuration:

        main_queue(
            queue.filename="main"
            queue.maxdiskspace="1G"
            queue.saveonshutdown="on"
            queue.lowwatermark="150000"
            queue.highwatermark="180000"
            queue.size="200000"
            queue.type="LinkedList"
            queue.fulldelaymark="-1"
            queue.lightdelaymark="-1"
        )

        Resulted in this debug trace:

        Sep 29 13:07:22 host11 rsyslog[57150]: 3242.567069705:main
        thread    : main Q: queue.c: params: type 1, enq-only 0, disk
        assisted 1, spoolDir '/data', maxFileSz 16777216, maxQSize
        200000, lqsize 0, pqsize 0, child 0, *full delay 90000, light
        delay 140000*, deq batch size 1024, high wtrmrk 180000, low
        wtrmrk 150000, discardmrk 196000, max wrkr 1, min msgs f.
        wrkr 200000

        *Why full is lower than light? *

        Sep 29 13:07:22 host11 rsyslog[57150]: 3242.567182561:main
        thread    : main Q[DA]: queue.c: params: type 2, enq-only 0,
        disk assisted 0, spoolDir '/data', maxFileSz 16777216,
        maxQSize 0, lqsize 0, pqsize 0, child 1, full delay -1, light
        delay -1, deq batch size 8, high wtrmrk 0, low wtrmrk 1,
        discardmrk 0, max wrkr 1, min msgs f. wrkr 0


        This other configuration,

                queue.filename="relp"
                queue.maxdiskspace="1G"
                queue.saveonshutdown="on"
                queue.lowwatermark="100000"
                queue.highwatermark="600000"
                queue.size="1000000"
                queue.type="LinkedList"
                queue.fulldelaymark="5000000"
                queue.lightdelaymark="5000000"

        resulted in:

        Sep 29 13:07:22 host11 rsyslog[57150]: 3242.566625167:main
        thread    : relp: queue.c: params: type 1, enq-only 0, disk
        assisted 1, spoolDir '/data', maxFileSz 16777216, maxQSize
        1000000, lqsize 0, pqsize 0, child 0, *full delay 300000,
        light delay 700000*, deq batch size 1024, high wtrmrk 600000,
        low wtrmrk 100000, discardmrk 980000, max wrkr 1, min msgs f.
        wrkr 1000000

        *What are the valid values for delay marks?*
        *If a full delay behave as closing the socket...when it would
        be set to normal again?*

        Sep 29 13:07:22 host11 rsyslog[57150]: 3242.566787556:main
        thread    : relp[DA]: queue.c: params: type 2, enq-only 0,
        disk assisted 0, spoolDir '/data', maxFileSz 16777216,
        maxQSize 0, lqsize 0, pqsize 0, child 1, full delay -1, light
        delay -1, deq batch size 8, high wtrmrk 0, low wtrmrk 1,
        discardmrk 0, max wrkr 1, min msgs f. wrkr 0



        I think this doubts would be quite useful for other people
        having this kind of issues.

        Regards



    _______________________________________________
    rsyslog mailing list
    http://lists.adiscon.net/mailman/listinfo/rsyslog
    <http://lists.adiscon.net/mailman/listinfo/rsyslog>
    http://www.rsyslog.com/professional-services/
    <http://www.rsyslog.com/professional-services/>
    What's up with rsyslog? Follow https://twitter.com/rgerhards
    NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
    myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
    POST if you DON'T LIKE THAT.



_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Reply via email to