GitHub user blbradley opened a pull request:

    https://github.com/apache/kafka/pull/3083

    KAFKA-1955: [WIP] Disk based buffer in Producer

    Based on patch from @jkreps in [this JIRA 
ticket](https://issues.apache.org/jira/browse/KAFKA-1955).
    
    - [ ] Get some unit tests that would cover disk-backed usage
    - [ ] Do some manual performance testing of this usage and understand the 
impact on throughput.
    - [ ] Do some manual testing of failure cases (i.e. if the broker goes down 
for 30 seconds we should be able to keep taking writes) and observe how well 
the producer handles the catch up time when it has a large backlog to get rid 
of.
    - [ ] Add a new configuration for the producer to enable this, something 
like use.file.buffers=true/false.
    - [ ] Add documentation that covers these new options.
    
    I've brought the patch into sync with trunk. Testing is next, which I've 
started on. I am flexible on how this can be implemented.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/blbradley/kafka kafka-disk-buffer

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/3083.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3083
    
----
commit 6b29fc95c394283ff4f2410ad37f7c8fcbd0d8d7
Author: Brandon Bradley <bradleytas...@gmail.com>
Date:   2017-05-17T17:12:53Z

    WIP: KAFKA-1955 August 8th 2015 rebase

commit 75d2af1d7f8dda4e2fe41da60455d813d655edd0
Author: Brandon Bradley <bradleytas...@gmail.com>
Date:   2017-05-17T22:43:47Z

    Merge branch 'trunk' into kafka-disk-buffer
    
    patch works against trunk test suite

commit d3c765db789eef2fe71eca7a45dbca72e356f346
Author: Brandon Bradley <bradleytas...@gmail.com>
Date:   2017-05-17T23:14:34Z

    fix imports, add whitespace from diff

commit b58118c6413a5e900f5c1ebee112bd24e8d4b119
Author: Brandon Bradley <bradleytas...@gmail.com>
Date:   2017-05-17T23:34:04Z

    simple file buffer test

commit cd389f073eca18effa6449d9934aea0f90e84139
Author: Brandon Bradley <bradleytas...@gmail.com>
Date:   2017-05-17T23:35:21Z

    failing unallocated memory check

commit 49b6860e6c3be4bac62937dc835d5b6f97c7ff11
Author: Brandon Bradley <bradleytas...@gmail.com>
Date:   2017-05-18T00:35:29Z

    allocate buffer dynamically, passing tests

commit ed7aab5357fe9d7805dcb305d0318fb4ea770550
Author: Brandon Bradley <bradleytas...@gmail.com>
Date:   2017-05-18T00:46:10Z

    failing allocated memory check

commit 875ac83096199e35307a7ef47772907607aba1f1
Author: Brandon Bradley <bradleytas...@gmail.com>
Date:   2017-05-18T00:56:47Z

    do not add to free list during allocation

commit 4223e14896f4609d5bef80e97ee6d9982d2127a5
Author: Brandon Bradley <bradleytas...@gmail.com>
Date:   2017-05-18T01:20:46Z

    add license

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Reply via email to