I'd second Tom here. Create a ram disk or just let Kafka write to disk. Use
compression and batch messages and the OS fscache would take care of the rest.
Kafka is pretty fast and you probably won't notice.
_____________________________
From: Tom Crayford <tcrayf...@heroku.com>
Sent: Friday, July 15, 2016 12:12 AM
Subject: Re: Using Kafka without persisting message to disk
To: Users <users@kafka.apache.org>
Hi Jack,
No, kafka doesn't support not writing to disk. If you're really 100% sure
of yourself you could use a ramdisk and mount Kafka on it, but that's not
supported. I'd recommend "just" writing to disk, it's plenty fast enough
for nearly all use cases.
Thanks
Tom Crayford
Heroku Kafka
On Thu, Jul 14, 2016 at 7:33 PM, Jack Huang <jackhu...@mz.com> wrote:
> Hi all,
>
> Is there a way to make a topic to be stored in memory only and not writing
> to disk? If not, what's the best way to minimize writing to disk? For this
> application we only need the notion of partitions and a short retention
> time (1hr or so) from Kafka. We want to use Kafka because we want to keep
> the flexibility to add persistence back if we need to.
>
> Thanks,
> Jack
>
