A cheaper approach is to use ephemeral disk and have the replicas across
AZs. You can lose up to 2 AZs and still avoid data loss. You mitigate
complete region failure using mirror maker to replicate to another region.
On Tue, Dec 3, 2013 at 8:41 AM, Philip O'Toole wrote:
> Yeah, we use provis
Yeah, we use provisioned IOPs EBS so we have good performance.
We run 3 ZK nodes, one in each AZ. Quorum 2.
Philip
On Dec 3, 2013, at 7:31 AM, Timothy Chen wrote:
> Hi Philip,
>
> So I wonder if you guys hit disk perf problems with EBS? It seems quite
> common in the past but I haven't tri
Hi Philip,
So I wonder if you guys hit disk perf problems with EBS? It seems quite common
in the past but I haven't tried recently.
Also can you share how you guys deployed zookeeper in AWS so that a qurom is
always available?
Tim
Sent from my iPhone
> On Dec 2, 2013, at 5:15 PM, Steve Morin
Philip this is definitely useful
> On Dec 2, 2013, at 14:55, Surendranauth Hiraman
> wrote:
>
> S Ahmed,
>
> This combination of Kafka and Storm to process streaming data is becoming
> pretty common. Definitely worth looking at.
>
> The throughput will vary depending on your workload (cpu usa
S Ahmed,
This combination of Kafka and Storm to process streaming data is becoming
pretty common. Definitely worth looking at.
The throughput will vary depending on your workload (cpu usage, etc.) and
if you're talking to a backend, of course. But it scales very well.
-Suren
On Mon, Dec 2, 20
Interesting. So twitter storm is used to basically process the messages on
kafka? I'll have to read-up on storm b/c I always thought the use case
was a bit different.
On Sun, Dec 1, 2013 at 9:59 PM, Joe Stein wrote:
> Awesome Philip, thanks for sharing!
>
> On Sun, Dec 1, 2013 at 9:17 PM, P
Awesome Philip, thanks for sharing!
On Sun, Dec 1, 2013 at 9:17 PM, Philip O'Toole wrote:
> A couple of us here at Loggly recently spoke at AWS reinvent, on how we
> use Kafka 0.72 in our ingestion pipeline. The slides are at the link below,
> and may be of interest to people on this list.
>
>
>
A couple of us here at Loggly recently spoke at AWS reinvent, on how we use
Kafka 0.72 in our ingestion pipeline. The slides are at the link below, and may
be of interest to people on this list.
http://www.slideshare.net/AmazonWebServices/infrastructure-at-scale-apache-kafka-twitter-storm-elast
