Hi again,
Thought I'd share some numbers on performance comparison between Kafka
Connect and Apache Spark.
In my case, for ingesting data from Kafka to HDFS (HDP), Kafka Connect
performs ~25% faster than Apache Spark. Both consumers read from a single,
non-partitioned Kafka topic containing 5 mil
Whoever this is: would you please abstain from such anonymous unsubstantiated
claims? I cannot comment on the truthfulness of your critique, common courtesy
would entail that I know at least your name to ask for supporting evidence.
Regards, Roland
Sent from my iPhone
> On 3. May 2017, at 06:
Be careful with Flink. IME it's got a long way to go. The core model is
fantastic but there's a lot of low-hanging fruit that needs to be fixed.
On Tuesday, May 2, 2017 at 4:03:12 PM UTC-6, Evan Chan wrote:
>
> Hi Shiva,
>
> Spark will likely be too high latency for you. Practical minimal batc
Hi Shiva,
Spark will likely be too high latency for you. Practical minimal batch
size is a couple seconds.
Think of Akka as a best fit for if you want to deploy individual apps each
reading from some set of fixed Kafka partitions. Each one could then write
to HDFS. However you would need to
On Fri, Apr 28, 2017 at 4:52 PM, Shiva Ramagopal wrote:
> Viktor,
>
>
> On Fri, Apr 28, 2017 at 5:03 PM, Viktor Klang
> wrote:
>
>>
>>
>> On Fri, Apr 28, 2017 at 1:12 PM, Shiva Ramagopal
>> wrote:
>>
>>> Hi Viktor,
>>>
>>> On Fri, Apr 28, 2017 at 2:55 PM, Viktor Klang
>>> wrote:
>>>
Hi Sh
Viktor,
On Fri, Apr 28, 2017 at 5:03 PM, Viktor Klang
wrote:
>
>
> On Fri, Apr 28, 2017 at 1:12 PM, Shiva Ramagopal
> wrote:
>
>> Hi Viktor,
>>
>> On Fri, Apr 28, 2017 at 2:55 PM, Viktor Klang
>> wrote:
>>
>>> Hi Shiva,
>>>
>>> On Fri, Apr 28, 2017 at 11:20 AM, Shiva Ramagopal
>>> wrote:
>>>
On Fri, Apr 28, 2017 at 1:12 PM, Shiva Ramagopal wrote:
> Hi Viktor,
>
> On Fri, Apr 28, 2017 at 2:55 PM, Viktor Klang
> wrote:
>
>> Hi Shiva,
>>
>> On Fri, Apr 28, 2017 at 11:20 AM, Shiva Ramagopal
>> wrote:
>>
>>> I'm looking to compare Kafka Streams vs Akka Streams in two areas:
>>>
>>> 1. F
Hi Viktor,
On Fri, Apr 28, 2017 at 2:55 PM, Viktor Klang
wrote:
> Hi Shiva,
>
> On Fri, Apr 28, 2017 at 11:20 AM, Shiva Ramagopal
> wrote:
>
>> I'm looking to compare Kafka Streams vs Akka Streams in two areas:
>>
>> 1. For ingesting between Kafka and HDFS/RDBMS
>>
>> Requirements are mainly ar
Hi Shiva,
On Fri, Apr 28, 2017 at 11:20 AM, Shiva Ramagopal wrote:
> I'm looking to compare Kafka Streams vs Akka Streams in two areas:
>
> 1. For ingesting between Kafka and HDFS/RDBMS
>
> Requirements are mainly around performance and latency. A Kafka topic can
> have several million events, e
I'm looking to compare Kafka Streams vs Akka Streams in two areas:
1. For ingesting between Kafka and HDFS/RDBMS
Requirements are mainly around performance and latency. A Kafka topic can
have several million events, each corresponding to a database change
capture. When ingesting this topic into H
On Thu, Apr 27, 2017 at 10:39 AM, Shiva Ramagopal wrote:
> Hi,
>
> I have read through multiple articles describing the SMACK stack but I'm
> having difficulty understanding the role of Akka in the stack. How does
> Akka fit in?
>
Akka is for building the application itself.
>
> Also I would l
Hi,
I have read through multiple articles describing the SMACK stack but I'm
having difficulty understanding the role of Akka in the stack. How does
Akka fit in?
Also I would like to know of experiences in using Akka Streams vs Kafka
Connect for ingesting from Kafka into HDFS (Hive) and RDBMS. Ha
12 matches
Mail list logo
