Re: Kafka over Satellite links

[Mathias Herberts]

Sat links induce roughly 250ms of latency, you have to tweak TCP Windows so
you can saturate the link as much as you can, but besides that Sat links
are rather common.
On Mar 4, 2016 7:26 AM, "Jan"  wrote:

> Thanks for the input.
> The IoT application could have a http ingestion receiver to funnel data
> into Kafka. However, I am thinking about Kafka's ability to be able to
> injest traffic directly from IoT devices.
> Does anyone have any recommendations while using Kafka Connect between
> producer &  consumer sides over Satellite links.   Does the Satellite
> link really make any difference v/s a slow Internet link ?
> Input/ suggestions would be much appreciated.
> ThanksJan
>
>
> On Thursday, March 3, 2016 1:16 AM, Christian Csar 
> wrote:
>
>
>  I would not do that. I admit I may be a bit biased due to working for
> Buddy Platform (IoT backend stuff including telemetry collection), but
> you want to send the data via some protocol (HTTP? MQTT? COAP?) to the
> central hub and then have those servers put the data into Kafka. Now
> if you want to use Kafka there are the various HTTP front ends that
> will basically put the data into Kafka for you without the client
> needing to deal with the partition management part. But putting data
> into Kafka directly really seems like a bad idea even if it's a large
> number of messages per second per node, even if the security parts
> work out for you.
>
> Christian
>
> On Wed, Mar 2, 2016 at 9:52 PM, Jan  wrote:
> > Hi folks;
> > does anyone know of Kafka's ability to work over Satellite links. We
> have a IoT Telemetry application that uses Satellite communication to send
> data from remote sites to a Central hub.
> > Any help/ input/ links/ gotchas would be much appreciated.
> > Regards,Jan
>
>
>

Re: Security in 0.8.2 beta

[Mathias Herberts]

Simply encrypt your messages with a PSK between producers and consumers.
On Nov 12, 2014 4:38 AM, Kashyap Mhaisekar kashya...@gmail.com wrote:

 Hi,
 Is there a way to secure the topics created in Kafka 0.8.2 beta? The need
 is to ensure no one is asked to read data from the topic without
 authorization.

 Regards
 Kashyap

Re: powered by kafka

[Mathias Herberts]

Hi there,

my company Cityzen Data uses Kafka as well, we provide a paltform for
collecting, storing and analyzing machine data.

http://www.cityzendata.com/
@CityzenData

Mathias.

On Mon, Nov 10, 2014 at 11:57 AM, Pierre-Yves Ritschard
p...@spootnik.org wrote:
 I guess I should mention that exoscale (https://exoscale.ch) is powered by
 kafka as well.

 Cheers,
- pyr

 On Sun, Nov 9, 2014 at 7:36 PM, Gwen Shapira gshap...@cloudera.com wrote:

 I'm not Jay, but fixed it anyways ;)

 Gwen

 On Sun, Nov 9, 2014 at 10:34 AM, vipul jhawar vipul.jha...@gmail.com
 wrote:

  Hi Jay
 
  Thanks for posting the update.
 
  However, i checked the page history and the hyperlink is pointing to the
  wrong domain.
  Exponential refers to www.exponential.com. I sent the twitter handle,
  should have sent the domain.
  Please correct.
 
  Thanks
 
  On Sat, Nov 8, 2014 at 3:45 PM, vipul jhawar vipul.jha...@gmail.com
  wrote:
 
   Exponential @exponentialinc is using kafka in production to power the
   events ingestion pipeline for real time analytics and log feed
  consumption.
  
   Please post on powered by kafka wiki -
   https://cwiki.apache.org/confluence/display/KAFKA/Powered+By
  
   Thanks
   Vipul
   http://in.linkedin.com/in/vjhawar/
  
 

Producer thread safety

[Mathias Herberts]

Hi,

I'm testing Kafka 0.8 and I encounter an error when accessing a Producer
from mutliple threads.

I've always thought Producer was thread safe but it seems this is no longer
the case. This is the stack trace of the error I'm getting:

java.util.ConcurrentModificationException
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:819)
at java.util.ArrayList$Itr.next(ArrayList.java:791)
at
scala.collection.JavaConversions$JIteratorWrapper.next(JavaConversions.scala:575)
at scala.collection.Iterator$class.foreach(Iterator.scala:772)
at
scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:573)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:73)
at
scala.collection.JavaConversions$JListWrapper.foreach(JavaConversions.scala:615)
at kafka.producer.Producer.recordStats(Producer.scala:80)
at kafka.producer.Producer.send(Producer.scala:72)
at kafka.javaapi.producer.Producer.send(Producer.scala:41)

Mathias.

Re: Producer thread safety

[Mathias Herberts]

Please ignore this report, the problem was triggered by another thread
clearing the list currently being sent.

Sorry for the noise.


On Thu, Sep 19, 2013 at 11:34 AM, Mathias Herberts 
mathias.herbe...@gmail.com wrote:

 Hi,

 I'm testing Kafka 0.8 and I encounter an error when accessing a Producer
 from mutliple threads.

 I've always thought Producer was thread safe but it seems this is no
 longer the case. This is the stack trace of the error I'm getting:

 java.util.ConcurrentModificationException
 at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:819)
 at java.util.ArrayList$Itr.next(ArrayList.java:791)
 at
 scala.collection.JavaConversions$JIteratorWrapper.next(JavaConversions.scala:575)
 at scala.collection.Iterator$class.foreach(Iterator.scala:772)
 at
 scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:573)
 at scala.collection.IterableLike$class.foreach(IterableLike.scala:73)
 at
 scala.collection.JavaConversions$JListWrapper.foreach(JavaConversions.scala:615)
 at kafka.producer.Producer.recordStats(Producer.scala:80)
 at kafka.producer.Producer.send(Producer.scala:72)
 at kafka.javaapi.producer.Producer.send(Producer.scala:41)

 Mathias.