Re: JNI linker issue on ARM (Raspberry PI)

[John Roesler]

Ah, yes, before messing with rocksjni. As Sophie said, you'll probably just 
want to try using 2.6.0. It should work "out of the box".

Thanks,
-John

On Mon, Aug 24, 2020, at 21:08, Steve Jones wrote:
> Version: *kafka_2.12-2.5.0*
> 
> RocksDB compiles fine, Kafka compiles fine, I'm just trying to work out
> where the linking is done and if that can be changed to a static rather
> than a dynamic link.  Any hints appreciated, and I'll document out what
> I've found.   Its very stable on the Raspberry Pi as a broker, and for
> streams if the streams is run on another machine.
> 
> Steve
> 
> On Mon, 24 Aug 2020 at 17:49, Sophie Blee-Goldman 
> wrote:
> 
> > Yeah, if you weren't already running it, try upgrading Streams to 2.6. They
> > recently
> > added support for "all" platforms to rocksdb and we updated the dependency
> > to
> > get this fix in 2.6. See KAFKA-9225
> > 
> >
> > If you already were running 2.6, then, that's unfortunate. You might have
> > some luck
> > asking the rocksdb folks if all else fails
> >
> > On Mon, Aug 24, 2020 at 5:46 PM John Roesler  wrote:
> >
> > > Hi Steve,
> > >
> > > Which version of Streams is this? I vaguely recall that we updated to a
> > > version of Rocks that’s compiled for ARM, and I think some people have
> > used
> > > it on ARM, but I might be misremembering.
> > >
> > > I’m afraid I can’t be much help in debugging this, but maybe some others
> > > on the list have more context. If all else fails, you can probably narrow
> > > it down to the Java RocksDB library. If you create a standalone java
> > > program using the same rocks dependency that we do, then you can pretty
> > > confidently raise it with the Rocks folks.
> > >
> > > If you want to side-step the issue while debugging this, plugging in a
> > > different store implementation (like the in-memory one) would probably
> > > work, although I’m sure memory is scarce on a Raspberry Pi.
> > >
> > > I hope this helps,
> > > -John
> > >
> > > On Mon, Aug 24, 2020, at 19:26, Steve Jones wrote:
> > > > I'm trying to install Kafka Streams on a Raspberry PI, it works fine
> > as a
> > > > broker, works fine as both a producer and consumer, but when I try and
> > > run
> > > > streams on the PI rather than on the Mac there is a linker issue:
> > > >
> > > > Exception in thread
> > > > "main-broker-f53264a1-0c70-445f-bf3f-bf634a9a1ed2-StreamThread-1"
> > > > java.lang.UnsatisfiedLinkError:
> > > /tmp/librocksdbjni15158764823832728522.so:
> > > > /tmp/librocksdbjni15158764823832728522.so: cannot open shared object
> > > file:
> > > > No such file or directory (Possible cause: can't load IA 32 .so on a
> > ARM
> > > > platform)
> > > >
> > > > at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
> > > >
> > > > at
> > > java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2452)
> > > >
> > > > at
> > > >
> > >
> > java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2508)
> > > >
> > > > at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2704)
> > > >
> > > > at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2637)
> > > >
> > > >
> > > > I've recompiled rocksdb on the RaspberryPI and added that to the
> > loadpath
> > > > but the same error occurs.  I've done the Google search but not found
> > > > anything around streams on ARM (Raspberry PI) and what needs to be
> > > > recompiled/linked for this to work.
> > > >
> > > >
> > > > Help appreciated.
> > > >
> > > >
> > > > Steve Jones
> > > >
> > >
> >
>

Re: JNI linker issue on ARM (Raspberry PI)

[Steve Jones]

Version: *kafka_2.12-2.5.0*

RocksDB compiles fine, Kafka compiles fine, I'm just trying to work out
where the linking is done and if that can be changed to a static rather
than a dynamic link.  Any hints appreciated, and I'll document out what
I've found.   Its very stable on the Raspberry Pi as a broker, and for
streams if the streams is run on another machine.

Steve

On Mon, 24 Aug 2020 at 17:49, Sophie Blee-Goldman 
wrote:

> Yeah, if you weren't already running it, try upgrading Streams to 2.6. They
> recently
> added support for "all" platforms to rocksdb and we updated the dependency
> to
> get this fix in 2.6. See KAFKA-9225
> 
>
> If you already were running 2.6, then, that's unfortunate. You might have
> some luck
> asking the rocksdb folks if all else fails
>
> On Mon, Aug 24, 2020 at 5:46 PM John Roesler  wrote:
>
> > Hi Steve,
> >
> > Which version of Streams is this? I vaguely recall that we updated to a
> > version of Rocks that’s compiled for ARM, and I think some people have
> used
> > it on ARM, but I might be misremembering.
> >
> > I’m afraid I can’t be much help in debugging this, but maybe some others
> > on the list have more context. If all else fails, you can probably narrow
> > it down to the Java RocksDB library. If you create a standalone java
> > program using the same rocks dependency that we do, then you can pretty
> > confidently raise it with the Rocks folks.
> >
> > If you want to side-step the issue while debugging this, plugging in a
> > different store implementation (like the in-memory one) would probably
> > work, although I’m sure memory is scarce on a Raspberry Pi.
> >
> > I hope this helps,
> > -John
> >
> > On Mon, Aug 24, 2020, at 19:26, Steve Jones wrote:
> > > I'm trying to install Kafka Streams on a Raspberry PI, it works fine
> as a
> > > broker, works fine as both a producer and consumer, but when I try and
> > run
> > > streams on the PI rather than on the Mac there is a linker issue:
> > >
> > > Exception in thread
> > > "main-broker-f53264a1-0c70-445f-bf3f-bf634a9a1ed2-StreamThread-1"
> > > java.lang.UnsatisfiedLinkError:
> > /tmp/librocksdbjni15158764823832728522.so:
> > > /tmp/librocksdbjni15158764823832728522.so: cannot open shared object
> > file:
> > > No such file or directory (Possible cause: can't load IA 32 .so on a
> ARM
> > > platform)
> > >
> > > at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
> > >
> > > at
> > java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2452)
> > >
> > > at
> > >
> >
> java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2508)
> > >
> > > at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2704)
> > >
> > > at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2637)
> > >
> > >
> > > I've recompiled rocksdb on the RaspberryPI and added that to the
> loadpath
> > > but the same error occurs.  I've done the Google search but not found
> > > anything around streams on ARM (Raspberry PI) and what needs to be
> > > recompiled/linked for this to work.
> > >
> > >
> > > Help appreciated.
> > >
> > >
> > > Steve Jones
> > >
> >
>

Re: JNI linker issue on ARM (Raspberry PI)

[Sophie Blee-Goldman]

Yeah, if you weren't already running it, try upgrading Streams to 2.6. They
recently
added support for "all" platforms to rocksdb and we updated the dependency
to
get this fix in 2.6. See KAFKA-9225


If you already were running 2.6, then, that's unfortunate. You might have
some luck
asking the rocksdb folks if all else fails

On Mon, Aug 24, 2020 at 5:46 PM John Roesler  wrote:

> Hi Steve,
>
> Which version of Streams is this? I vaguely recall that we updated to a
> version of Rocks that’s compiled for ARM, and I think some people have used
> it on ARM, but I might be misremembering.
>
> I’m afraid I can’t be much help in debugging this, but maybe some others
> on the list have more context. If all else fails, you can probably narrow
> it down to the Java RocksDB library. If you create a standalone java
> program using the same rocks dependency that we do, then you can pretty
> confidently raise it with the Rocks folks.
>
> If you want to side-step the issue while debugging this, plugging in a
> different store implementation (like the in-memory one) would probably
> work, although I’m sure memory is scarce on a Raspberry Pi.
>
> I hope this helps,
> -John
>
> On Mon, Aug 24, 2020, at 19:26, Steve Jones wrote:
> > I'm trying to install Kafka Streams on a Raspberry PI, it works fine as a
> > broker, works fine as both a producer and consumer, but when I try and
> run
> > streams on the PI rather than on the Mac there is a linker issue:
> >
> > Exception in thread
> > "main-broker-f53264a1-0c70-445f-bf3f-bf634a9a1ed2-StreamThread-1"
> > java.lang.UnsatisfiedLinkError:
> /tmp/librocksdbjni15158764823832728522.so:
> > /tmp/librocksdbjni15158764823832728522.so: cannot open shared object
> file:
> > No such file or directory (Possible cause: can't load IA 32 .so on a ARM
> > platform)
> >
> > at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
> >
> > at
> java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2452)
> >
> > at
> >
> java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2508)
> >
> > at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2704)
> >
> > at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2637)
> >
> >
> > I've recompiled rocksdb on the RaspberryPI and added that to the loadpath
> > but the same error occurs.  I've done the Google search but not found
> > anything around streams on ARM (Raspberry PI) and what needs to be
> > recompiled/linked for this to work.
> >
> >
> > Help appreciated.
> >
> >
> > Steve Jones
> >
>

Re: JNI linker issue on ARM (Raspberry PI)

[John Roesler]

Hi Steve,

Which version of Streams is this? I vaguely recall that we updated to a version 
of Rocks that’s compiled for ARM, and I think some people have used it on ARM, 
but I might be misremembering. 

I’m afraid I can’t be much help in debugging this, but maybe some others on the 
list have more context. If all else fails, you can probably narrow it down to 
the Java RocksDB library. If you create a standalone java program using the 
same rocks dependency that we do, then you can pretty confidently raise it with 
the Rocks folks.

If you want to side-step the issue while debugging this, plugging in a 
different store implementation (like the in-memory one) would probably work, 
although I’m sure memory is scarce on a Raspberry Pi. 

I hope this helps,
-John

On Mon, Aug 24, 2020, at 19:26, Steve Jones wrote:
> I'm trying to install Kafka Streams on a Raspberry PI, it works fine as a
> broker, works fine as both a producer and consumer, but when I try and run
> streams on the PI rather than on the Mac there is a linker issue:
> 
> Exception in thread
> "main-broker-f53264a1-0c70-445f-bf3f-bf634a9a1ed2-StreamThread-1"
> java.lang.UnsatisfiedLinkError: /tmp/librocksdbjni15158764823832728522.so:
> /tmp/librocksdbjni15158764823832728522.so: cannot open shared object file:
> No such file or directory (Possible cause: can't load IA 32 .so on a ARM
> platform)
> 
> at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
> 
> at java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2452)
> 
> at
> java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2508)
> 
> at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2704)
> 
> at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2637)
> 
> 
> I've recompiled rocksdb on the RaspberryPI and added that to the loadpath
> but the same error occurs.  I've done the Google search but not found
> anything around streams on ARM (Raspberry PI) and what needs to be
> recompiled/linked for this to work.
> 
> 
> Help appreciated.
> 
> 
> Steve Jones
>

JNI linker issue on ARM (Raspberry PI)

[Steve Jones]

I'm trying to install Kafka Streams on a Raspberry PI, it works fine as a
broker, works fine as both a producer and consumer, but when I try and run
streams on the PI rather than on the Mac there is a linker issue:

Exception in thread
"main-broker-f53264a1-0c70-445f-bf3f-bf634a9a1ed2-StreamThread-1"
java.lang.UnsatisfiedLinkError: /tmp/librocksdbjni15158764823832728522.so:
/tmp/librocksdbjni15158764823832728522.so: cannot open shared object file:
No such file or directory (Possible cause: can't load IA 32 .so on a ARM
platform)

at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)

at java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2452)

at
java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2508)

at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2704)

at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2637)


I've recompiled rocksdb on the RaspberryPI and added that to the loadpath
but the same error occurs.  I've done the Google search but not found
anything around streams on ARM (Raspberry PI) and what needs to be
recompiled/linked for this to work.


Help appreciated.


Steve Jones

Kafka and Zookeeper stable release

[Nancy Feng (defeng)]

Hi, I’m upgrading cloud infrastructure kafka(2.3.0) and zookeeper(3.4.9) to 
newer version. May I ask recommendation of the best latest stable versions for 
the both I could use?

Kafka 2.6.0 or 2.5.1?
How about compatible zookeeper version?

Thanks,
Nancy

Not able to connect to bootstrap server when one broker down

[Prateek Rajput]

Hi everyone,
I am new to Kafka, and recently started working on kafka in my company. We
recently migrated our client and cluster from the *0.10.x* version to
*2.3.0*. I am facing this issue quite often.
I have provided all brokers in *bootstrap.servers* config to instantiate
the producer client but while using this client for batch publishing,
sometimes some of my mappers get stuck.
I debugged and found that one broker was down (for some maintenance
activity). Now it was getting stuck because the mapper's client was trying
to connect to that node only for the very first time. And it was failing
with NoRouteToHost Exception.
I have read that the very first time the client will select a random broker
and will try to connect with that broker to get the meta-data of the whole
cluster. Is there any way so that on such exceptions it can switch to
another node dynamically and should not try to connect to the same box
again and again.

*Regards,*
*Prateek Rajput* 

-- 



*-*


*This email and any files transmitted with it are confidential and 
intended solely for the use of the individual or entity to whom they are 
addressed. If you have received this email in error, please notify the 
system manager. This message contains confidential information and is 
intended only for the individual named. If you are not the named addressee, 
you should not disseminate, distribute or copy this email. Please notify 
the sender immediately by email if you have received this email by mistake 
and delete this email from your system. If you are not the intended 
recipient, you are notified that disclosing, copying, distributing or 
taking any action in reliance on the contents of this information is 
strictly prohibited.*

 

*Any views or opinions presented in this 
email are solely those of the author and do not necessarily represent those 
of the organization. Any information on shares, debentures or similar 
instruments, recommended product pricing, valuations and the like are for 
information purposes only. It is not meant to be an instruction or 
recommendation, as the case may be, to buy or to sell securities, products, 
services nor an offer to buy or sell securities, products or services 
unless specifically stated to be so on behalf of the Flipkart group. 
Employees of the Flipkart group of companies are expressly required not to 
make defamatory statements and not to infringe or authorise any 
infringement of copyright or any other legal right by email communications. 
Any such communication is contrary to organizational policy and outside the 
scope of the employment of the individual concerned. The organization will 
not accept any liability in respect of such communication, and the employee 
responsible will be personally liable for any damages or other liability 
arising.*

 

*Our organization accepts no liability for the 
content of this email, or for the consequences of any actions taken on the 
basis of the information *provided,* unless that information is 
subsequently confirmed in writing. If you are not the intended recipient, 
you are notified that disclosing, copying, distributing or taking any 
action in reliance on the contents of this information is strictly 
prohibited.*


_-_

Request to be Assignee on JIRA

[xaka...@mail.ru.INVALID]

Hi, guys! I'd like to provide a PR for KAFKA-10426. But I'm not in the Assignee 
list. Could you, please, add me?
But I'm not sure that I'm writing to the right mailing list.
My JIRA full name is Goltseva Taisiia.


---
С уважением, Таисия Дорошенкова

Re: Request to be Assignee on JIRA

[Bill Bejeck]

Done.  Thanks for your interest in Apache Kafka.

-Bill

On Mon, Aug 24, 2020 at 8:34 AM xaka...@mail.ru.INVALID
 wrote:

> Hi, guys! I'd like to provide a PR for KAFKA-10426. But I'm not in the
> Assignee list. Could you, please, add me?
> But I'm not sure that I'm writing to the right mailing list.
> My JIRA full name is Goltseva Taisiia.
>
>
> ---
> С уважением, Таисия Дорошенкова

Request to be Assignee on JIRA

[xaka...@mail.ru.INVALID]

Hi, guys! I'd like to provide a PR for KAFKA-10426. But I'm not in the Assignee 
list. Could you, please, add me?
But I'm not sure that I'm writing to the right mailing list.
My JIRA full name is Goltseva Taisiia.


---
С уважением, Таисия Дорошенкова