Re: Why two netty libs?

Hi, Thanks much for the answers. Learning Spark every day! Pozdrawiam, Jacek Laskowski https://about.me/JacekLaskowski The Internals of Spark SQL https://bit.ly/spark-sql-internals The Internals of Spark Structured Streaming https://bit.ly/spark-structured-streaming The Internals of Apache K

Re: Why two netty libs?

Yes that's right. I don't think Spark's usage of ZK needs any ZK server, so it's safe to exclude in Spark (at least, so far so good!) On Wed, Sep 4, 2019 at 8:06 AM Steve Loughran wrote: > > Zookeeper client is/was netty 3, AFAIK, so if you want to use it for > anything, it ends up on the CP > >

Re: Why two netty libs?

Zookeeper client is/was netty 3, AFAIK, so if you want to use it for anything, it ends up on the CP On Tue, Sep 3, 2019 at 5:18 PM Shixiong(Ryan) Zhu wrote: > Yep, historical reasons. And Netty 4 is under another namespace, so we can > use Netty 3 and Netty 4 in the same JVM. > > On Tue, Sep 3,

Re: Why two netty libs?

Yep, historical reasons. And Netty 4 is under another namespace, so we can use Netty 3 and Netty 4 in the same JVM. On Tue, Sep 3, 2019 at 6:15 AM Sean Owen wrote: > It was for historical reasons; some other transitive dependencies needed > it. > I actually was just able to exclude Netty 3 last

Re: Why two netty libs?

It was for historical reasons; some other transitive dependencies needed it. I actually was just able to exclude Netty 3 last week from master. Spark uses Netty 4. On Tue, Sep 3, 2019 at 6:59 AM Jacek Laskowski wrote: > > Hi, > > Just noticed that Spark 2.4.x uses two netty deps of different vers

Why two netty libs?

Hi, Just noticed that Spark 2.4.x uses two netty deps of different versions. Why? jars/netty-all-4.1.17.Final.jar jars/netty-3.9.9.Final.jar Shouldn't one be excluded or perhaps shaded? Pozdrawiam, Jacek Laskowski https://about.me/JacekLaskowski The Internals of Spark SQL https://bit.ly/sp