Re: [DISCUSSION] Upgrading core dependencies

2017-02-11 Thread Andrew Purtell
Minor point, but I maintain we don't want to make coprocessors like osgi or built on osgi. I think we still want to scope them as extension mixins, not an inner platform. We see the limitations (limited API compatibility guarantees for internals by definition) over on Phoenix but it's the right

Re: [DISCUSSION] Upgrading core dependencies

2017-02-08 Thread Jerry He
I was more on the thinking to avoid/reduce pulling in hbase dependencies into hbase-spark, and that maybe even hbase-spark can depend on shaded client and server -- it will be easier and more feasible if the shaded client becomes the default as you mentioned. Your idea that hbase-spark itself beco

Re: [DISCUSSION] Upgrading core dependencies

2017-02-08 Thread Nick Dimiduk
On Wed, Feb 8, 2017 at 10:24 AM Jerry He wrote: > Yeah. Talking about the dependency, the hbase-spark module already has > dependency on hbase-server (coming from the spark bulk load producing > hfiles). > This is not very good. We have to be careful not entangling it more. > Also, there is alre

Re: [DISCUSSION] Upgrading core dependencies

2017-02-08 Thread Josh Elser
(late to the party, but..) +1 Nick sums this up better than I could have. Nick Dimiduk wrote: For the client: I'm a fan of shaded client modules by default and minimizing the exposure of that surface area of 3rd party libs (none, if possible). For example, Elastic Search has a similar set of ch

Re: [DISCUSSION] Upgrading core dependencies

2017-02-08 Thread Jerry He
Yeah. Talking about the dependency, the hbase-spark module already has dependency on hbase-server (coming from the spark bulk load producing hfiles). This is not very good. We have to be careful not entangling it more. Also, there is already problem running the hbase-spark due to dependency confli

Re: [DISCUSSION] Upgrading core dependencies

2017-02-07 Thread Stack
On Tue, Feb 7, 2017 at 7:48 PM, Ted Yu wrote: > bq. Better to start using the new API's available in Java 8 > > +1 to the above. > If no new Guava construct is introduced and we replace current Guava usage > with Java 8 counterpart(s), in the future we can get rid of Guava > dependency. > > I don

Re: [DISCUSSION] Upgrading core dependencies

2017-02-07 Thread Stack
Thanks Nick and Duo. See below. On Tue, Feb 7, 2017 at 6:50 PM, Nick Dimiduk wrote: > For the client: I'm a fan of shaded client modules by default and > minimizing the exposure of that surface area of 3rd party libs (none, if > possible). For example, Elastic Search has a similar set of challe

Re: [DISCUSSION] Upgrading core dependencies

2017-02-07 Thread Duo Zhang
For coprocessors, our compatibility matrix says that we can break everything, so I think dependency is not the first thing they need to consider when upgrading between major versions? 2017-02-08 11:48 GMT+08:00 Ted Yu : > bq. Better to start using the new API's available in Java 8 > > +1 to the a

Re: [DISCUSSION] Upgrading core dependencies

2017-02-07 Thread Ted Yu
bq. Better to start using the new API's available in Java 8 +1 to the above. If no new Guava construct is introduced and we replace current Guava usage with Java 8 counterpart(s), in the future we can get rid of Guava dependency. On Tue, Feb 7, 2017 at 6:50 PM, Nick Dimiduk wrote: > For the cli

Re: [DISCUSSION] Upgrading core dependencies

2017-02-07 Thread Nick Dimiduk
For the client: I'm a fan of shaded client modules by default and minimizing the exposure of that surface area of 3rd party libs (none, if possible). For example, Elastic Search has a similar set of challenges, the solve it by advocating users shade from step 1. It's addressed first thing in the do

Re: [DISCUSSION] Upgrading core dependencies

2017-02-07 Thread Duo Zhang
I think we can upgrade these bad guys every time when major version changes and keep the version unchanged during the whole major version, For the shading things, upgrading between major versions is always painful, so I think shaded client and shaded server is enough for users? Maybe the current s

[DISCUSSION] Upgrading core dependencies

2017-02-07 Thread Stack
Here's an old thorny issue that won't go away. I'd like to hear what folks are thinking these times. My immediate need is that I want to upgrade Guava [1]. I want to move us to guava 21.0, the latest release [2]. We currently depend on guava 12.0. Hadoop's guava -- 11.0 -- is also on our CLASSPATH