On Wed, Mar 18, 2015 at 4:32 PM, Andrew Purtell <[email protected]> wrote:
> A better analogy would be Pig deciding to ship DataFu UDFs in a > convenience artifact. > A swaying argument. It still seems to me a bit like putting the cart in front of the horse, given the dependency DAG. Does the eventual inclusion of a Phoenix Query Server (PHOENIX-971) impact your opinion on this? On Wed, Mar 18, 2015 at 2:45 PM, Nick Dimiduk <[email protected]> wrote: > > > > > HBase shipping a tgz containing Phoenix sounds backwards to me. It seems > > like Hadoop project shipping a tgz with Hive included as a convenience. > > Such convenience packages would be great, but I don't think it's HBase > > project's responsibility to do so any more than it would be Hadoop > > project's responsibility in my above example. Upstreamers packaging > > downstreamers sounds backwards to me. > > > > On the other hand, I think it makes a lot of sense for *someone* to ship > a > > "batteries included"Phoenix tgz with Hadoop and HBase. If anyone, that > > someone should be Phoenix. I don't think it that unreasonable for Phoenix > > to do this. > > > > Or have a 3rd party packaging, as I mentioned before. I don't know what > > BigTop packages, but such a bundle seems reasonable for that project to > > produce, as one of many "batteries included" Apache products. > > > > On Wednesday, March 18, 2015, lars hofhansl <[email protected]> wrote: > > > > > Again... 'bit late chiming in. > > > > > > I'd be +1 on this. We'd essentially just ship HBase with an extra > shell.I > > > don't think Phoenix can do this, because it does not usually ship the > > bits > > > needed to run HBase, but just the bits to add to HBase to have > > > Phoenix.Phoenix should not be in the business to maintain an HBase > > > distribution. > > > One could argue that we should not be in the business to maintain a > > > Phoenix distribution. But we wouldn't really. We'd just add the Phoenix > > jar > > > to our binary distro, pre-hook the coprocessors, and add the SQLLine, > and > > > psql.On the HBase side I'd suggest we'd add a script to dev-support to > > > package up the HBase+Phoenix distro, and maybe we add a new target to > the > > > pom to pull a Phoenix version. > > > > > > What exactly we'd ship should be discussed (all the Phoenix tools, or > > just > > > some?) Giving HBase a nice SQL shell seems cool to me, and that's were > > some > > > more of the more hairy issues might be found. > > > > > > -- Lars > > > > > > From: Andrew Purtell <[email protected] <javascript:;>> > > > To: "[email protected] <javascript:;>" <[email protected] > > > <javascript:;>> > > > Sent: Tuesday, March 17, 2015 12:44 PM > > > Subject: [DISCUSSION] "Convenience binaries" bundling Phoenix > > > > > > Consider if the HBase project starts releasing new "convenience > > binaries", > > > in addition to the existing ones, in which we bundle a > > recent/vetted/stable > > > version of Phoenix, with the site file changes for loading their > > > coprocessors already patched in (to hbase-default.xml) For now this > would > > > be done for 0.98 only, since that's the only release line supported by > an > > > actively developed Phoenix version. We could also do this for 0.94 > > releases > > > with Phoenix 3 if the 0.94 RM wants, but I doubt there would be any > > demand > > > for this, Phoenix 3 is inactive because that community has all moved to > > 4, > > > I'd imagine that carries over here. > > > > > > Advantages: > > > > > > - HBase would ship with a SQL access option. There's the Phoenix JDBC > > > driver of course, and we'd also bundle the psql and sqlline exec > wrappers > > > from the Phoenix binary distribution. We'd have both the jruby shell > and > > a > > > SQL shell, this is a powerful combination. > > > > > > - HBase ships with a library that assists users in making efficient > > queries > > > if their data is typed, but this doesn't include the server side > > > optimizations that the Phoenix coprocessors provide, and in that case > no > > > hand rolling is necessary. > > > > > > - HBase would ship with secondary indexes. These would not cover all > > > possible use cases and requirements, let's stipulate that now and hope > > this > > > doesn't kick off another circular discussion on that front. > > Unquestionably > > > this is a compelling Phoenix feature so some use cases obviously can > > > benefit, and if users find the combined distribution useful enough we > > don't > > > have to discuss secondary indexes in HBase core again. > > > > > > - We will have done the necessary integration work for the combined > > result > > > to be easy to use. Apache software cat herders will appreciate this. > > > > > > - It's totally optional, simply ignore the new binary packages if you > > don't > > > care. This is not a Grand Unification proposal. > > > > > > Concerns: > > > > > > - More work for the RM. Unquestionably. > > > > > > - Concerns about the quality of the combined convenience artifact: Is > > there > > > an implied warranty? Could we disclaim? Should we disclaim? If not, how > > > does HBase do QA on this. Related to the above concern about RM > > bandwidth. > > > Maybe Phoenix could help. > > > > > > - Increased coupling between the projects. Frankly, I think this > already > > > there, we just don't see it until we trip over issues that could have > > been > > > avoided with more communication between projects. Pushing on Phoenix > for > > > bits for a monthly HBase release cadence will surface issues faster and > > > improve communication between the projects. This benefits Phoenix with > > more > > > QA bandwidth. This benefits HBase because we see Phoenix bringing in a > > > significant number of users. > > > > > > - We may want to revisit again normalizing type support in HBase's > client > > > library and Phoenix, eventually. > > > > > > I could add more items to the advantage or concern lists but mainly > want > > to > > > float the idea for feedback at this time. > > > > > > Thoughts? > > > > > > -- > > > Best regards, > > > > > > - Andy > > > > > > Problems worthy of attack prove their worth by hitting back. - Piet > Hein > > > (via Tom White) > > > > > > > > > > > > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) >
