> Only shell we could > swap in w/o annoying users would be SQL (ducks); [...] You would > also have to manage expectations: i.e. that the SQL would be extremely > basic and that the REPL does not come with an idling 100 node cluster > ready to take up any involved queries.
If we are talking about adding a SQL shell, it doesn't have to be extremely basic, we can revisit my proposal from the other month about embedding Phoenix. On Wed, May 13, 2015 at 4:50 PM, Stack <st...@duboce.net> wrote: > On Wed, May 13, 2015 at 1:38 PM, Andrew Purtell <apurt...@apache.org> > wrote: > > > > That's only if you assume HBase users actually understand Ruby though, > > no? > > > > Yes, and this is a fair point. > > > > The language REPL in which we're embedding the shell DSL doesn't have to > be > > Ruby. JavaScript would work, maybe vi > > a Nashorn. Python would work, via > > Jython. Or Scala's REPL, even. > > > > > > > JRuby had best interpreter and the others considered had extra baggage > doing a DSL (For history, see HBASE-487). Having to put tabs in your shell > doing loops and no jline at the time made jython a non-runner though I > wanted it to win (Has anything changed here?) JS is probably way better > than when I looked at it back then? > > Regardless, whether jython, JS, scala, or custom, IMO, we'd just be pissing > of users if we swap one idiosyncratic for another. Only shell we could > swap in w/o annoying users would be SQL (ducks); and even here you'd > probably have to implement a bunch of operators on the other side of > sqllines bang operator (or equivalent) to do the hbase specifics. You would > also have to manage expectations: i.e. that the SQL would be extremely > basic and that the REPL does not come with an idling 100 node cluster > ready to take up any involved queries. > > St.Ack > > > > > > On Wed, May 13, 2015 at 1:10 PM, Josh Elser <josh.el...@gmail.com> > wrote: > > > > > Andrew Purtell wrote: > > > > > >> This is because the Accumulo shell is a custom built shell? If so, we > > had > > >> one of those once and replaced it with the IRB based one. We didn't > > settle > > >> on JRuby right away but, at the time, the consensus was we didn't want > > to > > >> be in the business of maintaining yet another REPL when specialist > open > > >> source communities have already done that. Why has this changed? Has > it? > > >> > > > > > > Accumulo shell is just a Java program that uses JLine. It has no > control > > > flow structures, variables, or anything like that. > > > > > > Why is the Accumulo shell easier to script with? Does it have control > > flow > > >> constructs? Variable assignment? Or is it easier somehow because it > does > > >> not have those things? Even the system shell has those. Hmm... Are we > > >> saying that any control flow or variables/substitution should be done > by > > >> calling our shell from a bash or sh script? Wouldn't that be super > slow > > >> given Java's startup time for every invocation of our shell? A bit > like > > >> trying to script the AWS command line utilities, which is > excruciating. > > >> > > > > > > This is what I was trying to get at: the lack of typical language > > > constructs is both beneficial and irritating. I think it depends on > what > > > you really want to do. Does that mean trying to maintain two shells is > > > worth it? I'm not sure, but I'm also not sure where the happy medium > is. > > > > > > If you exec the shell to just run one command at a time, yes. It is > slow. > > > You can redirect input or provide a file of commands to run at one time > > > which don't suffer from the JVM startup problem. > > > > > > I can see why a custom shell built with Java would easier to test > where > > >> the > > >> rest of the project is using Surefire/JUnit. That's a developer > > >> convenience > > >> concern though, not a user convenience one. > > >> > > > > > > That's only if you assume HBase users actually understand Ruby though, > > no? > > > I would think that something which acts like your normal unix shell > would > > > be familiar to users w/o the need to understand some other language. > > > > > > > > > > > >> On Wed, May 13, 2015 at 10:41 AM, Sean Busbey<bus...@cloudera.com> > > >> wrote: > > >> > > >> Pros: > > >>> > > >>> * It's easier to test and maintain > > >>> * it's easier to script with > > >>> * its interactive mode "feels" mode focused on the task of > interacting > > >>> with > > >>> a cluster to me (maybe this is just acting like a psql or mysql > shell) > > >>> > > >>> Cons > > >>> > > >>> * adding custom commands requires knowing java > > >>> > > >>> -- > > >>> Sean > > >>> On May 13, 2015 12:31 PM, "Andrew Purtell"<apurt...@apache.org> > > wrote: > > >>> > > >>> Why is the Accumulo shell superior? > > >>>> > > >>>> Is it scriptable? > > >>>> > > >>>> > > >>>> On Wed, May 13, 2015 at 10:28 AM, Sean Busbey<bus...@cloudera.com> > > >>>> > > >>> wrote: > > >>> > > >>>> I would love to rip out the JRuby shell entirely and make something > > >>>>> > > >>>> closer > > >>>> > > >>>>> to the Accumulo shell, but I expect that will > > >>>>> > > >>>>> 1) be way more work > > >>>>> > > >>>>> 2) be even less compatible for those that rely on customizations. > > >>>>> > > >>>>> I figured given time we could get a preview "user shell" (rather > than > > >>>>> "power shell" via irb) together in 2.0 and aim for default in 3.0. > > >>>>> > > >>>>> -- > > >>>>> Sean > > >>>>> On May 13, 2015 12:19 PM, "Stack"<st...@duboce.net> wrote: > > >>>>> > > >>>>> Nice writeup Sean. > > >>>>>> > > >>>>>> Yeah, +1 to new jruby in hbase 2.0. We'd need to be careful > license > > >>>>>> > > >>>>> is > > >>> > > >>>> still amenable and hopefully jruby 9k will be slimmer than jruby > > >>>>>> > > >>>>> 1.7+. > > >>> > > >>>> But if we are going to do a significant shell refactor for hbase > 2.0, > > >>>>>> should we consider doing something more radical; e.g. a new shell? > > If > > >>>>>> interest, could start up a new thread so don't distract from this > > >>>>>> > > >>>>> one. > > >>> > > >>>> St.Ack > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> On Wed, May 13, 2015 at 9:22 AM, Sean Busbey<bus...@cloudera.com> > > >>>>>> > > >>>>> wrote: > > >>>>> > > >>>>>> Hi folks! > > >>>>>>> > > >>>>>>> If you weren't aware, our current shell relies on Ruby, > > >>>>>>> > > >>>>>> specifically > > >>> > > >>>> the > > >>>>> > > >>>>>> REPL program IRB from JRuby. When we launched 1.0 we were on JRuby > > >>>>>>> > > >>>>>> 1.6 > > >>>> > > >>>>> with > > >>>>>> > > >>>>>>> defaults, which means we're stuck on Ruby 1.8. > > >>>>>>> > > >>>>>>> For those that don't already know, Ruby 1.8 is super old and has > > >>>>>>> > > >>>>>> been > > >>> > > >>>> walking off into the sunset for a few years now. Most (but not > > >>>>>>> > > >>>>>> all!) > > >>> > > >>>> formal > > >>>>>> > > >>>>>>> support systems for running Ruby have EOLed 1.8 and there are > > >>>>>>> > > >>>>>> numerous > > >>>> > > >>>>> known issues with it. > > >>>>>>> > > >>>>>>> Right now there's an open ticket to get us on JRuby 1.7 so that > our > > >>>>>>> > > >>>>>> shell > > >>>>> > > >>>>>> can work on PPC systems[1]. That version of JRuby defaults to Ruby > > >>>>>>> > > >>>>>> 1.9 > > >>>> > > >>>>> but > > >>>>>> > > >>>>>>> can be run in Ruby 1.8 mode. There are some implementation > details > > >>>>>>> outstanding, but I'm hoping that ticket can work out such that it > > >>>>>>> > > >>>>>> can > > >>> > > >>>> land > > >>>>>> > > >>>>>>> in branch-1. > > >>>>>>> > > >>>>>>> For HBase 2.0, I'd like us to plan for a little farther out in > the > > >>>>>>> > > >>>>>> future > > >>>>> > > >>>>>> than just updating to Ruby 1.9 (though that would be a fine > > >>>>>>> > > >>>>>> incremental > > >>>> > > >>>>> step with some non-trivial work attached). The "current" version of > > >>>>>>> > > >>>>>> Ruby > > >>>>> > > >>>>>> is > > >>>>>> > > >>>>>>> 2.2. Much like the move from 1.8 -> 1.9 it is not backwards > > >>>>>>> > > >>>>>> compatible. > > >>>> > > >>>>> JRuby's next major maintenance release line is "version 9000"[2] > > >>>>>>> > > >>>>>> and > > >>> > > >>>> it > > >>>> > > >>>>> will start out *only* supporting Ruby 2.2. Right now JRuby 9000 is > > >>>>>>> > > >>>>>> in > > >>> > > >>>> its > > >>>>> > > >>>>>> second "preview" release. They still have a few feature complete > > >>>>>>> > > >>>>>> items > > >>>> > > >>>>> to > > >>>>> > > >>>>>> address before they hit their first GA release. > > >>>>>>> > > >>>>>>> I'd like us to move to Ruby 2.2 via JRuby 9000 for HBase 2.0. > This > > >>>>>>> > > >>>>>> will > > >>>>> > > >>>>>> cause operator pain to folks with advanced scripts based on Ruby > > >>>>>>> > > >>>>>> 1.8, > > >>> > > >>>> but > > >>>>> > > >>>>>> it should allow us to update versions to avoid e.g. perf, > > >>>>>>> > > >>>>>> correctness, > > >>>> > > >>>>> and > > >>>>>> > > >>>>>>> security issues much more easily over the lifetime of 2.0. > > >>>>>>> > > >>>>>>> What do folks think? Would JRuby 9000 need to hit a GA release > > >>>>>>> > > >>>>>> prior > > >>> > > >>>> to > > >>>> > > >>>>> HBase 2.0 getting released for us to adopt it? Or would it only > > >>>>>>> > > >>>>>> need > > >>> > > >>>> enough > > >>>>>> > > >>>>>>> of Ruby 2.2 to run our current shell? > > >>>>>>> > > >>>>>>> > > >>>>>>> [1]: https://issues.apache.org/jira/browse/HBASE-13338 > > >>>>>>> [2]: > > >>>>>>> > > >>>>>> http://www.slideshare.net/CharlesNutter/over-9000-jruby-in-2015 > > >>> > > >>>> -- >
