Re: On coprocessor API evolution

2014-05-17 Thread lars hofhansl
That would be separate use case, though. It's easy to say coprocessors are too invasive, but that is what they were designed for: A way to extend region server and HMaster without requiring subclassing the main classes and taking over completely. I'm all in favor of adding new APIs for other

Re: On coprocessor API evolution

2014-05-17 Thread lars hofhansl
We've seen similar issues with Filters. Those are good rules to follow. From: Vladimir Rodionov To: "dev@hbase.apache.org" Sent: Friday, May 16, 2014 10:59 AM Subject: Re: On coprocessor API evolution 1) Have default implementations (abstract classes) for

Re: On coprocessor API evolution

2014-05-17 Thread lars hofhansl
But note that such a high level interface will probably not allow access at level necessary for Phoenix. You can't eat your cake and keep it. For some performance you deep access to HBase internals (RegionScanners, for example). At the same time, these *are* internal. I suppose to least we can

Re: jetty 9 on hbase? (HADOOP-10075)

2014-05-17 Thread Demai Ni
Sounds good. I will open one if I can make it works on 96.x. Currently my change on Infoserver seems ok. RESTserver compiled, but I can start rest. Need to play more tomorrow Demai on the run On May 16, 2014, at 9:34 PM, Ted Yu wrote: > You may want to open a JIRA with your findings. > > Ch

Re: 0.94.20 soon

2014-05-17 Thread lars hofhansl
Looking. Seems like a bug we should fix. From: qiang tian To: "dev@hbase.apache.org" ; lars hofhansl Sent: Saturday, May 17, 2014 6:39 PM Subject: Re: 0.94.20 soon Hi Lars,Andrew, Could you take a look at HBASE-11096? Thanks. On Friday, May 16, 2014,

[jira] [Resolved] (HBASE-3324) LRU block cache configuration improvements

2014-05-17 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-3324. -- Resolution: Later Part dup of HBASE-3306 and stale. Resolving. > LRU block cache configuration improvements >

[jira] [Resolved] (HBASE-5285) runtime exception -- cached an already cached block -- during compaction

2014-05-17 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-5285. -- Resolution: Cannot Reproduce Haven't see this in years. Correct me if I am wrong. > runtime exception -- cach

[jira] [Resolved] (HBASE-4909) Detailed Block Cache Metrics

2014-05-17 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-4909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-4909. -- Resolution: Fixed Subsumed by HBASE-4027 > Detailed Block Cache Metrics > > >

Re: 0.94.20 soon

2014-05-17 Thread Esteban Gutierrez
Hi Lars, Do you think HBASE-6990 makes sense to include in 0.94.20? regards, esteban. -- Cloudera, Inc. On Sat, May 17, 2014 at 6:39 PM, qiang tian wrote: > Hi Lars,Andrew, > Could you take a look at HBASE-11096? > Thanks. > > > > On Friday, May 16, 2014, lars hofhansl wrote: > > > I'd li

Re: On coprocessor API evolution

2014-05-17 Thread Andrew Purtell
You should be telling those customers that use of coprocessors "voids the warranty". They are a convenience for HBase project developers and advanced users, not a license for random devs to upload code into the server and then expect vendor support. It should be obvious on the face of it that is no

Re: On coprocessor API evolution

2014-05-17 Thread Andrew Purtell
A related issue is HBASE-2396, which envisions building a sandbox inside an embedded scripting environment on top of the coprocessor hooks for 'safer' execution of such things we might call user code triggers, see the description on the issue and then the large-ish comment 6 or 7 down from the top.

[jira] [Created] (HBASE-11198) test-patch.sh should handle the case where trunk patch is attached along with patch for 0.98

2014-05-17 Thread Ted Yu (JIRA)
Ted Yu created HBASE-11198: -- Summary: test-patch.sh should handle the case where trunk patch is attached along with patch for 0.98 Key: HBASE-11198 URL: https://issues.apache.org/jira/browse/HBASE-11198 Proj

Re: 0.94.20 soon

2014-05-17 Thread qiang tian
Hi Lars,Andrew, Could you take a look at HBASE-11096? Thanks. On Friday, May 16, 2014, lars hofhansl wrote: > I'd like to do an RC for 0.94.20 soon. Looks like it'd be a nice and small > release, only 7 fixes so far. 0.94 is maturing. :) > Anything that should get pulled in? > > Thanks. > > --

Re: How can we process complex logic on hbase

2014-05-17 Thread Jean-Marc Spaggiari
Moving the discussion to the "user" list. Hi Vikas, You can use coprocessors to do something similar, but there is some drawback. You need to be pretty careful with them. That's the way to create process similar to stored procs. Else, to MR jobs? JM 2014-05-08 5:47 GMT-04:00 Vikas Jadhav : >

Re: On coprocessor API evolution

2014-05-17 Thread Kevin O'dell
Andrew, HBase-4047 is a great idea(even if it is three years old). I have had numerous customers implement Co-Procs and take down every RS in a spectacular fashion from JVM crashes to performance crawling so slow that jobs fail out. I will raise this internally and see if we can get some extr

Re: On coprocessor API evolution

2014-05-17 Thread Andrew Purtell
Great, see HBASE-4047. In the best of the open source tradition, there hasn't been anyone sufficiently motivated to do the work necessary (current use cases are "good enough"), but that someone can always come along. Perhaps that is yourself. On Sat, May 17, 2014 at 5:39 AM, Michael Segel wrote:

Re: On coprocessor API evolution

2014-05-17 Thread Michael Segel
You have to understand… I do see the importance of the hook to allow for a trigger to implement 3rd party code on the server side. No argument there. Its just how the current implementation doesn’t sandbox the code so that it limits the potential for harm to the RS. In simple terms you can i

Re: On coprocessor API evolution

2014-05-17 Thread Michael Segel
Andrew, Since you focused on my term ‘end user’ let me try this one more time because you clearly, don’t grok the problem. Running third party code as a coprocessor in the same jvm at the RS creates the risk that said code could cause the RS to fail and cause a cascading failure. That’s an u

Re: On coprocessor API evolution

2014-05-17 Thread qiang tian
My small 2 cents...:-) Hook/coprocessor is useful mechanism to interacting with a system for things that cannot be done via API. For end user, the tradeoff factors like performance, security, reliability etc can be control by upper layer' policy. e.g. In RDBMS, the end user has limited usage cas

Re: On coprocessor API evolution

2014-05-17 Thread Andrew Purtell
The idea that coprocessors are meant for running untrusted end user code is a strawman of your own construction. You show up with this whenever coprocessors comes up on the list and lecture on our supposed incompetence and inability to see straight. It's not really that helpful. I understand you

Re: On coprocessor API evolution

2014-05-17 Thread Michael Segel
Andrew, Is ‘magical fairy dust’ a reference to some new synthetic drug you take at raves? But lets get back to reality. Lets try this again; simply put… the coprocessor runs on the same JVM as the RS, therefore you have an unacceptable level of risk. That inherent risk means that you cannot