Todd,
I’m saying that you’re raising a straw man.
Now correct me if I’m wrong… but coprocessor code is split in to two camps. One
is System coprocessors which are defined in the hbase-site.xml, right?
What do you call the other group of coprocessor code?
(Sorry my memory is going. Killed too
On Wed, May 21, 2014 at 5:16 AM, Michael Segel wrote:
> And they accuse me of raising a straw man. ;-)
>
I wasn't arguing for or against coprocessors being out of process. I think
both sides of this "argument" are in fact in agreement: if you think of
coprocessors as a place to run *user* code, t
And they accuse me of raising a straw man. ;-)
Todd, really? A parent/child relationship can be secured… how depends on how
you communicate.
You could always encrypt the data… in the messaging… ;-)
On May 19, 2014, at 11:37 PM, Todd Lipcon wrote:
> On Mon, May 19, 2014 at 12:05 PM, Vladimir
On Mon, May 19, 2014 at 12:05 PM, Vladimir Rodionov
wrote:
> Michael S.
> >> To the best of my knowledge, MapR’s M7 doesn’t have coprocessors. I’ll
> wager that when they do, it will work and not have these issues. I believe
> that they are writing their stuff in C/C++, if so, then they’d have an
n more work for
> someone.
> >>>>
> >>>> -Mike
> >>>>
> >>>>> On May 18, 2014, at 9:58 PM, lars hofhansl wrote:
> >>>>>
> >>>>> Coprocessors are a means to extend HBase. Nothing more, nothing less.
> &
gt;>>
>>>>> Coprocessors are a means to extend HBase. Nothing more, nothing less.
>> They are not stored procedures or triggers.
>>>>> Not sure in how many other ways we can/need to phrase that.
>>>>>
>>>>>
>>>>
;>>
> >>> I agree that there should be a simple way to disable user coprocessors
> (or at least disable loading from HDFS) for the security conscious. Let's
> do that, it's simple.
> >>>
> >>> There is nothing to "fix" since it
m HDFS) for the security conscious. Let's do
>>> that, it's simple.
>>>
>>> There is nothing to "fix" since it ain't broken. It's only seems broken
>>> when you do not understand what it was designed for.
>>>
>>>
ings in a sandbox, more like stored
>> procedures and triggers... Sure, let's do that too. But realize that is a
>> *new* use case, and that we'll keep the old stuff.
>>
>> -- Lars
>>
>>
>> From: Michael Segel
>&g
ars
>
>
> From: Michael Segel
> To: dev@hbase.apache.org; lars hofhansl
> Sent: Sunday, May 18, 2014 10:21 AM
> Subject: Re: On coprocessor API evolution
>
>
> It doesn’t matter.
>
> Sure we can follow Vlad’s rules… but you still
triggers... Sure, let's do that too. But realize that is a
> *new* use case, and that we'll keep the old stuff.
>
> -- Lars
>
>
> From: Michael Segel >
> To: dev@hbase.apache.org ; lars hofhansl
>
> >
> Sent: Sunday,
but to me at least it does not seem
> > to
> > > be Phoenix. But maybe I am wrong.
> > >
> > >
> > > -- Lars
> > >
> > >
> > >
> > >
> > > From: James Taylor >
> > > T
riggers... Sure, let's do that too. But realize that is a *new*
use case, and that we'll keep the old stuff.
-- Lars
From: Michael Segel
To: dev@hbase.apache.org; lars hofhansl
Sent: Sunday, May 18, 2014 10:21 AM
Subject: Re: On coprocessor API evol
Thanks James.
If Phoenix works on the new APIs, the likelihood is high that most other things
would work as well. :)
-- Lars
- Original Message -
From: James Taylor
To: "dev@hbase.apache.org"
Cc:
Sent: Sunday, May 18, 2014 11:50 AM
Subject: Re: On coprocessor API
s a use case for HBASE-11125, but to me at least it does not seem
> to
> > be Phoenix. But maybe I am wrong.
> >
> >
> > -- Lars
> >
> >
> >
> > ____
> > From: James Taylor >
> > To: "dev@hbase.apache.org &qu
And you should consult a lawyer before you make a statement like that…
These are exposed APIs and Cloudera, Hortonworks, MapR, Pivotal, even Intel… if
they still have licensed customers.. all have to support their releases.
BTW, I think I’m the only person who’s given a talk trying to explai
From: Vladimir Rodionov
> To: "dev@hbase.apache.org"
> Sent: Friday, May 16, 2014 10:59 AM
> Subject: Re: On coprocessor API evolution
>
>
> 1) Have default implementations (abstract classes) for every interface from
> Coprocessor API.
> 2) Advise coprocessor
gt; Sent: Friday, May 16, 2014 10:59 AM
> Subject: Re: On coprocessor API evolution
>
>
> 1) Have default implementations (abstract classes) for every interface from
> Coprocessor API.
> 2) Advise coprocessor users not to implement interface directly but sub
> class default impl.
&
_
> From: James Taylor >
> To: "dev@hbase.apache.org "
> >
> Sent: Thursday, May 15, 2014 6:47 PM
> Subject: Re: On coprocessor API evolution
>
>
> +1 to HBASE-11125. Current incarnation of coprocessors expose too much of
> the guts of the imple
2 AM
Subject: Re: On coprocessor API evolution
Andrew,
HBase-4047 is a great idea(even if it is three years old). I have had
numerous customers implement Co-Procs and take down every RS in a
spectacular fashion from JVM crashes to performance crawling so slow that
jobs fail out. I will raise thi
We've seen similar issues with Filters. Those are good rules to follow.
From: Vladimir Rodionov
To: "dev@hbase.apache.org"
Sent: Friday, May 16, 2014 10:59 AM
Subject: Re: On coprocessor API evolution
1) Have default implementations (abstra
rom: James Taylor
To: "dev@hbase.apache.org"
Sent: Thursday, May 15, 2014 6:47 PM
Subject: Re: On coprocessor API evolution
+1 to HBASE-11125. Current incarnation of coprocessors expose too much of
the guts of the implementation.
On Wed, May 14, 2014 at 6:13 PM, Andrew Purtel
You should be telling those customers that use of coprocessors "voids the
warranty". They are a convenience for HBase project developers and advanced
users, not a license for random devs to upload code into the server and
then expect vendor support. It should be obvious on the face of it that is
no
A related issue is HBASE-2396, which envisions building a sandbox inside an
embedded scripting environment on top of the coprocessor hooks for 'safer'
execution of such things we might call user code triggers, see the
description on the issue and then the large-ish comment 6 or 7 down from
the top.
Andrew,
HBase-4047 is a great idea(even if it is three years old). I have had
numerous customers implement Co-Procs and take down every RS in a
spectacular fashion from JVM crashes to performance crawling so slow that
jobs fail out. I will raise this internally and see if we can get some
extr
Great, see HBASE-4047. In the best of the open source tradition, there
hasn't been anyone sufficiently motivated to do the work necessary (current
use cases are "good enough"), but that someone can always come along.
Perhaps that is yourself.
On Sat, May 17, 2014 at 5:39 AM, Michael Segel wrote:
You have to understand…
I do see the importance of the hook to allow for a trigger to implement 3rd
party code on the server side.
No argument there.
Its just how the current implementation doesn’t sandbox the code so that it
limits the potential for harm to the RS.
In simple terms you can i
Andrew,
Since you focused on my term ‘end user’ let me try this one more time because
you clearly, don’t grok the problem.
Running third party code as a coprocessor in the same jvm at the RS creates the
risk that said code could cause the RS to fail and cause a cascading failure.
That’s an u
My small 2 cents...:-)
Hook/coprocessor is useful mechanism to interacting with a system for
things that cannot be done via API. For end user, the tradeoff factors
like performance, security, reliability etc can be control by upper layer'
policy.
e.g. In RDBMS, the end user has limited usage cas
The idea that coprocessors are meant for running untrusted end user code is a
strawman of your own construction. You show up with this whenever coprocessors
comes up on the list and lecture on our supposed incompetence and inability to
see straight. It's not really that helpful. I understand you
Andrew,
Is ‘magical fairy dust’ a reference to some new synthetic drug you take at
raves?
But lets get back to reality.
Lets try this again; simply put… the coprocessor runs on the same JVM as the
RS, therefore you have an unacceptable level of risk.
That inherent risk means that you cannot
On Fri, May 16, 2014 at 1:47 AM, Stack wrote:
> I had just read a user mail on the phoenix list where someone thought that
> phoenix had been broken going from 0.98.1 to 0.98.2 (apparently its fine).
>
> Lets write up agreement. We've talked this topic a bunch.
>
> 1. No guarantees minor version
Michael,
As you know, we have implemented security features with coprocessors
precisely because they can be interposed on internal actions to make
authoritative decisions in-process. Coprocessors are a way to have
composable internal extensions. They don't have and probably never will
have magic f
Thanks Vladimir, I added your points to the discussion on HBASE-11125.
On Sat, May 17, 2014 at 1:59 AM, Vladimir Rodionov
wrote:
> 1) Have default implementations (abstract classes) for every interface from
> Coprocessor API.
> 2) Advise coprocessor users not to implement interface directly but
1) Have default implementations (abstract classes) for every interface from
Coprocessor API.
2) Advise coprocessor users not to implement interface directly but sub
class default impl.
3) Preserve backward compatibility by adding only new hooks/methods
4) DO NOT CHANGE existing API (no method renam
Nicolas:
Can you give an example of using @since to tag new hooks ?
I searched hadoop and hbase codebase but didn't seem to find such
annotation.
Cheers
On Fri, May 16, 2014 at 1:18 AM, Nicolas Liochon wrote:
> Hi,
>
> (With Apache still lagging on mails, it may be difficult to have a
> discu
+1 to HBASE-11125. Current incarnation of coprocessors expose too much of
the guts of the implementation.
On Wed, May 14, 2014 at 6:13 PM, Andrew Purtell wrote:
> Because coprocessor APIs are so tightly bound with internals, if we apply
> suggested rules like as mentioned on HBASE-11054:
>
>
Until you move the coprocessor out of the RS space and into its own sandbox…
saying security and coprocessor in the same sentence is a joke.
Oh wait… you were serious… :-(
I’d say there’s a significant rethink on coprocessors that’s required.
Anyone running a secure (kerberos) cluster, will wan
Hi,
(With Apache still lagging on mails, it may be difficult to have a
discussion...)
For 1.0+, I think that registering observer as proposed in 11125 works well.
For 0.98, could we do something like this?
- new coprocessor hooks can be added between minor releases
- existing coprocessors hooks
On Wed, May 14, 2014 at 6:13 PM, Andrew Purtell wrote:
> Because coprocessor APIs are so tightly bound with internals, if we apply
> suggested rules like as mentioned on HBASE-11054:
>
> I'd say policy should be no changes to method apis across minor
> versions
>
> This will lock coprocesso
Because coprocessor APIs are so tightly bound with internals, if we apply
suggested rules like as mentioned on HBASE-11054:
I'd say policy should be no changes to method apis across minor
versions
This will lock coprocessor based components to the limitations of the API
as we encounter them
41 matches
Mail list logo
