date:20160421

Re: Review Request 46542: SENTRY-583: Add boundary condition test coverage to HDFS synchronization test suite around max #of groups

2016-04-21 Thread Anne Yu


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46542/
---

(Updated April 21, 2016, 11:47 p.m.)


Review request for sentry, Hao Hao and Lenni Kuff.


Changes
---

I also added one more test case to test one scenario when there is an external 
path configured in the prefix.


Bugs: SENTRY-583
https://issues.apache.org/jira/browse/SENTRY-583


Repository: sentry


Description
---

Add one boundary HDFS SyncUp test to run on a real cluster;
Also add some basic APIs for createing more HDFS SyncUp tests.


Diffs (updated)
-

  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestDbHdfsBase.java
 PRE-CREATION 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestDbHdfsMaxGroups.java
 PRE-CREATION 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithStaticConfiguration.java
 8515a2ba1fc08104a788ea054f2b871905359273 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/fs/DFSFactory.java
 e1881b4bfaa5923fd0dab2096f670b22b944f116 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/fs/MiniDFS.java
 77af4329642807283e6cd3d2e3be837c5d3e0dfb 

Diff: https://reviews.apache.org/r/46542/diff/


Testing
---

Tested on a unmanaged real cluster.


 T E S T S
---
Running org.apache.sentry.tests.e2e.hdfs.TestDbHdfsMaxGroups
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 50.615 sec - in 
org.apache.sentry.tests.e2e.hdfs.TestDbHdfsMaxGroups

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0


Thanks,

Anne Yu

Re: FAQs?

2016-04-21 Thread Bhooshan Mogal

Never mind. I created an account on Confluence. I'll add the page later.

On Thu, Apr 21, 2016 at 2:54 PM, Bhooshan Mogal 
wrote:

> Thanks. Yes, it would be helpful to throw an appropriate error message.
> The problem though is we have to presume that the service and the client
> use different sentry-site.xmls. In the service process,  you do not have
> access to the client sentry-site.xml and vice versa. So I don't think there
> is a way to throw the right message here, because the inconsistency is in
> two different processes, and the communication between them is thrift
> (which we do not control). I'll still try and see if there is a code path
> that we can intercept and log a message. If anyone has suggestions, I can
> try it them out as well.
>
> Regarding the FAQ page, how do I get access to confluence (my Apache
> credentials do not seem to work).
>
> -
> Bhooshan
>
> On Thu, Apr 21, 2016 at 2:43 PM, Anne Yu  wrote:
>
>> Hi Bhooshan,
>>
>> IC, FAQ session could be very helpful. We have a upstream space for all
>> sentry docs (https://cwiki.apache.org/confluence/display/SENTRY/Home).
>> You
>> might want to add a dedicated page for FAQ.
>>
>> Besides if "sentry.service.security.mode" is inconsistent between server
>> and client, the client APIs could be improved and surface such issues into
>> exception or log error, instead of wiki page. If you have any idea could
>> contribute to it. That's what I meant.
>>
>> Thanks,
>> Anne
>>
>> On Thu, Apr 21, 2016 at 10:43 AM, Bhooshan Mogal <
>> bhooshan.mo...@gmail.com>
>> wrote:
>>
>> > Thanks Anne. Like I said, I'm pretty sure what's going on. But it seems
>> > very unlikely that we could add a fix for this in Sentry, since the
>> error
>> > occurs in the thrift communication between SentryService and
>> > SentryGenericServiceClientDefultImpl. To me, this seemed like a good
>> > candidate for some FAQs section we there is one on wiki - so its at
>> least
>> > discoverable when someone does a search for this error with Sentry.
>> >
>> > -
>> > Bhooshan
>> >
>> > On Thu, Apr 21, 2016 at 10:27 AM, Anne Yu  wrote:
>> >
>> > > Hi Bhooshan,
>> > >
>> > > Thanks for reporting this issue. If you have clear idea of what's
>> going
>> > on,
>> > > since you've spent time debugging the code, please feel free to
>> create a
>> > > jira (https://issues.apache.org/jira/browse/SENTRY/) then post a fix.
>> > > Committers will be very happy to do the code review.
>> > >
>> > > Best,
>> > > Anne
>> > >
>> > > On Wed, Apr 20, 2016 at 9:17 PM, Bhooshan Mogal <
>> > bhooshan.mo...@gmail.com>
>> > > wrote:
>> > >
>> > > > Hi folks,
>> > > >
>> > > > I suddenly started seeing the following error in the Sentry Service
>> > today
>> > > > and ended up debugging it for a long time across Sentry, Thrift and
>> my
>> > > own
>> > > > code.
>> > > >
>> > > > 16/04/21 02:15:51 ERROR server.TThreadPoolServer: Thrift error
>> occurred
>> > > > during processing of message.
>> > > > org.apache.thrift.protocol.TProtocolException: Missing version in
>> > > > readMessageBegin, old client?
>> > > > at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:228)
>> > > > at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:92)
>> > > > at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
>> > > > at
>> > > >
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> > > > at
>> > > >
>> > > >
>> > >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> > > > at java.lang.Thread.run(Thread.java:745)
>> > > >
>> > > >
>> > > >
>> > > > It turns out that my (Sentry) client had
>> sentry.service.security.mode
>> > set
>> > > > to *kerberos* while the (Sentry) server had it set to *none. *But
>> the
>> > > error
>> > > > message threw me off completely.
>> > > >
>> > > > I tried to see if there was a way we could catch this scenario and
>> > throw
>> > > a
>> > > > better error, but its difficult because as either the client or
>> server,
>> > > its
>> > > > not possible to know what the other is configured as. However, is
>> there
>> > > an
>> > > > FAQs section where such issues can be documented?
>> > > >
>> > > >
>> > > > Best,
>> > > > Bhooshan
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Bhooshan
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Anne
>> > >
>> >
>> >
>> >
>> > --
>> > Bhooshan
>> >
>>
>>
>>
>> --
>> Anne
>>
>
>
>
> --
> Bhooshan
>



-- 
Bhooshan

Re: FAQs?

2016-04-21 Thread Bhooshan Mogal

Thanks. Yes, it would be helpful to throw an appropriate error message. The
problem though is we have to presume that the service and the client use
different sentry-site.xmls. In the service process,  you do not have access
to the client sentry-site.xml and vice versa. So I don't think there is a
way to throw the right message here, because the inconsistency is in two
different processes, and the communication between them is thrift (which we
do not control). I'll still try and see if there is a code path that we can
intercept and log a message. If anyone has suggestions, I can try it them
out as well.

Regarding the FAQ page, how do I get access to confluence (my Apache
credentials do not seem to work).

-
Bhooshan

On Thu, Apr 21, 2016 at 2:43 PM, Anne Yu  wrote:

> Hi Bhooshan,
>
> IC, FAQ session could be very helpful. We have a upstream space for all
> sentry docs (https://cwiki.apache.org/confluence/display/SENTRY/Home). You
> might want to add a dedicated page for FAQ.
>
> Besides if "sentry.service.security.mode" is inconsistent between server
> and client, the client APIs could be improved and surface such issues into
> exception or log error, instead of wiki page. If you have any idea could
> contribute to it. That's what I meant.
>
> Thanks,
> Anne
>
> On Thu, Apr 21, 2016 at 10:43 AM, Bhooshan Mogal  >
> wrote:
>
> > Thanks Anne. Like I said, I'm pretty sure what's going on. But it seems
> > very unlikely that we could add a fix for this in Sentry, since the error
> > occurs in the thrift communication between SentryService and
> > SentryGenericServiceClientDefultImpl. To me, this seemed like a good
> > candidate for some FAQs section we there is one on wiki - so its at least
> > discoverable when someone does a search for this error with Sentry.
> >
> > -
> > Bhooshan
> >
> > On Thu, Apr 21, 2016 at 10:27 AM, Anne Yu  wrote:
> >
> > > Hi Bhooshan,
> > >
> > > Thanks for reporting this issue. If you have clear idea of what's going
> > on,
> > > since you've spent time debugging the code, please feel free to create
> a
> > > jira (https://issues.apache.org/jira/browse/SENTRY/) then post a fix.
> > > Committers will be very happy to do the code review.
> > >
> > > Best,
> > > Anne
> > >
> > > On Wed, Apr 20, 2016 at 9:17 PM, Bhooshan Mogal <
> > bhooshan.mo...@gmail.com>
> > > wrote:
> > >
> > > > Hi folks,
> > > >
> > > > I suddenly started seeing the following error in the Sentry Service
> > today
> > > > and ended up debugging it for a long time across Sentry, Thrift and
> my
> > > own
> > > > code.
> > > >
> > > > 16/04/21 02:15:51 ERROR server.TThreadPoolServer: Thrift error
> occurred
> > > > during processing of message.
> > > > org.apache.thrift.protocol.TProtocolException: Missing version in
> > > > readMessageBegin, old client?
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:228)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:92)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
> > > > at
> > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > > > at
> > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > > > at java.lang.Thread.run(Thread.java:745)
> > > >
> > > >
> > > >
> > > > It turns out that my (Sentry) client had sentry.service.security.mode
> > set
> > > > to *kerberos* while the (Sentry) server had it set to *none. *But the
> > > error
> > > > message threw me off completely.
> > > >
> > > > I tried to see if there was a way we could catch this scenario and
> > throw
> > > a
> > > > better error, but its difficult because as either the client or
> server,
> > > its
> > > > not possible to know what the other is configured as. However, is
> there
> > > an
> > > > FAQs section where such issues can be documented?
> > > >
> > > >
> > > > Best,
> > > > Bhooshan
> > > >
> > > >
> > > >
> > > > --
> > > > Bhooshan
> > > >
> > >
> > >
> > >
> > > --
> > > Anne
> > >
> >
> >
> >
> > --
> > Bhooshan
> >
>
>
>
> --
> Anne
>



-- 
Bhooshan

Re: FAQs?

2016-04-21 Thread Bhooshan Mogal

Thanks Anne. Like I said, I'm pretty sure what's going on. But it seems
very unlikely that we could add a fix for this in Sentry, since the error
occurs in the thrift communication between SentryService and
SentryGenericServiceClientDefultImpl. To me, this seemed like a good
candidate for some FAQs section we there is one on wiki - so its at least
discoverable when someone does a search for this error with Sentry.

-
Bhooshan

On Thu, Apr 21, 2016 at 10:27 AM, Anne Yu  wrote:

> Hi Bhooshan,
>
> Thanks for reporting this issue. If you have clear idea of what's going on,
> since you've spent time debugging the code, please feel free to create a
> jira (https://issues.apache.org/jira/browse/SENTRY/) then post a fix.
> Committers will be very happy to do the code review.
>
> Best,
> Anne
>
> On Wed, Apr 20, 2016 at 9:17 PM, Bhooshan Mogal 
> wrote:
>
> > Hi folks,
> >
> > I suddenly started seeing the following error in the Sentry Service today
> > and ended up debugging it for a long time across Sentry, Thrift and my
> own
> > code.
> >
> > 16/04/21 02:15:51 ERROR server.TThreadPoolServer: Thrift error occurred
> > during processing of message.
> > org.apache.thrift.protocol.TProtocolException: Missing version in
> > readMessageBegin, old client?
> > at
> >
> >
> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:228)
> > at
> >
> >
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:92)
> > at
> >
> >
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > at java.lang.Thread.run(Thread.java:745)
> >
> >
> >
> > It turns out that my (Sentry) client had sentry.service.security.mode set
> > to *kerberos* while the (Sentry) server had it set to *none. *But the
> error
> > message threw me off completely.
> >
> > I tried to see if there was a way we could catch this scenario and throw
> a
> > better error, but its difficult because as either the client or server,
> its
> > not possible to know what the other is configured as. However, is there
> an
> > FAQs section where such issues can be documented?
> >
> >
> > Best,
> > Bhooshan
> >
> >
> >
> > --
> > Bhooshan
> >
>
>
>
> --
> Anne
>



-- 
Bhooshan

Re: FAQs?

2016-04-21 Thread Anne Yu

Hi Bhooshan,

Thanks for reporting this issue. If you have clear idea of what's going on,
since you've spent time debugging the code, please feel free to create a
jira (https://issues.apache.org/jira/browse/SENTRY/) then post a fix.
Committers will be very happy to do the code review.

Best,
Anne

On Wed, Apr 20, 2016 at 9:17 PM, Bhooshan Mogal 
wrote:

> Hi folks,
>
> I suddenly started seeing the following error in the Sentry Service today
> and ended up debugging it for a long time across Sentry, Thrift and my own
> code.
>
> 16/04/21 02:15:51 ERROR server.TThreadPoolServer: Thrift error occurred
> during processing of message.
> org.apache.thrift.protocol.TProtocolException: Missing version in
> readMessageBegin, old client?
> at
>
> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:228)
> at
>
> org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:92)
> at
>
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>
>
>
> It turns out that my (Sentry) client had sentry.service.security.mode set
> to *kerberos* while the (Sentry) server had it set to *none. *But the error
> message threw me off completely.
>
> I tried to see if there was a way we could catch this scenario and throw a
> better error, but its difficult because as either the client or server, its
> not possible to know what the other is configured as. However, is there an
> FAQs section where such issues can be documented?
>
>
> Best,
> Bhooshan
>
>
>
> --
> Bhooshan
>



-- 
Anne

Re: Review Request 46542: SENTRY-583: Add boundary condition test coverage to HDFS synchronization test suite around max #of groups

Re: FAQs?

Re: FAQs?

Re: FAQs?

Re: FAQs?

5 matches

Site Navigation

Mail list logo

Footer information