[jira] [Updated] (RANGER-1895) Simplify Storm dependencies
[ https://issues.apache.org/jira/browse/RANGER-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1895: Description: This task is to simplify the Apache Storm dependencies for Ranger. We can avoid explicitly listing some of the dependencies, as they get pulled in via other dependencies. (was: This task is to simplify the Apache Storm dependencies for Ranger. We are shipping the hadoop-hdfs jar, which is not required. Secondly, we can avoid explicitly listing some of the dependencies, as they get pulled in via other dependencies.) > Simplify Storm dependencies > --- > > Key: RANGER-1895 > URL: https://issues.apache.org/jira/browse/RANGER-1895 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh >Priority: Minor > Fix For: 1.0.0 > > > This task is to simplify the Apache Storm dependencies for Ranger. We can > avoid explicitly listing some of the dependencies, as they get pulled in via > other dependencies. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RANGER-1898) Simplify Knox plugin dependency management
[ https://issues.apache.org/jira/browse/RANGER-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1898: Description: This task is to simplify the dependency management for the Knox plugin for Ranger. Tested that the distribution ships the same jars as before. Also tested a use-case involving Ranger + Knox. was: This task is to simplify the dependency management for the Knox plugin for Ranger. In addition, the hadoop-hdfs jar is removed from the distribution, as it is not required. Tested that the distribution ships the same jars as before, apart from hadoop-hdfs. Also tested a use-case involving Ranger + Knox. > Simplify Knox plugin dependency management > -- > > Key: RANGER-1898 > URL: https://issues.apache.org/jira/browse/RANGER-1898 > Project: Ranger > Issue Type: Bug > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh >Priority: Minor > Fix For: 1.0.0 > > Attachments: > 0001-RANGER-1898-Simplify-Knox-plugin-dependency-manageme.patch > > > This task is to simplify the dependency management for the Knox plugin for > Ranger. > Tested that the distribution ships the same jars as before. Also tested a > use-case involving Ranger + Knox. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RANGER-1898) Simplify Knox plugin dependency management
[ https://issues.apache.org/jira/browse/RANGER-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1898: Attachment: 0001-RANGER-1898-Simplify-Knox-plugin-dependency-manageme.patch > Simplify Knox plugin dependency management > -- > > Key: RANGER-1898 > URL: https://issues.apache.org/jira/browse/RANGER-1898 > Project: Ranger > Issue Type: Bug > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh >Priority: Minor > Fix For: 1.0.0 > > Attachments: > 0001-RANGER-1898-Simplify-Knox-plugin-dependency-manageme.patch > > > This task is to simplify the dependency management for the Knox plugin for > Ranger. > Tested that the distribution ships the same jars as before. Also tested a > use-case involving Ranger + Knox. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Review Request 63949: RANGER-1898 - Simplify Knox plugin dependency management
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63949/ --- (Updated Nov. 27, 2017, 3:52 p.m.) Review request for ranger. Bugs: RANGER-1898 https://issues.apache.org/jira/browse/RANGER-1898 Repository: ranger Description (updated) --- This task is to simplify the dependency management for the Knox plugin for Ranger. Diffs (updated) - knox-agent/pom.xml 52f15cd0 Diff: https://reviews.apache.org/r/63949/diff/2/ Changes: https://reviews.apache.org/r/63949/diff/1-2/ Testing (updated) --- Tested that the distribution ships the same jars as before. Also tested a use-case involving Ranger + Knox. Thanks, Colm O hEigeartaigh
[jira] [Updated] (RANGER-1895) Simplify Storm dependencies
[ https://issues.apache.org/jira/browse/RANGER-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1895: Attachment: 0001-RANGER-1895-Simplify-Storm-dependencies.patch > Simplify Storm dependencies > --- > > Key: RANGER-1895 > URL: https://issues.apache.org/jira/browse/RANGER-1895 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh >Priority: Minor > Fix For: 1.0.0 > > Attachments: 0001-RANGER-1895-Simplify-Storm-dependencies.patch > > > This task is to simplify the Apache Storm dependencies for Ranger. We can > avoid explicitly listing some of the dependencies, as they get pulled in via > other dependencies. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Review Request 63919: RANGER-1895 - Simplify Storm dependencies
> On Nov. 27, 2017, 6:49 p.m., Ramesh Mani wrote: > > storm-agent/pom.xml > > Lines 95-96 (original) > > <https://reviews.apache.org/r/63919/diff/3/?file=1901538#file1901538line95> > > > > Colm, did you verify that audit to solr is working as expected? We may > > need this solr jar for Ranger audit dependency Yes I did. The solr jar comes in via the org.apache.ranger:ranger-plugins-audit dependency, so it is still there on the classpath. The distribution still contains the Solr jar in the lib directory as before. - Colm --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63919/#review191915 ------- On Nov. 27, 2017, 3:27 p.m., Colm O hEigeartaigh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63919/ > --- > > (Updated Nov. 27, 2017, 3:27 p.m.) > > > Review request for ranger. > > > Bugs: RANGER-1895 > https://issues.apache.org/jira/browse/RANGER-1895 > > > Repository: ranger > > > Description > --- > > This task is to simplify the Apache Storm dependencies for Ranger. We are > shipping the hadoop-hdfs jar, which is not required. Secondly, we can avoid > explicitly listing some of the dependencies, as they get pulled in via other > dependencies. > > > Diffs > - > > ranger-storm-plugin-shim/pom.xml de1972d4 > storm-agent/pom.xml 6e74e5b3 > > > Diff: https://reviews.apache.org/r/63919/diff/3/ > > > Testing > --- > > Tested the plugin works OK with Apache Storm 1.1.1. The only change in the > distribution is that it doesn't include the Hadoop HDFS jar. > > > Thanks, > > Colm O hEigeartaigh > >
Re: Review Request 61062: RANGER-1707 : fix hdfs traverse check
> On Nov. 21, 2017, 4 p.m., Colm O hEigeartaigh wrote:
> > You could put some spaces into "for (int i=0;i<pathSegments.length;i++) {"
> > There's also an indentation issue on line 201 of RangerHdfsAuthorizerTest.
> > Other spacing issue here "ancestorIndex,plugin"
> >
> > > for (FsAction action : Arrays.asList(FsAction.EXECUTE, FsAction.READ,
> > > FsAction.WRITE)) {
> >
> > I think the FsAction.EXECUTE is not necessary here, as we are checking
> > EXECUTE already in "traverseOnlyCheck".
>
> Zsombor Gegesy wrote:
> The trick is, that there are different inodes used for the checks:
>
> final AuthzStatus status = isAccessAllowed(nodeToCheck, nodeAttribs,
> FsAction.EXECUTE, user, groups, plugin, auditHandler);
> if (status == AuthzStatus.NOT_DETERMINED) {
> return isAnyAccessAllowed(inode, inode, user, groups, plugin,
> auditHandler);
> }
>
> First, we use 'nodeToCheck', which can be a parent or ancestor node, and
> in the loop, we use 'inode' which refers to the actual file.
OK understood thanks. The indentation issue is still there, now on line 224 of
RangerHdfsAuthorizerTest (single tab character indent)
- Colm
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61062/#review191583
---
On Nov. 21, 2017, 4:34 p.m., Zsombor Gegesy wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61062/
> ---
>
> (Updated Nov. 21, 2017, 4:34 p.m.)
>
>
> Review request for ranger.
>
>
> Bugs: RANGER-1707
> https://issues.apache.org/jira/browse/RANGER-1707
>
>
> Repository: ranger
>
>
> Description
> ---
>
> Fix hdfs traverse check, which problem was hidden before hdfs 2.8.0, where
> the traverse checks are called
> before reading and writing files, so if a policy is just about reading
> /tmp/somedir/somefile
> it means, that traverse should be allowed to get to that file. Adding
> more tests to highlight the issue
>
>
> Diffs
> -
>
> hdfs-agent/pom.xml 9f6206013
>
> hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java
> af4d9b5c2
>
> hdfs-agent/src/test/java/org/apache/ranger/services/hdfs/RangerHdfsAuthorizerTest.java
> PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/61062/diff/2/
>
>
> Testing
> ---
>
> Tested locally
> https://travis-ci.org/gzsombor/ranger/builds/256331500
>
>
> Thanks,
>
> Zsombor Gegesy
>
>
Re: Build issues - inconsistent
I can't reproduce the test failures in Kafka, HBase or Hive. However, I can reproduce the build errors in security-admin. It's caused by this bug in Maven: https://issues.apache.org/jira/browse/MNG-6298 The solution is to use Apache Maven 3.5.0 instead of 3.5.2. Colm. On Tue, Nov 21, 2017 at 5:19 PM, Nigel Jones <nigel.l.jo...@gmail.com> wrote: > Consistent in failing, but it's unclear how repeatable any specific error > is... I'll try some more - this was the test from just 4 attempts so I > wanted to check I was doing things correct. > > My invocation in each case is > mvn -DskipTests=false clean compile package install assembly:assembly > > Maven versions were > a) 3.5.2 (local mac) > b) 3.5.0 (azure ubuntu 17.10) > c) 3.3.9 (cloudfoundry/openstack ubuntu 16.04) > > We hit funny maven issues in Atlas - jenkins is now at 3.5.0. I proposed a > validation check in ATLAS-2245 to persuade everyone to upgrade. > > I'm assuming lots of people here use macOS. Whilst my OS level is forward > I doubt that's the issue. Is everyone still building with java 1.7? And > what version of maven? Any other dependent components? > > Thanks > Nigel. > > On 2017-11-21 15:48, Colm O hEigeartaigh <cohei...@apache.org> wrote: > > What maven version are you using? Are the test failures consistent or > > sporadic? > > > > Colm. > > > > On Tue, Nov 21, 2017 at 3:09 PM, Nigel Jones <nigel.l.jo...@gmail.com> > > wrote: > > > > > I'm experiencing some issues getting the ranger build to work. > > > a) macOS (10.13.2 b4), oracle 8 *(152) > > > > > > [INFO] HBase Security Plugin .. FAILURE > [11:37 > > > min] > > > > > > [INFO] Running org.apache.ranger.authorization.hbase. > > > HBaseRangerAuthorizationTest > > > Formatting using clusterid: testClusterID > > > [ERROR] Tests run: 20, Failures: 0, Errors: 1, Skipped: 0, Time > elapsed: > > > 690.138 s <<< FAILURE! - in org.apache.ranger.authorization.hbase. > > > HBaseRangerAuthorizationTest > > > [ERROR] testTagBasedTablePolicy(org.apache.ranger.authorization.hbase. > HBaseRangerAuthorizationTest) > > > Time elapsed: 612.923 s <<< ERROR! > > > org.apache.hadoop.hbase.exceptions.TimeoutIOException: > > > java.util.concurrent.TimeoutException: The procedure 18 is still > running > > > at org.apache.ranger.authorization.hbase. > > > HBaseRangerAuthorizationTest.testTagBasedTablePolicy( > > > HBaseRangerAuthorizationTest.java:780) > > > Caused by: java.util.concurrent.TimeoutException: The procedure 18 is > > > still running > > > at org.apache.ranger.authorization.hbase. > > > HBaseRangerAuthorizationTest.testTagBasedTablePolicy( > > > HBaseRangerAuthorizationTest.java:780) > > > > > > > > > >> This looks like potentially a timeout, though at 690s maybe in > reality > > > a logical issue > > > > > > b) Ubuntu 17.10, MS Azure, openjdk 8 (151) > > > > > > [INFO] Hive Security Plugin ... FAILURE [ > > > 41.084 s] > > > ERROR StatusLogger No log4j2 configuration file found. Using default > > > configuration: logging only errors to the console. > > > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > > > 30.286 s <<< FAILURE! - in org.apache.ranger.services. > > > hive.HIVERangerAuthorizerTest > > > [ERROR] org.apache.ranger.services.hive.HIVERangerAuthorizerTest Time > > > elapsed: 30.286 s <<< ERROR! > > > java.sql.SQLException: Could not open client transport with JDBC Uri: > > > jdbc:hive2://localhost:36781: java.net.ConnectException: Connection > refused > > > (Connection refused) > > > at org.apache.ranger.services.hive.HIVERangerAuthorizerTest. > setup( > > > HIVERangerAuthorizerTest.java:103) > > > Caused by: org.apache.thrift.transport.TTransportException: > > > java.net.ConnectException: Connection refused (Connection refused) > > > at org.apache.ranger.services.hive.HIVERangerAuthorizerTest. > setup( > > > HIVERangerAuthorizerTest.java:103) > > > Caused by: java.net.ConnectException: Connection refused (Connection > > > refused) > > > at org.apache.ranger.services.hive.HIVERangerAuthorizerTest. > setup( > > > HIVERangerAuthorizerTest.java:103) > > > > > > > > > >> This one is clearly a jdbc connection issue, perhaps hive didn't > start > >
Re: Review Request 63981: RANGER-1903:Simplify the maven dependency management of the Hdfs plugin for Ranger.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63981/#review191791 --- Ship it! Ship It! - Colm O hEigeartaigh On Nov. 23, 2017, 3:19 a.m., pengjianhua wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63981/ > --- > > (Updated Nov. 23, 2017, 3:19 a.m.) > > > Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan > Neethiraj, Velmurugan Periasamy, and Qiang Zhang. > > > Bugs: RANGER-1903 > https://issues.apache.org/jira/browse/RANGER-1903 > > > Repository: ranger > > > Description > --- > > The hdfs-agent need not dependency the hbase-server ,hive-common and > hive-service jar. So they should be removed from the pom.xml file of the > hdfs-agent. > > > Diffs > - > > hdfs-agent/pom.xml 9f62060 > > > Diff: https://reviews.apache.org/r/63981/diff/2/ > > > Testing > --- > > I had carefully tested the hdfs plugin, hive plugin and hbase plugin after > modified the issue. > > > Thanks, > > pengjianhua > >
Re: Review Request 63987: RANGER-1905 : fix maven assembly creation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63987/#review191793 --- How can I reproduce the error as reported on the JIRA? I unzipped the Ranger admin + installed and started it without any errors in the logs. - Colm O hEigeartaigh On Nov. 22, 2017, 12:23 p.m., Zsombor Gegesy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63987/ > --- > > (Updated Nov. 22, 2017, 12:23 p.m.) > > > Review request for ranger. > > > Bugs: RANGER-1905 > https://issues.apache.org/jira/browse/RANGER-1905 > > > Repository: ranger > > > Description > --- > > As ranger-plugins-audit doesn't exclude guava from hadoop common, an older > guava > is leaked into embeddedwebserver classpath, which somehow filtered out by the > assembly plugin. > If we explicitly exclude the old guava, and include the new (17.0) guava in > ranger-plugins-audit module, the generated tar.gz/zip distributions will > contain the necessary guava-17.0.jar in ews/lib path. > > > Diffs > - > > agents-audit/pom.xml c8bd1d8f3 > src/main/assembly/plugin-atlas.xml fd988116d > > > Diff: https://reviews.apache.org/r/63987/diff/2/ > > > Testing > --- > > Tested with executing: > > mvn -DskipTests package assembly:assembly; > unzip -Z1 target/ranger-1.0.0-SNAPSHOT-admin.zip | sort > FILES.txt > > And comparing the results with different versions. With the patch, the guava > jar appears correctly. > > > Thanks, > > Zsombor Gegesy > >
[jira] [Updated] (RANGER-1895) Simplify Storm dependencies
[ https://issues.apache.org/jira/browse/RANGER-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1895: Attachment: 0001-RANGER-1895-Simplify-Storm-dependencies.patch Also removing unused deps from the shim > Simplify Storm dependencies > --- > > Key: RANGER-1895 > URL: https://issues.apache.org/jira/browse/RANGER-1895 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh >Priority: Minor > Fix For: 1.0.0 > > Attachments: 0001-RANGER-1895-Simplify-Storm-dependencies.patch > > > This task is to simplify the Apache Storm dependencies for Ranger. We are > shipping the hadoop-hdfs jar, which is not required. Secondly, we can avoid > explicitly listing some of the dependencies, as they get pulled in via other > dependencies. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RANGER-1895) Simplify Storm dependencies
[ https://issues.apache.org/jira/browse/RANGER-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1895: Attachment: (was: 0001-RANGER-1895-Simplify-Storm-dependencies.patch) > Simplify Storm dependencies > --- > > Key: RANGER-1895 > URL: https://issues.apache.org/jira/browse/RANGER-1895 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh >Priority: Minor > Fix For: 1.0.0 > > Attachments: 0001-RANGER-1895-Simplify-Storm-dependencies.patch > > > This task is to simplify the Apache Storm dependencies for Ranger. We are > shipping the hadoop-hdfs jar, which is not required. Secondly, we can avoid > explicitly listing some of the dependencies, as they get pulled in via other > dependencies. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Review Request 63919: RANGER-1895 - Simplify Storm dependencies
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63919/ --- (Updated Nov. 23, 2017, 12:02 p.m.) Review request for ranger. Bugs: RANGER-1895 https://issues.apache.org/jira/browse/RANGER-1895 Repository: ranger Description --- This task is to simplify the Apache Storm dependencies for Ranger. We are shipping the hadoop-hdfs jar, which is not required. Secondly, we can avoid explicitly listing some of the dependencies, as they get pulled in via other dependencies. Diffs (updated) - ranger-storm-plugin-shim/pom.xml de1972d4 src/main/assembly/storm-agent.xml 64224ec7 storm-agent/pom.xml 6e74e5b3 Diff: https://reviews.apache.org/r/63919/diff/2/ Changes: https://reviews.apache.org/r/63919/diff/1-2/ Testing --- Tested the plugin works OK with Apache Storm 1.1.1. The only change in the distribution is that it doesn't include the Hadoop HDFS jar. Thanks, Colm O hEigeartaigh
Review Request 64051: RANGER-1906 - Simplify Atlas plugin dependency management
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64051/ --- Review request for ranger. Bugs: RANGER-1906 https://issues.apache.org/jira/browse/RANGER-1906 Repository: ranger Description --- This task is to simplify the dependency management for the Atlas plugin. In particular, the hadoop/solr dependencies should just be imported via the ranger-plugin-commons dependency. No changes are made to the resulting jars for the Atlas distribution. Diffs - plugin-atlas/pom.xml 957b4ce3 ranger-atlas-plugin-shim/pom.xml a207d16b src/main/assembly/plugin-atlas.xml fd988116 Diff: https://reviews.apache.org/r/64051/diff/1/ Testing --- Thanks, Colm O hEigeartaigh
[jira] [Updated] (RANGER-1906) Simplify Atlas plugin dependency management
[ https://issues.apache.org/jira/browse/RANGER-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1906: Attachment: 0001-RANGER-1906-Simplify-Atlas-plugin-dependency-managem.patch > Simplify Atlas plugin dependency management > --- > > Key: RANGER-1906 > URL: https://issues.apache.org/jira/browse/RANGER-1906 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh >Priority: Minor > Fix For: 1.0.0 > > Attachments: > 0001-RANGER-1906-Simplify-Atlas-plugin-dependency-managem.patch > > > This task is to simplify the dependency management for the Atlas plugin. In > particular, the hadoop/solr dependencies should just be imported via the > ranger-plugin-commons dependency. > No changes are made to the resulting jars for the Atlas distribution. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Review Request 61062: RANGER-1707 : fix hdfs traverse check
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61062/#review191736 --- Ship it! Ship It! - Colm O hEigeartaigh On Nov. 22, 2017, 12:39 p.m., Zsombor Gegesy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61062/ > --- > > (Updated Nov. 22, 2017, 12:39 p.m.) > > > Review request for ranger. > > > Bugs: RANGER-1707 > https://issues.apache.org/jira/browse/RANGER-1707 > > > Repository: ranger > > > Description > --- > > Fix hdfs traverse check, which problem was hidden before hdfs 2.8.0, where > the traverse checks are called > before reading and writing files, so if a policy is just about reading > /tmp/somedir/somefile > it means, that traverse should be allowed to get to that file. Adding > more tests to highlight the issue > > > Diffs > - > > hdfs-agent/pom.xml 9f6206013 > > hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java > af4d9b5c2 > > hdfs-agent/src/test/java/org/apache/ranger/services/hdfs/RangerHdfsAuthorizerTest.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/61062/diff/3/ > > > Testing > --- > > Tested locally > https://travis-ci.org/gzsombor/ranger/builds/256331500 > > > Thanks, > > Zsombor Gegesy > >
[jira] [Created] (RANGER-1906) Simplify Atlas plugin dependency management
Colm O hEigeartaigh created RANGER-1906: --- Summary: Simplify Atlas plugin dependency management Key: RANGER-1906 URL: https://issues.apache.org/jira/browse/RANGER-1906 Project: Ranger Issue Type: Improvement Components: plugins Reporter: Colm O hEigeartaigh Assignee: Colm O hEigeartaigh Priority: Minor Fix For: 1.0.0 This task is to simplify the dependency management for the Atlas plugin. In particular, the hadoop/solr dependencies should just be imported via the ranger-plugin-commons dependency. No changes are made to the resulting jars for the Atlas distribution. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1908) Build (unit tests) failed in HBase Security Plugin
[ https://issues.apache.org/jira/browse/RANGER-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266633#comment-16266633 ] Colm O hEigeartaigh commented on RANGER-1908: - Hi Nigel, Could you try applying this patch locally and see if you still see the test failure? https://issues.apache.org/jira/browse/RANGER-1893 The admin.disableTable is synchronous, and so the table should be disabled after the call returns, so this looks to me like a bug possibly in the old version of HBase we are using. > Build (unit tests) failed in HBase Security Plugin > -- > > Key: RANGER-1908 > URL: https://issues.apache.org/jira/browse/RANGER-1908 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Nigel Jones > Attachments: ranger-20171124a.log > > > MacOS > Java 1.8 (152, oracle) > Maven 3.5.0 > Build fails at > [INFO] HBase Security Plugin .. FAILURE [11:36 > min] > The failing test is: > [INFO] Running > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest > Formatting using clusterid: testClusterID > [ERROR] Tests run: 20, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > 689.962 s <<< FAILURE! - in > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest > [ERROR] > testTagBasedTablePolicy(org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest) > Time elapsed: 611.879 s <<< ERROR! > org.apache.hadoop.hbase.exceptions.TimeoutIOException: > java.util.concurrent.TimeoutException: The procedure 18 is still running > at > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.testTagBasedTablePolicy(HBaseRangerAuthorizationTest.java:780) > Caused by: java.util.concurrent.TimeoutException: The procedure 18 is still > running > at > org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.testTagBasedTablePolicy(HBaseRangerAuthorizationTest.java:780) > The next part of the test is ok: > [INFO] Running > org.apache.ranger.authorization.hbase.RangerAuthorizationFilterTest > [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.505 > s - in org.apache.ranger.authorization.hbase.RangerAuthorizationFilterTest -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Review Request 64051: RANGER-1906 - Simplify Atlas plugin dependency management
> On Nov. 30, 2017, 6:52 a.m., Mehul Parikh wrote: > > @Colm : Can you please confirm if Audit to HDFS and Audit to Solr are > > working after removal of these dependencies? Hi Mehul, I haven't actually removed any dependencies with this patch as such - the distribution jars are exactly the same. The Solr jar is bundled via the agents-common dependency, and the Atlas plugin takes the Hadoop jars from Atlas itself instead of via the lib directory in the distribution. - Colm --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64051/#review192242 --- On Nov. 28, 2017, 11:44 a.m., Colm O hEigeartaigh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64051/ > --- > > (Updated Nov. 28, 2017, 11:44 a.m.) > > > Review request for ranger. > > > Bugs: RANGER-1906 > https://issues.apache.org/jira/browse/RANGER-1906 > > > Repository: ranger > > > Description > --- > > This task is to simplify the dependency management for the Atlas plugin. In > particular, the hadoop/solr dependencies should just be imported via the > ranger-plugin-commons dependency. > > No changes are made to the resulting jars for the Atlas distribution. > > > Diffs > - > > plugin-atlas/pom.xml 957b4ce3 > ranger-atlas-plugin-shim/pom.xml a207d16b > src/main/assembly/plugin-atlas.xml fd988116 > > > Diff: https://reviews.apache.org/r/64051/diff/2/ > > > Testing > --- > > > Thanks, > > Colm O hEigeartaigh > >
Re: Review Request 64228: Traverse check in RangerHdfsAuthorizer works incorrectly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64228/#review192459 --- LGTM. Zsombor? - Colm O hEigeartaigh On Nov. 30, 2017, 10:38 p.m., Abhay Kulkarni wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64228/ > --- > > (Updated Nov. 30, 2017, 10:38 p.m.) > > > Review request for ranger, Colm O hEigeartaigh, Zsombor Gegesy, Madhan > Neethiraj, Ramesh Mani, and Velmurugan Periasamy. > > > Bugs: RANGER-1707 > https://issues.apache.org/jira/browse/RANGER-1707 > > > Repository: ranger > > > Description > --- > > Traversal check in RangerHdfsAuthorizer works incorrectly, when it is asked > for access to /a/b/c.txt, it only checks that if there are a policy which > grants EXEC to /a/b, but if it there aren't any, then it doesn't check, if > there is a policy which grants READ, WRITE or EXEC to /a/b/c.txt explicitly, > which would mean, that the path is accessible to the user. > This hasn't noticed by the current unit tests, because HDFS before 2.8.0 > doesn't called the traversal check before reading or writing a file, however > it will cause problem with 2.8.0, where FSDirectory.resolvePath will perform > a mandatory traversal check. > > This patch is based on the patch submitted for review > (https://reviews.apache.org/r/61062/) with following modifications. > 1. If traversal check (check for EXECUTE on the parent/ancestor if resource > is a file) does not fail with explicit DENY by Ranger Authorizer, then it is > presumed to have succeeded without any further checks and no audit record > created. If it fails with DENY, then the authorization fails and an audit > record is created. > 2. Test policies in hdfs-policies.json and test cases > (RangerHdfsAuthorizerTest) are modified to test for explicit DENY case. > > > Diffs > - > > hdfs-agent/pom.xml 87ba777 > > hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java > af4d9b5 > > hdfs-agent/src/test/java/org/apache/ranger/services/hdfs/RangerHdfsAuthorizerTest.java > PRE-CREATION > hdfs-agent/src/test/resources/hdfs-policies.json 056231f > > > Diff: https://reviews.apache.org/r/64228/diff/1/ > > > Testing > --- > > Unit tested with HDFS versions 2.7.1 and 3.0.0. > > > Thanks, > > Abhay Kulkarni > >
Re: Review Request 64051: RANGER-1906 - Simplify Atlas plugin dependency management
> On Nov. 30, 2017, 6:52 a.m., Mehul Parikh wrote: > > @Colm : Can you please confirm if Audit to HDFS and Audit to Solr are > > working after removal of these dependencies? > > Colm O hEigeartaigh wrote: > Hi Mehul, > > I haven't actually removed any dependencies with this patch as such - the > distribution jars are exactly the same. The Solr jar is bundled via the > agents-common dependency, and the Atlas plugin takes the Hadoop jars from > Atlas itself instead of via the lib directory in the distribution. @Mehul, As a sanity test I've verified that Atlas auditing to Solr works as expected. - Colm --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64051/#review192242 --- On Nov. 28, 2017, 11:44 a.m., Colm O hEigeartaigh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64051/ > --- > > (Updated Nov. 28, 2017, 11:44 a.m.) > > > Review request for ranger. > > > Bugs: RANGER-1906 > https://issues.apache.org/jira/browse/RANGER-1906 > > > Repository: ranger > > > Description > --- > > This task is to simplify the dependency management for the Atlas plugin. In > particular, the hadoop/solr dependencies should just be imported via the > ranger-plugin-commons dependency. > > No changes are made to the resulting jars for the Atlas distribution. > > > Diffs > - > > plugin-atlas/pom.xml 957b4ce3 > ranger-atlas-plugin-shim/pom.xml a207d16b > src/main/assembly/plugin-atlas.xml fd988116 > > > Diff: https://reviews.apache.org/r/64051/diff/2/ > > > Testing > --- > > > Thanks, > > Colm O hEigeartaigh > >
Re: Review Request 63949: RANGER-1898 - Simplify Knox plugin dependency management
> On Nov. 28, 2017, 6:39 a.m., bhavik patel wrote: > > @Colm : Can you please confirm that audit to solr is working as expected? > > because you are also removing the ranger-plugins-audit dependency. > > Colm O hEigeartaigh wrote: > The ranger-plugins-audit dependency is on the classpath via the > agents-common module. However, I've removed that change just to make things > easier to review - as my main goal is to simplify third-party dependencies in > the poms. For the record, I have verified that auditing to Solr works correctly. - Colm --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63949/#review191998 --- On Nov. 28, 2017, 10:47 a.m., Colm O hEigeartaigh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63949/ > --- > > (Updated Nov. 28, 2017, 10:47 a.m.) > > > Review request for ranger. > > > Bugs: RANGER-1898 > https://issues.apache.org/jira/browse/RANGER-1898 > > > Repository: ranger > > > Description > --- > > This task is to simplify the dependency management for the Knox plugin for > Ranger. > > > Diffs > - > > knox-agent/pom.xml 52f15cd0 > > > Diff: https://reviews.apache.org/r/63949/diff/3/ > > > Testing > --- > > Tested that the distribution ships the same jars as before. Also tested a > use-case involving Ranger + Knox. > > > Thanks, > > Colm O hEigeartaigh > >
[jira] [Created] (RANGER-1927) Upgrade Hive to 2.3.2
Colm O hEigeartaigh created RANGER-1927: --- Summary: Upgrade Hive to 2.3.2 Key: RANGER-1927 URL: https://issues.apache.org/jira/browse/RANGER-1927 Project: Ranger Issue Type: Improvement Components: plugins Reporter: Colm O hEigeartaigh Assignee: Colm O hEigeartaigh Priority: Minor Fix For: 1.0.0 We should upgrade Hive to a more recent release. Hive 2.3.2 is backwards compatible to 2.1.0 as far as Ranger is concerned, so no other changes are required. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Review Request 64525: RANGER-1924:Simplify ranger-hdfs plugin remove gson.jar in ranger/src/main/assembly/hdfs-agent.xml
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64525/#review193662 --- Ship it! Ship It! - Colm O hEigeartaigh On Dec. 13, 2017, 1:49 a.m., pengjianhua wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64525/ > --- > > (Updated Dec. 13, 2017, 1:49 a.m.) > > > Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan > Neethiraj, Velmurugan Periasamy, and Qiang Zhang. > > > Bugs: RANGER-1924 > https://issues.apache.org/jira/browse/RANGER-1924 > > > Repository: ranger > > > Description > --- > > Simplify ranger-hdfs plugin remove gson.jar in > ranger/src/main/assembly/hdfs-agent.xml and ranger/hdfs-agent/pom.xml > and Simplify ranger-hive plugin remove gson.jar in > ranger/src/main/assembly/hive-agent.xml and ranger/hive-agent/pom.xml > Description: > Remove com.google.code.gson:gson* in > ranger/src/main/assembly/hdfs-agent.xml > Remove com.google.code.gson:gson* in > ranger/src/main/assembly/hive-agent.xml > Remove " > com.google.code.gson > gson > " from ranger/hdfs-agent/pom.xml and > ranger/hive-agent/pom.xml > > Reason: > 1.Hadoop already has gson.jar removed from gson.jar to prevent conflicts > 2.Observing ranger-yarn plugin has also removed gson.jar in > ranger/src/main/assembly/plugin-yarn.xml. > 3.Hive already has gson.jar removed from gson.jar to prevent conflicts > > Result: the ranger-hdfs plugin also works fine after removing gson.jar. > the ranger-hive plugin also works fine after removing gson.jar > > > Diffs > - > > hdfs-agent/pom.xml 0b6773d > hive-agent/pom.xml c195443 > src/main/assembly/hdfs-agent.xml 5279a9a > src/main/assembly/hive-agent.xml ca65c80 > > > Diff: https://reviews.apache.org/r/64525/diff/2/ > > > Testing > --- > > Tested it. > > > Thanks, > > pengjianhua > >
[jira] [Updated] (RANGER-1927) Upgrade Hive to 2.3.2
[ https://issues.apache.org/jira/browse/RANGER-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1927: Attachment: 0001-RANGER-1927-Upgrade-Hive-to-2.3.2.patch > Upgrade Hive to 2.3.2 > - > > Key: RANGER-1927 > URL: https://issues.apache.org/jira/browse/RANGER-1927 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh >Priority: Minor > Fix For: 1.0.0 > > Attachments: 0001-RANGER-1927-Upgrade-Hive-to-2.3.2.patch > > > We should upgrade Hive to a more recent release. Hive 2.3.2 is backwards > compatible to 2.1.0 as far as Ranger is concerned, so no other changes are > required. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Review Request 64572: RANGER-1927 - Upgrade Hive to 2.3.2
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64572/ --- Review request for ranger. Bugs: RANGER-1927 https://issues.apache.org/jira/browse/RANGER-1927 Repository: ranger Description --- We should upgrade Hive to a more recent release. Hive 2.3.2 is backwards compatible to 2.1.0 as far as Ranger is concerned, so no other changes are required. Diffs - pom.xml 3086a1e5 Diff: https://reviews.apache.org/r/64572/diff/1/ Testing --- Not required, as no changes are made to the distribution. Thanks, Colm O hEigeartaigh
Moving master to Java 1.8?
Hi all, Before I fix the currently failing Hive tests with JDK 7, I'm wondering if it's time to move Ranger master to require Java 1.8. Apache Hadoop 3.0.0 requires Java 1.8 - so we will have to make the switch if we want to pick up Hadoop 3.0.0 for Ranger 1.0.0. Apache Knox 0.14.0 is also Java 8 only, so we can't upgrade to that as things stand. Thoughts? Colm. -- Colm O hEigeartaigh Talend Community Coder http://coders.talend.com
Review Request 64549: RANGER-1926 - Upgrade Storm dependency to 1.1.1
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64549/ --- Review request for ranger. Bugs: RANGER-1926 https://issues.apache.org/jira/browse/RANGER-1926 Repository: ranger Description --- This task is to upgrade the Storm dependency to 1.1.1. No further changes are required, as it's backwards compatible with 1.0.x. Diffs - pom.xml 3086a1e5 Diff: https://reviews.apache.org/r/64549/diff/1/ Testing --- Not required as no changes are made to the distribution jars. Thanks, Colm O hEigeartaigh
[jira] [Updated] (RANGER-1926) Upgrade Storm dependency to 1.1.1
[ https://issues.apache.org/jira/browse/RANGER-1926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1926: Attachment: 0001-RANGER-1926-Upgrade-Storm-dependency-to-1.1.1.patch > Upgrade Storm dependency to 1.1.1 > - > > Key: RANGER-1926 > URL: https://issues.apache.org/jira/browse/RANGER-1926 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh >Priority: Minor > Fix For: 1.0.0 > > Attachments: 0001-RANGER-1926-Upgrade-Storm-dependency-to-1.1.1.patch > > > This task is to upgrade the Storm dependency to 1.1.1. No further changes are > required, as it's backwards compatible with 1.0.x. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Review Request 64525: RANGER-1924:Simplify ranger-hdfs plugin remove gson.jar in ranger/src/main/assembly/hdfs-agent.xml
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64525/#review193578 --- The Hive plugin also ships gson, which looks like it could be removed as well. - Colm O hEigeartaigh On Dec. 12, 2017, 2:43 a.m., pengjianhua wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64525/ > --- > > (Updated Dec. 12, 2017, 2:43 a.m.) > > > Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan > Neethiraj, Velmurugan Periasamy, and Qiang Zhang. > > > Bugs: RANGER-1924 > https://issues.apache.org/jira/browse/RANGER-1924 > > > Repository: ranger > > > Description > --- > > Simplify ranger-hdfs plugin remove gson.jar in > ranger/src/main/assembly/hdfs-agent.xml > Description: > Remove com.google.code.gson:gson* in > ranger/src/main/assembly/hdfs-agent.xml > Reason: > 1.Hadoop already has gson.jar removed from gson.jar to prevent conflicts > 2.Observing ranger-yarn plugin has also removed gson.jar in > ranger/src/main/assembly/plugin-yarn.xml. > Result: the ranger-hdfs plugin also works fine after removing gson.jar. > > > Diffs > - > > src/main/assembly/hdfs-agent.xml 5279a9a > > > Diff: https://reviews.apache.org/r/64525/diff/1/ > > > Testing > --- > > Tested it. > > > Thanks, > > pengjianhua > >
[jira] [Created] (RANGER-1926) Upgrade Storm dependency to 1.1.1
Colm O hEigeartaigh created RANGER-1926: --- Summary: Upgrade Storm dependency to 1.1.1 Key: RANGER-1926 URL: https://issues.apache.org/jira/browse/RANGER-1926 Project: Ranger Issue Type: Improvement Components: plugins Reporter: Colm O hEigeartaigh Assignee: Colm O hEigeartaigh Priority: Minor Fix For: 1.0.0 This task is to upgrade the Storm dependency to 1.1.1. No further changes are required, as it's backwards compatible with 1.0.x. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Review Request 64525: RANGER-1924:Simplify ranger-hdfs plugin remove gson.jar in ranger/src/main/assembly/hdfs-agent.xml
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64525/#review193580 --- And also, you could remove the gson dependency from the hdfs-plugin pom. - Colm O hEigeartaigh On Dec. 12, 2017, 2:43 a.m., pengjianhua wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64525/ > --- > > (Updated Dec. 12, 2017, 2:43 a.m.) > > > Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan > Neethiraj, Velmurugan Periasamy, and Qiang Zhang. > > > Bugs: RANGER-1924 > https://issues.apache.org/jira/browse/RANGER-1924 > > > Repository: ranger > > > Description > --- > > Simplify ranger-hdfs plugin remove gson.jar in > ranger/src/main/assembly/hdfs-agent.xml > Description: > Remove com.google.code.gson:gson* in > ranger/src/main/assembly/hdfs-agent.xml > Reason: > 1.Hadoop already has gson.jar removed from gson.jar to prevent conflicts > 2.Observing ranger-yarn plugin has also removed gson.jar in > ranger/src/main/assembly/plugin-yarn.xml. > Result: the ranger-hdfs plugin also works fine after removing gson.jar. > > > Diffs > - > > src/main/assembly/hdfs-agent.xml 5279a9a > > > Diff: https://reviews.apache.org/r/64525/diff/1/ > > > Testing > --- > > Tested it. > > > Thanks, > > pengjianhua > >
[jira] [Updated] (RANGER-1935) Upgrade Ranger to support Apache Hadoop 3.0.0
[ https://issues.apache.org/jira/browse/RANGER-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1935: Attachment: 0001-RANGER-1935-Upgrade-Ranger-to-support-Apache-Hadoop-.patch > Upgrade Ranger to support Apache Hadoop 3.0.0 > - > > Key: RANGER-1935 > URL: https://issues.apache.org/jira/browse/RANGER-1935 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh > Fix For: 1.0.0 > > Attachments: > 0001-RANGER-1935-Upgrade-Ranger-to-support-Apache-Hadoop-.patch > > > This task is to upgrade Ranger to support Apache Hadoop 3.0.0. Here are some > notes about the upgrade: > a) The Hive plugin needs the Hadoop 3.0.0 jars to run the tests properly, as > Hive only supports the older Hadoop version, so an exclusion and some > additional 3.0.0 dependencies need to be added. > b) The Storm plugin bundles the hadoop-auth jars in storm-core (although they > really should be renamed here). Therefore, we have no option but to package > Storm with the Hadoop 2.7.x jars, until such time that Storm upgrades the > Hadoop dependency. > This is an initial patch to get some feedback. If there is broad agreement on > the upgrade I will test the distributions properly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (RANGER-1935) Upgrade Ranger to support Apache Hadoop 3.0.0
Colm O hEigeartaigh created RANGER-1935: --- Summary: Upgrade Ranger to support Apache Hadoop 3.0.0 Key: RANGER-1935 URL: https://issues.apache.org/jira/browse/RANGER-1935 Project: Ranger Issue Type: Improvement Components: plugins Reporter: Colm O hEigeartaigh Assignee: Colm O hEigeartaigh Fix For: 1.0.0 This task is to upgrade Ranger to support Apache Hadoop 3.0.0. Here are some notes about the upgrade: a) The Hive plugin needs the Hadoop 3.0.0 jars to run the tests properly, as Hive only supports the older Hadoop version, so an exclusion and some additional 3.0.0 dependencies need to be added. b) The Storm plugin bundles the hadoop-auth jars in storm-core (although they really should be renamed here). Therefore, we have no option but to package Storm with the Hadoop 2.7.x jars, until such time that Storm upgrades the Hadoop dependency. This is an initial patch to get some feedback. If there is broad agreement on the upgrade I will test the distributions properly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Review Request 64677: RANGER-1934:Optimize the init method in BaseAuditHandler class to avoid ArrayIndexOutOfBoundsException
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64677/#review194136
---
Better to have "!tokens.isEmpty()" than "tokens.size() > 0". Also, the
indentation inside the if statement is wrong "if (providerName == null) {".
- Colm O hEigeartaigh
On Dec. 19, 2017, 1:40 a.m., pengjianhua wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64677/
> ---
>
> (Updated Dec. 19, 2017, 1:40 a.m.)
>
>
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan
> Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
>
>
> Bugs: RANGER-1934
> https://issues.apache.org/jira/browse/RANGER-1934
>
>
> Repository: ranger
>
>
> Description
> ---
>
> Optimize the init method in BaseAuditHandler class to avoid
> ArrayIndexOutOfBoundsException
> The follow in the init method " List tokens =
> MiscUtil.toArray(propPrefix, ".");
> String finalToken = tokens.get(tokens.size() - 1);".
> in the init method we should add " if (tokens.size() > 1)" to avoid
> ArrayIndexOutOfBoundsException.
>
>
> Diffs
> -
>
>
> agents-audit/src/main/java/org/apache/ranger/audit/provider/BaseAuditHandler.java
> b095000
>
>
> Diff: https://reviews.apache.org/r/64677/diff/2/
>
>
> Testing
> ---
>
> Tested it.
>
>
> Thanks,
>
> pengjianhua
>
>
Re: Review Request 59523: RANGER-689 - "For Solr plugin, use resources folders for adding Ranger properties".
> On Aug. 11, 2017, 11:55 a.m., Colm O hEigeartaigh wrote: > > *bump* > > Colm O hEigeartaigh wrote: > Requesting review, please. Could someone take a look please? - Colm --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59523/#review182682 --- On May 24, 2017, 11:08 a.m., Colm O hEigeartaigh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59523/ > --- > > (Updated May 24, 2017, 11:08 a.m.) > > > Review request for ranger. > > > Bugs: RANGER-689 > https://issues.apache.org/jira/browse/RANGER-689 > > > Repository: ranger > > > Description > --- > > Put the Ranger properties into the SOLR resource folder instead of the webapp. > > > Diffs > - > > agents-common/scripts/enable-agent.sh 76ba8f0d > > > Diff: https://reviews.apache.org/r/59523/diff/1/ > > > Testing > --- > > Tested with Solr 6.5.1. > > > Thanks, > > Colm O hEigeartaigh > >
Re: Review Request 64677: RANGER-1934:Optimize the init method in BaseAuditHandler class to avoid ArrayIndexOutOfBoundsException
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64677/#review194223 --- Ship it! Ship It! - Colm O hEigeartaigh On Dec. 20, 2017, 5:54 a.m., pengjianhua wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64677/ > --- > > (Updated Dec. 20, 2017, 5:54 a.m.) > > > Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan > Neethiraj, Velmurugan Periasamy, and Qiang Zhang. > > > Bugs: RANGER-1934 > https://issues.apache.org/jira/browse/RANGER-1934 > > > Repository: ranger > > > Description > --- > > Optimize the init method in BaseAuditHandler class to avoid > ArrayIndexOutOfBoundsException > The follow in the init method " List tokens = > MiscUtil.toArray(propPrefix, "."); > String finalToken = tokens.get(tokens.size() - 1);". > in the init method we should add " if (tokens.size() > 1)" to avoid > ArrayIndexOutOfBoundsException. > > > Diffs > - > > > agents-audit/src/main/java/org/apache/ranger/audit/provider/BaseAuditHandler.java > b095000 > > > Diff: https://reviews.apache.org/r/64677/diff/3/ > > > Testing > --- > > Tested it. > > > Thanks, > > pengjianhua > >
[jira] [Created] (RANGER-1940) Upgrade to Knox 0.14.0
Colm O hEigeartaigh created RANGER-1940: --- Summary: Upgrade to Knox 0.14.0 Key: RANGER-1940 URL: https://issues.apache.org/jira/browse/RANGER-1940 Project: Ranger Issue Type: Improvement Components: plugins Reporter: Colm O hEigeartaigh Assignee: Colm O hEigeartaigh Fix For: 1.0.0 This task is to upgrade to Knox 0.14.0. We can take advantage of the changes made to the GatewayTestDriver to simplify the test configuration as a result. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Review Request 64634: RANGER-1929:The ranger should support the View policy.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64634/#review194225 --- Ship it! Ship It! - Colm O hEigeartaigh On Dec. 20, 2017, 6:23 a.m., pengjianhua wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64634/ > --- > > (Updated Dec. 20, 2017, 6:23 a.m.) > > > Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan > Neethiraj, Velmurugan Periasamy, and Qiang Zhang. > > > Bugs: RANGER-1929 > https://issues.apache.org/jira/browse/RANGER-1929 > > > Repository: ranger > > > Description > --- > > Currently we can only edit the policy without viewing the policy. We must use > editing funtion of policy when only need to query the detail for policy. So > we should supply the function of the query detail for policy. > > > Diffs > - > > security-admin/src/main/webapp/scripts/modules/globalize/message/en.js > b8de5c3ba > > security-admin/src/main/webapp/scripts/views/policies/RangerPolicyConditions.js > PRE-CREATION > security-admin/src/main/webapp/scripts/views/policies/RangerPolicyDetail.js > PRE-CREATION > > security-admin/src/main/webapp/scripts/views/policies/RangerPolicyTableLayout.js > 09e2e1669 > security-admin/src/main/webapp/styles/xa.css 22eedf644 > > security-admin/src/main/webapp/templates/policies/RangerPolicyConditions_tmpl.html > PRE-CREATION > > security-admin/src/main/webapp/templates/policies/RangerPolicyDetail_tmpl.html > PRE-CREATION > > > Diff: https://reviews.apache.org/r/64634/diff/5/ > > > Testing > --- > > > Thanks, > > pengjianhua > >
Review Request 64746: RANGER-1940 - Upgrade to Knox 0.14.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64746/ --- Review request for ranger. Bugs: RANGER-1940 https://issues.apache.org/jira/browse/RANGER-1940 Repository: ranger Description --- This task is to upgrade to Knox 0.14.0. We can take advantage of the changes made to the GatewayTestDriver to simplify the test configuration as a result. Diffs - knox-agent/src/test/java/org/apache/ranger/services/knox/KnoxRangerTest.java 53e66df4 pom.xml 255b02aa Diff: https://reviews.apache.org/r/64746/diff/1/ Testing --- Thanks, Colm O hEigeartaigh
[jira] [Updated] (RANGER-1940) Upgrade to Knox 0.14.0
[ https://issues.apache.org/jira/browse/RANGER-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1940: Attachment: 0001-RANGER-1940-Upgrade-to-Knox-0.14.0.patch > Upgrade to Knox 0.14.0 > -- > > Key: RANGER-1940 > URL: https://issues.apache.org/jira/browse/RANGER-1940 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh > Fix For: 1.0.0 > > Attachments: 0001-RANGER-1940-Upgrade-to-Knox-0.14.0.patch > > > This task is to upgrade to Knox 0.14.0. We can take advantage of the changes > made to the GatewayTestDriver to simplify the test configuration as a result. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (RANGER-1936) Upgrade Ranger to use Java 1.8
Colm O hEigeartaigh created RANGER-1936: --- Summary: Upgrade Ranger to use Java 1.8 Key: RANGER-1936 URL: https://issues.apache.org/jira/browse/RANGER-1936 Project: Ranger Issue Type: Improvement Components: admin, plugins Reporter: Colm O hEigeartaigh Assignee: Colm O hEigeartaigh Fix For: 1.0.0 As discussed on the mailing list, this task is to upgrade Ranger to use Java 1.8. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RANGER-1936) Upgrade Ranger to use Java 1.8
[ https://issues.apache.org/jira/browse/RANGER-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1936: Attachment: 0001-RANGER-1936-Upgrade-Ranger-to-use-Java-1.8.patch > Upgrade Ranger to use Java 1.8 > -- > > Key: RANGER-1936 > URL: https://issues.apache.org/jira/browse/RANGER-1936 > Project: Ranger > Issue Type: Improvement > Components: admin, plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh > Fix For: 1.0.0 > > Attachments: 0001-RANGER-1936-Upgrade-Ranger-to-use-Java-1.8.patch > > > As discussed on the mailing list, this task is to upgrade Ranger to use Java > 1.8. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Review Request 64708: RANGER-1936 - Upgrade Ranger to use Java 1.8
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64708/ --- Review request for ranger. Bugs: RANGER-1936 https://issues.apache.org/jira/browse/RANGER-1936 Repository: ranger Description --- As discussed on the mailing list, this task is to upgrade Ranger to use Java 1.8. Diffs - pom.xml a537a65a Diff: https://reviews.apache.org/r/64708/diff/1/ Testing --- Thanks, Colm O hEigeartaigh
Re: Re: Moving master to Java 1.8?
OK thanks all, as there is consensus I've updated the Jenkins job to use Java 1.8 and submitted the following patch: https://issues.apache.org/jira/browse/RANGER-1936 Colm. On Tue, Dec 19, 2017 at 1:17 AM, <peng.jian...@zte.com.cn> wrote: > +1 > > > > > > > Jianhua Peng > 原始邮件 > *发件人：* <sneet...@apache.org>; > *收件人：* <dev@ranger.apache.org>; > *抄送人：* <cohei...@apache.org>; > *日 期 ：*2017年12月19日 05:00 > *主 题 ：**Re: Moving master to Java 1.8?* > +1 > > Thanks, > Selva- > > > On Dec 18, 2017, at 2:52 PM, Velmurugan Periasamy <v...@apache.org > > wrote: > > > > Colm this is a good idea. +1 > > > > From: Ramesh Mani <rm...@hortonworks.com> > > Reply-To: "dev@ranger.apache.org" <dev@ranger.apache.org> > > Date: Monday, December 18, 2017 at 1:08 PM > > To: "dev@ranger.apache.org" <dev@ranger.apache.org>, "coh > ei...@apache.org" > > <cohei...@apache.org> > > Subject: Re: Moving master to Java 1.8? > > > > Colm, > > > > I having being thinking on this, when we move to Hbase 2. > 0 version we need > > Java 1.8 for the Ranger hbase plugin to compile. > > > > +1 for this. > > > > Thanks, > > Ramesh > > > > On 12/16/17, 10:18 AM, "Colm O hEigeartaigh" <cohei...@apache.org > > wrote: > > > >> Hi all, > >> > >> Before I fix the currently failing Hive tests with JDK 7, > I'm wondering if > >> it's time to move Ranger master to require Java 1.8. > Apache Hadoop 3.0.0 > >> requires Java 1.8 - so we will have to make the switch > if we want to pick > >> up Hadoop 3.0.0 for Ranger 1.0.0. Apache Knox 0.14.0 is > also Java 8 only, > >> so we can't upgrade to that as things stand. > >> > >> Thoughts? > >> > >> Colm. > >> > >> > >> -- > >> Colm O hEigeartaigh > >> > >> Talend Community Coder > >> http://coders.talend.com > > > > > > > > > > > -- Colm O hEigeartaigh Talend Community Coder http://coders.talend.com
Re: Review Request 63789: RANGER-1887:serviceDef.getResources().get(0).getName(); case IndexOutOfBoundsException in RangerServiceTag.class And print error in RangerServiceStorm
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63789/#review191312 --- The "String tagResourceName = null" definition could be moved into the if statement, e.g. String tagResourceName = serviceDef.getResources().get(0).getName(); - Colm O hEigeartaigh On Nov. 17, 2017, 6:16 a.m., Qiang Zhang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63789/ > --- > > (Updated Nov. 17, 2017, 6:16 a.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, > Selvamohan Neethiraj, sam rome, and Velmurugan Periasamy. > > > Bugs: RANGER-1887 > https://issues.apache.org/jira/browse/RANGER-1887 > > > Repository: ranger > > > Description > --- > > String tagResourceName = serviceDef.getResources().get(0).getName(); case > java.lang.IndexOutOfBoundsException in RangerServiceTag.class > And print log error for RangerServiceStorm.class > > > Diffs > - > > > agents-common/src/main/java/org/apache/ranger/services/tag/RangerServiceTag.java > ce6002b > > storm-agent/src/main/java/org/apache/ranger/services/storm/RangerServiceStorm.java > d4dbd7b > > > Diff: https://reviews.apache.org/r/63789/diff/3/ > > > Testing > --- > > Tested it > > > Thanks, > > Qiang Zhang > >
[jira] [Updated] (RANGER-1894) Fix HDFS tests to work with Hadoop 3.0.0
[ https://issues.apache.org/jira/browse/RANGER-1894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1894: Attachment: 0001-RANGER-1894-Fix-HDFS-tests-to-work-with-Hadoop-3.0.0.patch > Fix HDFS tests to work with Hadoop 3.0.0 > > > Key: RANGER-1894 > URL: https://issues.apache.org/jira/browse/RANGER-1894 > Project: Ranger > Issue Type: Test > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh > Fix For: 1.0.0 > > Attachments: > 0001-RANGER-1894-Fix-HDFS-tests-to-work-with-Hadoop-3.0.0.patch > > > The HDFS tests fail when run with Hadoop 3.0.0. This is due to a change in > permissions in Hadoop 3.0.0 - all operations (e.g. read/write) must also now > have an "execute" permission for all path components, apart from the last one. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Review Request 63908: RANGER-1894 - Fix HDFS tests to work with Hadoop 3.0.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63908/ --- Review request for ranger. Bugs: RANGER-1894 https://issues.apache.org/jira/browse/RANGER-1894 Repository: ranger Description --- The HDFS tests fail when run with Hadoop 3.0.0. This is due to a change in permissions in Hadoop 3.0.0 - all operations (e.g. read/write) must also now have an "execute" permission for all path components, apart from the last one. Diffs - hdfs-agent/src/test/java/org/apache/ranger/services/hdfs/HDFSRangerTest.java a7215ce9 hdfs-agent/src/test/resources/hdfs-policies.json 056231fc Diff: https://reviews.apache.org/r/63908/diff/1/ Testing --- Thanks, Colm O hEigeartaigh
[jira] [Created] (RANGER-1894) Fix HDFS tests to work with Hadoop 3.0.0
Colm O hEigeartaigh created RANGER-1894: --- Summary: Fix HDFS tests to work with Hadoop 3.0.0 Key: RANGER-1894 URL: https://issues.apache.org/jira/browse/RANGER-1894 Project: Ranger Issue Type: Test Components: plugins Reporter: Colm O hEigeartaigh Assignee: Colm O hEigeartaigh Fix For: 1.0.0 The HDFS tests fail when run with Hadoop 3.0.0. This is due to a change in permissions in Hadoop 3.0.0 - all operations (e.g. read/write) must also now have an "execute" permission for all path components, apart from the last one. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Review Request 63870: RANGER-1861:There are errors when the "ranger-kms start" command was executed.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63870/#review191172 --- How does your change prevent the error? From what I can see the change is that it doesn't load the DEFAULT_CONFIG_FILENAME if an argument is specified. - Colm O hEigeartaigh On Nov. 16, 2017, 10:03 a.m., pengjianhua wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63870/ > --- > > (Updated Nov. 16, 2017, 10:03 a.m.) > > > Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan > Neethiraj, Velmurugan Periasamy, and Qiang Zhang. > > > Bugs: RANGER-1861 > https://issues.apache.org/jira/browse/RANGER-1861 > > > Repository: ranger > > > Description > --- > > There are errors when the "ranger-kms start" command was executed. > > 2017-10-28 23:23:33,339 ERROR XMLUtils - Error loading : > java.lang.IllegalArgumentException: InputStream cannot be null > at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:117) > at org.apache.ranger.plugin.util.XMLUtils.loadConfig(XMLUtils.java:64) > at org.apache.ranger.plugin.util.XMLUtils.loadConfig(XMLUtils.java:48) > at > org.apache.ranger.server.tomcat.EmbeddedServer.(EmbeddedServer.java:77) > at > org.apache.ranger.server.tomcat.EmbeddedServer.main(EmbeddedServer.java:69) > > > Diffs > - > > > embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java > 40eed861 > > > Diff: https://reviews.apache.org/r/63870/diff/1/ > > > Testing > --- > > > Thanks, > > pengjianhua > >
[jira] [Updated] (RANGER-1849) Remove PasswordGenerator and FileStoreUtil
[ https://issues.apache.org/jira/browse/RANGER-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1849: Attachment: 0001-RANGER-1849-Remove-PasswordGenerator-and-FileStoreUt.patch > Remove PasswordGenerator and FileStoreUtil > -- > > Key: RANGER-1849 > URL: https://issues.apache.org/jira/browse/RANGER-1849 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh >Priority: Trivial > Fix For: 1.0.0 > > Attachments: > 0001-RANGER-1849-Remove-PasswordGenerator-and-FileStoreUt.patch > > > PasswordGenerator is not used in any of the scripts and should be removed. > Same goes for FileStoreUtil. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Review Request 63182: RANGER-1849 - Remove PasswordGenerator and FileStoreUtil
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63182/ --- (Updated Nov. 16, 2017, 12:18 p.m.) Review request for ranger. Bugs: RANGER-1849 https://issues.apache.org/jira/browse/RANGER-1849 Repository: ranger Description --- PasswordGenerator is not used in any of the scripts and should be removed. Same goes for FileStoreUtil. Diffs (updated) - agents-common/src/main/java/org/apache/ranger/plugin/store/file/FileStoreUtil.java 3f408d76 agents-installer/src/main/java/org/apache/ranger/utils/install/PasswordGenerator.java a829957a Diff: https://reviews.apache.org/r/63182/diff/2/ Changes: https://reviews.apache.org/r/63182/diff/1-2/ Testing --- Thanks, Colm O hEigeartaigh
[jira] [Commented] (RANGER-1738) RangerYarnAuthorizer not compatible with Hadoop-3.0.0
[
https://issues.apache.org/jira/browse/RANGER-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255203#comment-16255203
]
Colm O hEigeartaigh commented on RANGER-1738:
-
The problem with switching Ranger to Hadoop 3.0.0, is that a lot of the
components rely on Hadoop 2.7.x, and it ends up breaking a lot of the tests
(HBase, Hive, etc.), as we end up having different Hadoop versions on the
classpath. Perhaps we should wait to update to Hadoop 3.0.0 until more of the
components release new major versions depending on Hadoop 3.0.0? Until then I
could just add a new plugin for Yarn 3.0.0 so we can claim to support Hadoop
3.0.0. WDYT?
> RangerYarnAuthorizer not compatible with Hadoop-3.0.0
> -
>
> Key: RANGER-1738
> URL: https://issues.apache.org/jira/browse/RANGER-1738
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 0.7.1
>Reporter: Hong Shen
>Assignee: Colm O hEigeartaigh
> Fix For: 1.0.0
>
> Attachments:
> 0001-RANGER-1738-RangerYarnAuthorizer-not-compatible-with.patch
>
>
> In the newest hadoop version 3.0.0, YarnAuthorizationProvider has changed.
> The new YarnAuthorizationProvider.java has change the methods checkPermission
> and setPermission,
> {code:title=YarnAuthorizationProvider.java|borderStyle=solid}
> /**
>* Check if user has the permission to access the target object.
>*
>* @param accessRequest
>* the request object which contains all the access context info.
>* @return true if user can access the object, otherwise false.
>*/
> public abstract boolean checkPermission(AccessRequest accessRequest);
> /**
>* Set permissions for the target object.
>*
>* @param permissions
>*A list of permissions on the target object.
>* @param ugi User who sets the permissions.
>*/
> public abstract void setPermission(List permissions,
> UserGroupInformation ugi);
> {code}
> But the RangerYarnAuthorizer extends YarnAuthorizationProvider impletement
> the old method.
> {code:title=RangerYarnAuthorizer.java|borderStyle=solid}
> @Override
> public void setPermission(PrivilegedEntity entity, Map<AccessType,
> AccessControlList> permission, UserGroupInformation ugi) {
>...
> @Override
> public boolean checkPermission(AccessType accessType, PrivilegedEntity
> entity, UserGroupInformation ugi) {
> {code}
> I think yarn plugin should also impletement the new method. I will add a
> patch for it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (RANGER-1849) Remove PasswordGenerator and FileStoreUtil
[ https://issues.apache.org/jira/browse/RANGER-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1849: Summary: Remove PasswordGenerator and FileStoreUtil (was: Remove PasswordGenerator) > Remove PasswordGenerator and FileStoreUtil > -- > > Key: RANGER-1849 > URL: https://issues.apache.org/jira/browse/RANGER-1849 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh >Priority: Trivial > Fix For: 1.0.0 > > > PasswordGenerator is not used in any of the scripts and should be removed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RANGER-1849) Remove PasswordGenerator and FileStoreUtil
[ https://issues.apache.org/jira/browse/RANGER-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1849: Description: PasswordGenerator is not used in any of the scripts and should be removed. Same goes for FileStoreUtil. (was: PasswordGenerator is not used in any of the scripts and should be removed.) > Remove PasswordGenerator and FileStoreUtil > -- > > Key: RANGER-1849 > URL: https://issues.apache.org/jira/browse/RANGER-1849 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh >Priority: Trivial > Fix For: 1.0.0 > > > PasswordGenerator is not used in any of the scripts and should be removed. > Same goes for FileStoreUtil. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Review Request 63182: RANGER-1849 - Remove PasswordGenerator and FileStoreUtil
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63182/ --- (Updated Nov. 16, 2017, 12:11 p.m.) Review request for ranger. Summary (updated) - RANGER-1849 - Remove PasswordGenerator and FileStoreUtil Bugs: RANGER-1849 https://issues.apache.org/jira/browse/RANGER-1849 Repository: ranger Description (updated) --- PasswordGenerator is not used in any of the scripts and should be removed. Same goes for FileStoreUtil. Diffs - agents-installer/src/main/java/org/apache/ranger/utils/install/PasswordGenerator.java a829957a Diff: https://reviews.apache.org/r/63182/diff/1/ Testing --- Thanks, Colm O hEigeartaigh
[jira] [Updated] (RANGER-1895) Simplify Storm dependencies
[ https://issues.apache.org/jira/browse/RANGER-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1895: Attachment: 0001-RANGER-1895-Simplify-Storm-dependencies.patch > Simplify Storm dependencies > --- > > Key: RANGER-1895 > URL: https://issues.apache.org/jira/browse/RANGER-1895 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh >Priority: Minor > Fix For: 1.0.0 > > Attachments: 0001-RANGER-1895-Simplify-Storm-dependencies.patch > > > This task is to simplify the Apache Storm dependencies for Ranger. We are > shipping the hadoop-hdfs jar, which is not required. Secondly, we can avoid > explicitly listing some of the dependencies, as they get pulled in via other > dependencies. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Review Request 63919: RANGER-1895 - Simplify Storm dependencies
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63919/ --- Review request for ranger. Bugs: RANGER-1895 https://issues.apache.org/jira/browse/RANGER-1895 Repository: ranger Description --- This task is to simplify the Apache Storm dependencies for Ranger. We are shipping the hadoop-hdfs jar, which is not required. Secondly, we can avoid explicitly listing some of the dependencies, as they get pulled in via other dependencies. Diffs - src/main/assembly/storm-agent.xml 64224ec7 storm-agent/pom.xml 6e74e5b3 Diff: https://reviews.apache.org/r/63919/diff/1/ Testing --- Tested the plugin works OK with Apache Storm 1.1.1. The only change in the distribution is that it doesn't include the Hadoop HDFS jar. Thanks, Colm O hEigeartaigh
[jira] [Created] (RANGER-1895) Simplify Storm dependencies
Colm O hEigeartaigh created RANGER-1895: --- Summary: Simplify Storm dependencies Key: RANGER-1895 URL: https://issues.apache.org/jira/browse/RANGER-1895 Project: Ranger Issue Type: Improvement Components: plugins Reporter: Colm O hEigeartaigh Assignee: Colm O hEigeartaigh Priority: Minor Fix For: 1.0.0 This task is to simplify the Apache Storm dependencies for Ranger. We are shipping the hadoop-hdfs jar, which is not required. Secondly, we can avoid explicitly listing some of the dependencies, as they get pulled in via other dependencies. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Review Request 63867: RANGER-1848:Implement getLinkList/getJobList in SqoopClient for Ranger Sqoop2 plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63867/#review191359 --- I think the Commons Collections 4 dependency should be avoided as it is not really required. - Colm O hEigeartaigh On Nov. 16, 2017, 7:13 a.m., Qiang Zhang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63867/ > --- > > (Updated Nov. 16, 2017, 7:13 a.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, > Selvamohan Neethiraj, sam rome, Venkat Ranganathan, and Velmurugan Periasamy. > > > Bugs: RANGER-1848 > https://issues.apache.org/jira/browse/RANGER-1848 > > > Repository: ranger > > > Description > --- > > Implement getLinkList/getJobList in SqoopClient for Ranger Sqoop2 plugin. > It affects link and job resource lookup when creating sqoop policy in > RangerAdmin. > > > Diffs > - > > plugin-sqoop/pom.xml 58d10a4 > > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java > 640d5db > > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopJobResponse.java > PRE-CREATION > > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopJobsResponse.java > PRE-CREATION > > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopLinkResponse.java > PRE-CREATION > > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopLinksResponse.java > PRE-CREATION > pom.xml 589cd6a > src/main/assembly/admin-web.xml aa37426 > > > Diff: https://reviews.apache.org/r/63867/diff/1/ > > > Testing > --- > > Tested sqoop link and job resource lookup. > > > Thanks, > > Qiang Zhang > >
Re: Review Request 63699: RANGER-1882 - Compilation error in kms module with Hadoop 2.8.x
> On Nov. 9, 2017, 6:22 p.m., Alejandro Fernandez wrote: > > kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSAuthenticationFilter.java > > Line 74 (original), 73 (patched) > > <https://reviews.apache.org/r/63699/diff/1/?file=1886105#file1886105line74> > > > > Which other versions of Hadoop are supported? Right now, only Hadoop 2.7.x. With this change both Hadoop 2.8.x and 2.7.x will be supported. - Colm --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63699/#review190623 ------- On Nov. 9, 2017, 1:17 p.m., Colm O hEigeartaigh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63699/ > --- > > (Updated Nov. 9, 2017, 1:17 p.m.) > > > Review request for ranger. > > > Bugs: RANGER-1882 > https://issues.apache.org/jira/browse/RANGER-1882 > > > Repository: ranger > > > Description > --- > > There is a compilation failure in the KMS module when compiled with Hadoop > 2.8.x. The fix is just to use a constant String which works with both Hadoop > 2.7.x, 2.8.x. > > > Diffs > - > > > kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSAuthenticationFilter.java > ada9a56f > > > Diff: https://reviews.apache.org/r/63699/diff/1/ > > > Testing > --- > > > Thanks, > > Colm O hEigeartaigh > >
Re: Plugin Shim question
OK thanks Ramesh. I think having a super-shim is probably a bit overkill at this stage, as the user can easily just choose which version of Hadoop they want to use. We can always revisit that at a later stage. Colm. On Thu, Nov 9, 2017 at 7:02 PM, Ramesh Mani <rm...@hortonworks.com> wrote: > > Colm, > > Yes look like we need to have multiple shims. > > To support multiple version of Hadoop, we may need a super shim which will > find which version of Hadoop ( based on method signature) is used and then > instantiate the corresponding sub shim RangerYarnAuthorizer. > > This may be the case for HDFS plugin also in there is a different > behaviors or signature change in authorization hook. > > Thanks, > Ramesh > > > On 11/9/17, 4:22 AM, "Colm O hEigeartaigh" <cohei...@apache.org> wrote: > > >Hi all, > > > >I'm working on adding support for Hadoop 3.0.0 to the Yarn component > >(RANGER-1738). > > > >YarnAuthorizationProvider has some updated methods in Hadoop 3.0.0. It's > >easy to work around this in the RangerYarnAuthorizer though, so that it > >supports both Hadoop 2.7.x, 2.8.x and 3.0.0. > > > >The problem is in the plugin shim code for RangerYarnAuthorizer. It > >delegates the calls to the underlying YarnAuthorizationProvider instance. > >This means it's not possible to support both 2.7.x and 3.0.0 as it is in > >the plugin version of RangerYarnAuthorizer. > > > >Any ideas on this? The only way I can think of supporting it is to have > >separate plugin shims for Hadoop 2 + 3. > > > >Colm. > > > > > >-- > >Colm O hEigeartaigh > > > >Talend Community Coder > >http://coders.talend.com > > -- Colm O hEigeartaigh Talend Community Coder http://coders.talend.com
[jira] [Updated] (RANGER-1738) RangerYarnAuthorizer not compatible with Hadoop-3.0.0
[
https://issues.apache.org/jira/browse/RANGER-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colm O hEigeartaigh updated RANGER-1738:
Attachment: 0001-RANGER-1738-RangerYarnAuthorizer-not-compatible-with.patch
A patch for this issue. I introduced a new plugin for Yarn with Hadoop 3.0.0,
which simply extends the 2.x plugin with the new methods. A new plugin shim is
also included.
> RangerYarnAuthorizer not compatible with Hadoop-3.0.0
> -
>
> Key: RANGER-1738
> URL: https://issues.apache.org/jira/browse/RANGER-1738
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 0.7.1
>Reporter: Hong Shen
>Assignee: Colm O hEigeartaigh
> Fix For: 1.0.0
>
> Attachments:
> 0001-RANGER-1738-RangerYarnAuthorizer-not-compatible-with.patch
>
>
> In the newest hadoop version 3.0.0, YarnAuthorizationProvider has changed.
> The new YarnAuthorizationProvider.java has change the methods checkPermission
> and setPermission,
> {code:title=YarnAuthorizationProvider.java|borderStyle=solid}
> /**
>* Check if user has the permission to access the target object.
>*
>* @param accessRequest
>* the request object which contains all the access context info.
>* @return true if user can access the object, otherwise false.
>*/
> public abstract boolean checkPermission(AccessRequest accessRequest);
> /**
>* Set permissions for the target object.
>*
>* @param permissions
>*A list of permissions on the target object.
>* @param ugi User who sets the permissions.
>*/
> public abstract void setPermission(List permissions,
> UserGroupInformation ugi);
> {code}
> But the RangerYarnAuthorizer extends YarnAuthorizationProvider impletement
> the old method.
> {code:title=RangerYarnAuthorizer.java|borderStyle=solid}
> @Override
> public void setPermission(PrivilegedEntity entity, Map<AccessType,
> AccessControlList> permission, UserGroupInformation ugi) {
>...
> @Override
> public boolean checkPermission(AccessType accessType, PrivilegedEntity
> entity, UserGroupInformation ugi) {
> {code}
> I think yarn plugin should also impletement the new method. I will add a
> patch for it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Re: Review Request 63700: [RANGER-1881] Reduce code duplication in kms, where the console handling code is repeated 4 times.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63700/#review190819 --- Ship it! Ship It! - Colm O hEigeartaigh On Nov. 9, 2017, 1:45 p.m., Zsombor Gegesy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63700/ > --- > > (Updated Nov. 9, 2017, 1:45 p.m.) > > > Review request for ranger. > > > Bugs: RANGER-1881 > https://issues.apache.org/jira/browse/RANGER-1881 > > > Repository: ranger > > > Description > --- > > Move the password handling functionality into a common method > > > Diffs > - > > kms/src/main/java/org/apache/hadoop/crypto/key/ConsoleUtil.java > PRE-CREATION > kms/src/main/java/org/apache/hadoop/crypto/key/DB2HSMMKUtil.java edbb299aa > kms/src/main/java/org/apache/hadoop/crypto/key/HSM2DBMKUtil.java 427e09857 > kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java > 22dce0f5f > kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java > 1abbf8e3c > > > Diff: https://reviews.apache.org/r/63700/diff/1/ > > > Testing > --- > > Run the command line tools. > > Travis tests: > https://travis-ci.org/gzsombor/ranger/builds/299618293 > > > Thanks, > > Zsombor Gegesy > >
Re: Review Request 63789: RANGER-1887:serviceDef.getResources().get(0).getName(); case IndexOutOfBoundsException in RangerServiceTag.class And print error in RangerServiceStorm
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63789/#review190928 --- It's better to write "!serviceDef.getResources().isEmpty()" than "serviceDef.getResources().size() > 0". What happens if tagResourceName is null. Will "defaultPolicy.getResources().get(tagResourceName)" throw a NPE? - Colm O hEigeartaigh On Nov. 14, 2017, 8:15 a.m., Qiang Zhang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63789/ > --- > > (Updated Nov. 14, 2017, 8:15 a.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, > Selvamohan Neethiraj, sam rome, and Velmurugan Periasamy. > > > Bugs: RANGER-1887 > https://issues.apache.org/jira/browse/RANGER-1887 > > > Repository: ranger > > > Description > --- > > String tagResourceName = serviceDef.getResources().get(0).getName(); case > java.lang.IndexOutOfBoundsException in RangerServiceTag.class > And print log error for RangerServiceStorm.class > > > Diffs > - > > > agents-common/src/main/java/org/apache/ranger/services/tag/RangerServiceTag.java > ce6002b > > storm-agent/src/main/java/org/apache/ranger/services/storm/RangerServiceStorm.java > d4dbd7b > > > Diff: https://reviews.apache.org/r/63789/diff/1/ > > > Testing > --- > > Tested it > > > Thanks, > > Qiang Zhang > >
Re: Review Request 63921: RANGER-1896 : Remove deprecated extractedCommonCriterias call from the SearchUtil and fix the UserREST controller
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63921/#review191490 --- Ship it! Ship It! - Colm O hEigeartaigh On Nov. 17, 2017, 4:48 p.m., Zsombor Gegesy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63921/ > --- > > (Updated Nov. 17, 2017, 4:48 p.m.) > > > Review request for ranger. > > > Bugs: RANGER-1896 > https://issues.apache.org/jira/browse/RANGER-1896 > > > Repository: ranger > > > Description > --- > > Deprecated code removed, and the newer extractCommonCriterias method is used > > > Diffs > - > > security-admin/src/main/java/org/apache/ranger/common/SearchUtil.java > ad307339a > security-admin/src/main/java/org/apache/ranger/rest/UserREST.java 4bf18991a > security-admin/src/test/java/org/apache/ranger/rest/TestUserREST.java > 373b6f348 > > > Diff: https://reviews.apache.org/r/63921/diff/1/ > > > Testing > --- > > Tested with calling : > http://rangerhost/service/users?sortBy=userId > http://rangerhost/service/users?sortBy=loginId > http://rangerhost/service/users?sortBy=emailAddress > http://rangerhost/service/users?sortBy=firstName > http://rangerhost/service/users?sortBy=lastName > > > Thanks, > > Zsombor Gegesy > >
[jira] [Created] (RANGER-1898) Simplify Knox plugin dependency management
Colm O hEigeartaigh created RANGER-1898: --- Summary: Simplify Knox plugin dependency management Key: RANGER-1898 URL: https://issues.apache.org/jira/browse/RANGER-1898 Project: Ranger Issue Type: Bug Components: plugins Reporter: Colm O hEigeartaigh Assignee: Colm O hEigeartaigh Priority: Minor Fix For: 1.0.0 This task is to simplify the dependency management for the Knox plugin for Ranger. In addition, the hadoop-hdfs jar is removed from the distribution, as it is not required. Tested that the distribution ships the same jars as before, apart from hadoop-hdfs. Also tested a use-case involving Ranger + Knox. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RANGER-1898) Simplify Knox plugin dependency management
[ https://issues.apache.org/jira/browse/RANGER-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1898: Attachment: 0001-RANGER-1898-Simplify-Knox-plugin-dependency-manageme.patch > Simplify Knox plugin dependency management > -- > > Key: RANGER-1898 > URL: https://issues.apache.org/jira/browse/RANGER-1898 > Project: Ranger > Issue Type: Bug > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh >Priority: Minor > Fix For: 1.0.0 > > Attachments: > 0001-RANGER-1898-Simplify-Knox-plugin-dependency-manageme.patch > > > This task is to simplify the dependency management for the Knox plugin for > Ranger. In addition, the hadoop-hdfs jar is removed from the distribution, as > it is not required. > Tested that the distribution ships the same jars as before, apart from > hadoop-hdfs. Also tested a use-case involving Ranger + Knox. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Review Request 63949: RANGER-1898 - Simplify Knox plugin dependency management
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63949/ --- Review request for ranger. Bugs: RANGER-1898 https://issues.apache.org/jira/browse/RANGER-1898 Repository: ranger Description --- This task is to simplify the dependency management for the Knox plugin for Ranger. In addition, the hadoop-hdfs jar is removed from the distribution, as it is not required. Diffs - knox-agent/pom.xml 52f15cd0 src/main/assembly/knox-agent.xml 8357d498 Diff: https://reviews.apache.org/r/63949/diff/1/ Testing --- Tested that the distribution ships the same jars as before, apart from hadoop-hdfs. Also tested a use-case involving Ranger + Knox. Thanks, Colm O hEigeartaigh
Re: Review Request 63870: RANGER-1861:There are errors when the "ranger-kms start" command was executed.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63870/#review191494 --- Ship it! Ship It! - Colm O hEigeartaigh On Nov. 16, 2017, 10:03 a.m., pengjianhua wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63870/ > --- > > (Updated Nov. 16, 2017, 10:03 a.m.) > > > Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan > Neethiraj, Velmurugan Periasamy, and Qiang Zhang. > > > Bugs: RANGER-1861 > https://issues.apache.org/jira/browse/RANGER-1861 > > > Repository: ranger > > > Description > --- > > There are errors when the "ranger-kms start" command was executed. > > 2017-10-28 23:23:33,339 ERROR XMLUtils - Error loading : > java.lang.IllegalArgumentException: InputStream cannot be null > at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:117) > at org.apache.ranger.plugin.util.XMLUtils.loadConfig(XMLUtils.java:64) > at org.apache.ranger.plugin.util.XMLUtils.loadConfig(XMLUtils.java:48) > at > org.apache.ranger.server.tomcat.EmbeddedServer.(EmbeddedServer.java:77) > at > org.apache.ranger.server.tomcat.EmbeddedServer.main(EmbeddedServer.java:69) > > > Diffs > - > > > embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java > 40eed861 > > > Diff: https://reviews.apache.org/r/63870/diff/1/ > > > Testing > --- > > > Thanks, > > pengjianhua > >
Re: Review Request 63867: RANGER-1848:Implement getLinkList/getJobList in SqoopClient for Ranger Sqoop2 plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63867/#review191495 --- In SqoopClient, I think it would be better to return Collections.emptyList() instead of null. - Colm O hEigeartaigh On Nov. 20, 2017, 8:13 a.m., Qiang Zhang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63867/ > --- > > (Updated Nov. 20, 2017, 8:13 a.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, > Selvamohan Neethiraj, sam rome, Venkat Ranganathan, and Velmurugan Periasamy. > > > Bugs: RANGER-1848 > https://issues.apache.org/jira/browse/RANGER-1848 > > > Repository: ranger > > > Description > --- > > Implement getLinkList/getJobList in SqoopClient for Ranger Sqoop2 plugin. > It affects link and job resource lookup when creating sqoop policy in > RangerAdmin. > > > Diffs > - > > > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java > 640d5db > > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopJobResponse.java > PRE-CREATION > > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopJobsResponse.java > PRE-CREATION > > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopLinkResponse.java > PRE-CREATION > > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopLinksResponse.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/63867/diff/2/ > > > Testing > --- > > Tested sqoop link and job resource lookup. > > > Thanks, > > Qiang Zhang > >
Re: Review Request 61062: RANGER-1707 : fix hdfs traverse check
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61062/#review191583
---
You could put some spaces into "for (int i=0;i<pathSegments.length;i++) {"
There's also an indentation issue on line 201 of RangerHdfsAuthorizerTest.
Other spacing issue here "ancestorIndex,plugin"
> for (FsAction action : Arrays.asList(FsAction.EXECUTE, FsAction.READ,
> FsAction.WRITE)) {
I think the FsAction.EXECUTE is not necessary here, as we are checking EXECUTE
already in "traverseOnlyCheck".
- Colm O hEigeartaigh
On July 22, 2017, 10:31 a.m., Zsombor Gegesy wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61062/
> ---
>
> (Updated July 22, 2017, 10:31 a.m.)
>
>
> Review request for ranger.
>
>
> Bugs: RANGER-1707
> https://issues.apache.org/jira/browse/RANGER-1707
>
>
> Repository: ranger
>
>
> Description
> ---
>
> Fix hdfs traverse check, which problem was hidden before hdfs 2.8.0, where
> the traverse checks are called
> before reading and writing files, so if a policy is just about reading
> /tmp/somedir/somefile
> it means, that traverse should be allowed to get to that file. Adding
> more tests to highlight the issue
>
>
> Diffs
> -
>
> hdfs-agent/pom.xml 9f62060
>
> hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java
> d28685a
>
> hdfs-agent/src/test/java/org/apache/ranger/services/hdfs/RangerHdfsAuthorizerTest.java
> PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/61062/diff/1/
>
>
> Testing
> ---
>
> Tested locally
> https://travis-ci.org/gzsombor/ranger/builds/256331500
>
>
> Thanks,
>
> Zsombor Gegesy
>
>
[jira] [Updated] (RANGER-1896) Remove deprecated extractedCommonCriterias call from the SearchUtil
[ https://issues.apache.org/jira/browse/RANGER-1896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1896: Fix Version/s: 1.0.0 > Remove deprecated extractedCommonCriterias call from the SearchUtil > > > Key: RANGER-1896 > URL: https://issues.apache.org/jira/browse/RANGER-1896 > Project: Ranger > Issue Type: Bug > Components: admin >Affects Versions: 0.7.1 >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy > Fix For: 1.0.0 > > Attachments: RANGER-1896.patch > > > There is a deprecated extractedCommonCriterias call in SearchUtil, which is > called from UserREST controller, which can be removed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Review Request 63867: RANGER-1848:Implement getLinkList/getJobList in SqoopClient for Ranger Sqoop2 plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63867/#review191587 --- Ship it! Ship It! - Colm O hEigeartaigh On Nov. 21, 2017, 3:33 a.m., Qiang Zhang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63867/ > --- > > (Updated Nov. 21, 2017, 3:33 a.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, > Selvamohan Neethiraj, sam rome, Venkat Ranganathan, and Velmurugan Periasamy. > > > Bugs: RANGER-1848 > https://issues.apache.org/jira/browse/RANGER-1848 > > > Repository: ranger > > > Description > --- > > Implement getLinkList/getJobList in SqoopClient for Ranger Sqoop2 plugin. > It affects link and job resource lookup when creating sqoop policy in > RangerAdmin. > > > Diffs > - > > > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java > 640d5db > > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopJobResponse.java > PRE-CREATION > > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopJobsResponse.java > PRE-CREATION > > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopLinkResponse.java > PRE-CREATION > > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopLinksResponse.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/63867/diff/3/ > > > Testing > --- > > Tested sqoop link and job resource lookup. > > > Thanks, > > Qiang Zhang > >
[jira] [Resolved] (RANGER-1896) Remove deprecated extractedCommonCriterias call from the SearchUtil
[ https://issues.apache.org/jira/browse/RANGER-1896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh resolved RANGER-1896. - Resolution: Fixed > Remove deprecated extractedCommonCriterias call from the SearchUtil > > > Key: RANGER-1896 > URL: https://issues.apache.org/jira/browse/RANGER-1896 > Project: Ranger > Issue Type: Bug > Components: admin >Affects Versions: 0.7.1 >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy > Fix For: 1.0.0 > > Attachments: RANGER-1896.patch > > > There is a deprecated extractedCommonCriterias call in SearchUtil, which is > called from UserREST controller, which can be removed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Build issues - inconsistent
m2/repository/com/webcohesion/ > enunciate/enunciate-docs/2.8.0/enunciate-docs-2.8.0.jar!/ > com/webcohesion/enunciate/modules/docs/docs.fmt" in macro > "processResourceGroup" at line 556, column 11] > [ERROR] ~ Reached through: #nested [in template > "jar:file:/Users/jonesn/.m2/repository/com/webcohesion/ > enunciate/enunciate-docs/2.8.0/enunciate-docs-2.8.0.jar!/ > com/webcohesion/enunciate/modules/docs/docs.fmt" in macro "boilerplate" > at line 141, column 9] > [ERROR] ~ Reached through: @boilerplate title=(title + ": " + > re... [in template "jar:file:/Users/jonesn/.m2/ > repository/com/webcohesion/enunciate/enunciate-docs/2.8. > 0/enunciate-docs-2.8.0.jar!/com/webcohesion/enunciate/modules/docs/docs.fmt" > in macro "processResourceGroup" at line 518, column 5] > [ERROR] - Reached through: @file name=(resourceGroup.slug + > ".ht... [in template "jar:file:/Users/jonesn/.m2/ > repository/com/webcohesion/enunciate/enunciate-docs/2.8. > 0/enunciate-docs-2.8.0.jar!/com/webcohesion/enunciate/modules/docs/docs.fmt" > in macro "processResourceGroup" at line 517, column 3] > [ERROR] - Reached through: @processResourceGroup > resourceGroup=r... [in template "jar:file:/Users/jonesn/.m2/ > repository/com/webcohesion/enunciate/enunciate-docs/2.8. > 0/enunciate-docs-2.8.0.jar!/com/webcohesion/enunciate/modules/docs/docs.fmt" > at line 238, column 11] > [ERROR] ~ Reached through: #nested [in template > "jar:file:/Users/jonesn/.m2/repository/com/webcohesion/ > enunciate/enunciate-docs/2.8.0/enunciate-docs-2.8.0.jar!/ > com/webcohesion/enunciate/modules/docs/docs.fmt" in macro "boilerplate" > at line 141, column 9] > [ERROR] ~ Reached through: @boilerplate pagenav=pagenav [in > template "jar:file:/Users/jonesn/.m2/repository/com/webcohesion/ > enunciate/enunciate-docs/2.8.0/enunciate-docs-2.8.0.jar!/ > com/webcohesion/enunciate/modules/docs/docs.fmt" at line 192, column 3] > [ERROR] - Reached through: @file name=indexPageName [in template > "jar:file:/Users/jonesn/.m2/repository/com/webcohesion/ > enunciate/enunciate-docs/2.8.0/enunciate-docs-2.8.0.jar!/ > com/webcohesion/enunciate/modules/docs/docs.fmt" at line 181, column 1] > [ERROR] : InvocationTargetException: > javax/annotation/security/RolesAllowed: > javax.annotation.security.RolesAllowed > [ERROR] -> [Help 1] > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with the > -e switch. > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > [ERROR] > [ERROR] For more information about the errors and possible solutions, > please read the following articles: > [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ > MojoExecutionException > [ERROR] > [ERROR] After correcting the problems, you can resume the build with the > command > [ERROR] mvn -rf :security-admin-web > > > >> --- > > So that's 4 attempts, 4 different results. I could open JIRAs, but I'm > concerned at the lack of consistency. What do I need to check in my dev > environments? Are there additional prereqs not checked for in maven? > > My Atlas builds are more consistent (occasional timeout, but generally > sharing results with others) > > Many thanks > Nigel Jones > nigel.l.jo...@gmail.com > -- Colm O hEigeartaigh Talend Community Coder http://coders.talend.com
Re: Review Request 63789: RANGER-1887:serviceDef.getResources().get(0).getName(); case IndexOutOfBoundsException in RangerServiceTag.class And print error in RangerServiceStorm
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63789/#review191170 --- The indentation of the for loop needs to be shifted to the right due to the new if statement. - Colm O hEigeartaigh On Nov. 14, 2017, 11:09 a.m., Qiang Zhang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63789/ > --- > > (Updated Nov. 14, 2017, 11:09 a.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, > Selvamohan Neethiraj, sam rome, and Velmurugan Periasamy. > > > Bugs: RANGER-1887 > https://issues.apache.org/jira/browse/RANGER-1887 > > > Repository: ranger > > > Description > --- > > String tagResourceName = serviceDef.getResources().get(0).getName(); case > java.lang.IndexOutOfBoundsException in RangerServiceTag.class > And print log error for RangerServiceStorm.class > > > Diffs > - > > > agents-common/src/main/java/org/apache/ranger/services/tag/RangerServiceTag.java > ce6002b > > storm-agent/src/main/java/org/apache/ranger/services/storm/RangerServiceStorm.java > d4dbd7b > > > Diff: https://reviews.apache.org/r/63789/diff/2/ > > > Testing > --- > > Tested it > > > Thanks, > > Qiang Zhang > >
[jira] [Commented] (RANGER-1738) RangerYarnAuthorizer not compatible with Hadoop-3.0.0
[
https://issues.apache.org/jira/browse/RANGER-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253284#comment-16253284
]
Colm O hEigeartaigh commented on RANGER-1738:
-
[~rmani], [~vperiasamy], OK so what you are suggesting here is to drop the
Hadoop 2.7.x dependency from Ranger altogether? This could work, but 3.0.0 is
currently a "beta" - I'm not sure when the plans are to ship a final 3.0.0
version. Would we be happy to release Ranger 1.0.0 off a beta Hadoop version?
It might be less risky just to leave the dependency on Hadoop 2.7.x, but add
some exclusions in the Yarn 3.0.0 plugin to exclude 2.7.x jars. The
distribution for the new plugin doesn't include any Hadoop jars in the "lib"
directory.
[~vperiasamy], It's not possible to have RangerYarnAuthorizer work with
multiple versions, as with Hadoop 3.0.0 you have to implement methods that take
arguments that are only available in Hadoop 3.0.0.
> RangerYarnAuthorizer not compatible with Hadoop-3.0.0
> -
>
> Key: RANGER-1738
> URL: https://issues.apache.org/jira/browse/RANGER-1738
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 0.7.1
> Reporter: Hong Shen
>Assignee: Colm O hEigeartaigh
> Fix For: 1.0.0
>
> Attachments:
> 0001-RANGER-1738-RangerYarnAuthorizer-not-compatible-with.patch
>
>
> In the newest hadoop version 3.0.0, YarnAuthorizationProvider has changed.
> The new YarnAuthorizationProvider.java has change the methods checkPermission
> and setPermission,
> {code:title=YarnAuthorizationProvider.java|borderStyle=solid}
> /**
>* Check if user has the permission to access the target object.
>*
>* @param accessRequest
>* the request object which contains all the access context info.
>* @return true if user can access the object, otherwise false.
>*/
> public abstract boolean checkPermission(AccessRequest accessRequest);
> /**
>* Set permissions for the target object.
>*
>* @param permissions
>*A list of permissions on the target object.
>* @param ugi User who sets the permissions.
>*/
> public abstract void setPermission(List permissions,
> UserGroupInformation ugi);
> {code}
> But the RangerYarnAuthorizer extends YarnAuthorizationProvider impletement
> the old method.
> {code:title=RangerYarnAuthorizer.java|borderStyle=solid}
> @Override
> public void setPermission(PrivilegedEntity entity, Map<AccessType,
> AccessControlList> permission, UserGroupInformation ugi) {
>...
> @Override
> public boolean checkPermission(AccessType accessType, PrivilegedEntity
> entity, UserGroupInformation ugi) {
> {code}
> I think yarn plugin should also impletement the new method. I will add a
> patch for it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Review Request 63834: RANGER-1893 - Update HBase dependency to 1.2.6
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63834/ --- Review request for ranger. Bugs: RANGER-1893 https://issues.apache.org/jira/browse/RANGER-1893 Repository: ranger Description --- We currently rely on an old HBase dependency (1.1.3). This task is to upgrade it to 1.2.6. The plugin works correctly with no changes since 1.1.x. Diffs - pom.xml 589cd6ac Diff: https://reviews.apache.org/r/63834/diff/1/ Testing --- Thanks, Colm O hEigeartaigh
[jira] [Created] (RANGER-1893) Update HBase dependency to 1.2.6
Colm O hEigeartaigh created RANGER-1893: --- Summary: Update HBase dependency to 1.2.6 Key: RANGER-1893 URL: https://issues.apache.org/jira/browse/RANGER-1893 Project: Ranger Issue Type: Improvement Components: plugins Reporter: Colm O hEigeartaigh Assignee: Colm O hEigeartaigh Fix For: 1.0.0 We currently rely on an old HBase dependency (1.1.3). This task is to upgrade it to 1.2.6. The plugin works correctly with no changes since 1.1.x. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RANGER-1893) Update HBase dependency to 1.2.6
[ https://issues.apache.org/jira/browse/RANGER-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1893: Attachment: 0001-RANGER-1893-Update-HBase-dependency-to-1.2.6.patch > Update HBase dependency to 1.2.6 > > > Key: RANGER-1893 > URL: https://issues.apache.org/jira/browse/RANGER-1893 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh > Fix For: 1.0.0 > > Attachments: 0001-RANGER-1893-Update-HBase-dependency-to-1.2.6.patch > > > We currently rely on an old HBase dependency (1.1.3). This task is to upgrade > it to 1.2.6. The plugin works correctly with no changes since 1.1.x. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RANGER-1893) Update HBase dependency to 1.2.6
[ https://issues.apache.org/jira/browse/RANGER-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1893: Attachment: 0001-RANGER-1893-Update-HBase-dependency-to-1.2.6.patch Also removing some unused properties > Update HBase dependency to 1.2.6 > > > Key: RANGER-1893 > URL: https://issues.apache.org/jira/browse/RANGER-1893 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh > Fix For: 1.0.0 > > Attachments: 0001-RANGER-1893-Update-HBase-dependency-to-1.2.6.patch > > > We currently rely on an old HBase dependency (1.1.3). This task is to upgrade > it to 1.2.6. The plugin works correctly with no changes since 1.1.x. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RANGER-1893) Update HBase dependency to 1.2.6
[ https://issues.apache.org/jira/browse/RANGER-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1893: Attachment: (was: 0001-RANGER-1893-Update-HBase-dependency-to-1.2.6.patch) > Update HBase dependency to 1.2.6 > > > Key: RANGER-1893 > URL: https://issues.apache.org/jira/browse/RANGER-1893 > Project: Ranger > Issue Type: Improvement > Components: plugins > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh > Fix For: 1.0.0 > > Attachments: 0001-RANGER-1893-Update-HBase-dependency-to-1.2.6.patch > > > We currently rely on an old HBase dependency (1.1.3). This task is to upgrade > it to 1.2.6. The plugin works correctly with no changes since 1.1.x. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Review Request 63834: RANGER-1893 - Update HBase dependency to 1.2.6
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63834/ --- (Updated Nov. 15, 2017, 3:14 p.m.) Review request for ranger. Changes --- Also removing some unused properties Bugs: RANGER-1893 https://issues.apache.org/jira/browse/RANGER-1893 Repository: ranger Description --- We currently rely on an old HBase dependency (1.1.3). This task is to upgrade it to 1.2.6. The plugin works correctly with no changes since 1.1.x. Diffs (updated) - pom.xml 589cd6ac Diff: https://reviews.apache.org/r/63834/diff/2/ Changes: https://reviews.apache.org/r/63834/diff/1-2/ Testing --- Thanks, Colm O hEigeartaigh
Re: Review Request 63534: RANGER-1870:Modify the logic for buildUnixUserList and parseMembers method in UnixUserGroupBuilder class
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63534/#review190018
---
Ship it!
Ship It!
- Colm O hEigeartaigh
On Nov. 3, 2017, 8:36 a.m., Qiang Zhang wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63534/
> ---
>
> (Updated Nov. 3, 2017, 8:36 a.m.)
>
>
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani,
> Selvamohan Neethiraj, sam rome, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1870
> https://issues.apache.org/jira/browse/RANGER-1870
>
>
> Repository: ranger
>
>
> Description
> ---
>
> Modify the logic for buildUnixUserList and parseMembers method in
> UnixUserGroupBuilder class
> 1.In the buildUnixUserList method:
> int len = tokens.length;
> if (len = 3) groupId = tokens [3]; case ArrayIndexOutOfBoundsException.
> Solution: change from "if (len < 3) {" to "if (len < 4) {"
>
> 2.Another similar error in the parseMembers method:
> String[] tokens = line.split(":");
> if (tokens.length = 2) String groupId = tokens [2]; case
> ArrayIndexOutOfBoundsException.
> Solution:change from “if (tokens.length < 2)” to “if (tokens.length < 3)”
>
>
> Diffs
> -
>
>
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/UnixUserGroupBuilder.java
> 045fe3f
>
>
> Diff: https://reviews.apache.org/r/63534/diff/1/
>
>
> Testing
> ---
>
> tested it.
>
>
> Thanks,
>
> Qiang Zhang
>
>
Re: Review Request 63578: anger README.txt should be add tar and sqoop-plugin desc
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63578/#review190167 --- Change (i) to (h). Also it should be "needs" and not "need" (also change this for the Yarn plugin). - Colm O hEigeartaigh On Nov. 6, 2017, 12:11 p.m., wang yuan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63578/ > --- > > (Updated Nov. 6, 2017, 12:11 p.m.) > > > Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, > Selvamohan Neethiraj, Velmurugan Periasamy, and Qiang Zhang. > > > Bugs: RANGER-1874 > https://issues.apache.org/jira/browse/RANGER-1874 > > > Repository: ranger > > > Description > --- > > ranger README.txt should be add tar and sqoop-plugin desc > ==add tar desc: > ranger--atlas-plugin.tar.gz > ranger--ranger-tools.tar.gz > ranger--sqoop-plugin.tar.gz > ranger--tagsync.tar.gz > ==add pulgin desc: > SQOOP plugin need to be installed on SQOOP2 Resource Manager hosts > > > Diffs > - > > README.txt 536d3986 > > > Diff: https://reviews.apache.org/r/63578/diff/1/ > > > Testing > --- > > > Thanks, > > wang yuan > >
Re: Review Request 63403: RANGER-1862:generalName.get(1) cause IndexOutOfBoundsException in NiFiClient
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63403/#review189789 --- Ship it! Ship It! - Colm O hEigeartaigh On Nov. 1, 2017, 8:12 a.m., Qiang Zhang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63403/ > --- > > (Updated Nov. 1, 2017, 8:12 a.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, > Selvamohan Neethiraj, sam rome, and Velmurugan Periasamy. > > > Bugs: RANGER-1862 > https://issues.apache.org/jira/browse/RANGER-1862 > > > Repository: ranger > > > Description > --- > > when generalName.size() <=1 , generalName.get(1) cause > IndexOutOfBoundsException in NiFiClient.java > > > Diffs > - > > > plugin-nifi/src/main/java/org/apache/ranger/services/nifi/client/NiFiClient.java > c03bc12 > > > Diff: https://reviews.apache.org/r/63403/diff/3/ > > > Testing > --- > > tested it > > > Thanks, > > Qiang Zhang > >
[jira] [Resolved] (RANGER-1142) Sqoop import error
[ https://issues.apache.org/jira/browse/RANGER-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh resolved RANGER-1142. - Resolution: Not A Problem Did you mean to file this JIRA at the Sqoop project? > Sqoop import error > -- > > Key: RANGER-1142 > URL: https://issues.apache.org/jira/browse/RANGER-1142 > Project: Ranger > Issue Type: Improvement >Reporter: anusha >Priority: Major > > Trying to import data in Sqoop from MS SQL Server database which has Windows > Authentication. > Trying to run the following command on the hive command prompt. > sqoop import --driver="com.microsoft.jdbc.sqlserver.SQLServerDriver" > --connect="jdbc:microsoft:sqlserver://**.**.**.** :1433/DB_Schema;Integrated > Security=TRUE" > --table dbo.TABLE_NAME > Error Received : > Error 1: > --driver is set to an explicit driver however appropriate connection manager > is not being set (via --connection-manager). Sqoop is going to fall back to > org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which > connection manager should be used next time. > Error 2 : > Got exception running Sqoop: java.lang.RuntimeException: Could not load db > driver class: com.microsoft.jdbc.sqlserver.SQLServerDriver > java.lang.RuntimeException: Could not load db driver class: > com.microsoft.jdbc.sqlserver.SQLServerDriver > Could some one guide me . > versions : > Sqoop 1.4.6 > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (RANGER-1140) sqoop import
[ https://issues.apache.org/jira/browse/RANGER-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh closed RANGER-1140. --- > sqoop import > -- > > Key: RANGER-1140 > URL: https://issues.apache.org/jira/browse/RANGER-1140 > Project: Ranger > Issue Type: Bug > Components: plugins >Reporter: anusha >Priority: Major > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (RANGER-1142) Sqoop import error
[ https://issues.apache.org/jira/browse/RANGER-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh closed RANGER-1142. --- > Sqoop import error > -- > > Key: RANGER-1142 > URL: https://issues.apache.org/jira/browse/RANGER-1142 > Project: Ranger > Issue Type: Improvement >Reporter: anusha >Priority: Major > > Trying to import data in Sqoop from MS SQL Server database which has Windows > Authentication. > Trying to run the following command on the hive command prompt. > sqoop import --driver="com.microsoft.jdbc.sqlserver.SQLServerDriver" > --connect="jdbc:microsoft:sqlserver://**.**.**.** :1433/DB_Schema;Integrated > Security=TRUE" > --table dbo.TABLE_NAME > Error Received : > Error 1: > --driver is set to an explicit driver however appropriate connection manager > is not being set (via --connection-manager). Sqoop is going to fall back to > org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which > connection manager should be used next time. > Error 2 : > Got exception running Sqoop: java.lang.RuntimeException: Could not load db > driver class: com.microsoft.jdbc.sqlserver.SQLServerDriver > java.lang.RuntimeException: Could not load db driver class: > com.microsoft.jdbc.sqlserver.SQLServerDriver > Could some one guide me . > versions : > Sqoop 1.4.6 > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (RANGER-1140) sqoop import
[ https://issues.apache.org/jira/browse/RANGER-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh resolved RANGER-1140. - Resolution: Not A Problem > sqoop import > -- > > Key: RANGER-1140 > URL: https://issues.apache.org/jira/browse/RANGER-1140 > Project: Ranger > Issue Type: Bug > Components: plugins >Reporter: anusha >Priority: Major > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RANGER-1830) Write unit test for RANGER-1810
[ https://issues.apache.org/jira/browse/RANGER-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16236194#comment-16236194 ] Colm O hEigeartaigh commented on RANGER-1830: - I can take this issue if you have not already started it? > Write unit test for RANGER-1810 > --- > > Key: RANGER-1830 > URL: https://issues.apache.org/jira/browse/RANGER-1830 > Project: Ranger > Issue Type: Sub-task > Components: plugins >Affects Versions: 1.0.0, master >Reporter: Qiang Zhang >Assignee: Qiang Zhang >Priority: Minor > Labels: newbie, patch > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Review Request 63578: anger README.txt should be add tar and sqoop-plugin desc
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63578/#review190281 --- Please remove the tab character before (h) and replace it with spaces - Colm O hEigeartaigh On Nov. 7, 2017, 1:26 a.m., wang yuan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63578/ > --- > > (Updated Nov. 7, 2017, 1:26 a.m.) > > > Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, > Selvamohan Neethiraj, Velmurugan Periasamy, and Qiang Zhang. > > > Bugs: RANGER-1874 > https://issues.apache.org/jira/browse/RANGER-1874 > > > Repository: ranger > > > Description > --- > > ranger README.txt should be add tar and sqoop-plugin desc > ==add tar desc: > ranger--atlas-plugin.tar.gz > ranger--ranger-tools.tar.gz > ranger--sqoop-plugin.tar.gz > ranger--tagsync.tar.gz > ==add pulgin desc: > SQOOP plugin need to be installed on SQOOP2 Resource Manager hosts > > > Diffs > - > > README.txt 536d3986 > > > Diff: https://reviews.apache.org/r/63578/diff/2/ > > > Testing > --- > > > Thanks, > > wang yuan > >
Re: Review Request 63608: RANGER-1830:Write unit test for RANGER-1810 sqoop plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63608/#review190284 --- Looks good to me. Just a few trivial changes: a) In getRandomLinkName and getRandomJobName there is no need to store the variable - just return it directly, e.g. return RandomStringUtils.randomAlphanumeric(10) + "-link"; b) Change new ArrayList() to new ArrayList<>() - Colm O hEigeartaigh On Nov. 7, 2017, 6:56 a.m., Qiang Zhang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63608/ > --- > > (Updated Nov. 7, 2017, 6:56 a.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, > Selvamohan Neethiraj, sam rome, Venkat Ranganathan, and Velmurugan Periasamy. > > > Bugs: RANGER-1830 > https://issues.apache.org/jira/browse/RANGER-1830 > > > Repository: ranger > > > Description > --- > > Write unit test for RANGER-1810 sqoop plugin > > > Diffs > - > > plugin-sqoop/pom.xml 0c52e66 > > plugin-sqoop/src/test/java/org/apache/ranger/authorization/sqoop/authorizer/RangerAdminClientImpl.java > PRE-CREATION > > plugin-sqoop/src/test/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizerTest.java > PRE-CREATION > plugin-sqoop/src/test/resources/log4j.properties PRE-CREATION > plugin-sqoop/src/test/resources/ranger-sqoop-security.xml PRE-CREATION > plugin-sqoop/src/test/resources/sqoop-policies.json PRE-CREATION > plugin-sqoop/src/test/resources/sqoop.properties PRE-CREATION > plugin-sqoop/src/test/resources/sqoop_bootstrap.properties PRE-CREATION > > > Diff: https://reviews.apache.org/r/63608/diff/1/ > > > Testing > --- > > Tested > > > Thanks, > > Qiang Zhang > >
[jira] [Assigned] (RANGER-1738) RangerYarnAuthorizer not compatible with Hadoop-3.0.0
[
https://issues.apache.org/jira/browse/RANGER-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colm O hEigeartaigh reassigned RANGER-1738:
---
Assignee: Colm O hEigeartaigh
> RangerYarnAuthorizer not compatible with Hadoop-3.0.0
> -
>
> Key: RANGER-1738
> URL: https://issues.apache.org/jira/browse/RANGER-1738
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 0.7.1
>Reporter: Hong Shen
>Assignee: Colm O hEigeartaigh
>
> In the newest hadoop version 3.0.0, YarnAuthorizationProvider has changed.
> The new YarnAuthorizationProvider.java has change the methods checkPermission
> and setPermission,
> {code:title=YarnAuthorizationProvider.java|borderStyle=solid}
> /**
>* Check if user has the permission to access the target object.
>*
>* @param accessRequest
>* the request object which contains all the access context info.
>* @return true if user can access the object, otherwise false.
>*/
> public abstract boolean checkPermission(AccessRequest accessRequest);
> /**
>* Set permissions for the target object.
>*
>* @param permissions
>*A list of permissions on the target object.
>* @param ugi User who sets the permissions.
>*/
> public abstract void setPermission(List permissions,
> UserGroupInformation ugi);
> {code}
> But the RangerYarnAuthorizer extends YarnAuthorizationProvider impletement
> the old method.
> {code:title=RangerYarnAuthorizer.java|borderStyle=solid}
> @Override
> public void setPermission(PrivilegedEntity entity, Map<AccessType,
> AccessControlList> permission, UserGroupInformation ugi) {
>...
> @Override
> public boolean checkPermission(AccessType accessType, PrivilegedEntity
> entity, UserGroupInformation ugi) {
> {code}
> I think yarn plugin should also impletement the new method. I will add a
> patch for it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (RANGER-1203) Ranger YARN Plugin supports Fair Scheduler
[ https://issues.apache.org/jira/browse/RANGER-1203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245540#comment-16245540 ] Colm O hEigeartaigh commented on RANGER-1203: - This issue can be marked as resolved. I've tested a local copy of Ranger, with the changes applied for Hadoop 3.0.0, and authorization works as expected with the Fair Scheduler. > Ranger YARN Plugin supports Fair Scheduler > -- > > Key: RANGER-1203 > URL: https://issues.apache.org/jira/browse/RANGER-1203 > Project: Ranger > Issue Type: New Feature > Components: plugins >Affects Versions: 0.6.1 >Reporter: Qiang Zhang >Assignee: Qiang Zhang > Labels: patch > > Currently,hadoop yarn Capacity Scheduler provides a YarnAuthorizationProvider > interface.Ranger yarn-plug-in can implement this interface. > Also Ranger supports the json packet parsing of Capacity Scheduler. > However,ranger does not support Fair Scheduler. > Increased functionality: > Ranger supports the json packet parsing of fair scheduler. > Hadoop yarn Fair Scheduler will also provide a YarnAuthorizationProvider > interface. > Ranger Yarn plugin adds authority control by fair scheduler, it relies on the > fair scheduler pluggable licensing feature patch that hadoop yarn supports. > Improvement issue of hadoop > yarn：https://issues.apache.org/jira/browse/YARN-4997 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Plugin Shim question
Hi all, I'm working on adding support for Hadoop 3.0.0 to the Yarn component (RANGER-1738). YarnAuthorizationProvider has some updated methods in Hadoop 3.0.0. It's easy to work around this in the RangerYarnAuthorizer though, so that it supports both Hadoop 2.7.x, 2.8.x and 3.0.0. The problem is in the plugin shim code for RangerYarnAuthorizer. It delegates the calls to the underlying YarnAuthorizationProvider instance. This means it's not possible to support both 2.7.x and 3.0.0 as it is in the plugin version of RangerYarnAuthorizer. Any ideas on this? The only way I can think of supporting it is to have separate plugin shims for Hadoop 2 + 3. Colm. -- Colm O hEigeartaigh Talend Community Coder http://coders.talend.com
[jira] [Resolved] (RANGER-1236) RangerYarnAuthorizer should be updated since YarnAuthorizationProvider modified in YARN-4571
[ https://issues.apache.org/jira/browse/RANGER-1236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh resolved RANGER-1236. - Resolution: Duplicate > RangerYarnAuthorizer should be updated since YarnAuthorizationProvider > modified in YARN-4571 > > > Key: RANGER-1236 > URL: https://issues.apache.org/jira/browse/RANGER-1236 > Project: Ranger > Issue Type: Bug >Reporter: Tao Jie >Assignee: Qiang Zhang > > Today Ranger Yarn Plugin supports authorization for Yarn by implements > interface YarnAuthorizationProvider which is based on Hadoop-2.7.0. > Since interface YarnAuthorizationProvider has been modified in YARN-4571, > RangerYarnAuthorizer should be updated accordingly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Review Request 63699: RANGER-1882 - Compilation error in kms module with Hadoop 2.8.x
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63699/ --- Review request for ranger. Bugs: RANGER-1882 https://issues.apache.org/jira/browse/RANGER-1882 Repository: ranger Description --- There is a compilation failure in the KMS module when compiled with Hadoop 2.8.x. The fix is just to use a constant String which works with both Hadoop 2.7.x, 2.8.x. Diffs - kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSAuthenticationFilter.java ada9a56f Diff: https://reviews.apache.org/r/63699/diff/1/ Testing --- Thanks, Colm O hEigeartaigh
[jira] [Updated] (RANGER-1882) Compilation error in kms module with Hadoop 2.8.x
[ https://issues.apache.org/jira/browse/RANGER-1882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh updated RANGER-1882: Attachment: 0001-RANGER-1882-Compilation-error-in-kms-module-with-Had.patch > Compilation error in kms module with Hadoop 2.8.x > - > > Key: RANGER-1882 > URL: https://issues.apache.org/jira/browse/RANGER-1882 > Project: Ranger > Issue Type: Improvement > Components: kms > Reporter: Colm O hEigeartaigh > Assignee: Colm O hEigeartaigh >Priority: Minor > Fix For: 1.0.0 > > Attachments: > 0001-RANGER-1882-Compilation-error-in-kms-module-with-Had.patch > > > There is a compilation failure in the KMS module when compiled with Hadoop > 2.8.x. The fix is just to use a constant String which works with both Hadoop > 2.7.x, 2.8.x. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
