Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-25 Thread Qiang Zhang


> On Oct. 24, 2017, 10:24 a.m., Nitin Galave wrote:
> > security-admin/src/main/webapp/scripts/models/BackboneFormDataType.js
> > Lines 85 (patched)
> > 
> >
> > I think adding individual conditions for the components to add text 
> > field with an icon( i.e information) on service form page instead we should 
> > provide text field with icon option itself in the service definition.
> > For example :
> > "configs":
> > [
> > {
> > "itemId": 2,
> > "name": "sqoop.url",
> > "type": "string",
> > "mandatory": true,
> > "defaultValue": "",
> > "validationRegEx":"",
> > "validationMessage": "",
> > "uiHint":"{ \"infoIcon\":true , \"info\": 
> > \"Enter information about configuration param\"}",
> > "label": "Sqoop URL"
> > }
> > ]
> 
> Colm O hEigeartaigh wrote:
> It looks like your comment was missed Nitin, could you create a separate 
> JIRA for it?

Ok. I will create a new JIRA to fixed it. Thanks very much!


- Qiang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review189038
---


On Oct. 24, 2017, 9:02 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated Oct. 24, 2017, 9:02 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
>   plugin-sqoop/pom.xml PRE-CREATION 
>   plugin-sqoop/scripts/install.properties PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
>   ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   security-admin/src/main/webapp/scripts/models/BackboneFormDataType.js 
> afcc290 
>   security-admin/src/main/webapp/scripts/

Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-25 Thread Colm O hEigeartaigh


> On Oct. 24, 2017, 10:24 a.m., Nitin Galave wrote:
> > security-admin/src/main/webapp/scripts/models/BackboneFormDataType.js
> > Lines 85 (patched)
> > 
> >
> > I think adding individual conditions for the components to add text 
> > field with an icon( i.e information) on service form page instead we should 
> > provide text field with icon option itself in the service definition.
> > For example :
> > "configs":
> > [
> > {
> > "itemId": 2,
> > "name": "sqoop.url",
> > "type": "string",
> > "mandatory": true,
> > "defaultValue": "",
> > "validationRegEx":"",
> > "validationMessage": "",
> > "uiHint":"{ \"infoIcon\":true , \"info\": 
> > \"Enter information about configuration param\"}",
> > "label": "Sqoop URL"
> > }
> > ]

It looks like your comment was missed Nitin, could you create a separate JIRA 
for it?


- Colm


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review189038
---


On Oct. 24, 2017, 9:02 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated Oct. 24, 2017, 9:02 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
>   plugin-sqoop/pom.xml PRE-CREATION 
>   plugin-sqoop/scripts/install.properties PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
>   ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   security-admin/src/main/webapp/scripts/models/BackboneFormDataType.js 
> afcc290 
>   security-admin/src/main/webapp/scripts/modules/globalize/message/en.js 
> 5900c55 
>   src/main/assembly/admin-web.xml 4dc52fd 
>   src/main

Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-24 Thread Nitin Galave

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review189038
---




security-admin/src/main/webapp/scripts/models/BackboneFormDataType.js
Lines 85 (patched)


I think adding individual conditions for the components to add text field 
with an icon( i.e information) on service form page instead we should provide 
text field with icon option itself in the service definition.
For example :
"configs":
[
{
"itemId": 2,
"name": "sqoop.url",
"type": "string",
"mandatory": true,
"defaultValue": "",
"validationRegEx":"",
"validationMessage": "",
"uiHint":"{ \"infoIcon\":true , \"info\": \"Enter 
information about configuration param\"}",
"label": "Sqoop URL"
}
]


- Nitin Galave


On Oct. 24, 2017, 9:02 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated Oct. 24, 2017, 9:02 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
>   plugin-sqoop/pom.xml PRE-CREATION 
>   plugin-sqoop/scripts/install.properties PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
>   ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   security-admin/src/main/webapp/scripts/models/BackboneFormDataType.js 
> afcc290 
>   security-admin/src/main/webapp/scripts/modules/globalize/message/en.js 
> 5900c55 
>   src/main/assembly/admin-web.xml 4dc52fd 
>   src/main/assembly/plugin-sqoop.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62710/diff/5/
> 
> 
> Testing
> ---
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>



Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-24 Thread Qiang Zhang


> On 十月 24, 2017, 8:30 a.m., pengjianhua wrote:
> > Please rebuild the patch.
> > error: patch failed: 
> > security-admin/src/main/webapp/scripts/modules/globalize/message/en.js:446
> > error: 
> > security-admin/src/main/webapp/scripts/modules/globalize/message/en.js: 
> > patch does not apply

OK, thanks.


- Qiang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review189032
---


On 十月 24, 2017, 9:02 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated 十月 24, 2017, 9:02 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
>   plugin-sqoop/pom.xml PRE-CREATION 
>   plugin-sqoop/scripts/install.properties PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
>   ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   security-admin/src/main/webapp/scripts/models/BackboneFormDataType.js 
> afcc290 
>   security-admin/src/main/webapp/scripts/modules/globalize/message/en.js 
> 5900c55 
>   src/main/assembly/admin-web.xml 4dc52fd 
>   src/main/assembly/plugin-sqoop.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62710/diff/5/
> 
> 
> Testing
> ---
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>



Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-24 Thread Qiang Zhang

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/
---

(Updated 十月 24, 2017, 9:02 a.m.)


Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, 
Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, 
sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.


Changes
---

Update to the latest code.


Bugs: RANGER-1810
https://issues.apache.org/jira/browse/RANGER-1810


Repository: ranger


Description
---

Apache Sqoop is a tool designed for efficiently transferring bulk data between 
Apache Hadoop and structured datastores such as relational databases. You can 
use Sqoop to import data from external structured datastores into Hadoop 
Distributed File System or related systems like Hive and HBase. Conversely, 
Sqoop can be used to extract data from Hadoop and export it to external 
structured datastores such as relational databases and enterprise data 
warehouses.It successfully graduated from the Incubator in March of 2012 and is 
now a Top-Level Apache project.
The Ranger will further expand the influence in the hadoop ecosystem if it 
supports sqoop authorization. So we should develop sqoop plugin to enable, 
monitor and manage apache Sqoop2.

Our test specialists have rigorously tested this feature.


Diffs (updated)
-

  agents-common/scripts/enable-agent.sh d31a264 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
 9463ab8 
  agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
PRE-CREATION 
  plugin-sqoop/.gitignore PRE-CREATION 
  plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
  plugin-sqoop/pom.xml PRE-CREATION 
  plugin-sqoop/scripts/install.properties PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
 PRE-CREATION 
  pom.xml 3958014 
  ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
  ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
  
ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
 PRE-CREATION 
  security-admin/src/main/webapp/scripts/models/BackboneFormDataType.js afcc290 
  security-admin/src/main/webapp/scripts/modules/globalize/message/en.js 
5900c55 
  src/main/assembly/admin-web.xml 4dc52fd 
  src/main/assembly/plugin-sqoop.xml PRE-CREATION 


Diff: https://reviews.apache.org/r/62710/diff/5/

Changes: https://reviews.apache.org/r/62710/diff/4-5/


Testing
---

Our test specialists have rigorously tested this feature.


Thanks,

Qiang Zhang



Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-24 Thread pengjianhua

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review189032
---



Please rebuild the patch.
error: patch failed: 
security-admin/src/main/webapp/scripts/modules/globalize/message/en.js:446
error: security-admin/src/main/webapp/scripts/modules/globalize/message/en.js: 
patch does not apply

- pengjianhua


On 十月 23, 2017, 9 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated 十月 23, 2017, 9 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
>   plugin-sqoop/pom.xml PRE-CREATION 
>   plugin-sqoop/scripts/install.properties PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
>   ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   security-admin/src/main/webapp/scripts/models/BackboneFormDataType.js 
> 3f8697e 
>   security-admin/src/main/webapp/scripts/modules/globalize/message/en.js 
> 811db0c 
>   src/main/assembly/admin-web.xml 4dc52fd 
>   src/main/assembly/plugin-sqoop.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62710/diff/4/
> 
> 
> Testing
> ---
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>



Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-23 Thread pengjianhua

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review188933
---


Ship it!




Ship It!

- pengjianhua


On 十月 23, 2017, 9 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated 十月 23, 2017, 9 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
>   plugin-sqoop/pom.xml PRE-CREATION 
>   plugin-sqoop/scripts/install.properties PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
>   ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   security-admin/src/main/webapp/scripts/models/BackboneFormDataType.js 
> 3f8697e 
>   security-admin/src/main/webapp/scripts/modules/globalize/message/en.js 
> 811db0c 
>   src/main/assembly/admin-web.xml 4dc52fd 
>   src/main/assembly/plugin-sqoop.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62710/diff/4/
> 
> 
> Testing
> ---
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>



Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-23 Thread Colm O hEigeartaigh

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review188931
---


Ship it!




Ship It!

- Colm O hEigeartaigh


On Oct. 23, 2017, 9 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated Oct. 23, 2017, 9 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
>   plugin-sqoop/pom.xml PRE-CREATION 
>   plugin-sqoop/scripts/install.properties PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
>   ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   security-admin/src/main/webapp/scripts/models/BackboneFormDataType.js 
> 3f8697e 
>   security-admin/src/main/webapp/scripts/modules/globalize/message/en.js 
> 811db0c 
>   src/main/assembly/admin-web.xml 4dc52fd 
>   src/main/assembly/plugin-sqoop.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62710/diff/4/
> 
> 
> Testing
> ---
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>



Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-23 Thread Qiang Zhang


> On 十月 20, 2017, 3:48 p.m., Colm O hEigeartaigh wrote:
> > Looks good thanks. Just three more changes:
> > 
> > a) Add a "hover over" hint for "Sqoop URL" in the Admin Console (e.g. 
> > should be "http://localhost:12000";)
> > b) Remove "password" from the UI as it's not used
> > c) Remove "password" from SqoopClient as it's not used.

OK, thanks.


- Qiang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review188844
---


On 十月 23, 2017, 9 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated 十月 23, 2017, 9 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
>   plugin-sqoop/pom.xml PRE-CREATION 
>   plugin-sqoop/scripts/install.properties PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
>   ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   security-admin/src/main/webapp/scripts/models/BackboneFormDataType.js 
> 3f8697e 
>   security-admin/src/main/webapp/scripts/modules/globalize/message/en.js 
> 811db0c 
>   src/main/assembly/admin-web.xml 4dc52fd 
>   src/main/assembly/plugin-sqoop.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62710/diff/4/
> 
> 
> Testing
> ---
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>



Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-23 Thread Qiang Zhang

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/
---

(Updated 十月 23, 2017, 9 a.m.)


Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, 
Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, 
sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.


Changes
---

a) Add a "hover over" hint for "Sqoop URL" in the Admin Console.
b) Remove "password" from the UI.
c) Remove "password" from SqoopClient.  
d) Change accessTypes of ranger-servicedef-sqoop.json from "Read" to "READ", 
from "Write" to "WRITE".


Bugs: RANGER-1810
https://issues.apache.org/jira/browse/RANGER-1810


Repository: ranger


Description
---

Apache Sqoop is a tool designed for efficiently transferring bulk data between 
Apache Hadoop and structured datastores such as relational databases. You can 
use Sqoop to import data from external structured datastores into Hadoop 
Distributed File System or related systems like Hive and HBase. Conversely, 
Sqoop can be used to extract data from Hadoop and export it to external 
structured datastores such as relational databases and enterprise data 
warehouses.It successfully graduated from the Incubator in March of 2012 and is 
now a Top-Level Apache project.
The Ranger will further expand the influence in the hadoop ecosystem if it 
supports sqoop authorization. So we should develop sqoop plugin to enable, 
monitor and manage apache Sqoop2.

Our test specialists have rigorously tested this feature.


Diffs (updated)
-

  agents-common/scripts/enable-agent.sh d31a264 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
 9463ab8 
  agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
PRE-CREATION 
  plugin-sqoop/.gitignore PRE-CREATION 
  plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
  plugin-sqoop/pom.xml PRE-CREATION 
  plugin-sqoop/scripts/install.properties PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
 PRE-CREATION 
  pom.xml 3958014 
  ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
  ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
  
ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
 PRE-CREATION 
  security-admin/src/main/webapp/scripts/models/BackboneFormDataType.js 3f8697e 
  security-admin/src/main/webapp/scripts/modules/globalize/message/en.js 
811db0c 
  src/main/assembly/admin-web.xml 4dc52fd 
  src/main/assembly/plugin-sqoop.xml PRE-CREATION 


Diff: https://reviews.apache.org/r/62710/diff/4/

Changes: https://reviews.apache.org/r/62710/diff/3-4/


Testing
---

Our test specialists have rigorously tested this feature.


Thanks,

Qiang Zhang



Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-20 Thread Colm O hEigeartaigh

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review188844
---



Looks good thanks. Just three more changes:

a) Add a "hover over" hint for "Sqoop URL" in the Admin Console (e.g. should be 
"http://localhost:12000";)
b) Remove "password" from the UI as it's not used
c) Remove "password" from SqoopClient as it's not used.

- Colm O hEigeartaigh


On Oct. 20, 2017, 7:16 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated Oct. 20, 2017, 7:16 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
>   plugin-sqoop/pom.xml PRE-CREATION 
>   plugin-sqoop/scripts/install.properties PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
>   ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   src/main/assembly/admin-web.xml 4dc52fd 
>   src/main/assembly/plugin-sqoop.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62710/diff/3/
> 
> 
> Testing
> ---
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>



Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-20 Thread Qiang Zhang

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/
---

(Updated 十月 20, 2017, 7:16 a.m.)


Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, 
Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, 
sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.


Changes
---

Update sqoop2 from 1.99.6 to 1.99.7 as it is the latest version


Bugs: RANGER-1810
https://issues.apache.org/jira/browse/RANGER-1810


Repository: ranger


Description
---

Apache Sqoop is a tool designed for efficiently transferring bulk data between 
Apache Hadoop and structured datastores such as relational databases. You can 
use Sqoop to import data from external structured datastores into Hadoop 
Distributed File System or related systems like Hive and HBase. Conversely, 
Sqoop can be used to extract data from Hadoop and export it to external 
structured datastores such as relational databases and enterprise data 
warehouses.It successfully graduated from the Incubator in March of 2012 and is 
now a Top-Level Apache project.
The Ranger will further expand the influence in the hadoop ecosystem if it 
supports sqoop authorization. So we should develop sqoop plugin to enable, 
monitor and manage apache Sqoop2.

Our test specialists have rigorously tested this feature.


Diffs (updated)
-

  agents-common/scripts/enable-agent.sh d31a264 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
 9463ab8 
  agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
PRE-CREATION 
  plugin-sqoop/.gitignore PRE-CREATION 
  plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
  plugin-sqoop/pom.xml PRE-CREATION 
  plugin-sqoop/scripts/install.properties PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
 PRE-CREATION 
  pom.xml 3958014 
  ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
  ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
  
ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
 PRE-CREATION 
  src/main/assembly/admin-web.xml 4dc52fd 
  src/main/assembly/plugin-sqoop.xml PRE-CREATION 


Diff: https://reviews.apache.org/r/62710/diff/3/

Changes: https://reviews.apache.org/r/62710/diff/2-3/


Testing
---

Our test specialists have rigorously tested this feature.


Thanks,

Qiang Zhang



Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-20 Thread Qiang Zhang


> On 十月 16, 2017, 12:51 p.m., Colm O hEigeartaigh wrote:
> > In addition to comment above, here are some initial remarks:
> > 
> > a) We should update to 1.99.7 as it is the latest version
> > b) getLinkList/getJobList are not implemented in SqoopClient.
> > c) There is a problem with the Sqoop REST API that I haven't been able to 
> > figure out. "http://localhost:12000/sqoop/v1/connectors"; returns a 404 with 
> > 1.99.7 (but http://localhost:12000/sqoop/v1/connector/kafka-connector works 
> > fine), so I can't get "Test Connection" to work. Same goes for the v1/jobs 
> > + v1/links mentioned in the REST API.
> > d) It was not picking up conf/ranger-sqoop-security.xml + hence the plugin 
> > wasn't working. I had to copy ranger-sqoop-security.xml to the root 
> > directory of the Sqoop installation. Any idea why this is?
> > e) After changing the policy in Ranger and having it synced down, it only 
> > "applies" once I exited the sqoop shell and restarted it. Any idea why this 
> > is? I guess it is caching the old policy somehow?

a)OK, thanks. I have updated my sqoop from 1.99.6 to 1.99.7.

b)Yes, I will give another patch to implement getLinkList/getJobList after this 
patch recevied,
because there is some work and optimization to do for connector/link/job, and 
it only affects resource lookup.
We are now focusing on Ranger's access control over Sqoop.

c)Yes, the Sqoop REST API is different between two versions.
1.99.6 : http://localhost:12000/sqoop/v1/connectors
1.99.7 : http://localhost:12000/sqoop/v1/connector/all
Same goes for the jobs and links.

d)I think it's because the conf directory is not in sqoop's classpath,
so ranger configuration files can not be read in JVM,
add conf to sqoop classpath can resolve this problem.
Please add the follow configuration to function sqoop_server_classpath_set of 
script bin/sqoop.sh:
CLASSPATH="$CLASSPATH":"$sqoop_home/conf"

e)Yes, the connector will be cached in sqoop client, 
when Ranger change the user's connector permissions,
the show connector command is not the latest result.

Execute the follow command can clear sqoop client cache:
set server --host zdh-11 --port 12000 --webapp sqoop

Please refer the following sqoop client code which cache connectors:
org.apache.sqoop.client.SqoopClient.getConnector(String)
public MConnector getConnector(String connectorName) {
  // Firstly try if we have this connector already in cache
  MConnector connector = getConnectorFromCache(connectorName);
  if(connector != null) return connector;

  // If the connector wasn't in cache and we have all connectors,
  // it simply do not exists.
  if(isAllConnectors) return null;

  // Retrieve all connectors from server
  getConnectors();
  return getConnectorFromCache(connectorName);
}


- Qiang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review188127
---


On 十月 12, 2017, 9:02 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated 十月 12, 2017, 9:02 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoo

Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-20 Thread Qiang Zhang


> On 十月 12, 2017, 3:33 p.m., Colm O hEigeartaigh wrote:
> > I think there is a problem with the installation scripts. The lib and conf 
> > dirs are defined as:
> > 
> > HCOMPONENT_LIB_DIR=${HCOMPONENT_INSTALL_DIR}/server/webapps/sqoop/WEB-INF/lib
> > HCOMPONENT_CONF_DIR=${HCOMPONENT_INSTALL_DIR}/server/conf
> > 
> > However these are not the standard locations if you download a Sqoop2 
> > server (instead just $sqoop_home/server/lib + $sqoop_home/conf).

OK, thanks.


- Qiang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review187801
---


On 十月 12, 2017, 9:02 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated 十月 12, 2017, 9:02 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
>   plugin-sqoop/pom.xml PRE-CREATION 
>   plugin-sqoop/scripts/install.properties PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
>   ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   src/main/assembly/admin-web.xml 0e97818 
>   src/main/assembly/plugin-sqoop.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62710/diff/2/
> 
> 
> Testing
> ---
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>



Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-16 Thread Colm O hEigeartaigh

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review188127
---



In addition to comment above, here are some initial remarks:

a) We should update to 1.99.7 as it is the latest version
b) getLinkList/getJobList are not implemented in SqoopClient.
c) There is a problem with the Sqoop REST API that I haven't been able to 
figure out. "http://localhost:12000/sqoop/v1/connectors"; returns a 404 with 
1.99.7 (but http://localhost:12000/sqoop/v1/connector/kafka-connector works 
fine), so I can't get "Test Connection" to work. Same goes for the v1/jobs + 
v1/links mentioned in the REST API.
d) It was not picking up conf/ranger-sqoop-security.xml + hence the plugin 
wasn't working. I had to copy ranger-sqoop-security.xml to the root directory 
of the Sqoop installation. Any idea why this is?
e) After changing the policy in Ranger and having it synced down, it only 
"applies" once I exited the sqoop shell and restarted it. Any idea why this is? 
I guess it is caching the old policy somehow?

- Colm O hEigeartaigh


On Oct. 12, 2017, 9:02 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated Oct. 12, 2017, 9:02 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
>   plugin-sqoop/pom.xml PRE-CREATION 
>   plugin-sqoop/scripts/install.properties PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
>   ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   src/main/assembly/admin-web.xml 0e97818 
>   src/main/assembly/plugin-sqoop.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62710/diff/2/
> 
> 
> Testing
> ---
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>



Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-12 Thread Colm O hEigeartaigh

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review187801
---



I think there is a problem with the installation scripts. The lib and conf dirs 
are defined as:

HCOMPONENT_LIB_DIR=${HCOMPONENT_INSTALL_DIR}/server/webapps/sqoop/WEB-INF/lib
HCOMPONENT_CONF_DIR=${HCOMPONENT_INSTALL_DIR}/server/conf

However these are not the standard locations if you download a Sqoop2 server 
(instead just $sqoop_home/server/lib + $sqoop_home/conf).

- Colm O hEigeartaigh


On Oct. 12, 2017, 9:02 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated Oct. 12, 2017, 9:02 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
>   plugin-sqoop/pom.xml PRE-CREATION 
>   plugin-sqoop/scripts/install.properties PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
>   ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   src/main/assembly/admin-web.xml 0e97818 
>   src/main/assembly/plugin-sqoop.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62710/diff/2/
> 
> 
> Testing
> ---
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>



Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-12 Thread Qiang Zhang


> On 十月 2, 2017, 6:13 p.m., Alejandro Fernandez wrote:
> > plugin-sqoop/scripts/install.properties
> > Lines 39 (patched)
> > 
> >
> > Should all of these commented lines be removed?

These commented lines should not be removed, they are useful for user 
configuration parameters.


> On 十月 2, 2017, 6:13 p.m., Alejandro Fernandez wrote:
> > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
> > Lines 85 (patched)
> > 
> >
> > If any errors are found, should this raise an exception instead of just 
> > logging?

There is no need to throw an exception, and it doesn't affect creating policies 
in Ranger.
Notice:" You can still save the repository and start creating policies, but you 
would not be able to use autocomplete for resource names. Check 
ranger_admin.log for more info."


> On 十月 2, 2017, 6:13 p.m., Alejandro Fernandez wrote:
> > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
> > Lines 103 (patched)
> > 
> >
> > Are there any paths in which the response object needs to be .closed()?

The response will be closed in method 
org.apache.ranger.services.sqoop.client.SqoopClient.getSqoopConnectorResponse(ClientResponse)
 finally block.


- Qiang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review186860
---


On 十月 12, 2017, 9:02 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated 十月 12, 2017, 9:02 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
>   plugin-sqoop/pom.xml PRE-CREATION 
>   plugin-sqoop/scripts/install.properties PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
>   ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java

Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-12 Thread Qiang Zhang


> On 十月 2, 2017, 6:46 p.m., Ramesh Mani wrote:
> > plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
> > Lines 99 (patched)
> > 
> >
> > can privilege be null? Is it test for this case?
> > Please add some unit test also for various scenarios.

OK,I will submit another patch for some unit test.
And according to Sqoop AuthorizationEngine.checkPrivilege method,
the privilege would not be null, but privileges would be an empty list.
Sqoop code reference, please notice SQOOP-2256 commont:
org.apache.sqoop.security.authorization.AuthorizationEngine.checkPrivilege(MPrivilege...)
  private static void checkPrivilege(MPrivilege... privileges) {
AuthorizationHandler handler = 
AuthorizationManager.getAuthorizationHandler();
UserGroupInformation user = HttpUserGroupInformation.get();
String user_name = user == null ? StringUtils.EMPTY : 
user.getShortUserName();
MPrincipal principal = new MPrincipal(user_name, MPrincipal.TYPE.USER);

// SQOOP-2256: Hack code, do not check privilege when the user is the 
creator
// If the user is the owner/creator of this resource, then privilege will
// not be checked. It is a hack code for the time being. The concept of
// "Owner" will be added in the future and this code will be removed.
ArrayList privilegesNeedCheck = new ArrayList();
for (MPrivilege privilege : privileges) {
  Repository repository = RepositoryManager.getInstance().getRepository();
  if 
(MResource.TYPE.LINK.name().equalsIgnoreCase(privilege.getResource().getType()))
 {
MLink link = 
repository.findLink(Long.valueOf(privilege.getResource().getName()));
if (!user_name.equals(link.getCreationUser())) {
  privilegesNeedCheck.add(privilege);
}
  } else if 
(MResource.TYPE.JOB.name().equalsIgnoreCase(privilege.getResource().getType())) 
{
MJob job = 
repository.findJob(Long.valueOf(privilege.getResource().getName()));
if (!user_name.equals(job.getCreationUser())) {
  privilegesNeedCheck.add(privilege);
}
  } else {
privilegesNeedCheck.add(privilege);
  }
}

handler.checkPrivileges(principal, privilegesNeedCheck);
  }
}


- Qiang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review186865
---


On 十月 12, 2017, 9:02 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated 十月 12, 2017, 9:02 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
>   plugin-sqoop/pom.xml PRE-CREATION 
>   plugin-sqoop/scripts/install.properties PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.j

Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-12 Thread Qiang Zhang

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/
---

(Updated 十月 12, 2017, 9:02 a.m.)


Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, 
Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, 
sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.


Changes
---

1.Fix testConnection failed after sqooop enable ranger access control.
2.According to SQOOP-2256, log when checkPrivilege input prameter privileges is 
empty instead of throw expection.
3.Ohter small modifications for optimization.


Bugs: RANGER-1810
https://issues.apache.org/jira/browse/RANGER-1810


Repository: ranger


Description
---

Apache Sqoop is a tool designed for efficiently transferring bulk data between 
Apache Hadoop and structured datastores such as relational databases. You can 
use Sqoop to import data from external structured datastores into Hadoop 
Distributed File System or related systems like Hive and HBase. Conversely, 
Sqoop can be used to extract data from Hadoop and export it to external 
structured datastores such as relational databases and enterprise data 
warehouses.It successfully graduated from the Incubator in March of 2012 and is 
now a Top-Level Apache project.
The Ranger will further expand the influence in the hadoop ecosystem if it 
supports sqoop authorization. So we should develop sqoop plugin to enable, 
monitor and manage apache Sqoop2.

Our test specialists have rigorously tested this feature.


Diffs (updated)
-

  agents-common/scripts/enable-agent.sh d31a264 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
 9463ab8 
  agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
PRE-CREATION 
  plugin-sqoop/.gitignore PRE-CREATION 
  plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
  plugin-sqoop/pom.xml PRE-CREATION 
  plugin-sqoop/scripts/install.properties PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
 PRE-CREATION 
  pom.xml 3958014 
  ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
  ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
  
ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
 PRE-CREATION 
  src/main/assembly/admin-web.xml 0e97818 
  src/main/assembly/plugin-sqoop.xml PRE-CREATION 


Diff: https://reviews.apache.org/r/62710/diff/2/

Changes: https://reviews.apache.org/r/62710/diff/1-2/


Testing
---

Our test specialists have rigorously tested this feature.


Thanks,

Qiang Zhang



Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-02 Thread Ramesh Mani

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review186865
---




plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
Lines 99 (patched)


can privilege be null? Is it test for this case?
Please add some unit test also for various scenarios.


- Ramesh Mani


On Sept. 30, 2017, 8:34 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated Sept. 30, 2017, 8:34 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java 
> 58cdd35 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
>   plugin-sqoop/pom.xml PRE-CREATION 
>   plugin-sqoop/scripts/install.properties PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
>   ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   src/main/assembly/admin-web.xml 0e97818 
>   src/main/assembly/plugin-sqoop.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62710/diff/1/
> 
> 
> Testing
> ---
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>



Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-02 Thread Alejandro Fernandez


> On Oct. 2, 2017, 6:13 p.m., Alejandro Fernandez wrote:
> >

Awesome contribution.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review186860
---


On Sept. 30, 2017, 8:34 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated Sept. 30, 2017, 8:34 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java 
> 58cdd35 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
>   plugin-sqoop/pom.xml PRE-CREATION 
>   plugin-sqoop/scripts/install.properties PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
>   ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   src/main/assembly/admin-web.xml 0e97818 
>   src/main/assembly/plugin-sqoop.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62710/diff/1/
> 
> 
> Testing
> ---
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>



Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-10-02 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review186860
---




plugin-sqoop/scripts/install.properties
Lines 39 (patched)


Should all of these commented lines be removed?



plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
Lines 49 (patched)


Add JavaDoc to all new classes and methods.



plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
Lines 85 (patched)


If any errors are found, should this raise an exception instead of just 
logging?



plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
Lines 103 (patched)


Are there any paths in which the response object needs to be .closed()?



plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
Lines 21 (patched)


Add JavaDoc


- Alejandro Fernandez


On Sept. 30, 2017, 8:34 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated Sept. 30, 2017, 8:34 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java 
> 58cdd35 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
>   plugin-sqoop/pom.xml PRE-CREATION 
>   plugin-sqoop/scripts/install.properties PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
>   ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   src/main/assembly/admin-web.xml 0e97818 
>   src/main/assembly/plugin-sqoop.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62710/diff/1/
> 
> 
> Testing
> ---
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>



Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-09-30 Thread Qiang Zhang

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/
---

(Updated Sept. 30, 2017, 8:34 a.m.)


Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, 
Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, 
sam  rome, and Velmurugan Periasamy.


Bugs: RANGER-1810
https://issues.apache.org/jira/browse/RANGER-1810


Repository: ranger


Description (updated)
---

Apache Sqoop is a tool designed for efficiently transferring bulk data between 
Apache Hadoop and structured datastores such as relational databases. You can 
use Sqoop to import data from external structured datastores into Hadoop 
Distributed File System or related systems like Hive and HBase. Conversely, 
Sqoop can be used to extract data from Hadoop and export it to external 
structured datastores such as relational databases and enterprise data 
warehouses.It successfully graduated from the Incubator in March of 2012 and is 
now a Top-Level Apache project.
The Ranger will further expand the influence in the hadoop ecosystem if it 
supports sqoop authorization. So we should develop sqoop plugin to enable, 
monitor and manage apache Sqoop2.

Our test specialists have rigorously tested this feature.


Diffs
-

  agents-common/scripts/enable-agent.sh d31a264 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
 9463ab8 
  agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java 
58cdd35 
  agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
PRE-CREATION 
  plugin-sqoop/.gitignore PRE-CREATION 
  plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
  plugin-sqoop/pom.xml PRE-CREATION 
  plugin-sqoop/scripts/install.properties PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
 PRE-CREATION 
  pom.xml 3958014 
  ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
  ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
  
ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
 PRE-CREATION 
  src/main/assembly/admin-web.xml 0e97818 
  src/main/assembly/plugin-sqoop.xml PRE-CREATION 


Diff: https://reviews.apache.org/r/62710/diff/1/


Testing
---

Our test specialists have rigorously tested this feature.


Thanks,

Qiang Zhang



Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-09-30 Thread Qiang Zhang

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/
---

(Updated Sept. 30, 2017, 8:17 a.m.)


Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, 
Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, 
sam  rome, and Velmurugan Periasamy.


Bugs: RANGER-1810
https://issues.apache.org/jira/browse/RANGER-1810


Repository: ranger


Description (updated)
---

Apache Sqoop2 is a tool designed for efficiently transferring data betweeen 
structured, semi-structured and unstructured data sources. Relational databases 
are examples of structured data sources with well defined schema for the data 
they store. Cassandra, Hbase are examples of semi-structured data sources and 
HDFS is an example of unstructured data source that Sqoop can support.
Ranger should support plugin to enable, monitor and manage apache Sqoop2

Our test specialists have rigorously tested this feature.


Diffs
-

  agents-common/scripts/enable-agent.sh d31a264 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
 9463ab8 
  agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java 
58cdd35 
  agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
PRE-CREATION 
  plugin-sqoop/.gitignore PRE-CREATION 
  plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
  plugin-sqoop/pom.xml PRE-CREATION 
  plugin-sqoop/scripts/install.properties PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
 PRE-CREATION 
  pom.xml 3958014 
  ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
  ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
  
ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
 PRE-CREATION 
  src/main/assembly/admin-web.xml 0e97818 
  src/main/assembly/plugin-sqoop.xml PRE-CREATION 


Diff: https://reviews.apache.org/r/62710/diff/1/


Testing (updated)
---

Our test specialists have rigorously tested this feature.


Thanks,

Qiang Zhang



Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

2017-09-30 Thread Qiang Zhang

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/
---

Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, 
Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan Neethiraj, sam  rome, 
and Velmurugan Periasamy.


Bugs: RANGER-1810
https://issues.apache.org/jira/browse/RANGER-1810


Repository: ranger


Description
---

Apache Sqoop2 is a tool designed for efficiently transferring data betweeen 
structured, semi-structured and unstructured data sources. Relational databases 
are examples of structured data sources with well defined schema for the data 
they store. Cassandra, Hbase are examples of semi-structured data sources and 
HDFS is an example of unstructured data source that Sqoop can support.
Ranger should support plugin to enable, monitor and manage apache Sqoop2


Diffs
-

  agents-common/scripts/enable-agent.sh d31a264 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
 9463ab8 
  agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java 
58cdd35 
  agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
PRE-CREATION 
  plugin-sqoop/.gitignore PRE-CREATION 
  plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
  plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
  plugin-sqoop/pom.xml PRE-CREATION 
  plugin-sqoop/scripts/install.properties PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
 PRE-CREATION 
  
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
 PRE-CREATION 
  pom.xml 3958014 
  ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
  ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
  
ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
 PRE-CREATION 
  src/main/assembly/admin-web.xml 0e97818 
  src/main/assembly/plugin-sqoop.xml PRE-CREATION 


Diff: https://reviews.apache.org/r/62710/diff/1/


Testing
---

Tested


Thanks,

Qiang Zhang