[GitHub] incubator-hawq pull request #1253: HAWQ-1485. Use user/password instead of c...

2017-06-12 Thread interma 
Github user interma commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1253#discussion_r121583805
  
--- Diff: 
ranger-plugin/admin-plugin/src/main/java/org/apache/hawq/ranger/service/HawqClient.java
 ---
@@ -91,125 +88,42 @@
 }
 
 public HawqClient(String serviceName, Map 
connectionProperties) throws Exception {
-super(serviceName,connectionProperties);
+super(serviceName, connectionProperties);
 this.connectionProperties = connectionProperties;
-initHawq();
 }
-
-public void initHawq() throws Exception {
-   if(connectionProperties.containsKey(AUTHENTICATION)) {
-   isKerberosAuth = 
connectionProperties.get(AUTHENTICATION).equals(KERBEROS);
-   }
-   if (isKerberosAuth) {
-   LOG.info("Secured Mode: JDBC Connection done with 
preAuthenticated Subject");
-   
-   // do kinit in hawqclient by principal name and password
-   final String userName = getConfigHolder().getUserName();
-   final String password = getConfigHolder().getPassword();
-   
-   String[] kinitcmd ={
-   "/bin/sh",
-   "-c",
-   "echo '"+password+"' | kinit " + userName
-   };
-   java.lang.Runtime rt = java.lang.Runtime.getRuntime();
-   if (LOG.isDebugEnabled()) {
-   LOG.debug("kinit command: "+"echo 
'"+password+"' | kinit " + userName);
-   }
-   java.lang.Process p = rt.exec(kinitcmd);
-   
-   Subject.doAs(getLoginSubject(), new 
PrivilegedExceptionAction(){
-   public Void run() throws Exception {
-   final String lookupPricipalName = 
getConfigHolder().getUserName();
-   final String serverprincipal = 
connectionProperties.get("principal");
-   initConnectionKerberos(serverprincipal, 
lookupPricipalName);
-   return null;
-   }});
-   }
-   else {
-   LOG.info("Trying to use UnSecure client with username 
"+ getConfigHolder().getUserName() +" and password");
-   final String userName = getConfigHolder().getUserName();
-   final String password = getConfigHolder().getPassword();
-   initConnection(userName, password);
-   }
-   }
-
-private void initConnectionKerberos(String serverPricipal, String 
userPrincipal) throws SQLException{
-   try {
-   String url = 
String.format("jdbc:postgresql://%s:%s/%s?kerberosServerName=%s=pgjdbc=%s",
 
-   connectionProperties.get("hostname"), 
-   connectionProperties.get("port"), 
DEFAULT_DATABASE, 
-   serverPricipal, userPrincipal
-   );
-   if (LOG.isDebugEnabled()) {
-   LOG.debug("InitConnectionKerberos "+ url);
-   }
-   con = DriverManager.getConnection(url); 
-   jdbc_url_template = 
String.format("jdbc:postgresql://%s:%s/%s?kerberosServerName=%s=pgjdbc=%s",
 
-   connectionProperties.get("hostname"), 
-   connectionProperties.get("port"), 
DEFAULT_DATABASE_TEMPLATE, 
-   serverPricipal, userPrincipal
-   );
-   } catch (SQLException e) {
- e.printStackTrace();
-  LOG.error("Unable to Connect to Hawq", e);
-  throw e;
-   } catch (SecurityException se) {
-   se.printStackTrace();
-   }
-   }
-
-   
-   private void initConnection(String userName, String password) throws 
SQLException  {
-   try {
-   String url = 
String.format("jdbc:postgresql://%s:%s/%s", 
connectionProperties.get("hostname"), connectionProperties.get("port"), 
DEFAULT_DATABASE);
-   if (LOG.isDebugEnabled()) {
-   LOG.debug("InitConnectionKerberos "+ url);
-   }
-   con = DriverManager.getConnection(url, userName, 
password);
-   jdbc_url_template = 
String.format("jdbc:postgresql://%s:%s/%s", 
connectionProperties.get("hostname"),

[GitHub] incubator-hawq pull request #1253: HAWQ-1485. Use user/password instead of c...

2017-06-12 Thread interma 
Github user interma commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1253#discussion_r121583655
  
--- Diff: 
ranger-plugin/admin-plugin/src/main/java/org/apache/hawq/ranger/service/RangerServiceHawq.java
 ---
@@ -25,41 +25,48 @@
 import org.apache.commons.logging.Log;
 import org.apache.commons.logging.LogFactory;
 import org.apache.ranger.plugin.service.ResourceLookupContext;
+import org.apache.ranger.plugin.util.PasswordUtils;
 
 import java.util.*;
+import java.io.IOException;
 
 public class RangerServiceHawq extends RangerBaseService {
 
 private static final Log LOG = 
LogFactory.getLog(RangerServiceHawq.class);
 
 public RangerServiceHawq() {
-   super();
-   }
-   
-   @Override
-   public void init(RangerServiceDef serviceDef, RangerService service) {
-   super.init(serviceDef, service);
-   }
-   
+super();
+}
+
+@Override
+public void init(RangerServiceDef serviceDef, RangerService service) {
+super.init(serviceDef, service);
+}
+
 @Override
 public HashMap validateConfig() throws Exception {
 boolean isDebugEnabled = LOG.isDebugEnabled();
 
-if(isDebugEnabled) {
+if (isDebugEnabled) {
 LOG.debug("==> RangerServiceHawq.validateConfig Service: 
(hawq)");
 }
 
 HashMap result = new HashMap<>();
-String serviceName = getServiceName();
 if (configs != null) {
-try  {
-HawqClient hawqClient = new HawqClient(serviceName, 
configs);
-result = hawqClient.checkConnection(configs);
-hawqClient.close();
-} catch (HadoopException e) {
-LOG.error("<== RangerServiceHawq.validateConfig Error:" + 
e);
-throw e;
+boolean retry = false;
+
+// try normal password (user input in webform)
+result = check_connection(configs);
+if (!(boolean)(result.get("connectivityStatus"))) {
+retry = true;
+}
+
+if (retry) {
+// try decrypt password
+decrypt_password(configs);
+result = check_connection(configs);
--- End diff --

Explain why try check_connection() twice in this validateConfig():

validateConfig() is called in ranger UI "test connection" buttion, and the 
password stored in ranger db is encrypted (I have tested).

There are two situations when user clicks "test connection" button:
1. User already inputed a new password, and the password is plain(not 
encrypted). 
2. User didn't change anything, just click button, but the password is 
encrypted (fetch directly from ranger db).

Since there is no way to identify these two situations, we just try twice: 
plain password and decrypted password, test connection is success if one of 
them is passed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1253: Use user/password jdbc method in Ranger l...

2017-06-12 Thread interma 
GitHub user interma opened a pull request:

https://github.com/apache/incubator-hawq/pull/1253

Use user/password jdbc method in Ranger lookup for HAWQ with Kerberos…

… enabled. 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/interma/interma-hawq hawq-1485

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/1253.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1253


commit 6810f9e7cf6ab01b70d2a6b37038f65955496853
Author: interma 
Date:   2017-06-13T04:09:40Z

Use user/password jdbc method in Ranger lookup for HAWQ with Kerberos 
enabled.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (HAWQ-1485) Use user/password instead of credentials cache in Ranger lookup for HAWQ with Kerberos enabled.

2017-06-12 Thread Hongxu Ma (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HAWQ-1485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hongxu Ma reassigned HAWQ-1485:
---

Assignee: Hongxu Ma  (was: Radar Lei)

> Use user/password instead of credentials cache in Ranger lookup for HAWQ with 
> Kerberos enabled.
> ---
>
> Key: HAWQ-1485
> URL: https://issues.apache.org/jira/browse/HAWQ-1485
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Security
>Reporter: Hongxu Ma
>Assignee: Hongxu Ma
> Fix For: 2.3.0.0-incubating
>
>
> When used credentials cache:
> Try error password in Ranger UI doesn't destroy the existed kerberos 
> credentials (created by last success kinit command)
> It's a strange behavior to user.
> So we should use user/password for kerberos authentication.
> Core logic:
> {code}
> Properties props = new Properties();
> if (connectionProperties.containsKey(AUTHENTICATION) && 
> connectionProperties.get(AUTHENTICATION).equals(KERBEROS)) {
> //kerberos mode
> props.setProperty("kerberosServerName", 
> connectionProperties.get("principal"));
> props.setProperty("jaasApplicationName", "pgjdbc");
> }
> String url = String.format("jdbc:postgresql://%s:%s/%s", 
> connectionProperties.get("hostname"), connectionProperties.get("port"), db);
> props.setProperty("user", connectionProperties.get("username"));
> props.setProperty("password", connectionProperties.get("password"));
> return DriverManager.getConnection(url, props);
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HAWQ-1485) Use user/password instead of credentials cache in Ranger lookup for HAWQ with Kerberos enabled.

2017-06-12 Thread Hongxu Ma (JIRA) 
Hongxu Ma created HAWQ-1485:
---

 Summary: Use user/password instead of credentials cache in Ranger 
lookup for HAWQ with Kerberos enabled.
 Key: HAWQ-1485
 URL: https://issues.apache.org/jira/browse/HAWQ-1485
 Project: Apache HAWQ
  Issue Type: Sub-task
  Components: Security
Reporter: Hongxu Ma
Assignee: Radar Lei
 Fix For: 2.3.0.0-incubating


When used credentials cache:
Try error password in Ranger UI doesn't destroy the existed kerberos 
credentials (created by last success kinit command)
It's a strange behavior to user.

So we should use user/password for kerberos authentication.
Core logic:
{code}
Properties props = new Properties();
if (connectionProperties.containsKey(AUTHENTICATION) && 
connectionProperties.get(AUTHENTICATION).equals(KERBEROS)) {
//kerberos mode
props.setProperty("kerberosServerName", 
connectionProperties.get("principal"));
props.setProperty("jaasApplicationName", "pgjdbc");
}

String url = String.format("jdbc:postgresql://%s:%s/%s", 
connectionProperties.get("hostname"), connectionProperties.get("port"), db);
props.setProperty("user", connectionProperties.get("username"));
props.setProperty("password", connectionProperties.get("password"));

return DriverManager.getConnection(url, props);
{code}




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HAWQ-1484) Spin PXF into a Separate Project for Data Access

2017-06-12 Thread Suminda Dharmasena (JIRA) 
Suminda Dharmasena created HAWQ-1484:


 Summary: Spin PXF into a Separate Project for Data Access
 Key: HAWQ-1484
 URL: https://issues.apache.org/jira/browse/HAWQ-1484
 Project: Apache HAWQ
  Issue Type: New Feature
Reporter: Suminda Dharmasena
Assignee: Radar Lei


Can the PXF be spinned into a seperate projects here they can be used as a 
basis for other data access projects.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] incubator-hawq issue #1251: HAWQ-1480 - Added feature for packing a core fil...

2017-06-12 Thread linwen 
Github user linwen commented on the issue:

https://github.com/apache/incubator-hawq/pull/1251
  
merged, this pr can be closed now. Thanks! 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-1435) docs - add usage info for pxf jdbc plug-in

2017-06-12 Thread Lisa Owen (JIRA) 

[ 
https://issues.apache.org/jira/browse/HAWQ-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047197#comment-16047197
 ] 

Lisa Owen commented on HAWQ-1435:
-

[~michael.andre.pearce] - please review.

> docs - add usage info for pxf jdbc plug-in
> --
>
> Key: HAWQ-1435
> URL: https://issues.apache.org/jira/browse/HAWQ-1435
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Lisa Owen
>Assignee: David Yozie
>
> create usage info for the new jdbc plug-in.  there is some good info in the 
> pxf-jdbc README.md. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HAWQ-1435) docs - add usage info for pxf jdbc plug-in

2017-06-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/HAWQ-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047191#comment-16047191
 ] 

ASF GitHub Bot commented on HAWQ-1435:
--

GitHub user lisakowen opened a pull request:

https://github.com/apache/incubator-hawq-docs/pull/124

HAWQ-1435 document new pxf jdbc plug-in

document the community-contributed PXF JDBC plug-in.  include a simple 
mysql example.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lisakowen/incubator-hawq-docs feature/pxf-jdbc

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq-docs/pull/124.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #124


commit a008125b2864c3acbc3b630030cb614a5ea2679f
Author: Lisa Owen 
Date:   2017-04-19T00:13:57Z

document new pxf jdbc plug-in




> docs - add usage info for pxf jdbc plug-in
> --
>
> Key: HAWQ-1435
> URL: https://issues.apache.org/jira/browse/HAWQ-1435
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Documentation
>Reporter: Lisa Owen
>Assignee: David Yozie
>
> create usage info for the new jdbc plug-in.  there is some good info in the 
> pxf-jdbc README.md. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] incubator-hawq issue #1251: HAWQ-1480 - Added feature for packing a core fil...

2017-06-12 Thread edespino 
Github user edespino commented on the issue:

https://github.com/apache/incubator-hawq/pull/1251
  
This PR (sans documentation which is included in PR #123) LGTM. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---