[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[ https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=782311&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-782311 ] ASF GitHub Bot logged work on HIVE-25929: - Author: ASF GitHub Bot Created on: 17/Jun/22 08:45 Start Date: 17/Jun/22 08:45 Worklog Time Spent: 10m Work Description: abstractdog closed pull request #3019: HIVE-25929: Let secret config properties to be propagated to Tez URL: https://github.com/apache/hive/pull/3019 Issue Time Tracking --- Worklog Id: (was: 782311) Time Spent: 3h 10m (was: 3h) > Let secret config properties to be propagated to Tez > > > Key: HIVE-25929 > URL: https://issues.apache.org/jira/browse/HIVE-25929 > Project: Hive > Issue Type: Bug >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > History in chronological order: > HIVE-10508: removed some passwords from config that's propagated to execution > engines > HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the > hardcoded list in HIVE-10508 > the problem with HIVE-9013 is it's about to introduce a common method for > removing sensitive data from Configuration, which absolutely makes sense in > most of the cases (set command showing sensitive data), but can cause issues > e.g. while using non-secure cloud connectors on a cluster, where instead of > the hadoop credential provider API (which is considered the secure and proper > way), passwords/secrets appear in the Configuration object (like: > "fs.azure.account.oauth2.client.secret") > 2 possible solutions: > 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" -> > which defaults to "hive.conf.hidden.list" (configurable, but maybe just more > confusing to users, having a new config property which should be understood > and maintained on a cluster) > 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced > by HIVE-10508 (convenient, less confusing for users, but cannot be configured) -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[
https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=782312&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-782312
]
ASF GitHub Bot logged work on HIVE-25929:
-
Author: ASF GitHub Bot
Created on: 17/Jun/22 08:45
Start Date: 17/Jun/22 08:45
Worklog Time Spent: 10m
Work Description: abstractdog commented on code in PR #3019:
URL: https://github.com/apache/hive/pull/3019#discussion_r899907375
##
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:
##
@@ -5455,6 +5455,22 @@ public static enum ConfVars {
+ ",hive.zookeeper.ssl.truststore.location"
+ ",hive.zookeeper.ssl.truststore.password",
"Comma separated list of configuration options which should not be
read by normal user like passwords"),
+HIVE_CONF_PROPAGATE_EXEC_ENGINES("hive.conf.propagate.exec.engines",
+"fs.s3.awsAccessKeyId"
Review Comment:
yes, if configs are dumped in logs then it's a security risk, but currently,
we have no other way to support 'less secure' option
('more secure' uses hadoop credential provider)
I don't have strong opinions about this one, so I'm closing the jira as
invalid, until we're not facing customer pressure to support it
Issue Time Tracking
---
Worklog Id: (was: 782312)
Time Spent: 3h 20m (was: 3h 10m)
> Let secret config properties to be propagated to Tez
>
>
> Key: HIVE-25929
> URL: https://issues.apache.org/jira/browse/HIVE-25929
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> History in chronological order:
> HIVE-10508: removed some passwords from config that's propagated to execution
> engines
> HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the
> hardcoded list in HIVE-10508
> the problem with HIVE-9013 is it's about to introduce a common method for
> removing sensitive data from Configuration, which absolutely makes sense in
> most of the cases (set command showing sensitive data), but can cause issues
> e.g. while using non-secure cloud connectors on a cluster, where instead of
> the hadoop credential provider API (which is considered the secure and proper
> way), passwords/secrets appear in the Configuration object (like:
> "fs.azure.account.oauth2.client.secret")
> 2 possible solutions:
> 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" ->
> which defaults to "hive.conf.hidden.list" (configurable, but maybe just more
> confusing to users, having a new config property which should be understood
> and maintained on a cluster)
> 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced
> by HIVE-10508 (convenient, less confusing for users, but cannot be configured)
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[
https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=782310&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-782310
]
ASF GitHub Bot logged work on HIVE-25929:
-
Author: ASF GitHub Bot
Created on: 17/Jun/22 08:44
Start Date: 17/Jun/22 08:44
Worklog Time Spent: 10m
Work Description: abstractdog commented on code in PR #3019:
URL: https://github.com/apache/hive/pull/3019#discussion_r899907375
##
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:
##
@@ -5455,6 +5455,22 @@ public static enum ConfVars {
+ ",hive.zookeeper.ssl.truststore.location"
+ ",hive.zookeeper.ssl.truststore.password",
"Comma separated list of configuration options which should not be
read by normal user like passwords"),
+HIVE_CONF_PROPAGATE_EXEC_ENGINES("hive.conf.propagate.exec.engines",
+"fs.s3.awsAccessKeyId"
Review Comment:
yes, if configs are dumped in logs then it's a security risk, but currently,
we have no other way to support 'less secure' option
('more secure' uses hadoop credential provider)
I don't have strong opinions about this one, so I'm closing this as invalid,
until we're not facing customer pressure to support it
Issue Time Tracking
---
Worklog Id: (was: 782310)
Time Spent: 3h (was: 2h 50m)
> Let secret config properties to be propagated to Tez
>
>
> Key: HIVE-25929
> URL: https://issues.apache.org/jira/browse/HIVE-25929
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h
> Remaining Estimate: 0h
>
> History in chronological order:
> HIVE-10508: removed some passwords from config that's propagated to execution
> engines
> HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the
> hardcoded list in HIVE-10508
> the problem with HIVE-9013 is it's about to introduce a common method for
> removing sensitive data from Configuration, which absolutely makes sense in
> most of the cases (set command showing sensitive data), but can cause issues
> e.g. while using non-secure cloud connectors on a cluster, where instead of
> the hadoop credential provider API (which is considered the secure and proper
> way), passwords/secrets appear in the Configuration object (like:
> "fs.azure.account.oauth2.client.secret")
> 2 possible solutions:
> 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" ->
> which defaults to "hive.conf.hidden.list" (configurable, but maybe just more
> confusing to users, having a new config property which should be understood
> and maintained on a cluster)
> 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced
> by HIVE-10508 (convenient, less confusing for users, but cannot be configured)
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[
https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=771068&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-771068
]
ASF GitHub Bot logged work on HIVE-25929:
-
Author: ASF GitHub Bot
Created on: 16/May/22 23:48
Start Date: 16/May/22 23:48
Worklog Time Spent: 10m
Work Description: rbalamohan commented on code in PR #3019:
URL: https://github.com/apache/hive/pull/3019#discussion_r874242811
##
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:
##
@@ -5455,6 +5455,22 @@ public static enum ConfVars {
+ ",hive.zookeeper.ssl.truststore.location"
+ ",hive.zookeeper.ssl.truststore.password",
"Comma separated list of configuration options which should not be
read by normal user like passwords"),
+HIVE_CONF_PROPAGATE_EXEC_ENGINES("hive.conf.propagate.exec.engines",
+"fs.s3.awsAccessKeyId"
Review Comment:
If configs are dumped in logs or available as files in the local directories
of AM or tasks, this would be visible to end users which can be a serious risk.
Issue Time Tracking
---
Worklog Id: (was: 771068)
Time Spent: 2h 50m (was: 2h 40m)
> Let secret config properties to be propagated to Tez
>
>
> Key: HIVE-25929
> URL: https://issues.apache.org/jira/browse/HIVE-25929
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> History in chronological order:
> HIVE-10508: removed some passwords from config that's propagated to execution
> engines
> HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the
> hardcoded list in HIVE-10508
> the problem with HIVE-9013 is it's about to introduce a common method for
> removing sensitive data from Configuration, which absolutely makes sense in
> most of the cases (set command showing sensitive data), but can cause issues
> e.g. while using non-secure cloud connectors on a cluster, where instead of
> the hadoop credential provider API (which is considered the secure and proper
> way), passwords/secrets appear in the Configuration object (like:
> "fs.azure.account.oauth2.client.secret")
> 2 possible solutions:
> 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" ->
> which defaults to "hive.conf.hidden.list" (configurable, but maybe just more
> confusing to users, having a new config property which should be understood
> and maintained on a cluster)
> 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced
> by HIVE-10508 (convenient, less confusing for users, but cannot be configured)
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[ https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=770795&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-770795 ] ASF GitHub Bot logged work on HIVE-25929: - Author: ASF GitHub Bot Created on: 16/May/22 12:07 Start Date: 16/May/22 12:07 Worklog Time Spent: 10m Work Description: zhangbutao commented on PR #3019: URL: https://github.com/apache/hive/pull/3019#issuecomment-1127587969 I think this fix make sense. I've had similar problems which tez can not get s3a secret parameter when running query against s3a object store. Issue Time Tracking --- Worklog Id: (was: 770795) Time Spent: 2h 40m (was: 2.5h) > Let secret config properties to be propagated to Tez > > > Key: HIVE-25929 > URL: https://issues.apache.org/jira/browse/HIVE-25929 > Project: Hive > Issue Type: Bug >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > History in chronological order: > HIVE-10508: removed some passwords from config that's propagated to execution > engines > HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the > hardcoded list in HIVE-10508 > the problem with HIVE-9013 is it's about to introduce a common method for > removing sensitive data from Configuration, which absolutely makes sense in > most of the cases (set command showing sensitive data), but can cause issues > e.g. while using non-secure cloud connectors on a cluster, where instead of > the hadoop credential provider API (which is considered the secure and proper > way), passwords/secrets appear in the Configuration object (like: > "fs.azure.account.oauth2.client.secret") > 2 possible solutions: > 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" -> > which defaults to "hive.conf.hidden.list" (configurable, but maybe just more > confusing to users, having a new config property which should be understood > and maintained on a cluster) > 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced > by HIVE-10508 (convenient, less confusing for users, but cannot be configured) -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[ https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=770779&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-770779 ] ASF GitHub Bot logged work on HIVE-25929: - Author: ASF GitHub Bot Created on: 16/May/22 11:28 Start Date: 16/May/22 11:28 Worklog Time Spent: 10m Work Description: abstractdog opened a new pull request, #3019: URL: https://github.com/apache/hive/pull/3019 ### What changes were proposed in this pull request? Prevent removing some config properties from Configuration object which are passed to execution engines. ### Why are the changes needed? Described in jira. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Unit test included. Issue Time Tracking --- Worklog Id: (was: 770779) Time Spent: 2.5h (was: 2h 20m) > Let secret config properties to be propagated to Tez > > > Key: HIVE-25929 > URL: https://issues.apache.org/jira/browse/HIVE-25929 > Project: Hive > Issue Type: Bug >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > History in chronological order: > HIVE-10508: removed some passwords from config that's propagated to execution > engines > HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the > hardcoded list in HIVE-10508 > the problem with HIVE-9013 is it's about to introduce a common method for > removing sensitive data from Configuration, which absolutely makes sense in > most of the cases (set command showing sensitive data), but can cause issues > e.g. while using non-secure cloud connectors on a cluster, where instead of > the hadoop credential provider API (which is considered the secure and proper > way), passwords/secrets appear in the Configuration object (like: > "fs.azure.account.oauth2.client.secret") > 2 possible solutions: > 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" -> > which defaults to "hive.conf.hidden.list" (configurable, but maybe just more > confusing to users, having a new config property which should be understood > and maintained on a cluster) > 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced > by HIVE-10508 (convenient, less confusing for users, but cannot be configured) -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[ https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=770543&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-770543 ] ASF GitHub Bot logged work on HIVE-25929: - Author: ASF GitHub Bot Created on: 15/May/22 00:26 Start Date: 15/May/22 00:26 Worklog Time Spent: 10m Work Description: github-actions[bot] closed pull request #3019: HIVE-25929: Let secret config properties to be propagated to Tez URL: https://github.com/apache/hive/pull/3019 Issue Time Tracking --- Worklog Id: (was: 770543) Time Spent: 2h 20m (was: 2h 10m) > Let secret config properties to be propagated to Tez > > > Key: HIVE-25929 > URL: https://issues.apache.org/jira/browse/HIVE-25929 > Project: Hive > Issue Type: Bug >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > History in chronological order: > HIVE-10508: removed some passwords from config that's propagated to execution > engines > HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the > hardcoded list in HIVE-10508 > the problem with HIVE-9013 is it's about to introduce a common method for > removing sensitive data from Configuration, which absolutely makes sense in > most of the cases (set command showing sensitive data), but can cause issues > e.g. while using non-secure cloud connectors on a cluster, where instead of > the hadoop credential provider API (which is considered the secure and proper > way), passwords/secrets appear in the Configuration object (like: > "fs.azure.account.oauth2.client.secret") > 2 possible solutions: > 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" -> > which defaults to "hive.conf.hidden.list" (configurable, but maybe just more > confusing to users, having a new config property which should be understood > and maintained on a cluster) > 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced > by HIVE-10508 (convenient, less confusing for users, but cannot be configured) -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[ https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=767542&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-767542 ] ASF GitHub Bot logged work on HIVE-25929: - Author: ASF GitHub Bot Created on: 07/May/22 08:33 Start Date: 07/May/22 08:33 Worklog Time Spent: 10m Work Description: abstractdog opened a new pull request, #3019: URL: https://github.com/apache/hive/pull/3019 ### What changes were proposed in this pull request? Prevent removing some config properties from Configuration object which are passed to execution engines. ### Why are the changes needed? Described in jira. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Unit test included. Issue Time Tracking --- Worklog Id: (was: 767542) Time Spent: 2h 10m (was: 2h) > Let secret config properties to be propagated to Tez > > > Key: HIVE-25929 > URL: https://issues.apache.org/jira/browse/HIVE-25929 > Project: Hive > Issue Type: Bug >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > History in chronological order: > HIVE-10508: removed some passwords from config that's propagated to execution > engines > HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the > hardcoded list in HIVE-10508 > the problem with HIVE-9013 is it's about to introduce a common method for > removing sensitive data from Configuration, which absolutely makes sense in > most of the cases (set command showing sensitive data), but can cause issues > e.g. while using non-secure cloud connectors on a cluster, where instead of > the hadoop credential provider API (which is considered the secure and proper > way), passwords/secrets appear in the Configuration object (like: > "fs.azure.account.oauth2.client.secret") > 2 possible solutions: > 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" -> > which defaults to "hive.conf.hidden.list" (configurable, but maybe just more > confusing to users, having a new config property which should be understood > and maintained on a cluster) > 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced > by HIVE-10508 (convenient, less confusing for users, but cannot be configured) -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[ https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=766355&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-766355 ] ASF GitHub Bot logged work on HIVE-25929: - Author: ASF GitHub Bot Created on: 05/May/22 00:19 Start Date: 05/May/22 00:19 Worklog Time Spent: 10m Work Description: github-actions[bot] closed pull request #3019: HIVE-25929: Let secret config properties to be propagated to Tez URL: https://github.com/apache/hive/pull/3019 Issue Time Tracking --- Worklog Id: (was: 766355) Time Spent: 2h (was: 1h 50m) > Let secret config properties to be propagated to Tez > > > Key: HIVE-25929 > URL: https://issues.apache.org/jira/browse/HIVE-25929 > Project: Hive > Issue Type: Bug >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > > History in chronological order: > HIVE-10508: removed some passwords from config that's propagated to execution > engines > HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the > hardcoded list in HIVE-10508 > the problem with HIVE-9013 is it's about to introduce a common method for > removing sensitive data from Configuration, which absolutely makes sense in > most of the cases (set command showing sensitive data), but can cause issues > e.g. while using non-secure cloud connectors on a cluster, where instead of > the hadoop credential provider API (which is considered the secure and proper > way), passwords/secrets appear in the Configuration object (like: > "fs.azure.account.oauth2.client.secret") > 2 possible solutions: > 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" -> > which defaults to "hive.conf.hidden.list" (configurable, but maybe just more > confusing to users, having a new config property which should be understood > and maintained on a cluster) > 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced > by HIVE-10508 (convenient, less confusing for users, but cannot be configured) -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[ https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=762475&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762475 ] ASF GitHub Bot logged work on HIVE-25929: - Author: ASF GitHub Bot Created on: 26/Apr/22 18:43 Start Date: 26/Apr/22 18:43 Worklog Time Spent: 10m Work Description: abstractdog opened a new pull request, #3019: URL: https://github.com/apache/hive/pull/3019 ### What changes were proposed in this pull request? Prevent removing some config properties from Configuration object which are passed to execution engines. ### Why are the changes needed? Described in jira. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Unit test included. Issue Time Tracking --- Worklog Id: (was: 762475) Time Spent: 1h 50m (was: 1h 40m) > Let secret config properties to be propagated to Tez > > > Key: HIVE-25929 > URL: https://issues.apache.org/jira/browse/HIVE-25929 > Project: Hive > Issue Type: Bug >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > History in chronological order: > HIVE-10508: removed some passwords from config that's propagated to execution > engines > HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the > hardcoded list in HIVE-10508 > the problem with HIVE-9013 is it's about to introduce a common method for > removing sensitive data from Configuration, which absolutely makes sense in > most of the cases (set command showing sensitive data), but can cause issues > e.g. while using non-secure cloud connectors on a cluster, where instead of > the hadoop credential provider API (which is considered the secure and proper > way), passwords/secrets appear in the Configuration object (like: > "fs.azure.account.oauth2.client.secret") > 2 possible solutions: > 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" -> > which defaults to "hive.conf.hidden.list" (configurable, but maybe just more > confusing to users, having a new config property which should be understood > and maintained on a cluster) > 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced > by HIVE-10508 (convenient, less confusing for users, but cannot be configured) -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[ https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=762072&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762072 ] ASF GitHub Bot logged work on HIVE-25929: - Author: ASF GitHub Bot Created on: 26/Apr/22 00:20 Start Date: 26/Apr/22 00:20 Worklog Time Spent: 10m Work Description: github-actions[bot] closed pull request #3019: HIVE-25929: Let secret config properties to be propagated to Tez URL: https://github.com/apache/hive/pull/3019 Issue Time Tracking --- Worklog Id: (was: 762072) Time Spent: 1h 40m (was: 1.5h) > Let secret config properties to be propagated to Tez > > > Key: HIVE-25929 > URL: https://issues.apache.org/jira/browse/HIVE-25929 > Project: Hive > Issue Type: Bug >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > History in chronological order: > HIVE-10508: removed some passwords from config that's propagated to execution > engines > HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the > hardcoded list in HIVE-10508 > the problem with HIVE-9013 is it's about to introduce a common method for > removing sensitive data from Configuration, which absolutely makes sense in > most of the cases (set command showing sensitive data), but can cause issues > e.g. while using non-secure cloud connectors on a cluster, where instead of > the hadoop credential provider API (which is considered the secure and proper > way), passwords/secrets appear in the Configuration object (like: > "fs.azure.account.oauth2.client.secret") > 2 possible solutions: > 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" -> > which defaults to "hive.conf.hidden.list" (configurable, but maybe just more > confusing to users, having a new config property which should be understood > and maintained on a cluster) > 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced > by HIVE-10508 (convenient, less confusing for users, but cannot be configured) -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[ https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=757724&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-757724 ] ASF GitHub Bot logged work on HIVE-25929: - Author: ASF GitHub Bot Created on: 18/Apr/22 00:21 Start Date: 18/Apr/22 00:21 Worklog Time Spent: 10m Work Description: github-actions[bot] commented on PR #3019: URL: https://github.com/apache/hive/pull/3019#issuecomment-1100978183 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. Issue Time Tracking --- Worklog Id: (was: 757724) Time Spent: 1.5h (was: 1h 20m) > Let secret config properties to be propagated to Tez > > > Key: HIVE-25929 > URL: https://issues.apache.org/jira/browse/HIVE-25929 > Project: Hive > Issue Type: Bug >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > History in chronological order: > HIVE-10508: removed some passwords from config that's propagated to execution > engines > HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the > hardcoded list in HIVE-10508 > the problem with HIVE-9013 is it's about to introduce a common method for > removing sensitive data from Configuration, which absolutely makes sense in > most of the cases (set command showing sensitive data), but can cause issues > e.g. while using non-secure cloud connectors on a cluster, where instead of > the hadoop credential provider API (which is considered the secure and proper > way), passwords/secrets appear in the Configuration object (like: > "fs.azure.account.oauth2.client.secret") > 2 possible solutions: > 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" -> > which defaults to "hive.conf.hidden.list" (configurable, but maybe just more > confusing to users, having a new config property which should be understood > and maintained on a cluster) > 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced > by HIVE-10508 (convenient, less confusing for users, but cannot be configured) -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[ https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=727512&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-727512 ] ASF GitHub Bot logged work on HIVE-25929: - Author: ASF GitHub Bot Created on: 15/Feb/22 18:57 Start Date: 15/Feb/22 18:57 Worklog Time Spent: 10m Work Description: abstractdog commented on pull request #3019: URL: https://github.com/apache/hive/pull/3019#issuecomment-1040191210 > LGTM, left few minor comments, thanks for the patch @abstractdog! Unfortunately I am not a committer so it won't count as +1 for merging, but hope it helps anyway :) thanks @asolimando for the review, it should help with moving this forward -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 727512) Time Spent: 1h 20m (was: 1h 10m) > Let secret config properties to be propagated to Tez > > > Key: HIVE-25929 > URL: https://issues.apache.org/jira/browse/HIVE-25929 > Project: Hive > Issue Type: Bug >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > History in chronological order: > HIVE-10508: removed some passwords from config that's propagated to execution > engines > HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the > hardcoded list in HIVE-10508 > the problem with HIVE-9013 is it's about to introduce a common method for > removing sensitive data from Configuration, which absolutely makes sense in > most of the cases (set command showing sensitive data), but can cause issues > e.g. while using non-secure cloud connectors on a cluster, where instead of > the hadoop credential provider API (which is considered the secure and proper > way), passwords/secrets appear in the Configuration object (like: > "fs.azure.account.oauth2.client.secret") > 2 possible solutions: > 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" -> > which defaults to "hive.conf.hidden.list" (configurable, but maybe just more > confusing to users, having a new config property which should be understood > and maintained on a cluster) > 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced > by HIVE-10508 (convenient, less confusing for users, but cannot be configured) -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[
https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=727368&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-727368
]
ASF GitHub Bot logged work on HIVE-25929:
-
Author: ASF GitHub Bot
Created on: 15/Feb/22 18:44
Start Date: 15/Feb/22 18:44
Worklog Time Spent: 10m
Work Description: abstractdog commented on a change in pull request #3019:
URL: https://github.com/apache/hive/pull/3019#discussion_r806738235
##
File path: common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
##
@@ -7002,6 +7018,16 @@ public void stripHiddenConfigurations(Configuration
conf) {
HiveConfUtil.stripConfigurations(conf, hiddenSet);
}
+ /**
+ * Strips hidden config entries from configuration, but takes care of
entries for execution engines.
+ */
+ public void stripHiddenConfigurationsForExecutionEngines(Configuration conf)
{
+Set propsToBePropagatedToExecEngines =
HiveConfUtil.getPropagateToExecutionEnginesList(conf);
+Set propsToBeStripped = new HashSet(hiddenSet);
Review comment:
done
##
File path: common/src/java/org/apache/hadoop/hive/conf/HiveConfUtil.java
##
@@ -84,12 +84,18 @@ public static StringBuilder dumpConfig(HiveConf conf) {
* @return The list of the configuration values to hide
*/
public static Set getHiddenSet(Configuration configuration) {
+return getConfigurationKeyset(configuration,
HiveConf.ConfVars.HIVE_CONF_HIDDEN_LIST);
+ }
+
+ public static Set getPropagateToExecutionEnginesList(Configuration
configuration) {
+return getConfigurationKeyset(configuration,
HiveConf.ConfVars.HIVE_CONF_PROPAGATE_EXEC_ENGINES);
+ }
+
+ private static Set getConfigurationKeyset(Configuration
configuration, ConfVars name) {
Set hiddenSet = new HashSet();
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 727368)
Time Spent: 1h 10m (was: 1h)
> Let secret config properties to be propagated to Tez
>
>
> Key: HIVE-25929
> URL: https://issues.apache.org/jira/browse/HIVE-25929
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> History in chronological order:
> HIVE-10508: removed some passwords from config that's propagated to execution
> engines
> HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the
> hardcoded list in HIVE-10508
> the problem with HIVE-9013 is it's about to introduce a common method for
> removing sensitive data from Configuration, which absolutely makes sense in
> most of the cases (set command showing sensitive data), but can cause issues
> e.g. while using non-secure cloud connectors on a cluster, where instead of
> the hadoop credential provider API (which is considered the secure and proper
> way), passwords/secrets appear in the Configuration object (like:
> "fs.azure.account.oauth2.client.secret")
> 2 possible solutions:
> 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" ->
> which defaults to "hive.conf.hidden.list" (configurable, but maybe just more
> confusing to users, having a new config property which should be understood
> and maintained on a cluster)
> 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced
> by HIVE-10508 (convenient, less confusing for users, but cannot be configured)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[ https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=726986&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-726986 ] ASF GitHub Bot logged work on HIVE-25929: - Author: ASF GitHub Bot Created on: 15/Feb/22 12:02 Start Date: 15/Feb/22 12:02 Worklog Time Spent: 10m Work Description: abstractdog commented on pull request #3019: URL: https://github.com/apache/hive/pull/3019#issuecomment-1040191210 > LGTM, left few minor comments, thanks for the patch @abstractdog! Unfortunately I am not a committer so it won't count as +1 for merging, but hope it helps anyway :) thanks @asolimando for the review, it should help with moving this forward -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 726986) Time Spent: 1h (was: 50m) > Let secret config properties to be propagated to Tez > > > Key: HIVE-25929 > URL: https://issues.apache.org/jira/browse/HIVE-25929 > Project: Hive > Issue Type: Bug >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > History in chronological order: > HIVE-10508: removed some passwords from config that's propagated to execution > engines > HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the > hardcoded list in HIVE-10508 > the problem with HIVE-9013 is it's about to introduce a common method for > removing sensitive data from Configuration, which absolutely makes sense in > most of the cases (set command showing sensitive data), but can cause issues > e.g. while using non-secure cloud connectors on a cluster, where instead of > the hadoop credential provider API (which is considered the secure and proper > way), passwords/secrets appear in the Configuration object (like: > "fs.azure.account.oauth2.client.secret") > 2 possible solutions: > 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" -> > which defaults to "hive.conf.hidden.list" (configurable, but maybe just more > confusing to users, having a new config property which should be understood > and maintained on a cluster) > 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced > by HIVE-10508 (convenient, less confusing for users, but cannot be configured) -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[
https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=726985&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-726985
]
ASF GitHub Bot logged work on HIVE-25929:
-
Author: ASF GitHub Bot
Created on: 15/Feb/22 12:01
Start Date: 15/Feb/22 12:01
Worklog Time Spent: 10m
Work Description: abstractdog commented on a change in pull request #3019:
URL: https://github.com/apache/hive/pull/3019#discussion_r806757215
##
File path: common/src/java/org/apache/hadoop/hive/conf/HiveConfUtil.java
##
@@ -84,12 +84,18 @@ public static StringBuilder dumpConfig(HiveConf conf) {
* @return The list of the configuration values to hide
*/
public static Set getHiddenSet(Configuration configuration) {
+return getConfigurationKeyset(configuration,
HiveConf.ConfVars.HIVE_CONF_HIDDEN_LIST);
+ }
+
+ public static Set getPropagateToExecutionEnginesList(Configuration
configuration) {
+return getConfigurationKeyset(configuration,
HiveConf.ConfVars.HIVE_CONF_PROPAGATE_EXEC_ENGINES);
+ }
+
+ private static Set getConfigurationKeyset(Configuration
configuration, ConfVars name) {
Set hiddenSet = new HashSet();
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 726985)
Time Spent: 50m (was: 40m)
> Let secret config properties to be propagated to Tez
>
>
> Key: HIVE-25929
> URL: https://issues.apache.org/jira/browse/HIVE-25929
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> History in chronological order:
> HIVE-10508: removed some passwords from config that's propagated to execution
> engines
> HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the
> hardcoded list in HIVE-10508
> the problem with HIVE-9013 is it's about to introduce a common method for
> removing sensitive data from Configuration, which absolutely makes sense in
> most of the cases (set command showing sensitive data), but can cause issues
> e.g. while using non-secure cloud connectors on a cluster, where instead of
> the hadoop credential provider API (which is considered the secure and proper
> way), passwords/secrets appear in the Configuration object (like:
> "fs.azure.account.oauth2.client.secret")
> 2 possible solutions:
> 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" ->
> which defaults to "hive.conf.hidden.list" (configurable, but maybe just more
> confusing to users, having a new config property which should be understood
> and maintained on a cluster)
> 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced
> by HIVE-10508 (convenient, less confusing for users, but cannot be configured)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[
https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=726982&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-726982
]
ASF GitHub Bot logged work on HIVE-25929:
-
Author: ASF GitHub Bot
Created on: 15/Feb/22 11:37
Start Date: 15/Feb/22 11:37
Worklog Time Spent: 10m
Work Description: abstractdog commented on a change in pull request #3019:
URL: https://github.com/apache/hive/pull/3019#discussion_r806738235
##
File path: common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
##
@@ -7002,6 +7018,16 @@ public void stripHiddenConfigurations(Configuration
conf) {
HiveConfUtil.stripConfigurations(conf, hiddenSet);
}
+ /**
+ * Strips hidden config entries from configuration, but takes care of
entries for execution engines.
+ */
+ public void stripHiddenConfigurationsForExecutionEngines(Configuration conf)
{
+Set propsToBePropagatedToExecEngines =
HiveConfUtil.getPropagateToExecutionEnginesList(conf);
+Set propsToBeStripped = new HashSet(hiddenSet);
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 726982)
Time Spent: 40m (was: 0.5h)
> Let secret config properties to be propagated to Tez
>
>
> Key: HIVE-25929
> URL: https://issues.apache.org/jira/browse/HIVE-25929
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> History in chronological order:
> HIVE-10508: removed some passwords from config that's propagated to execution
> engines
> HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the
> hardcoded list in HIVE-10508
> the problem with HIVE-9013 is it's about to introduce a common method for
> removing sensitive data from Configuration, which absolutely makes sense in
> most of the cases (set command showing sensitive data), but can cause issues
> e.g. while using non-secure cloud connectors on a cluster, where instead of
> the hadoop credential provider API (which is considered the secure and proper
> way), passwords/secrets appear in the Configuration object (like:
> "fs.azure.account.oauth2.client.secret")
> 2 possible solutions:
> 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" ->
> which defaults to "hive.conf.hidden.list" (configurable, but maybe just more
> confusing to users, having a new config property which should be understood
> and maintained on a cluster)
> 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced
> by HIVE-10508 (convenient, less confusing for users, but cannot be configured)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[
https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=725285&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-725285
]
ASF GitHub Bot logged work on HIVE-25929:
-
Author: ASF GitHub Bot
Created on: 11/Feb/22 17:18
Start Date: 11/Feb/22 17:18
Worklog Time Spent: 10m
Work Description: asolimando commented on a change in pull request #3019:
URL: https://github.com/apache/hive/pull/3019#discussion_r804855198
##
File path: common/src/java/org/apache/hadoop/hive/conf/HiveConfUtil.java
##
@@ -84,12 +84,18 @@ public static StringBuilder dumpConfig(HiveConf conf) {
* @return The list of the configuration values to hide
*/
public static Set getHiddenSet(Configuration configuration) {
+return getConfigurationKeyset(configuration,
HiveConf.ConfVars.HIVE_CONF_HIDDEN_LIST);
+ }
+
+ public static Set getPropagateToExecutionEnginesList(Configuration
configuration) {
+return getConfigurationKeyset(configuration,
HiveConf.ConfVars.HIVE_CONF_PROPAGATE_EXEC_ENGINES);
+ }
+
+ private static Set getConfigurationKeyset(Configuration
configuration, ConfVars name) {
Set hiddenSet = new HashSet();
Review comment:
```suggestion
Set hiddenSet = new HashSet<>();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 725285)
Time Spent: 0.5h (was: 20m)
> Let secret config properties to be propagated to Tez
>
>
> Key: HIVE-25929
> URL: https://issues.apache.org/jira/browse/HIVE-25929
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> History in chronological order:
> HIVE-10508: removed some passwords from config that's propagated to execution
> engines
> HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the
> hardcoded list in HIVE-10508
> the problem with HIVE-9013 is it's about to introduce a common method for
> removing sensitive data from Configuration, which absolutely makes sense in
> most of the cases (set command showing sensitive data), but can cause issues
> e.g. while using non-secure cloud connectors on a cluster, where instead of
> the hadoop credential provider API (which is considered the secure and proper
> way), passwords/secrets appear in the Configuration object (like:
> "fs.azure.account.oauth2.client.secret")
> 2 possible solutions:
> 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" ->
> which defaults to "hive.conf.hidden.list" (configurable, but maybe just more
> confusing to users, having a new config property which should be understood
> and maintained on a cluster)
> 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced
> by HIVE-10508 (convenient, less confusing for users, but cannot be configured)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[
https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=725284&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-725284
]
ASF GitHub Bot logged work on HIVE-25929:
-
Author: ASF GitHub Bot
Created on: 11/Feb/22 17:17
Start Date: 11/Feb/22 17:17
Worklog Time Spent: 10m
Work Description: asolimando commented on a change in pull request #3019:
URL: https://github.com/apache/hive/pull/3019#discussion_r804854330
##
File path: common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
##
@@ -7002,6 +7018,16 @@ public void stripHiddenConfigurations(Configuration
conf) {
HiveConfUtil.stripConfigurations(conf, hiddenSet);
}
+ /**
+ * Strips hidden config entries from configuration, but takes care of
entries for execution engines.
+ */
+ public void stripHiddenConfigurationsForExecutionEngines(Configuration conf)
{
+Set propsToBePropagatedToExecEngines =
HiveConfUtil.getPropagateToExecutionEnginesList(conf);
+Set propsToBeStripped = new HashSet(hiddenSet);
Review comment:
```suggestion
Set propsToBeStripped = new HashSet<>(hiddenSet);
```
You should be able to use the diamond operator here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 725284)
Time Spent: 20m (was: 10m)
> Let secret config properties to be propagated to Tez
>
>
> Key: HIVE-25929
> URL: https://issues.apache.org/jira/browse/HIVE-25929
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> History in chronological order:
> HIVE-10508: removed some passwords from config that's propagated to execution
> engines
> HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the
> hardcoded list in HIVE-10508
> the problem with HIVE-9013 is it's about to introduce a common method for
> removing sensitive data from Configuration, which absolutely makes sense in
> most of the cases (set command showing sensitive data), but can cause issues
> e.g. while using non-secure cloud connectors on a cluster, where instead of
> the hadoop credential provider API (which is considered the secure and proper
> way), passwords/secrets appear in the Configuration object (like:
> "fs.azure.account.oauth2.client.secret")
> 2 possible solutions:
> 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" ->
> which defaults to "hive.conf.hidden.list" (configurable, but maybe just more
> confusing to users, having a new config property which should be understood
> and maintained on a cluster)
> 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced
> by HIVE-10508 (convenient, less confusing for users, but cannot be configured)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-25929) Let secret config properties to be propagated to Tez
[ https://issues.apache.org/jira/browse/HIVE-25929?focusedWorklogId=725211&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-725211 ] ASF GitHub Bot logged work on HIVE-25929: - Author: ASF GitHub Bot Created on: 11/Feb/22 15:20 Start Date: 11/Feb/22 15:20 Worklog Time Spent: 10m Work Description: abstractdog opened a new pull request #3019: URL: https://github.com/apache/hive/pull/3019 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 725211) Remaining Estimate: 0h Time Spent: 10m > Let secret config properties to be propagated to Tez > > > Key: HIVE-25929 > URL: https://issues.apache.org/jira/browse/HIVE-25929 > Project: Hive > Issue Type: Bug >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > History in chronological order: > HIVE-10508: removed some passwords from config that's propagated to execution > engines > HIVE-9013: introduced hive.conf.hidden.list, which is used instead of the > hardcoded list in HIVE-10508 > the problem with HIVE-9013 is it's about to introduce a common method for > removing sensitive data from Configuration, which absolutely makes sense in > most of the cases (set command showing sensitive data), but can cause issues > e.g. while using non-secure cloud connectors on a cluster, where instead of > the hadoop credential provider API (which is considered the secure and proper > way), passwords/secrets appear in the Configuration object (like: > "fs.azure.account.oauth2.client.secret") > 2 possible solutions: > 1. introduce a new property like: "hive.conf.hidden.list.exec.engines" -> > which defaults to "hive.conf.hidden.list" (configurable, but maybe just more > confusing to users, having a new config property which should be understood > and maintained on a cluster) > 2. simply revert DAGUtils to use to old stripHivePasswordDetails introduced > by HIVE-10508 (convenient, less confusing for users, but cannot be configured) -- This message was sent by Atlassian Jira (v8.20.1#820001)
