[jira] [Reopened] (HADOOP-14241) Add ADLS sensitive config keys to default list

2017-04-19 Thread John Zhuge (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HADOOP-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Zhuge reopened HADOOP-14241:
-

Reopen to run pre-commit

> Add ADLS sensitive config keys to default list
> --
>
> Key: HADOOP-14241
> URL: https://issues.apache.org/jira/browse/HADOOP-14241
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs, fs/adl, security
>Affects Versions: 2.8.0
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Minor
> Fix For: 3.0.0-alpha3
>
> Attachments: HADOOP-14241.001.patch, HADOOP-14241.002.patch, 
> HADOOP-14241.branch-2.002.patch
>
>
> ADLS sensitive credential config keys should be added to the default list for 
> {{hadoop.security.sensitive-config-keys}}.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-14329) Thread leakage in Hadoop RollingAverages metric collection

2017-04-19 Thread Hrishikesh Gadre (JIRA) 
Hrishikesh Gadre created HADOOP-14329:
-

 Summary: Thread leakage in Hadoop RollingAverages metric collection
 Key: HADOOP-14329
 URL: https://issues.apache.org/jira/browse/HADOOP-14329
 Project: Hadoop Common
  Issue Type: Bug
  Components: metrics
Affects Versions: 3.0.0-alpha2
Reporter: Hrishikesh Gadre
Priority: Minor


The rolling averages metric implementation defines a static 
ScheduledExecutorService instance which is not shutdown properly. This results 
in leaking the corresponding thread. This issue was identified while testing 
Solr/Hadoop 3 integration.

Here is the relevant portion of the code involved,
https://github.com/apache/hadoop/blob/c1549352cf1d1b1b45d7b613f993d45649b8efcf/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/RollingAverages.java#L65



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2017-04-19 Thread Apache Jenkins Server 
For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/

[Apr 18, 2017 7:43:50 AM] (aajisaka) HADOOP-14318. Remove non-existent setfattr 
command option from
[Apr 18, 2017 8:14:22 AM] (aajisaka) HADOOP-14315. Python example in the rack 
awareness document doesn't work
[Apr 18, 2017 4:28:52 PM] (jzhuge) HDFS-11665. 
HttpFSServerWebServer$deprecateEnv may leak secret.
[Apr 18, 2017 4:29:16 PM] (jzhuge) HADOOP-14317. KMSWebServer$deprecateEnv may 
leak secret. Contributed by
[Apr 18, 2017 5:29:44 PM] (cdouglas) YARN-6451. Add RM monitor validating 
metrics invariants. Contributed by
[Apr 19, 2017 4:11:07 AM] (jzhuge) HDFS-11531. Expose hedged read metrics via 
libHDFS API. Contributed by




-1 overall


The following subsystems voted -1:
asflicense unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations 
   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting 
   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure 
   hadoop.hdfs.server.datanode.TestDataNodeMXBean 
   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport 
   hadoop.hdfs.TestDistributedFileSystem 
   hadoop.hdfs.TestPersistBlocks 
   hadoop.hdfs.server.datanode.TestDataNodeUUID 
   hadoop.yarn.server.resourcemanager.TestRMRestart 
   hadoop.yarn.server.TestContainerManagerSecurity 
   hadoop.yarn.server.TestMiniYarnClusterNodeUtilization 
   hadoop.yarn.server.TestDiskFailures 
   hadoop.mapred.TestJobEndNotifier 
   hadoop.mapred.TestMRTimelineEventHandling 
   hadoop.tools.TestDistCpSystem 
   hadoop.tools.TestHadoopArchiveLogsRunner 
   hadoop.metrics2.impl.TestKafkaMetrics 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/artifact/out/diff-compile-javac-root.txt
  [184K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/artifact/out/diff-checkstyle-root.txt
  [17M]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/artifact/out/diff-patch-pylint.txt
  [20K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/artifact/out/diff-patch-shelldocs.txt
  [12K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/artifact/out/whitespace-eol.txt
  [12M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/artifact/out/whitespace-tabs.txt
  [1.2M]

   javadoc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/artifact/out/diff-javadoc-javadoc-root.txt
  [2.2M]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [452K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
  [60K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
  [324K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
  [56K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
  [88K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/artifact/out/patch-unit-hadoop-tools_hadoop-distcp.txt
  [16K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/artifact/out/patch-unit-hadoop-tools_hadoop-archive-logs.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/artifact/out/patch-unit-hadoop-tools_hadoop-kafka.txt
  [8.0K]

   asflicense:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/381/artifact/out/patch-asflicense-problems.txt
  [4.0K]

Powered by Apache Yetus 0.5.0-SNAPSHOT   http://yetus.apache.org

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-14328) Hadoop build double jars some artifacts

2017-04-19 Thread Jonathan Eagles (JIRA) 
Jonathan Eagles created HADOOP-14328:


 Summary: Hadoop build double jars some artifacts
 Key: HADOOP-14328
 URL: https://issues.apache.org/jira/browse/HADOOP-14328
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Jonathan Eagles
Assignee: Jonathan Eagles






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-14327) KerberosAuthenticationHandler#authenticate throws meaningless exception with empty server principals

2017-04-19 Thread Wei-Chiu Chuang (JIRA) 
Wei-Chiu Chuang created HADOOP-14327:


 Summary: KerberosAuthenticationHandler#authenticate throws 
meaningless exception with empty server principals
 Key: HADOOP-14327
 URL: https://issues.apache.org/jira/browse/HADOOP-14327
 Project: Hadoop Common
  Issue Type: Bug
  Components: security
Affects Versions: 3.0.0-alpha2
Reporter: Wei-Chiu Chuang
Priority: Minor


If somehow KerberosAuthenticationHandler#authenticate gets an empty service 
principal set, it throws a useless exception like the following:

{noformat}
2017-04-19 10:11:39,812 DEBUG 
org.apache.hadoop.security.authentication.server.AuthenticationFilter: 
Authentication exception: 
org.apache.hadoop.security.authentication.client.AuthenticationExceptio
n
org.apache.hadoop.security.authentication.client.AuthenticationException: 
org.apache.hadoop.security.authentication.client.AuthenticationException
at 
org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.authenticate(KerberosAuthenticationHandler.java:452)
at 
org.apache.hadoop.security.authentication.server.MultiSchemeAuthenticationHandler.authenticate(MultiSchemeAuthenticationHandler.java:193)
at 
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:400)
at 
org.apache.hadoop.security.token.delegation.web.MultiSchemeDelegationTokenAuthenticationHandler.authenticate(MultiSchemeDelegationTokenAuthenticationHandler.java:180)
at 
org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
at 
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:532)
{noformat}

The following code has a logic error. If serverPrincipals is empty, token 
remains null in the end, but lastException is also null too, so throwing it is 
meaningless. It should throw with a more meaningful message.
{code:title=KerberosAuthenticationHandler#authenticate}
AuthenticationToken token = null;
Exception lastException = null;
for (String serverPrincipal : serverPrincipals) {
  try {
token = runWithPrincipal(serverPrincipal, clientToken,
base64, response);
  } catch (Exception ex) {
lastException = ex;
LOG.trace("Auth {} failed with {}", serverPrincipal, ex);
  } finally {
  if (token != null) {
LOG.trace("Auth {} successfully", serverPrincipal);
break;
}
  }
}
if (token != null) {
  return token;
} else {
  throw new AuthenticationException(lastException);
}
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Linux/ppc64le

2017-04-19 Thread Apache Jenkins Server 
For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/293/

[Apr 18, 2017 4:28:52 PM] (jzhuge) HDFS-11665. 
HttpFSServerWebServer$deprecateEnv may leak secret.
[Apr 18, 2017 4:29:16 PM] (jzhuge) HADOOP-14317. KMSWebServer$deprecateEnv may 
leak secret. Contributed by
[Apr 18, 2017 5:29:44 PM] (cdouglas) YARN-6451. Add RM monitor validating 
metrics invariants. Contributed by
[Apr 19, 2017 4:11:07 AM] (jzhuge) HDFS-11531. Expose hedged read metrics via 
libHDFS API. Contributed by
[Apr 19, 2017 9:21:44 AM] (stevel) HADOOP-14321. explicitly exclude s3a root 
dir ITests from parallel runs.




-1 overall


The following subsystems voted -1:
compile mvninstall unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc javac


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery 
   hadoop.hdfs.TestEncryptedTransfer 
   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer 
   hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes 
   hadoop.hdfs.web.TestWebHdfsTimeouts 
   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure210 
   hadoop.mapreduce.v2.hs.TestHistoryServerLeveldbStateStoreService 
   hadoop.mapred.TestShuffleHandler 
   hadoop.tools.TestHadoopArchiveLogsRunner 
   hadoop.metrics2.impl.TestKafkaMetrics 
   hadoop.yarn.applications.distributedshell.TestDistributedShell 
   hadoop.yarn.server.timeline.TestRollingLevelDB 
   hadoop.yarn.server.timeline.TestTimelineDataManager 
   hadoop.yarn.server.timeline.TestLeveldbTimelineStore 
   hadoop.yarn.server.timeline.recovery.TestLeveldbTimelineStateStore 
   hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore 
   
hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer 
   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer 
   hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore 
   hadoop.yarn.server.resourcemanager.TestRMRestart 
   
hadoop.yarn.server.resourcemanager.TestOpportunisticContainerAllocatorAMService 
   hadoop.yarn.server.TestMiniYarnClusterNodeUtilization 
   hadoop.yarn.server.TestContainerManagerSecurity 
   hadoop.yarn.server.timeline.TestLevelDBCacheTimelineStore 
   hadoop.yarn.server.timeline.TestOverrideTimelineStoreYarnClient 
   hadoop.yarn.server.timeline.TestEntityGroupFSTimelineStore 

Timed out junit tests :

   org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache 
  

   mvninstall:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/293/artifact/out/patch-mvninstall-root.txt
  [492K]

   compile:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/293/artifact/out/patch-compile-root.txt
  [20K]

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/293/artifact/out/patch-compile-root.txt
  [20K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/293/artifact/out/patch-compile-root.txt
  [20K]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/293/artifact/out/patch-unit-hadoop-assemblies.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/293/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [280K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/293/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt
  [16K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/293/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-nativetask.txt
  [52K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/293/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-shuffle.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/293/artifact/out/patch-unit-hadoop-tools_hadoop-archive-logs.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/293/artifact/out/patch-unit-hadoop-tools_hadoop-kafka.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/293/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/293/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt
  [52K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/293/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
  [16K]

[jira] [Resolved] (HADOOP-10604) CryptoFileSystem decorator using xAttrs and KeyProvider

2017-04-19 Thread Andrew Wang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HADOOP-10604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Wang resolved HADOOP-10604.
--
Resolution: Incomplete

I think we decided not to pursue this effort further, and focused instead on 
HDFS transparent encryption. Resolving.

> CryptoFileSystem decorator using xAttrs and KeyProvider
> ---
>
> Key: HADOOP-10604
> URL: https://issues.apache.org/jira/browse/HADOOP-10604
> Project: Hadoop Common
>  Issue Type: New Feature
>  Components: fs
>Affects Versions: fs-encryption (HADOOP-10150 and HDFS-6134)
>Reporter: Alejandro Abdelnur
>Assignee: Yi Liu
> Attachments: HADOOP-10604.1.patch, HADOOP-10604.patch
>
>
> A FileSystem implementation that wraps an existing filesystem and provides 
> encryption. It will require the underlying filesystem to support xAttrs. It  
> will use the KeyProvider API to retrieve encryption keys.
> This is mostly the work in the patch HADOOP-10150 minus the crypto streams



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: HADOOP-14316: Switching from Findbugs to Spotbugs

2017-04-19 Thread Allen Wittenauer 

> On Apr 19, 2017, at 10:52 AM, Wei-Chiu Chuang  wrote:
> That sounds scary. Would you mind to share the list of bugs that spotbugs 
> found? Sounds like some of them may warrant new blockers jiras for Hadoop 3.


I've added the list to the JIRA.
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: HADOOP-14316: Switching from Findbugs to Spotbugs

2017-04-19 Thread Wei-Chiu Chuang 
Hi Allen,
That sounds scary. Would you mind to share the list of bugs that spotbugs 
found? Sounds like some of them may warrant new blockers jiras for Hadoop 3.

Thank you,
Wei-Chiu Chuang

> On Apr 19, 2017, at 10:44 AM, Allen Wittenauer  
> wrote:
> 
> HADOOP-14316



signature.asc
Description: Message signed with OpenPGP using GPGMail

HADOOP-14316: Switching from Findbugs to Spotbugs

2017-04-19 Thread Allen Wittenauer 
Hey gang.

HADOOP-14316 enables the spotbugs back-end for the findbugs front-end.  
Spotbugs (https://spotbugs.github.io/) is the fork of findbugs that the 
community and some of the major contributors have made to move findbugs 
forward.  It is geared towards JDK8 and JDK9. 

Before I commit, I wanted to give a heads up to the community about 
this change.

After committal, there will be (approx) 62 new findbugs issues that 
will pop up in the source tree.  My quick pass over a handful of them indicates 
that a good number are legit/not false positives (and one of them, well, we got 
lucky it's an API that no one updates/uses). It will take the community to fix 
up those 62 problems, either by actually fixing them (better) or exempting them 
(not great, but sometimes required).

Thanks.
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-14326) S3A to support S3 reduced redundancy storage

2017-04-19 Thread Steve Loughran (JIRA) 
Steve Loughran created HADOOP-14326:
---

 Summary: S3A to support S3 reduced redundancy storage
 Key: HADOOP-14326
 URL: https://issues.apache.org/jira/browse/HADOOP-14326
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/s3
Affects Versions: 2.9.0
Reporter: Steve Loughran
Priority: Minor


S3Acan support lower-cost S3 storage through reduced replication

# add option to ask for this on PUT/COPY
# add a test (optional for other S3 implementations?)
# handle a 405 response when [an object has been 
lost|http://docs.aws.amazon.com/AmazonS3/latest/dev/RtnCodeLostData.html]. 
Perhaps uprate to an FNFE, or add a new exception.





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-14325) Stabilise S3A Server Side Encryption

2017-04-19 Thread Steve Loughran (JIRA) 
Steve Loughran created HADOOP-14325:
---

 Summary: Stabilise S3A Server Side Encryption
 Key: HADOOP-14325
 URL: https://issues.apache.org/jira/browse/HADOOP-14325
 Project: Hadoop Common
  Issue Type: Task
  Components: documentation, fs/s3, test
Affects Versions: 2.9.0
Reporter: Steve Loughran
Assignee: Steve Loughran


Round off the S3 SSE encryption support with everything needed to safely ship 
it.

The core code is in, along with tests, so this covers the details

* docs with examples, including JCEKS files
* keeping secrets secret
* any more tests, including scale ones (huge file, rename)
* I'll add a KMS test to my (github) spark suite



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-14324) Switch to fs.s3a.server-side-encryption.key as property for encryption secret

2017-04-19 Thread Steve Loughran (JIRA) 
Steve Loughran created HADOOP-14324:
---

 Summary: Switch to fs.s3a.server-side-encryption.key as property 
for encryption secret
 Key: HADOOP-14324
 URL: https://issues.apache.org/jira/browse/HADOOP-14324
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/s3
Reporter: Steve Loughran
Priority: Blocker


Before this ships, can we rename {{fs.s3a.server-side-encryption-key}} to 
{{fs.s3a.server-side-encryption.key}}.

This makes it consistent with all other .key secrets in S3A. so
* simplifies documentation
* reduces confusion "is it a - or a ."? This confusion is going to surface in 
config and support

I know that CDH is shipping with the old key, but it'll be easy for them to add 
a deprecation property to handle the migration. I do at least what the ASF 
release to be stable before it ships.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-14323) ITestS3GuardListConsistency failure w/ Local, authoritative metadata store

2017-04-19 Thread Aaron Fabbri (JIRA) 
Aaron Fabbri created HADOOP-14323:
-

 Summary: ITestS3GuardListConsistency failure w/ Local, 
authoritative metadata store
 Key: HADOOP-14323
 URL: https://issues.apache.org/jira/browse/HADOOP-14323
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: s3
Affects Versions: HADOOP-13345
Reporter: Aaron Fabbri
Priority: Minor


When doing some testing for HADOOP-14266 I noticed this test failure:

{noformat}
java.lang.NullPointerException: null
at 
org.apache.hadoop.fs.s3a.ITestS3GuardListConsistency.testListStatusWriteBack(ITestS3GuardListConsistency.java:317)
{noformat}

I was running with LocalMetadataStore and 
{{fs.s3a.metadatastore.authoritative}} set to true.  I haven't been testing 
this mode recently so not sure if this case ever worked.  Lower priority but we 
should fix it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org