[jira] [Created] (HADOOP-13574) Unnecessary file existence check causes problems with S3

2016-09-02 Thread Alex Zolotko (JIRA)
Alex Zolotko created HADOOP-13574:
-

 Summary: Unnecessary file existence check causes problems with S3
 Key: HADOOP-13574
 URL: https://issues.apache.org/jira/browse/HADOOP-13574
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs/s3
Reporter: Alex Zolotko
Priority: Minor


We recently got the following exception on production:

{noformat}
java.io.FileNotFoundException: Key 
'xxx/_temporary/0/_temporary/attempt_201609010631__m_001128_1128/part-01128'
 does not exist in S3
at 
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.handleServiceException(Jets3tNativeFileSystemStore.java:234)
at 
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.copy(Jets3tNativeFileSystemStore.java:201)
at sun.reflect.GeneratedMethodAccessor71.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:186)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at org.apache.hadoop.fs.s3native.$Proxy13.copy(Unknown Source)
at 
org.apache.hadoop.fs.s3native.NativeS3FileSystem.rename(NativeS3FileSystem.java:659)
at 
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitTask(FileOutputCommitter.java:435)
at 
org.apache.hadoop.mapred.FileOutputCommitter.commitTask(FileOutputCommitter.java:172)
at 
org.apache.hadoop.mapred.OutputCommitter.commitTask(OutputCommitter.java:291)
at 
org.apache.spark.mapred.SparkHadoopMapRedUtil$.performCommit$1(SparkHadoopMapRedUtil.scala:98)
at 
org.apache.spark.mapred.SparkHadoopMapRedUtil$.commitTask(SparkHadoopMapRedUtil.scala:124)
at 
org.apache.spark.SparkHadoopWriter.commit(SparkHadoopWriter.scala:107)
at 
org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1$$anonfun$13.apply(PairRDDFunctions.scala:1204)
at 
org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1$$anonfun$13.apply(PairRDDFunctions.scala:1183)
{noformat}

FileOutputCommitter.commitTask() does check that the file exists before trying 
to rename it, but due to S3's relaxed consistency guarantees the following 
fs.rename(taskAttemptPath, committedTaskPath) still fails.

Here's an excerpt from the Amazon S3 documentation 
([https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html]):

{quote}
Amazon S3 Data Consistency Model

Amazon S3 provides read-after-write consistency for PUTS of new objects in your 
S3 bucket in all regions with one caveat. The caveat is that if you make a HEAD 
or GET request to the key name (to find if the object exists) before creating 
the object, Amazon S3 provides eventual consistency for read-after-write.
{quote}

The problematic S3 object existence check, that causes S3 to fallback to 
eventual consistency, is in NativeS3FileSystem.create():

{code}
if (exists(f) && !overwrite) {
  throw new IOException("File already exists:"+f);
}
{code}

If the overwrite parameter is set to "true" (as in our case), calling exists(f) 
is unnecessary and only "upsets" S3.

The proposed fix is switch the order of the predicates:
{code}
if (!overwrite && exists(f)) {
  throw new IOException("File already exists:"+f);
}
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-13575) Website shows incorrect 25 January, 2016 as the release date for 2.7.3

2016-09-02 Thread Jacek Laskowski (JIRA)
Jacek Laskowski created HADOOP-13575:


 Summary: Website shows incorrect 25 January, 2016 as the release 
date for 2.7.3
 Key: HADOOP-13575
 URL: https://issues.apache.org/jira/browse/HADOOP-13575
 Project: Hadoop Common
  Issue Type: Bug
Affects Versions: 2.7.3
Reporter: Jacek Laskowski


http://hadoop.apache.org/releases.html shows 2.7.3 released on 25 January, 2016 
which should rather be 25 August, 2016.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

2016-09-02 Thread Kuhu Shukla
+1( non-binding)

* Successfully downloaded and built from source.
* Deployed to single node cluster.
*  Ran Sleep and Wordcount jobs.
Thanks Andrew for the effort!
Regards,Kuhu 

On Thursday, September 1, 2016 10:32 PM, Arun Suresh  
wrote:
 

 +1 (binding).

Thanks for driving this Andrew..

* Download and built from source.
* Setup a 5 mode cluster.
* Verified that MR works with opportunistic containers
* Verified that the AMRMClient supports 'allocationRequestId'

Cheers
-Arun

On Thu, Sep 1, 2016 at 4:31 PM, Aaron Fabbri  wrote:

> +1, non-binding.
>
> I built everything on OS X and ran the s3a contract tests successfully:
>
> mvn test -Dtest=org.apache.hadoop.fs.contract.s3a.\*
>
> ...
>
> Results :
>
>
> Tests run: 78, Failures: 0, Errors: 0, Skipped: 1
>
>
> [INFO]
> 
>
> [INFO] BUILD SUCCESS
>
> [INFO]
> 
>
> On Thu, Sep 1, 2016 at 3:39 PM, Andrew Wang 
> wrote:
>
> > Good point Allen, I forgot about `hadoop version`. Since it's populated
> by
> > a version-info.properties file, people can always cat that file.
> >
> > On Thu, Sep 1, 2016 at 3:21 PM, Allen Wittenauer <
> a...@effectivemachines.com
> > >
> > wrote:
> >
> > >
> > > > On Sep 1, 2016, at 3:18 PM, Allen Wittenauer <
> a...@effectivemachines.com
> > >
> > > wrote:
> > > >
> > > >
> > > >> On Sep 1, 2016, at 2:57 PM, Andrew Wang 
> > > wrote:
> > > >>
> > > >> Steve requested a git hash for this release. This led us into a
> brief
> > > >> discussion of our use of git tags, wherein we realized that although
> > > >> release tags are immutable (start with "rel/"), RC tags are not.
> This
> > is
> > > >> based on the HowToRelease instructions.
> > > >
> > > >      We should probably embed the git hash in one of the files that
> > > gets gpg signed.  That's an easy change to create-release.
> > >
> > >
> > >        (Well, one more easily accessible than 'hadoop version')
> >
>


   

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2016-09-02 Thread Apache Jenkins Server
For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/

[Sep 1, 2016 8:38:51 AM] (vvasudev) YARN-5596. Fix failing unit test in 
TestDockerContainerRuntime.
[Sep 1, 2016 6:03:06 PM] (liuml07) HADOOP-13375.
[Sep 1, 2016 6:32:26 PM] (cdouglas) FsPermission string constructor does not 
recognize sticky bit.
[Sep 1, 2016 8:44:44 PM] (kihwal) HADOOP-13465. Design Server.Call to be 
extensible for unified call
[Sep 1, 2016 9:44:01 PM] (kasha) YARN-5566. Client-side NM graceful decom is 
not triggered when jobs
[Sep 1, 2016 10:35:25 PM] (xiao) HDFS-10820. Reuse closeResponder to reset the 
response variable in




-1 overall


The following subsystems voted -1:
asflicense unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed CTEST tests :

   test_test_libhdfs_threaded_hdfs_static 
   test_test_libhdfs_zerocopy_hdfs_static 

Failed junit tests :

   hadoop.hdfs.TestFileChecksum 
   hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints 
   hadoop.hdfs.TestRollingUpgrade 
   hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength 
   hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices 
   hadoop.yarn.server.TestContainerManagerSecurity 
   hadoop.yarn.server.TestMiniYarnClusterNodeUtilization 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/diff-compile-javac-root.txt
  [168K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/diff-checkstyle-root.txt
  [16M]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/diff-patch-pylint.txt
  [16K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/diff-patch-shelldocs.txt
  [16K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/whitespace-eol.txt
  [12M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/whitespace-tabs.txt
  [1.3M]

   javadoc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/diff-javadoc-javadoc-root.txt
  [2.2M]

   CTEST:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/patch-hadoop-hdfs-project_hadoop-hdfs-native-client-ctest.txt
  [24K]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [152K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
  [268K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-nativetask.txt
  [124K]

   asflicense:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/patch-asflicense-problems.txt
  [4.0K]

Powered by Apache Yetus 0.4.0-SNAPSHOT   http://yetus.apache.org



-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-13576) Old Hadoop user mailing lists still exist

2016-09-02 Thread Sebb (JIRA)
Sebb created HADOOP-13576:
-

 Summary: Old Hadoop user mailing lists still exist
 Key: HADOOP-13576
 URL: https://issues.apache.org/jira/browse/HADOOP-13576
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Sebb
Priority: Minor


The mailing lists for 

(common,hdfs,mapreduce)-user@

are still active; i.e. users can subscribe and post messages.

If that was not the intention [1] I suggest raising an INFRA JIRA to get the 
old lists closed down and/or aliased to user@.

See also INFRA-12554.

[1] 
http://mail-archives.apache.org/mod_mbox/hadoop-user/201208.mbox/%3C0A0CA23A-313F-4BA7-ADC5-31A1DB76B4FA%40hortonworks.com%3E




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

2016-09-02 Thread Tsuyoshi Ozawa
Thanks for making this forward hardly, Andrew!

In fact, I hit an error similar to Eric's one regardless of
configuring yarn.app.mapreduce.am.env and mapreduce.admin.user.env...
does anyone hit same error?


2016-09-02 16:49:35,223 INFO mapreduce.Job: The url to track the job:
http://ip-172-31-7-124.ap-northeast-1.compute.internal:23188/proxy/application_1472832909785_0013/

2016-09-02 16:49:35,224 INFO mapreduce.Job: Running job: job_1472832909785_0013

2016-09-02 16:49:48,282 INFO mapreduce.Job: Job job_1472832909785_0013
running in uber mode : false

2016-09-02 16:49:48,283 INFO mapreduce.Job:  map 0% reduce 0%

2016-09-02 16:49:48,293 INFO mapreduce.Job: Job job_1472832909785_0013
failed with state FAILED due to: Application
application_1472832909785_0013 failed 2 times due to AM Container for
appattempt_1472832909785_0013_02 exited with  exitCode: 1

Failing this attempt.Diagnostics: Exception from container-launch.

Container id: container_e02_1472832909785_0013_02_01

Exit code: 1

Stack trace: ExitCodeException exitCode=1:

at org.apache.hadoop.util.Shell.runCommand(Shell.java:974)

at org.apache.hadoop.util.Shell.run(Shell.java:878)

at 
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1172)

at 
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:237)

at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:419)

at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:284)

at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:88)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Best,
- Tsuyoshi

On Wed, Aug 31, 2016 at 12:51 AM, Andrew Wang  wrote:
> Hi all,
>
> Thanks to the combined work of many, many contributors, here's an RC0 for
> 3.0.0-alpha1:
>
> http://home.apache.org/~wang/3.0.0-alpha1-RC0/
>
> alpha1 is the first in a series of planned alpha releases leading up to GA.
> The objective is to get an artifact out to downstreams for testing and to
> iterate quickly based on their feedback. So, please keep that in mind when
> voting; hopefully most issues can be addressed by future alphas rather than
> future RCs.
>
> Sorry for getting this out on a Tuesday, but I'd still like this vote to
> run the normal 5 days, thus ending Saturday (9/3) at 9AM PDT. I'll extend
> if we lack the votes.
>
> Please try it out and let me know what you think.
>
> Best,
> Andrew

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

2016-09-02 Thread Tsuyoshi Ozawa
I'm sorry, very big typo I did... s/hardly/very hard/

Best
- Tsuyoshi

On Sat, Sep 3, 2016 at 1:53 AM, Tsuyoshi Ozawa  wrote:
> Thanks for making this forward hardly, Andrew!
>
> In fact, I hit an error similar to Eric's one regardless of
> configuring yarn.app.mapreduce.am.env and mapreduce.admin.user.env...
> does anyone hit same error?
>
>
> 2016-09-02 16:49:35,223 INFO mapreduce.Job: The url to track the job:
> http://ip-172-31-7-124.ap-northeast-1.compute.internal:23188/proxy/application_1472832909785_0013/
>
> 2016-09-02 16:49:35,224 INFO mapreduce.Job: Running job: 
> job_1472832909785_0013
>
> 2016-09-02 16:49:48,282 INFO mapreduce.Job: Job job_1472832909785_0013
> running in uber mode : false
>
> 2016-09-02 16:49:48,283 INFO mapreduce.Job:  map 0% reduce 0%
>
> 2016-09-02 16:49:48,293 INFO mapreduce.Job: Job job_1472832909785_0013
> failed with state FAILED due to: Application
> application_1472832909785_0013 failed 2 times due to AM Container for
> appattempt_1472832909785_0013_02 exited with  exitCode: 1
>
> Failing this attempt.Diagnostics: Exception from container-launch.
>
> Container id: container_e02_1472832909785_0013_02_01
>
> Exit code: 1
>
> Stack trace: ExitCodeException exitCode=1:
>
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:974)
>
> at org.apache.hadoop.util.Shell.run(Shell.java:878)
>
> at 
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1172)
>
> at 
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:237)
>
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:419)
>
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:284)
>
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:88)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>
> at java.lang.Thread.run(Thread.java:745)
>
> Best,
> - Tsuyoshi
>
> On Wed, Aug 31, 2016 at 12:51 AM, Andrew Wang  
> wrote:
>> Hi all,
>>
>> Thanks to the combined work of many, many contributors, here's an RC0 for
>> 3.0.0-alpha1:
>>
>> http://home.apache.org/~wang/3.0.0-alpha1-RC0/
>>
>> alpha1 is the first in a series of planned alpha releases leading up to GA.
>> The objective is to get an artifact out to downstreams for testing and to
>> iterate quickly based on their feedback. So, please keep that in mind when
>> voting; hopefully most issues can be addressed by future alphas rather than
>> future RCs.
>>
>> Sorry for getting this out on a Tuesday, but I'd still like this vote to
>> run the normal 5 days, thus ending Saturday (9/3) at 9AM PDT. I'll extend
>> if we lack the votes.
>>
>> Please try it out and let me know what you think.
>>
>> Best,
>> Andrew

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



[jira] [Created] (HADOOP-13577) Download page should link the ASF mirrora for KEYS, sigs, hashes

2016-09-02 Thread Sebb (JIRA)
Sebb created HADOOP-13577:
-

 Summary: Download page should link the ASF mirrora for KEYS, sigs, 
hashes
 Key: HADOOP-13577
 URL: https://issues.apache.org/jira/browse/HADOOP-13577
 Project: Hadoop Common
  Issue Type: Bug
 Environment: http://hadoop.apache.org/releases.html
Reporter: Sebb


The download page currently points to 

https://dist.apache.org/repos/dist/release/hadoop/...

for KEYS, sigs and hashes.

However the dist SVN tree is not designed for this; such files must be 
downloaded from the ASF mirrors, i.e

https://www.apache.org/dist/hadoop/...

Please can you adjust the links accordingly?
The links should use HTTPS.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



RE: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

2016-09-02 Thread Zheng, Kai
Thanks Sammi.

My non-binding +1 to make the release candidate.

Regards,
Kai

-Original Message-
From: Chen, Sammi 
Sent: Friday, September 02, 2016 4:59 PM
To: Zheng, Kai ; Andrew Wang ; 
Arun Suresh 
Cc: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org; Chen, Sammi 

Subject: RE: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

+1 (non-binding).

Thanks for driving this Andrew!

* Download and built from source.
* Setup a 10 node cluster (1 name node + 9 data nodes)
* Verified normal HDFS file put/get operation with 3x replication
* With 2 data nodes failure, verified HDFS file put/get operation with 3x 
replication, file integrity is OK
* Enable Erasure Code policy "RS-DEFAULT-6-3-64k", verified HDFS file put/get 
operation
* Enable Erasure Code policy "RS-DEFAULT-6-3-64k", with 3 data nodes failure, 
verified HDFS file put/get operation, file integrity is OK

Cheers
-Sammi

-Original Message-
From: Zheng, Kai
Sent: Friday, September 02, 2016 3:25 PM
To: Chen, Sammi
Subject: FW: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

Hi Sammi,

Could you help provide our feedback? I know you did lots of tests. Thanks!

Regards,
Kai

-Original Message-
From: Arun Suresh [mailto:asur...@apache.org]
Sent: Friday, September 02, 2016 11:33 AM
To: Andrew Wang 
Cc: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

+1 (binding).

Thanks for driving this Andrew..

* Download and built from source.
* Setup a 5 mode cluster.
* Verified that MR works with opportunistic containers
* Verified that the AMRMClient supports 'allocationRequestId'

Cheers
-Arun

On Thu, Sep 1, 2016 at 4:31 PM, Aaron Fabbri  wrote:

> +1, non-binding.
>
> I built everything on OS X and ran the s3a contract tests successfully:
>
> mvn test -Dtest=org.apache.hadoop.fs.contract.s3a.\*
>
> ...
>
> Results :
>
>
> Tests run: 78, Failures: 0, Errors: 0, Skipped: 1
>
>
> [INFO]
> --
> --
>
> [INFO] BUILD SUCCESS
>
> [INFO]
> --
> --
>
> On Thu, Sep 1, 2016 at 3:39 PM, Andrew Wang 
> wrote:
>
> > Good point Allen, I forgot about `hadoop version`. Since it's 
> > populated
> by
> > a version-info.properties file, people can always cat that file.
> >
> > On Thu, Sep 1, 2016 at 3:21 PM, Allen Wittenauer <
> a...@effectivemachines.com
> > >
> > wrote:
> >
> > >
> > > > On Sep 1, 2016, at 3:18 PM, Allen Wittenauer <
> a...@effectivemachines.com
> > >
> > > wrote:
> > > >
> > > >
> > > >> On Sep 1, 2016, at 2:57 PM, Andrew Wang 
> > > >> 
> > > wrote:
> > > >>
> > > >> Steve requested a git hash for this release. This led us into a
> brief
> > > >> discussion of our use of git tags, wherein we realized that 
> > > >> although release tags are immutable (start with "rel/"), RC tags are 
> > > >> not.
> This
> > is
> > > >> based on the HowToRelease instructions.
> > > >
> > > >   We should probably embed the git hash in one of the files 
> > > > that
> > > gets gpg signed.  That's an easy change to create-release.
> > >
> > >
> > > (Well, one more easily accessible than 'hadoop version')
> >
>

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org


[jira] [Created] (HADOOP-13578) Add Codec for ZStandard Compression

2016-09-02 Thread churro morales (JIRA)
churro morales created HADOOP-13578:
---

 Summary: Add Codec for ZStandard Compression
 Key: HADOOP-13578
 URL: https://issues.apache.org/jira/browse/HADOOP-13578
 Project: Hadoop Common
  Issue Type: New Feature
Reporter: churro morales
 Fix For: 3.0.0-alpha1


ZStandard: https://github.com/facebook/zstd has been used in production for 6 
months by facebook now.  v1.0 was recently released.  Create a codec for this 
library.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org