[jira] [Created] (HADOOP-13574) Unnecessary file existence check causes problems with S3
Alex Zolotko created HADOOP-13574: - Summary: Unnecessary file existence check causes problems with S3 Key: HADOOP-13574 URL: https://issues.apache.org/jira/browse/HADOOP-13574 Project: Hadoop Common Issue Type: Bug Components: fs/s3 Reporter: Alex Zolotko Priority: Minor We recently got the following exception on production: {noformat} java.io.FileNotFoundException: Key 'xxx/_temporary/0/_temporary/attempt_201609010631__m_001128_1128/part-01128' does not exist in S3 at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.handleServiceException(Jets3tNativeFileSystemStore.java:234) at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.copy(Jets3tNativeFileSystemStore.java:201) at sun.reflect.GeneratedMethodAccessor71.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:186) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at org.apache.hadoop.fs.s3native.$Proxy13.copy(Unknown Source) at org.apache.hadoop.fs.s3native.NativeS3FileSystem.rename(NativeS3FileSystem.java:659) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitTask(FileOutputCommitter.java:435) at org.apache.hadoop.mapred.FileOutputCommitter.commitTask(FileOutputCommitter.java:172) at org.apache.hadoop.mapred.OutputCommitter.commitTask(OutputCommitter.java:291) at org.apache.spark.mapred.SparkHadoopMapRedUtil$.performCommit$1(SparkHadoopMapRedUtil.scala:98) at org.apache.spark.mapred.SparkHadoopMapRedUtil$.commitTask(SparkHadoopMapRedUtil.scala:124) at org.apache.spark.SparkHadoopWriter.commit(SparkHadoopWriter.scala:107) at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1$$anonfun$13.apply(PairRDDFunctions.scala:1204) at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1$$anonfun$13.apply(PairRDDFunctions.scala:1183) {noformat} FileOutputCommitter.commitTask() does check that the file exists before trying to rename it, but due to S3's relaxed consistency guarantees the following fs.rename(taskAttemptPath, committedTaskPath) still fails. Here's an excerpt from the Amazon S3 documentation ([https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html]): {quote} Amazon S3 Data Consistency Model Amazon S3 provides read-after-write consistency for PUTS of new objects in your S3 bucket in all regions with one caveat. The caveat is that if you make a HEAD or GET request to the key name (to find if the object exists) before creating the object, Amazon S3 provides eventual consistency for read-after-write. {quote} The problematic S3 object existence check, that causes S3 to fallback to eventual consistency, is in NativeS3FileSystem.create(): {code} if (exists(f) && !overwrite) { throw new IOException("File already exists:"+f); } {code} If the overwrite parameter is set to "true" (as in our case), calling exists(f) is unnecessary and only "upsets" S3. The proposed fix is switch the order of the predicates: {code} if (!overwrite && exists(f)) { throw new IOException("File already exists:"+f); } {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13575) Website shows incorrect 25 January, 2016 as the release date for 2.7.3
Jacek Laskowski created HADOOP-13575: Summary: Website shows incorrect 25 January, 2016 as the release date for 2.7.3 Key: HADOOP-13575 URL: https://issues.apache.org/jira/browse/HADOOP-13575 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.7.3 Reporter: Jacek Laskowski http://hadoop.apache.org/releases.html shows 2.7.3 released on 25 January, 2016 which should rather be 25 August, 2016. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0
+1( non-binding) * Successfully downloaded and built from source. * Deployed to single node cluster. * Ran Sleep and Wordcount jobs. Thanks Andrew for the effort! Regards,Kuhu On Thursday, September 1, 2016 10:32 PM, Arun Suresh wrote: +1 (binding). Thanks for driving this Andrew.. * Download and built from source. * Setup a 5 mode cluster. * Verified that MR works with opportunistic containers * Verified that the AMRMClient supports 'allocationRequestId' Cheers -Arun On Thu, Sep 1, 2016 at 4:31 PM, Aaron Fabbri wrote: > +1, non-binding. > > I built everything on OS X and ran the s3a contract tests successfully: > > mvn test -Dtest=org.apache.hadoop.fs.contract.s3a.\* > > ... > > Results : > > > Tests run: 78, Failures: 0, Errors: 0, Skipped: 1 > > > [INFO] > > > [INFO] BUILD SUCCESS > > [INFO] > > > On Thu, Sep 1, 2016 at 3:39 PM, Andrew Wang > wrote: > > > Good point Allen, I forgot about `hadoop version`. Since it's populated > by > > a version-info.properties file, people can always cat that file. > > > > On Thu, Sep 1, 2016 at 3:21 PM, Allen Wittenauer < > a...@effectivemachines.com > > > > > wrote: > > > > > > > > > On Sep 1, 2016, at 3:18 PM, Allen Wittenauer < > a...@effectivemachines.com > > > > > > wrote: > > > > > > > > > > > >> On Sep 1, 2016, at 2:57 PM, Andrew Wang > > > wrote: > > > >> > > > >> Steve requested a git hash for this release. This led us into a > brief > > > >> discussion of our use of git tags, wherein we realized that although > > > >> release tags are immutable (start with "rel/"), RC tags are not. > This > > is > > > >> based on the HowToRelease instructions. > > > > > > > > We should probably embed the git hash in one of the files that > > > gets gpg signed. That's an easy change to create-release. > > > > > > > > > (Well, one more easily accessible than 'hadoop version') > > >
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/ [Sep 1, 2016 8:38:51 AM] (vvasudev) YARN-5596. Fix failing unit test in TestDockerContainerRuntime. [Sep 1, 2016 6:03:06 PM] (liuml07) HADOOP-13375. [Sep 1, 2016 6:32:26 PM] (cdouglas) FsPermission string constructor does not recognize sticky bit. [Sep 1, 2016 8:44:44 PM] (kihwal) HADOOP-13465. Design Server.Call to be extensible for unified call [Sep 1, 2016 9:44:01 PM] (kasha) YARN-5566. Client-side NM graceful decom is not triggered when jobs [Sep 1, 2016 10:35:25 PM] (xiao) HDFS-10820. Reuse closeResponder to reset the response variable in -1 overall The following subsystems voted -1: asflicense unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed CTEST tests : test_test_libhdfs_threaded_hdfs_static test_test_libhdfs_zerocopy_hdfs_static Failed junit tests : hadoop.hdfs.TestFileChecksum hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints hadoop.hdfs.TestRollingUpgrade hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices hadoop.yarn.server.TestContainerManagerSecurity hadoop.yarn.server.TestMiniYarnClusterNodeUtilization cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/diff-compile-javac-root.txt [168K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/diff-checkstyle-root.txt [16M] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/diff-patch-pylint.txt [16K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/diff-patch-shelldocs.txt [16K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/whitespace-eol.txt [12M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/whitespace-tabs.txt [1.3M] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/diff-javadoc-javadoc-root.txt [2.2M] CTEST: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/patch-hadoop-hdfs-project_hadoop-hdfs-native-client-ctest.txt [24K] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [152K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt [12K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt [268K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-nativetask.txt [124K] asflicense: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/152/artifact/out/patch-asflicense-problems.txt [4.0K] Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13576) Old Hadoop user mailing lists still exist
Sebb created HADOOP-13576: - Summary: Old Hadoop user mailing lists still exist Key: HADOOP-13576 URL: https://issues.apache.org/jira/browse/HADOOP-13576 Project: Hadoop Common Issue Type: Bug Reporter: Sebb Priority: Minor The mailing lists for (common,hdfs,mapreduce)-user@ are still active; i.e. users can subscribe and post messages. If that was not the intention [1] I suggest raising an INFRA JIRA to get the old lists closed down and/or aliased to user@. See also INFRA-12554. [1] http://mail-archives.apache.org/mod_mbox/hadoop-user/201208.mbox/%3C0A0CA23A-313F-4BA7-ADC5-31A1DB76B4FA%40hortonworks.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0
Thanks for making this forward hardly, Andrew! In fact, I hit an error similar to Eric's one regardless of configuring yarn.app.mapreduce.am.env and mapreduce.admin.user.env... does anyone hit same error? 2016-09-02 16:49:35,223 INFO mapreduce.Job: The url to track the job: http://ip-172-31-7-124.ap-northeast-1.compute.internal:23188/proxy/application_1472832909785_0013/ 2016-09-02 16:49:35,224 INFO mapreduce.Job: Running job: job_1472832909785_0013 2016-09-02 16:49:48,282 INFO mapreduce.Job: Job job_1472832909785_0013 running in uber mode : false 2016-09-02 16:49:48,283 INFO mapreduce.Job: map 0% reduce 0% 2016-09-02 16:49:48,293 INFO mapreduce.Job: Job job_1472832909785_0013 failed with state FAILED due to: Application application_1472832909785_0013 failed 2 times due to AM Container for appattempt_1472832909785_0013_02 exited with exitCode: 1 Failing this attempt.Diagnostics: Exception from container-launch. Container id: container_e02_1472832909785_0013_02_01 Exit code: 1 Stack trace: ExitCodeException exitCode=1: at org.apache.hadoop.util.Shell.runCommand(Shell.java:974) at org.apache.hadoop.util.Shell.run(Shell.java:878) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1172) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:237) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:419) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:284) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:88) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Best, - Tsuyoshi On Wed, Aug 31, 2016 at 12:51 AM, Andrew Wang wrote: > Hi all, > > Thanks to the combined work of many, many contributors, here's an RC0 for > 3.0.0-alpha1: > > http://home.apache.org/~wang/3.0.0-alpha1-RC0/ > > alpha1 is the first in a series of planned alpha releases leading up to GA. > The objective is to get an artifact out to downstreams for testing and to > iterate quickly based on their feedback. So, please keep that in mind when > voting; hopefully most issues can be addressed by future alphas rather than > future RCs. > > Sorry for getting this out on a Tuesday, but I'd still like this vote to > run the normal 5 days, thus ending Saturday (9/3) at 9AM PDT. I'll extend > if we lack the votes. > > Please try it out and let me know what you think. > > Best, > Andrew - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0
I'm sorry, very big typo I did... s/hardly/very hard/ Best - Tsuyoshi On Sat, Sep 3, 2016 at 1:53 AM, Tsuyoshi Ozawa wrote: > Thanks for making this forward hardly, Andrew! > > In fact, I hit an error similar to Eric's one regardless of > configuring yarn.app.mapreduce.am.env and mapreduce.admin.user.env... > does anyone hit same error? > > > 2016-09-02 16:49:35,223 INFO mapreduce.Job: The url to track the job: > http://ip-172-31-7-124.ap-northeast-1.compute.internal:23188/proxy/application_1472832909785_0013/ > > 2016-09-02 16:49:35,224 INFO mapreduce.Job: Running job: > job_1472832909785_0013 > > 2016-09-02 16:49:48,282 INFO mapreduce.Job: Job job_1472832909785_0013 > running in uber mode : false > > 2016-09-02 16:49:48,283 INFO mapreduce.Job: map 0% reduce 0% > > 2016-09-02 16:49:48,293 INFO mapreduce.Job: Job job_1472832909785_0013 > failed with state FAILED due to: Application > application_1472832909785_0013 failed 2 times due to AM Container for > appattempt_1472832909785_0013_02 exited with exitCode: 1 > > Failing this attempt.Diagnostics: Exception from container-launch. > > Container id: container_e02_1472832909785_0013_02_01 > > Exit code: 1 > > Stack trace: ExitCodeException exitCode=1: > > at org.apache.hadoop.util.Shell.runCommand(Shell.java:974) > > at org.apache.hadoop.util.Shell.run(Shell.java:878) > > at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1172) > > at > org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:237) > > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:419) > > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:284) > > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:88) > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:745) > > Best, > - Tsuyoshi > > On Wed, Aug 31, 2016 at 12:51 AM, Andrew Wang > wrote: >> Hi all, >> >> Thanks to the combined work of many, many contributors, here's an RC0 for >> 3.0.0-alpha1: >> >> http://home.apache.org/~wang/3.0.0-alpha1-RC0/ >> >> alpha1 is the first in a series of planned alpha releases leading up to GA. >> The objective is to get an artifact out to downstreams for testing and to >> iterate quickly based on their feedback. So, please keep that in mind when >> voting; hopefully most issues can be addressed by future alphas rather than >> future RCs. >> >> Sorry for getting this out on a Tuesday, but I'd still like this vote to >> run the normal 5 days, thus ending Saturday (9/3) at 9AM PDT. I'll extend >> if we lack the votes. >> >> Please try it out and let me know what you think. >> >> Best, >> Andrew - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13577) Download page should link the ASF mirrora for KEYS, sigs, hashes
Sebb created HADOOP-13577: - Summary: Download page should link the ASF mirrora for KEYS, sigs, hashes Key: HADOOP-13577 URL: https://issues.apache.org/jira/browse/HADOOP-13577 Project: Hadoop Common Issue Type: Bug Environment: http://hadoop.apache.org/releases.html Reporter: Sebb The download page currently points to https://dist.apache.org/repos/dist/release/hadoop/... for KEYS, sigs and hashes. However the dist SVN tree is not designed for this; such files must be downloaded from the ASF mirrors, i.e https://www.apache.org/dist/hadoop/... Please can you adjust the links accordingly? The links should use HTTPS. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
RE: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0
Thanks Sammi. My non-binding +1 to make the release candidate. Regards, Kai -Original Message- From: Chen, Sammi Sent: Friday, September 02, 2016 4:59 PM To: Zheng, Kai ; Andrew Wang ; Arun Suresh Cc: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org; Chen, Sammi Subject: RE: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0 +1 (non-binding). Thanks for driving this Andrew! * Download and built from source. * Setup a 10 node cluster (1 name node + 9 data nodes) * Verified normal HDFS file put/get operation with 3x replication * With 2 data nodes failure, verified HDFS file put/get operation with 3x replication, file integrity is OK * Enable Erasure Code policy "RS-DEFAULT-6-3-64k", verified HDFS file put/get operation * Enable Erasure Code policy "RS-DEFAULT-6-3-64k", with 3 data nodes failure, verified HDFS file put/get operation, file integrity is OK Cheers -Sammi -Original Message- From: Zheng, Kai Sent: Friday, September 02, 2016 3:25 PM To: Chen, Sammi Subject: FW: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0 Hi Sammi, Could you help provide our feedback? I know you did lots of tests. Thanks! Regards, Kai -Original Message- From: Arun Suresh [mailto:asur...@apache.org] Sent: Friday, September 02, 2016 11:33 AM To: Andrew Wang Cc: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0 +1 (binding). Thanks for driving this Andrew.. * Download and built from source. * Setup a 5 mode cluster. * Verified that MR works with opportunistic containers * Verified that the AMRMClient supports 'allocationRequestId' Cheers -Arun On Thu, Sep 1, 2016 at 4:31 PM, Aaron Fabbri wrote: > +1, non-binding. > > I built everything on OS X and ran the s3a contract tests successfully: > > mvn test -Dtest=org.apache.hadoop.fs.contract.s3a.\* > > ... > > Results : > > > Tests run: 78, Failures: 0, Errors: 0, Skipped: 1 > > > [INFO] > -- > -- > > [INFO] BUILD SUCCESS > > [INFO] > -- > -- > > On Thu, Sep 1, 2016 at 3:39 PM, Andrew Wang > wrote: > > > Good point Allen, I forgot about `hadoop version`. Since it's > > populated > by > > a version-info.properties file, people can always cat that file. > > > > On Thu, Sep 1, 2016 at 3:21 PM, Allen Wittenauer < > a...@effectivemachines.com > > > > > wrote: > > > > > > > > > On Sep 1, 2016, at 3:18 PM, Allen Wittenauer < > a...@effectivemachines.com > > > > > > wrote: > > > > > > > > > > > >> On Sep 1, 2016, at 2:57 PM, Andrew Wang > > > >> > > > wrote: > > > >> > > > >> Steve requested a git hash for this release. This led us into a > brief > > > >> discussion of our use of git tags, wherein we realized that > > > >> although release tags are immutable (start with "rel/"), RC tags are > > > >> not. > This > > is > > > >> based on the HowToRelease instructions. > > > > > > > > We should probably embed the git hash in one of the files > > > > that > > > gets gpg signed. That's an easy change to create-release. > > > > > > > > > (Well, one more easily accessible than 'hadoop version') > > > - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13578) Add Codec for ZStandard Compression
churro morales created HADOOP-13578: --- Summary: Add Codec for ZStandard Compression Key: HADOOP-13578 URL: https://issues.apache.org/jira/browse/HADOOP-13578 Project: Hadoop Common Issue Type: New Feature Reporter: churro morales Fix For: 3.0.0-alpha1 ZStandard: https://github.com/facebook/zstd has been used in production for 6 months by facebook now. v1.0 was recently released. Create a codec for this library. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org