Also I am trying to run the abfs tests but all tests are getting skipped
even if I think I have right auth-keys.xml. I will debug this and update.
hadoop-azure expects azure-auth-keys.xml instead of auth-keys.xml?
I think it should be consistent with other FS modules..
Masatake Iwasaki
On 2020/05/13 0:33, Mukund Madhav Thakur wrote:
I compiled and ran s3 tests using
mvn clean verify -Ds3guard -Ddynamo -Dauth. I see some failures. I ran
these separately as well but it still fails for me.
Also I am trying to run the abfs tests but all tests are getting skipped
even if I think I have right auth-keys.xml. I will debug this and update.
[*ERROR*] *Failures: *
[*ERROR*] *
ITestS3AMiscOperations.testEmptyFileChecksums:147->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88
checksums expected:<etag: "4eb50cc3151f8d174a2ff79a7c16277e"> but
was:<etag: "cb190410cc285024b2066360c44c006a">*
[*ERROR*] *
ITestS3AMiscOperations.testNonEmptyFileChecksumsUnencrypted:199->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88
checksums expected:<etag: "90d890f880e286b6d62534dfde5ce720"> but
was:<etag: "e9181a730651153656d3570226667ef0">*
[*INFO*]
[*ERROR*] *Tests run: 12, Failures: 2, Errors: 0, Skipped: 0*
[*ERROR*] *Errors: *
[*ERROR*] *
ITestS3GuardToolDynamoDB.testDynamoDBInitDestroyCycle:224->AbstractS3GuardToolTestBase.exec:286->AbstractS3GuardToolTestBase.exec:308
ยป AWSServiceIO*
On Tue, May 5, 2020 at 11:48 PM Steve Loughran <ste...@cloudera.com.invalid>
wrote:
mvn -T 1 -Phadoop-3.2 -Dhadoop.version=3.1.4 -Psnapshots-and-staging
-Phadoop-cloud,yarn,kinesis-asl,yarn clean install -DskipTests
Then a test run of the cloud bits
mvn -T 1 -Phadoop-3.2 -Dhadoop.version=3.1.4 -Psnapshots-and-staging
-Phadoop-cloud,yarn,kinesis-asl test --pl hadoop-cloud
And I got a guava binding stack trace (joy!)
CommitterBindingSuite:
*** RUN ABORTED ***
java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
at
org.apache.spark.internal.io
.cloud.CommitterBindingSuite.newJob(CommitterBindingSuite.scala:89)
at
org.apache.spark.internal.io
.cloud.CommitterBindingSuite.$anonfun$new$1(CommitterBindingSuite.scala:55)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:186)
...
Fix: spark's guava version needs to be bumped up. There's a bigger patch
for that, but I've got a minor PR which lets someone change it on the maven
CLI to match the hadoop version, e.g. -Dguava.version=27.0-jre
https://issues.apache.org/jira/browse/SPARK-31644
There's nothing which can be done except highlight to all that this has
happened and what that stack trace means.
On Tue, 5 May 2020 at 02:08, Wei-Chiu Chuang <weic...@cloudera.com.invalid
wrote:
Gabor, I'm sorry there's a test failure in branch-3.1 HDFS-14599
<https://issues.apache.org/jira/browse/HDFS-14599>
1.
1. I just cherrypicked the fix to branch-3.2 branch-3.1. It's a
test
only fix so technically I could live with it. But it would be best
to add
the fix to 3.1.4 as well.
On Mon, May 4, 2020 at 3:20 PM Gabor Bota <gab...@apache.org> wrote:
Hi folks,
I have put together a release candidate (RC0) for Hadoop 3.1.4.
The RC is available at:
http://people.apache.org/~gabota/hadoop-3.1.4-RC0/
The RC tag in git is here:
https://github.com/apache/hadoop/releases/tag/release-3.1.4-RC0
The maven artifacts are staged at
https://repository.apache.org/content/repositories/orgapachehadoop-1266/
You can find my public key at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
and http://keys.gnupg.net/pks/lookup?op=get&search=0xB86249D83539B38C
Please try the release and vote. The vote will run for 5 weekdays,
until May 11. 2020. 23:00 CET.
Thanks,
Gabor
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
