steveloughran commented on pull request #2149:
URL: https://github.com/apache/hadoop/pull/2149#issuecomment-667037406
Overnight test run failures. This is with a change not pushed up, where
during teardown if FS != keep we do an audit of the store. Aim is to make sure
that 100% of the FS operations don't leave markers in delete mode, so as to be
fully compatible with existing releases.
```
[ERROR] testRunVerboseAudit(org.apache.hadoop.fs.s3a.tools.ITestMarkerTool)
Time elapsed: 3.298 s <<< FAILURE!
org.junit.ComparisonFailure: [Exit code of
marker(s3a://stevel-london/fork-0001/test/testRunVerboseAudit, true, 0) ->
ScanResult{exitCode=46, tracker=DirMarkerTracker{leafMarkers=2,
surplusMarkers=4,
lastDirChecked=s3a://stevel-london/fork-0001/test/testRunVerboseAudit,
filesFound=3, scanCount=28}, purgeSummary=null}] expected:<[0]> but was:<[46]>
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
org.apache.hadoop.fs.s3a.tools.ITestMarkerTool.markerTool(ITestMarkerTool.java:528)
at
org.apache.hadoop.fs.s3a.AbstractS3ATestBase.teardown(AbstractS3ATestBase.java:73)
at
org.apache.hadoop.fs.s3a.tools.ITestMarkerTool.teardown(ITestMarkerTool.java:109)
```
One unexpected seek failure (!)
```
[ERROR]
testReadSmallFile[0](org.apache.hadoop.fs.contract.s3a.ITestS3AContractSeek)
Time elapsed: 1.191 s <<< FAILURE!
java.lang.AssertionError: expected:<1024> but was:<433>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:631)
at
org.apache.hadoop.fs.contract.AbstractContractSeekTest.testReadSmallFile(AbstractContractSeekTest.java:575)
```
and the landsat select test is timing out in parallel (6) test runs, but not
standalone. This is consistent for me, new this week.
```
[ERROR]
testSelectSeekFullLandsat(org.apache.hadoop.fs.s3a.select.ITestS3SelectLandsat)
Time elapsed: 599.989 s <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 600000
milliseconds
at java.lang.Throwable.fillInStackTrace(Native Method)
at java.lang.Throwable.fillInStackTrace(Throwable.java:784)
at java.lang.Throwable.<init>(Throwable.java:251)
at
org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:253)
at
org.apache.log4j.helpers.PatternParser$LocationPatternConverter.convert(PatternParser.java:500)
at
org.apache.log4j.helpers.PatternConverter.format(PatternConverter.java:65)
at org.apache.log4j.PatternLayout.format(PatternLayout.java:506)
at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:310)
at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
at
org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
at org.apache.log4j.Category.callAppenders(Category.java:206)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.log(Category.java:856)
at org.slf4j.impl.Log4jLoggerAdapter.debug(Log4jLoggerAdapter.java:230)
at org.apache.hadoop.util.DurationInfo.close(DurationInfo.java:101)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:111)
at
org.apache.hadoop.fs.s3a.select.SelectInputStream.read(SelectInputStream.java:246)
at
org.apache.hadoop.fs.s3a.select.SelectInputStream.seek(SelectInputStream.java:324)
at
org.apache.hadoop.fs.FSDataInputStream.seek(FSDataInputStream.java:67)
at
org.apache.hadoop.fs.s3a.select.AbstractS3SelectTest.seek(AbstractS3SelectTest.java:701)
at
org.apache.hadoop.fs.s3a.select.ITestS3SelectLandsat.testSelectSeekFullLandsat(ITestS3SelectLandsat.java:427)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
```
really weird where it times out though -log4j. Deadlock?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
