[ https://issues.apache.org/jira/browse/HBASE-22607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995257#comment-16995257 ]
Mingliang Liu commented on HBASE-22607: --------------------------------------- [~AK2019] That is interesting. Can you reproduce this consistently? If so, the problem might be easier to debug. I can not debug here because I never see this with multiple runs. {code} git checkout rel/2.2.0 commit=$(git log master | grep -B 5 HBASE-22607 | grep commit | awk '{print $2}') git cherry-pick $commit mvn clean package mvn test -Dtest=TestExportSnapshotNoCluster {code} So I check the line number and it is not very clear which line error out in {{testSnapshotWithRefsExportFileSystemState(}}. I guess it's in LoC 216 of {{TestExportSnapshot}}. If so, the fs is created using new Configuration which is patched as in {{TestExportSnapshotNoCluster}}. {code:title=TestExportSnapshot.java:216} copyDir = copyDir.makeQualified(fs); {code} Could you try the addendum diff [^HBASE-22607.addendum.000.patch] ? Hopefully it will fix this. Otherwise we may have to debug further, which perhaps is orthogonal to this patch. > TestExportSnapshotNoCluster::testSnapshotWithRefsExportFileSystemState() > fails intermittently > --------------------------------------------------------------------------------------------- > > Key: HBASE-22607 > URL: https://issues.apache.org/jira/browse/HBASE-22607 > Project: HBase > Issue Type: Bug > Components: test > Affects Versions: 3.0.0, 2.2.0, 2.0.6 > Reporter: Mingliang Liu > Assignee: Mingliang Liu > Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9 > > Attachments: HBASE-22607.000.patch, HBASE-22607.001.patch, > HBASE-22607.002.patch, HBASE-22607.addendum.000.patch > > > In previous runs, test > {{TestExportSnapshotNoCluster.testSnapshotWithRefsExportFileSystemState}} > fails intermittently with {{java.net.ConnectException: Connection refused}} > exception, see build > [510|https://builds.apache.org/job/PreCommit-HBASE-Build/510/testReport/org.apache.hadoop.hbase.snapshot/TestExportSnapshotNoCluster/testSnapshotWithRefsExportFileSystemState/], > > [545|https://builds.apache.org/job/PreCommit-HBASE-Build/545/testReport/org.apache.hadoop.hbase.snapshot/TestExportSnapshotNoCluster/testSnapshotWithRefsExportFileSystemState/], > and > [556|https://builds.apache.org/job/PreCommit-HBASE-Build/556/testReport/org.apache.hadoop.hbase.snapshot/TestExportSnapshotNoCluster/testSnapshotWithRefsExportFileSystemState/]. > So one sample exception is like: > {quote} > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155) > at > org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346) > at com.sun.proxy.$Proxy20.getListing(Unknown Source) > at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1630) > at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1614) > at > org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:900) > at > org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:114) > at > org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:964) > at > org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:961) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:961) > at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1537) > at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1580) > at > org.apache.hadoop.hbase.util.CommonFSUtils.listStatus(CommonFSUtils.java:693) > at > org.apache.hadoop.hbase.util.FSTableDescriptors.getCurrentTableInfoStatus(FSTableDescriptors.java:448) > at > org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoPath(FSTableDescriptors.java:429) > at > org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoPath(FSTableDescriptors.java:410) > at > org.apache.hadoop.hbase.util.FSTableDescriptors.createTableDescriptorForTableDirectory(FSTableDescriptors.java:763) > at > org.apache.hadoop.hbase.snapshot.SnapshotTestingUtils$SnapshotMock.createTable(SnapshotTestingUtils.java:675) > at > org.apache.hadoop.hbase.snapshot.SnapshotTestingUtils$SnapshotMock.createSnapshot(SnapshotTestingUtils.java:653) > at > org.apache.hadoop.hbase.snapshot.SnapshotTestingUtils$SnapshotMock.createSnapshot(SnapshotTestingUtils.java:647) > at > org.apache.hadoop.hbase.snapshot.SnapshotTestingUtils$SnapshotMock.createSnapshotV2(SnapshotTestingUtils.java:637) > at > org.apache.hadoop.hbase.snapshot.TestExportSnapshotNoCluster.testSnapshotWithRefsExportFileSystemState(TestExportSnapshotNoCluster.java:80) > {quote} > This seems that, somehow the rootdir filesystem is not LocalFileSystem, but > on HDFS. I have not dig deeper why this happens since it's failing > intermittently and I can not reproduce it locally. Since this is testing > export snapshot tool without cluster, we can enforce it using > LocalFileSystem; no breaking change. -- This message was sent by Atlassian Jira (v8.3.4#803005)