[
https://issues.apache.org/jira/browse/LUCENE-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14563790#comment-14563790
]
Steve Rowe commented on LUCENE-6507:
------------------------------------
I see an HdfsLockFactoryTest failure on 5.2 after this commit:
[http://jenkins.sarowe.net/job/Lucene-Solr-tests-5.2-Java8/3/].
{noformat}
[junit4] Suite: org.apache.solr.store.hdfs.HdfsLockFactoryTest
[junit4] 2> Creating dataDir:
/var/lib/jenkins/jobs/Lucene-Solr-tests-5.2-Java8/workspace/solr/build/solr-core/test/J5/temp/solr.store.hdfs.HdfsLockFactoryTest
B48BC404BF6BB3F1-001/init-core-data-001
[junit4] 2> 123149 T2061 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl
(false) and clientAuth (false)
[junit4] 2> 124356 T2061 oahu.NativeCodeLoader.<clinit> WARN Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 125343 T2061 oahmi.MetricsConfig.loadFirst WARN Cannot locate
configuration: tried
hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 125705 T2061 oml.Slf4jLog.info Logging to
org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 125714 T2061 oahh.HttpRequestLog.getRequestLog WARN Jetty
request log can only be enabled using Log4j
[junit4] 2> 126082 T2061 oml.Slf4jLog.info jetty-6.1.26
[junit4] 2> 126200 T2061 oml.Slf4jLog.info Extract
jar:file:/var/lib/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs
to ./temp/Jetty_localhost_45038_hdfs____jfnzfi/webapp
[junit4] 2> 126577 T2061 oml.Slf4jLog.info NO JSP Support for /, did not
find org.apache.jasper.servlet.JspServlet
[junit4] 2> 127704 T2061 oml.Slf4jLog.info Started
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45038
[junit4] 2> 129764 T2061 oahh.HttpRequestLog.getRequestLog WARN Jetty
request log can only be enabled using Log4j
[junit4] 2> 129777 T2061 oml.Slf4jLog.info jetty-6.1.26
[junit4] 2> 129841 T2061 oml.Slf4jLog.info Extract
jar:file:/var/lib/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
to ./temp/Jetty_localhost_60765_datanode____fxr31f/webapp
[junit4] 2> 130228 T2061 oml.Slf4jLog.info NO JSP Support for /, did not
find org.apache.jasper.servlet.JspServlet
[junit4] 2> 131028 T2061 oml.Slf4jLog.info Started
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:60765
[junit4] 2> 131767 T2061 oahh.HttpRequestLog.getRequestLog WARN Jetty
request log can only be enabled using Log4j
[junit4] 2> 131769 T2061 oml.Slf4jLog.info jetty-6.1.26
[junit4] 2> 131799 T2061 oml.Slf4jLog.info Extract
jar:file:/var/lib/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
to ./temp/Jetty_localhost_45594_datanode____xb8eu/webapp
[junit4] 2> 132058 T2061 oml.Slf4jLog.info NO JSP Support for /, did not
find org.apache.jasper.servlet.JspServlet
[junit4] 2> 132856 T2061 oml.Slf4jLog.info Started
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45594
[junit4] 2> 135493 T2088 oahhsb.BlockManager.processReport BLOCK*
processReport: from storage DS-c7286d21-0c75-425a-b32a-cda888b89811 node
DatanodeRegistration(127.0.0.1,
datanodeUuid=183b04a7-dc21-4d3e-ad8c-52569c645c0d, infoPort=60765,
ipcPort=36559, storageInfo=lv=-56;cid=testClusterID;nsid=349984326;c=0),
blocks: 0, hasStaleStorages: true, processing time: 2 msecs
[junit4] 2> 135501 T2088 oahhsb.BlockManager.processReport BLOCK*
processReport: from storage DS-2cbc54c3-5182-44c8-b80a-a4af3d3bea02 node
DatanodeRegistration(127.0.0.1,
datanodeUuid=183b04a7-dc21-4d3e-ad8c-52569c645c0d, infoPort=60765,
ipcPort=36559, storageInfo=lv=-56;cid=testClusterID;nsid=349984326;c=0),
blocks: 0, hasStaleStorages: false, processing time: 0 msecs
[junit4] 2> 135493 T2097 oahhsb.BlockManager.processReport BLOCK*
processReport: from storage DS-3c1bd938-8f62-4608-a6d4-63f576e970cd node
DatanodeRegistration(127.0.0.1,
datanodeUuid=3e441a70-defc-4b2f-bf7f-95c351d97a39, infoPort=45594,
ipcPort=58299, storageInfo=lv=-56;cid=testClusterID;nsid=349984326;c=0),
blocks: 0, hasStaleStorages: true, processing time: 1 msecs
[junit4] 2> 135511 T2097 oahhsb.BlockManager.processReport BLOCK*
processReport: from storage DS-7b0f0027-583c-462f-9b9c-e6f13ca8a160 node
DatanodeRegistration(127.0.0.1,
datanodeUuid=3e441a70-defc-4b2f-bf7f-95c351d97a39, infoPort=45594,
ipcPort=58299, storageInfo=lv=-56;cid=testClusterID;nsid=349984326;c=0),
blocks: 0, hasStaleStorages: false, processing time: 0 msecs
[junit4] 2> 135786 T2061 oas.SolrTestCaseJ4.setUp ###Starting testBasic
[junit4] 2> 135956 T2061 oassh.HdfsDirectory.<init> WARN The NameNode is in
SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 141161 T2061 oas.SolrTestCaseJ4.tearDown ###Ending testBasic
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=HdfsLockFactoryTest
-Dtests.method=testBasic -Dtests.seed=B48BC404BF6BB3F1 -Dtests.slow=true
-Dtests.locale=de_GR -Dtests.timezone=Antarctica/Vostok -Dtests.asserts=true
-Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 5.48s J5 | HdfsLockFactoryTest.testBasic <<<
[junit4] > Throwable #1:
org.apache.lucene.store.LockObtainFailedException: this lock instance was
already obtained
[junit4] > at
__randomizedtesting.SeedInfo.seed([B48BC404BF6BB3F1:1F71D91160B735DF]:0)
[junit4] > at
org.apache.solr.store.hdfs.HdfsLockFactory$HdfsLock.obtain(HdfsLockFactory.java:71)
[junit4] > at
org.apache.solr.store.hdfs.HdfsLockFactoryTest.testBasic(HdfsLockFactoryTest.java:75)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 141271 T2061 oas.SolrTestCaseJ4.setUp ###Starting
testDoubleObtain
[junit4] 2> 141393 T2061 oassh.HdfsDirectory.close Closing hdfs directory
hdfs://localhost:40796/basedir/lock
[junit4] 2> 141394 T2061 oas.SolrTestCaseJ4.tearDown ###Ending
testDoubleObtain
[junit4] 2> 141394 T2061 oahhsd.DirectoryScanner.shutdown WARN
DirectoryScanner: shutdown has been called
[junit4] 2> 141435 T2061 oml.Slf4jLog.info Stopped
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 141436 T2133
oahh.HttpServer2$SelectChannelConnectorWithSafeStartup.isRunning WARN
HttpServer Acceptor: isRunning is false. Rechecking.
[junit4] 2> 141437 T2133
oahh.HttpServer2$SelectChannelConnectorWithSafeStartup.isRunning WARN
HttpServer Acceptor: isRunning is false
[junit4] 2> 141538 T2143 oahhsd.BPServiceActor.offerService WARN
BPOfferService for Block pool BP-1715631016-127.0.1.1-1432850493139 (Datanode
Uuid 3e441a70-defc-4b2f-bf7f-95c351d97a39) service to localhost/127.0.0.1:40796
interrupted
[junit4] 2> 141538 T2143 oahhsd.BPServiceActor.run WARN Ending block pool
service for: Block pool BP-1715631016-127.0.1.1-1432850493139 (Datanode Uuid
3e441a70-defc-4b2f-bf7f-95c351d97a39) service to localhost/127.0.0.1:40796
[junit4] 2> 141542 T2061 oahhsd.DirectoryScanner.shutdown WARN
DirectoryScanner: shutdown has been called
[junit4] 2> 141559 T2061 oml.Slf4jLog.info Stopped
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 141559 T2109
oahh.HttpServer2$SelectChannelConnectorWithSafeStartup.isRunning WARN
HttpServer Acceptor: isRunning is false. Rechecking.
[junit4] 2> 141571 T2109
oahh.HttpServer2$SelectChannelConnectorWithSafeStartup.isRunning WARN
HttpServer Acceptor: isRunning is false
[junit4] 2> 141580 T2116 oahhsd.BPServiceActor.offerService WARN
BPOfferService for Block pool BP-1715631016-127.0.1.1-1432850493139 (Datanode
Uuid 183b04a7-dc21-4d3e-ad8c-52569c645c0d) service to localhost/127.0.0.1:40796
interrupted
[junit4] 2> 141588 T2116 oahhsd.BPServiceActor.run WARN Ending block pool
service for: Block pool BP-1715631016-127.0.1.1-1432850493139 (Datanode Uuid
183b04a7-dc21-4d3e-ad8c-52569c645c0d) service to localhost/127.0.0.1:40796
[junit4] 2> 141598 T2087 oahhsb.DecommissionManager$Monitor.run WARN
Monitor interrupted: java.lang.InterruptedException: sleep interrupted
[junit4] 2> 141616 T2061 oml.Slf4jLog.info Stopped
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 141722 T2061 oahml.MethodMetric$2.snapshot ERROR Error invoking
method getBlocksTotal java.lang.reflect.InvocationTargetException
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
[junit4] 2> at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:497)
[junit4] 2> at
org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
[junit4] 2> at
org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
[junit4] 2> at
org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:387)
[junit4] 2> at
org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
[junit4] 2> at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:195)
[junit4] 2> at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
[junit4] 2> at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
[junit4] 2> at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
[junit4] 2> at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
[junit4] 2> at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
[junit4] 2> at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
[junit4] 2> at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
[junit4] 2> at
com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
[junit4] 2> at
org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
[junit4] 2> at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
[junit4] 2> at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
[junit4] 2> at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
[junit4] 2> at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
[junit4] 2> at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
[junit4] 2> at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
[junit4] 2> at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
[junit4] 2> at
org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
[junit4] 2> at
org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
[junit4] 2> at
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
[junit4] 2> at
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
[junit4] 2> at
org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
[junit4] 2> at
org.apache.solr.store.hdfs.HdfsLockFactoryTest.afterClass(HdfsLockFactoryTest.java:52)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
[junit4] 2> at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:497)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
[junit4] 2> at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: java.lang.NullPointerException
[junit4] 2> at
org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
[junit4] 2> at
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
[junit4] 2> at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
[junit4] 2> ... 54 more
[junit4] 2>
[junit4] 2> 141730 T2061 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at:
/var/lib/jenkins/jobs/Lucene-Solr-tests-5.2-Java8/workspace/solr/build/solr-core/test/J5/temp/solr.store.hdfs.HdfsLockFactoryTest
B48BC404BF6BB3F1-001
[junit4] 2> 18586 T2060 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will
linger awaiting termination of 2 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=CheapBastard,
sim=DefaultSimilarity, locale=de_GR, timezone=Antarctica/Vostok
[junit4] 2> NOTE: Linux 4.0.2 amd64/Oracle Corporation 1.8.0_45
(64-bit)/cpus=16,threads=3,free=274346960,total=521142272
[junit4] 2> NOTE: All tests run in this JVM: [TestDistributedMissingSort,
TestRandomMergePolicy, TestSolrConfigHandlerCloud,
TestSweetSpotSimilarityFactory, TestSolrJ, TestStressLucene,
TestBinaryResponseWriter, TestIndexSearcher, HdfsLockFactoryTest]
[junit4] Completed [164/497] on J5 in 28.44s, 2 tests, 1 error <<< FAILURES!
{noformat}
> NativeFSLock.close() can invalidate other locks
> -----------------------------------------------
>
> Key: LUCENE-6507
> URL: https://issues.apache.org/jira/browse/LUCENE-6507
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Simon Willnauer
> Priority: Blocker
> Fix For: 4.10.5, 5.2
>
> Attachments: LUCENE-6507.patch, LUCENE-6507.patch, LUCENE-6507.patch,
> LUCENE-6507.patch, LUCENE-6507.patch, LUCENE-6507.patch, LUCENE-6507.patch,
> LUCENE-6507.patch, LUCENE-6507.patch, LUCENE-6507.patch, LUCENE-6507.patch
>
>
> the lock API in Lucene is super trappy since the lock that we return form
> this API must first be obtained and if we can't obtain it the lock should not
> be closed since we might ie. close the underlying channel in the NativeLock
> case which releases all lock for this file on some operating systems. I think
> the makeLock method should try to obtain and only return a lock if we
> successfully obtained it. Not sure if it's possible everywhere but we should
> at least make the documentation clear here.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
