[
https://issues.apache.org/jira/browse/DRILL-4249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15123344#comment-15123344
]
Khurram Faraaz commented on DRILL-4249:
---------------------------------------
And this is from drillbit.out file, it looks to be related to mapr FS client.
{noformat}
2016-01-29 09:50:19,6724 ERROR Cidcache
fs/client/fileclient/cc/cidcache.cc:1586 Thread: 8396 MoveToNextCldb: No CLDB
entries, cannot run, sleeping 5 seconds!
2016-01-29 09:50:28,6736 ERROR Cidcache
fs/client/fileclient/cc/cidcache.cc:1586 Thread: 8396 MoveToNextCldb: No CLDB
entries, cannot run, sleeping 5 seconds!
2016-01-29 09:54:39
Full thread dump OpenJDK 64-Bit Server VM (25.65-b01 mixed mode):
"2954c936-769a-def3-e59b-70a29ace5af1:foreman" #95 daemon prio=10 os_prio=0
tid=0x00007fd3cc400000 nid=0x1bd1 runnable [0x00007fd3a1521000]
java.lang.Thread.State: RUNNABLE
at com.mapr.fs.jni.MapRClient.readdirplus(Native Method)
at com.mapr.fs.MapRClientImpl.listStatus(MapRClientImpl.java:353)
at com.mapr.fs.MapRFileSystem.listMapRStatus(MapRFileSystem.java:1403)
at com.mapr.fs.MapRFileSystem.listStatus(MapRFileSystem.java:1436)
at com.mapr.fs.MapRFileSystem.listStatus(MapRFileSystem.java:78)
at org.apache.hadoop.fs.Globber.listStatus(Globber.java:69)
at org.apache.hadoop.fs.Globber.glob(Globber.java:218)
at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1710)
at com.mapr.fs.MapRFileSystem.globStatus(MapRFileSystem.java:1270)
at
org.apache.drill.exec.store.dfs.DrillFileSystem.addRecursiveStatus(DrillFileSystem.java:767)
at
org.apache.drill.exec.store.dfs.DrillFileSystem.list(DrillFileSystem.java:754)
at
org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.isHomogeneous(WorkspaceSchemaFactory.java:664)
at
org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.dropTable(WorkspaceSchemaFactory.java:692)
at
org.apache.drill.exec.planner.sql.handlers.DropTableHandler.getPlan(DropTableHandler.java:72)
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:218)
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:909)
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:244)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
"qtp801808302-94" #94 prio=5 os_prio=0 tid=0x00007fd3c8e58000 nid=0x15cb
waiting on condition [0x00007fd3a2824000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000007a42065d0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:513)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:48)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:569)
at java.lang.Thread.run(Thread.java:745)
"BitServer-4" #23 daemon prio=10 os_prio=0 tid=0x00007fd3c89c9000 nid=0x3ac8
runnable [0x00007fd3a1221000]
java.lang.Thread.State: RUNNABLE
at io.netty.channel.epoll.Native.epollWait0(Native Method)
at io.netty.channel.epoll.Native.epollWait(Native.java:148)
at
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:180)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:205)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Thread.java:745)
"BitServer-3" #22 daemon prio=10 os_prio=0 tid=0x00007fd412f42800 nid=0x3ac7
runnable [0x00007fd3a1322000]
java.lang.Thread.State: RUNNABLE
at io.netty.channel.epoll.Native.epollWait0(Native Method)
at io.netty.channel.epoll.Native.epollWait(Native.java:148)
at
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:180)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:205)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Thread.java:745)
"Curator-PathChildrenCache-2" #86 daemon prio=5 os_prio=0
tid=0x00007fd3cc578000 nid=0x20ca waiting on condition [0x00007fd3a662a000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000007a4306170> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
"threadDeathWatcher-2-1" #85 daemon prio=1 os_prio=0 tid=0x00007fd3cc1e0000
nid=0x1b52 waiting on condition [0x00007fd3a7d21000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
io.netty.util.ThreadDeathWatcher$Watcher.run(ThreadDeathWatcher.java:137)
at
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
at java.lang.Thread.run(Thread.java:745)
"qtp801808302-82-acceptor-1-ServerConnector@60d4c06d{HTTP/1.1}{0.0.0.0:8047}"
#82 prio=5 os_prio=0 tid=0x0000000002457000 nid=0x19a1 waiting for monitor
entry [0x00007fd3a7e22000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:234)
- waiting to lock <0x00000007a3e06178> (a java.lang.Object)
at
org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:336)
at
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:467)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
at java.lang.Thread.run(Thread.java:745)
...
{noformat}
> DROP TABLE HANGS indefinitely
> -----------------------------
>
> Key: DRILL-4249
> URL: https://issues.apache.org/jira/browse/DRILL-4249
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
> Affects Versions: 1.4.0
> Environment: 4 node cluster on CentOS
> Reporter: Khurram Faraaz
>
> DROP TABLE hangs. Table was created using CTAS and the input to CTAS was a
> directory that had 2000 JSON files.
> There is no Exception or error message on sqlline prompt nor in drillbit.log
> Drill 1.4.0, git commitID: 32b85160
> hadoop fs -ls /tmp shows that the directory exists (at the time when DROP
> TABLE was hung)
> drwxr-xr-x - mapr mapr 10550677 2016-01-05 12:14 /tmp/tblMD332_wp
> {noformat}
> Snippet from drillbit.log, there's no further information after the below
> text in drillbit.log
> 2016-01-06 10:03:09,817 [297319a1-fb9d-8247-7d12-c8d551d56d2d:foreman] INFO
> o.a.drill.exec.work.foreman.Foreman - Query text for query id
> 297319a1-fb9d-8247-7d12-c8d551d56d2d: drop table tblMD332_wp
> 2016-01-06 10:03:11,516 [297319a1-fb9d-8247-7d12-c8d551d56d2d:foreman] INFO
> o.a.d.exec.store.dfs.FileSelection - FileSelection.create() took 6 ms
> 2016-01-06 10:03:11,517 [297319a1-fb9d-8247-7d12-c8d551d56d2d:foreman] INFO
> o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
> numFiles: 1
> {noformat}
> {noformat}
> Note that DROP TABLE commands hangs indefinitely.
> [root@centos-01 bin]# ./sqlline -u "jdbc:drill:schema=dfs.tmp -n mapr -p mapr"
> apache drill 1.4.0
> "got drill?"
> 0: jdbc:drill:schema=dfs.tmp> drop table tblMD332_wp;
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
