RE: Apache Drill- Error setting up remote intermediate fragment execution
Hi Sorabh, Thanks for your quick response. It resolved my problem . Best regards, _ Tilak Surneni -Original Message- From: Sorabh Hamirwasia [mailto:shamirwa...@mapr.com] Sent: Wednesday, April 11, 2018 6:09 PM To: user@drill.apache.org Subject: Re: Apache Drill- Error setting up remote intermediate fragment execution Sent too soon. Hi Surneni, This looks to be an issue with your network setup. The address or hostname known to a Drillbit about another Drillbit is not resolved correctly while trying to make connection. Nodes with failure: drillslave-7dccb67657-9qv4l, drillslave-7dccb67657-rhfgt Can you please verify if above 2 nodes can ping each other using the hostname? If not may be you can try to update the /etc/hosts file to provide a mapping between hostname and corresponding IP address. with org.apache.drill.exec.rpc.RpcException: Command failed while establishing connection. Failure type CONNECTION.. 2018-04-11 07:26:17,705 [BitServer-6] ERROR o.a.d.e.rpc.ConnectionMultiListener - Failed to establish connection java.util.concurrent.ExecutionException: java.nio.channels.UnresolvedAddressException at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:54) ~[netty-common-4.0.48.Final.jar:4.0.48.Final] Thanks, Sorabh From: Sorabh Hamirwasia Sent: Wednesday, April 11, 2018 9:04 AM To: user@drill.apache.org Subject: Re: Apache Drill- Error setting up remote intermediate fragment execution Hi Surneni, This looks to be an issue with your network setup. The address or hostname known to a Drillbit about another Drillbit is not resolved correctly while trying to make connection. Thanks, Sorabh From: Surneni Tilak <surneni.ti...@ericsson.com> Sent: Wednesday, April 11, 2018 8:30 AM To: user@drill.apache.org Subject: FW: Apache Drill- Error setting up remote intermediate fragment execution Hi Team, I have successfully launched a Drill cluster on 3 docker containers with Zoo keeper Quorum and accessing data from a remote node on which Drill is not installed . I am able to list all the drill bits from each node and run queries on a single drill bit , But I am facing issue when I am trying to run a query which distributes the query to the other drill bits by creating the intermediate fragments. I have gone through the documentation but It couldn't help. I got the below error message in Drill shell and the detailed error log mentioned below. Please help me on resolving this. Error in Drill Shell: Error: CONNECTION ERROR: Error setting up remote intermediate fragment execution Nodes with failures drillslave-7dccb67657-9qv4l, drillslave-7dccb67657-rhfgt [Error Id: 5d701a0d-988b-4845-83f6-b029ad6825a8 on drillmaster-76d6885c59-wpksz:31010] (state=,code=0) Error Log: 2018-04-11 07:26:17,391 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,401 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,404 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Execute d 1 out of 1 using 1 threads. Time: 2ms total, 2.798853ms avg, 2ms max. 2018-04-11 07:26:17,404 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Execute d 1 out of 1 using 1 threads. Earliest start: 0.699000 ?s, Latest start: 0.699000 ?s, Average start: 0.699000 ?s . 2018-04-11 07:26:17,408 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pr uning, pruning class: org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule 2018-04-11 07:26:17,408 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.e.p.l.partition.PruneScanRule - No partition columns a re projected from the scan..continue. Total pruning elapsed time: 0 m
Re: Apache Drill- Error setting up remote intermediate fragment execution
Sent too soon. Hi Surneni, This looks to be an issue with your network setup. The address or hostname known to a Drillbit about another Drillbit is not resolved correctly while trying to make connection. Nodes with failure: drillslave-7dccb67657-9qv4l, drillslave-7dccb67657-rhfgt Can you please verify if above 2 nodes can ping each other using the hostname? If not may be you can try to update the /etc/hosts file to provide a mapping between hostname and corresponding IP address. with org.apache.drill.exec.rpc.RpcException: Command failed while establishing connection. Failure type CONNECTION.. 2018-04-11 07:26:17,705 [BitServer-6] ERROR o.a.d.e.rpc.ConnectionMultiListener - Failed to establish connection java.util.concurrent.ExecutionException: java.nio.channels.UnresolvedAddressException at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:54) ~[netty-common-4.0.48.Final.jar:4.0.48.Final] Thanks, Sorabh From: Sorabh Hamirwasia Sent: Wednesday, April 11, 2018 9:04 AM To: user@drill.apache.org Subject: Re: Apache Drill- Error setting up remote intermediate fragment execution Hi Surneni, This looks to be an issue with your network setup. The address or hostname known to a Drillbit about another Drillbit is not resolved correctly while trying to make connection. Thanks, Sorabh From: Surneni Tilak <surneni.ti...@ericsson.com> Sent: Wednesday, April 11, 2018 8:30 AM To: user@drill.apache.org Subject: FW: Apache Drill- Error setting up remote intermediate fragment execution Hi Team, I have successfully launched a Drill cluster on 3 docker containers with Zoo keeper Quorum and accessing data from a remote node on which Drill is not installed . I am able to list all the drill bits from each node and run queries on a single drill bit , But I am facing issue when I am trying to run a query which distributes the query to the other drill bits by creating the intermediate fragments. I have gone through the documentation but It couldn't help. I got the below error message in Drill shell and the detailed error log mentioned below. Please help me on resolving this. Error in Drill Shell: Error: CONNECTION ERROR: Error setting up remote intermediate fragment execution Nodes with failures drillslave-7dccb67657-9qv4l, drillslave-7dccb67657-rhfgt [Error Id: 5d701a0d-988b-4845-83f6-b029ad6825a8 on drillmaster-76d6885c59-wpksz:31010] (state=,code=0) Error Log: 2018-04-11 07:26:17,391 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,401 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,404 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Execute d 1 out of 1 using 1 threads. Time: 2ms total, 2.798853ms avg, 2ms max. 2018-04-11 07:26:17,404 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Execute d 1 out of 1 using 1 threads. Earliest start: 0.699000 ?s, Latest start: 0.699000 ?s, Average start: 0.699000 ?s . 2018-04-11 07:26:17,408 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pr uning, pruning class: org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule 2018-04-11 07:26:17,408 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.e.p.l.partition.PruneScanRule - No partition columns a re projected from the scan..continue. Total pruning elapsed time: 0 ms 2018-04-11 07:26:17,611 [BitServer-3] ERROR o.a.d.e.rpc.ConnectionMultiListener - Failed to establish connection java.util.concurrent.ExecutionException: java.nio.channels.UnresolvedAddressException at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:54) ~[netty-common-4.0.48.Final.jar:4.0.48
Re: Apache Drill- Error setting up remote intermediate fragment execution
Hi Surneni, This looks to be an issue with your network setup. The address or hostname known to a Drillbit about another Drillbit is not resolved correctly while trying to make connection. Thanks, Sorabh From: Surneni Tilak <surneni.ti...@ericsson.com> Sent: Wednesday, April 11, 2018 8:30 AM To: user@drill.apache.org Subject: FW: Apache Drill- Error setting up remote intermediate fragment execution Hi Team, I have successfully launched a Drill cluster on 3 docker containers with Zoo keeper Quorum and accessing data from a remote node on which Drill is not installed . I am able to list all the drill bits from each node and run queries on a single drill bit , But I am facing issue when I am trying to run a query which distributes the query to the other drill bits by creating the intermediate fragments. I have gone through the documentation but It couldn't help. I got the below error message in Drill shell and the detailed error log mentioned below. Please help me on resolving this. Error in Drill Shell: Error: CONNECTION ERROR: Error setting up remote intermediate fragment execution Nodes with failures drillslave-7dccb67657-9qv4l, drillslave-7dccb67657-rhfgt [Error Id: 5d701a0d-988b-4845-83f6-b029ad6825a8 on drillmaster-76d6885c59-wpksz:31010] (state=,code=0) Error Log: 2018-04-11 07:26:17,391 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,401 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,404 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Execute d 1 out of 1 using 1 threads. Time: 2ms total, 2.798853ms avg, 2ms max. 2018-04-11 07:26:17,404 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Execute d 1 out of 1 using 1 threads. Earliest start: 0.699000 ?s, Latest start: 0.699000 ?s, Average start: 0.699000 ?s . 2018-04-11 07:26:17,408 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pr uning, pruning class: org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule 2018-04-11 07:26:17,408 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.e.p.l.partition.PruneScanRule - No partition columns a re projected from the scan..continue. Total pruning elapsed time: 0 ms 2018-04-11 07:26:17,611 [BitServer-3] ERROR o.a.d.e.rpc.ConnectionMultiListener - Failed to establish connection java.util.concurrent.ExecutionException: java.nio.channels.UnresolvedAddressException at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:54) ~[netty-common-4.0.48.Final.jar:4.0.48.Final] at org.apache.drill.exec.rpc.ConnectionMultiListener$ConnectionHandler.operationComplete(ConnectionMultiListener.java:89) [drill- rpc-1.12.0.jar:1.12.0] at org.apache.drill.exec.rpc.ConnectionMultiListener$ConnectionHandler.operationComplete(ConnectionMultiListener.java:76) [drill- rpc-1.12.0.jar:1.12.0] at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) [netty-common-4.0.48.Final.jar:4.0.48.Final] at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:481) [netty-common-4.0.48.Final.jar:4.0.48.Fina l] at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420) [netty-common-4.0.48.Final.jar:4.0.48.Final] at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:122) [netty-common-4.0.48.Final.jar:4.0.48.Final] at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.connect(AbstractNioChannel.java:241) [netty-transport-4.0.48.Final.j ar:4.0.48.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultChannelPipeline.java:1226) [netty-transport-4.0.48.Final.ja r:4.0.48
FW: Apache Drill- Error setting up remote intermediate fragment execution
Hi Team, I have successfully launched a Drill cluster on 3 docker containers with Zoo keeper Quorum and accessing data from a remote node on which Drill is not installed . I am able to list all the drill bits from each node and run queries on a single drill bit , But I am facing issue when I am trying to run a query which distributes the query to the other drill bits by creating the intermediate fragments. I have gone through the documentation but It couldn't help. I got the below error message in Drill shell and the detailed error log mentioned below. Please help me on resolving this. Error in Drill Shell: Error: CONNECTION ERROR: Error setting up remote intermediate fragment execution Nodes with failures drillslave-7dccb67657-9qv4l, drillslave-7dccb67657-rhfgt [Error Id: 5d701a0d-988b-4845-83f6-b029ad6825a8 on drillmaster-76d6885c59-wpksz:31010] (state=,code=0) Error Log: 2018-04-11 07:26:17,391 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,401 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus es() took 0 ms, numFiles: 1 2018-04-11 07:26:17,404 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Execute d 1 out of 1 using 1 threads. Time: 2ms total, 2.798853ms avg, 2ms max. 2018-04-11 07:26:17,404 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Execute d 1 out of 1 using 1 threads. Earliest start: 0.699000 ?s, Latest start: 0.699000 ?s, Average start: 0.699000 ?s . 2018-04-11 07:26:17,408 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pr uning, pruning class: org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule 2018-04-11 07:26:17,408 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO o.a.d.e.p.l.partition.PruneScanRule - No partition columns a re projected from the scan..continue. Total pruning elapsed time: 0 ms 2018-04-11 07:26:17,611 [BitServer-3] ERROR o.a.d.e.rpc.ConnectionMultiListener - Failed to establish connection java.util.concurrent.ExecutionException: java.nio.channels.UnresolvedAddressException at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:54) ~[netty-common-4.0.48.Final.jar:4.0.48.Final] at org.apache.drill.exec.rpc.ConnectionMultiListener$ConnectionHandler.operationComplete(ConnectionMultiListener.java:89) [drill- rpc-1.12.0.jar:1.12.0] at org.apache.drill.exec.rpc.ConnectionMultiListener$ConnectionHandler.operationComplete(ConnectionMultiListener.java:76) [drill- rpc-1.12.0.jar:1.12.0] at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) [netty-common-4.0.48.Final.jar:4.0.48.Final] at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:481) [netty-common-4.0.48.Final.jar:4.0.48.Fina l] at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420) [netty-common-4.0.48.Final.jar:4.0.48.Final] at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:122) [netty-common-4.0.48.Final.jar:4.0.48.Final] at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.connect(AbstractNioChannel.java:241) [netty-transport-4.0.48.Final.j ar:4.0.48.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultChannelPipeline.java:1226) [netty-transport-4.0.48.Final.ja r:4.0.48.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:539) [netty-transport-4.0.48.F inal.jar:4.0.48.Final] at io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:524) [netty-transport-4.0.48.Final.j ar:4.0.48.Final] at io.netty.channel.ChannelOutboundHandlerAdapter.connect(ChannelOutboundHandlerAdapter.java:47) [netty-transport-4.0.48.Final.ja
