Re: java.io.IOException: Function not implemented
Edson Ramiro wrote: May be it's a bug. I'm not the admin. : ( so, I'll talk to him and may be he install a 2.6.32.9 in another node to test : ) Thanks Edson Ramiro On 30 March 2010 20:00, Todd Lipcon t...@cloudera.com wrote: Hi Edson, I noticed that only the h01 nodes are running 2.6.32.9, the other broken DNs are 2.6.32.10. Is there some reason you are running a kernel that is literally 2 weeks old? I wouldn't be at all surprised if there were a bug here, or some issue with your Debian unstable distribution... If you are running the SCM trunk of the OS, you are part of the dev team. They will be grateful for the bugs you find and fix, but you get to find and fix them. In Ant one bugrep was touch stopped setting dates in the past, turned out that on the debian nightly builds, you couldn't touch any file into the past... -steve
Re: java.io.IOException: Function not implemented
Edson Ramiro wrote: I'm not involved with Debian community :( I think you are now...
Re: java.io.IOException: Function not implemented
Hi all, Thanks for help Todd and Steve, I configured Hadoop (0.20.2) again and I'm getting the same error (Function not implemented). Do you think it's a Hadoop bug? This is the situation: I've 28 nodes where just four are running the datanode. In all other nodes the tasktracker in running ok. The NN and JT are running ok. The configuration of the machines is the same, its a nfs shared home. In all machines the Java version is 1.6.0_17. This is the kernel version of the nodes, note that are two versions and in both the datanode doesn't work. Just in the h0* machines. ram...@lcpad:~/hadoop-0.20.2$ ./bin/slaves.sh uname -a | sort a01: Linux a01 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a02: Linux a02 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a03: Linux a03 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a04: Linux a04 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a05: Linux a05 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a06: Linux a06 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a07: Linux a07 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a09: Linux a09 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a10: Linux a10 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux ag06: Linux ag06 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux ag07: Linux ag07 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux bl02: Linux bl02 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux bl03: Linux bl03 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux bl04: Linux bl04 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux bl06: Linux bl06 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux bl07: Linux bl07 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux ct02: Linux ct02 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux ct03: Linux ct03 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux ct04: Linux ct04 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux ct06: Linux ct06 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux h01: Linux h01 2.6.32.9 #2 SMP Sat Mar 6 19:09:13 BRT 2010 x86_64 GNU/Linux h02: Linux h02 2.6.32.9 #2 SMP Sat Mar 6 19:09:13 BRT 2010 x86_64 GNU/Linux h03: Linux h03 2.6.32.9 #2 SMP Sat Mar 6 19:09:13 BRT 2010 x86_64 GNU/Linux h04: Linux h04 2.6.32.9 #2 SMP Sat Mar 6 19:09:13 BRT 2010 x86_64 GNU/Linux sd02: Linux sd02 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux sd05: Linux sd05 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux sd06: Linux sd06 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux sd07: Linux sd07 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux These are the java processes running on each clients. Jjust the h0* machines are running ok. ram...@lcpad:~/hadoop-0.20.2$ ./bin/slaves.sh pgrep -lc java | sort a01: 1 a02: 1 a03: 1 a04: 1 a05: 1 a06: 1 a07: 1 a09: 1 a10: 1 ag06: 1 ag07: 1 bl02: 1 bl03: 1 bl04: 1 bl06: 1 bl07: 1 ct02: 1 ct03: 1 ct04: 1 ct06: 1 h01: 2 h02: 2 h03: 2 h04: 2 sd02: 1 sd05: 1 sd06: 1 sd07: 1 This is my configuration: ram...@lcpad:~/hadoop-0.20.2$ cat conf/*site* ?xml version=1.0? ?xml-stylesheet type=text/xsl href=configuration.xsl? !-- Put site-specific property overrides in this file. -- configuration property namefs.default.name/name valuehdfs://lcpad:9000/value /property /configuration ?xml version=1.0? ?xml-stylesheet type=text/xsl href=configuration.xsl? !-- Put site-specific property overrides in this file. -- configuration property namedfs.replication/name value1/value /property /configuration ?xml version=1.0? ?xml-stylesheet type=text/xsl href=configuration.xsl? !-- Put site-specific property overrides in this file. -- configuration property namemapred.job.tracker/name valuelcpad:9001/value /property /configuration Thanks in Advance, Edson Ramiro On 30 March 2010 05:58, Steve Loughran ste...@apache.org wrote: Edson Ramiro wrote: I'm not involved with Debian community :( I think you are now...
Re: java.io.IOException: Function not implemented
Hi Edson, I noticed that only the h01 nodes are running 2.6.32.9, the other broken DNs are 2.6.32.10. Is there some reason you are running a kernel that is literally 2 weeks old? I wouldn't be at all surprised if there were a bug here, or some issue with your Debian unstable distribution... -Todd On Tue, Mar 30, 2010 at 3:54 PM, Edson Ramiro erlfi...@gmail.com wrote: Hi all, Thanks for help Todd and Steve, I configured Hadoop (0.20.2) again and I'm getting the same error (Function not implemented). Do you think it's a Hadoop bug? This is the situation: I've 28 nodes where just four are running the datanode. In all other nodes the tasktracker in running ok. The NN and JT are running ok. The configuration of the machines is the same, its a nfs shared home. In all machines the Java version is 1.6.0_17. This is the kernel version of the nodes, note that are two versions and in both the datanode doesn't work. Just in the h0* machines. ram...@lcpad:~/hadoop-0.20.2$ ./bin/slaves.sh uname -a | sort a01: Linux a01 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a02: Linux a02 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a03: Linux a03 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a04: Linux a04 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a05: Linux a05 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a06: Linux a06 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a07: Linux a07 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a09: Linux a09 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a10: Linux a10 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux ag06: Linux ag06 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux ag07: Linux ag07 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux bl02: Linux bl02 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux bl03: Linux bl03 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux bl04: Linux bl04 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux bl06: Linux bl06 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux bl07: Linux bl07 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux ct02: Linux ct02 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux ct03: Linux ct03 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux ct04: Linux ct04 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux ct06: Linux ct06 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux h01: Linux h01 2.6.32.9 #2 SMP Sat Mar 6 19:09:13 BRT 2010 x86_64 GNU/Linux h02: Linux h02 2.6.32.9 #2 SMP Sat Mar 6 19:09:13 BRT 2010 x86_64 GNU/Linux h03: Linux h03 2.6.32.9 #2 SMP Sat Mar 6 19:09:13 BRT 2010 x86_64 GNU/Linux h04: Linux h04 2.6.32.9 #2 SMP Sat Mar 6 19:09:13 BRT 2010 x86_64 GNU/Linux sd02: Linux sd02 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux sd05: Linux sd05 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux sd06: Linux sd06 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux sd07: Linux sd07 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux These are the java processes running on each clients. Jjust the h0* machines are running ok. ram...@lcpad:~/hadoop-0.20.2$ ./bin/slaves.sh pgrep -lc java | sort a01: 1 a02: 1 a03: 1 a04: 1 a05: 1 a06: 1 a07: 1 a09: 1 a10: 1 ag06: 1 ag07: 1 bl02: 1 bl03: 1 bl04: 1 bl06: 1 bl07: 1 ct02: 1 ct03: 1 ct04: 1 ct06: 1 h01: 2 h02: 2 h03: 2 h04: 2 sd02: 1 sd05: 1 sd06: 1 sd07: 1 This is my configuration: ram...@lcpad:~/hadoop-0.20.2$ cat conf/*site* ?xml version=1.0? ?xml-stylesheet type=text/xsl href=configuration.xsl? !-- Put site-specific property overrides in this file. -- configuration property namefs.default.name/name valuehdfs://lcpad:9000/value /property /configuration ?xml version=1.0? ?xml-stylesheet type=text/xsl href=configuration.xsl? !-- Put site-specific property overrides in this file. -- configuration property namedfs.replication/name value1/value /property /configuration ?xml version=1.0? ?xml-stylesheet type=text/xsl href=configuration.xsl? !-- Put site-specific property overrides in this file. -- configuration property namemapred.job.tracker/name valuelcpad:9001/value /property /configuration Thanks in Advance, Edson Ramiro On 30 March 2010 05:58, Steve Loughran ste...@apache.org wrote: Edson Ramiro wrote: I'm not involved with Debian community :( I think you are now... -- Todd Lipcon Software Engineer, Cloudera
Re: java.io.IOException: Function not implemented
May be it's a bug. I'm not the admin. : ( so, I'll talk to him and may be he install a 2.6.32.9 in another node to test : ) Thanks Edson Ramiro On 30 March 2010 20:00, Todd Lipcon t...@cloudera.com wrote: Hi Edson, I noticed that only the h01 nodes are running 2.6.32.9, the other broken DNs are 2.6.32.10. Is there some reason you are running a kernel that is literally 2 weeks old? I wouldn't be at all surprised if there were a bug here, or some issue with your Debian unstable distribution... -Todd On Tue, Mar 30, 2010 at 3:54 PM, Edson Ramiro erlfi...@gmail.com wrote: Hi all, Thanks for help Todd and Steve, I configured Hadoop (0.20.2) again and I'm getting the same error (Function not implemented). Do you think it's a Hadoop bug? This is the situation: I've 28 nodes where just four are running the datanode. In all other nodes the tasktracker in running ok. The NN and JT are running ok. The configuration of the machines is the same, its a nfs shared home. In all machines the Java version is 1.6.0_17. This is the kernel version of the nodes, note that are two versions and in both the datanode doesn't work. Just in the h0* machines. ram...@lcpad:~/hadoop-0.20.2$ ./bin/slaves.sh uname -a | sort a01: Linux a01 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a02: Linux a02 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a03: Linux a03 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a04: Linux a04 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a05: Linux a05 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a06: Linux a06 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a07: Linux a07 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a09: Linux a09 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux a10: Linux a10 2.6.27.11 #4 Fri Jan 16 22:32:46 BRST 2009 x86_64 GNU/Linux ag06: Linux ag06 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux ag07: Linux ag07 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux bl02: Linux bl02 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux bl03: Linux bl03 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux bl04: Linux bl04 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux bl06: Linux bl06 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux bl07: Linux bl07 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux ct02: Linux ct02 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux ct03: Linux ct03 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux ct04: Linux ct04 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux ct06: Linux ct06 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux h01: Linux h01 2.6.32.9 #2 SMP Sat Mar 6 19:09:13 BRT 2010 x86_64 GNU/Linux h02: Linux h02 2.6.32.9 #2 SMP Sat Mar 6 19:09:13 BRT 2010 x86_64 GNU/Linux h03: Linux h03 2.6.32.9 #2 SMP Sat Mar 6 19:09:13 BRT 2010 x86_64 GNU/Linux h04: Linux h04 2.6.32.9 #2 SMP Sat Mar 6 19:09:13 BRT 2010 x86_64 GNU/Linux sd02: Linux sd02 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux sd05: Linux sd05 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux sd06: Linux sd06 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux sd07: Linux sd07 2.6.32.10 #1 SMP Tue Mar 16 10:17:30 BRT 2010 x86_64 GNU/Linux These are the java processes running on each clients. Jjust the h0* machines are running ok. ram...@lcpad:~/hadoop-0.20.2$ ./bin/slaves.sh pgrep -lc java | sort a01: 1 a02: 1 a03: 1 a04: 1 a05: 1 a06: 1 a07: 1 a09: 1 a10: 1 ag06: 1 ag07: 1 bl02: 1 bl03: 1 bl04: 1 bl06: 1 bl07: 1 ct02: 1 ct03: 1 ct04: 1 ct06: 1 h01: 2 h02: 2 h03: 2 h04: 2 sd02: 1 sd05: 1 sd06: 1 sd07: 1 This is my configuration: ram...@lcpad:~/hadoop-0.20.2$ cat conf/*site* ?xml version=1.0? ?xml-stylesheet type=text/xsl href=configuration.xsl? !-- Put site-specific property overrides in this file. -- configuration property namefs.default.name/name valuehdfs://lcpad:9000/value /property /configuration ?xml version=1.0? ?xml-stylesheet type=text/xsl href=configuration.xsl? !-- Put site-specific property overrides in this file. -- configuration property namedfs.replication/name value1/value /property /configuration ?xml version=1.0? ?xml-stylesheet type=text/xsl href=configuration.xsl? !-- Put site-specific property overrides in this file. -- configuration property namemapred.job.tracker/name valuelcpad:9001/value /property /configuration Thanks in Advance, Edson Ramiro On 30 March 2010 05:58, Steve Loughran ste...@apache.org wrote: Edson Ramiro wrote: I'm not involved
java.io.IOException: Function not implemented
Hi all, I'm trying to install Hadoop on a cluster, but I'm getting this error. I'm using java version 1.6.0_17 and hadoop-0.20.1+169.56.tar.gz from Cloudera. Its running in a NFS home shared between the nodes and masters. The NameNode works well, but all nodes try to connect and fail. Any Idea ? Thanks in Advance. == logs/hadoop-ramiro-datanode-a05.log == 2010-03-29 15:56:00,168 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 0 time(s). 2010-03-29 15:56:01,172 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 1 time(s). 2010-03-29 15:56:02,176 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 2 time(s). 2010-03-29 15:56:03,180 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 3 time(s). 2010-03-29 15:56:04,184 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 4 time(s). 2010-03-29 15:56:05,188 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 5 time(s). 2010-03-29 15:56:06,192 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 6 time(s). 2010-03-29 15:56:07,196 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 7 time(s). 2010-03-29 15:56:08,200 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 8 time(s). 2010-03-29 15:56:09,204 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 9 time(s). 2010-03-29 15:56:09,204 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Call to lcpad/192.168.1.51:9000 failed on local exception: java.io.IOException: Function not implemented at org.apache.hadoop.ipc.Client.wrapException(Client.java:775) at org.apache.hadoop.ipc.Client.call(Client.java:743) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220) at $Proxy4.getProtocolVersion(Unknown Source) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:346) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:383) at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:314) at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:291) at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:278) at org.apache.hadoop.hdfs.server.datanode.DataNode.init(DataNode.java:225) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1309) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1264) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1272) at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1394) Caused by: java.io.IOException: Function not implemented at sun.nio.ch.EPollArrayWrapper.epollCreate(Native Method) at sun.nio.ch.EPollArrayWrapper.init(EPollArrayWrapper.java:68) at sun.nio.ch.EPollSelectorImpl.init(EPollSelectorImpl.java:52) at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:18) at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.get(SocketIOWithTimeout.java:407) at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:322) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:203) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:407) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:304) at org.apache.hadoop.ipc.Client$Connection.access$1700(Client.java:176) at org.apache.hadoop.ipc.Client.getConnection(Client.java:860) at org.apache.hadoop.ipc.Client.call(Client.java:720) ... 13 more Edson Ramiro
Re: java.io.IOException: Function not implemented
Hi Edson, What operating system are you on? What kernel version? Thanks -Todd On Mon, Mar 29, 2010 at 12:01 PM, Edson Ramiro erlfi...@gmail.com wrote: Hi all, I'm trying to install Hadoop on a cluster, but I'm getting this error. I'm using java version 1.6.0_17 and hadoop-0.20.1+169.56.tar.gz from Cloudera. Its running in a NFS home shared between the nodes and masters. The NameNode works well, but all nodes try to connect and fail. Any Idea ? Thanks in Advance. == logs/hadoop-ramiro-datanode-a05.log == 2010-03-29 15:56:00,168 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 0 time(s). 2010-03-29 15:56:01,172 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 1 time(s). 2010-03-29 15:56:02,176 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 2 time(s). 2010-03-29 15:56:03,180 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 3 time(s). 2010-03-29 15:56:04,184 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 4 time(s). 2010-03-29 15:56:05,188 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 5 time(s). 2010-03-29 15:56:06,192 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 6 time(s). 2010-03-29 15:56:07,196 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 7 time(s). 2010-03-29 15:56:08,200 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 8 time(s). 2010-03-29 15:56:09,204 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 9 time(s). 2010-03-29 15:56:09,204 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Call to lcpad/192.168.1.51:9000 failed on local exception: java.io.IOException: Function not implemented at org.apache.hadoop.ipc.Client.wrapException(Client.java:775) at org.apache.hadoop.ipc.Client.call(Client.java:743) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220) at $Proxy4.getProtocolVersion(Unknown Source) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:346) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:383) at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:314) at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:291) at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:278) at org.apache.hadoop.hdfs.server.datanode.DataNode.init(DataNode.java:225) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1309) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1264) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1272) at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1394) Caused by: java.io.IOException: Function not implemented at sun.nio.ch.EPollArrayWrapper.epollCreate(Native Method) at sun.nio.ch.EPollArrayWrapper.init(EPollArrayWrapper.java:68) at sun.nio.ch.EPollSelectorImpl.init(EPollSelectorImpl.java:52) at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:18) at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.get(SocketIOWithTimeout.java:407) at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:322) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:203) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:407) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:304) at org.apache.hadoop.ipc.Client$Connection.access$1700(Client.java:176) at org.apache.hadoop.ipc.Client.getConnection(Client.java:860) at org.apache.hadoop.ipc.Client.call(Client.java:720) ... 13 more Edson Ramiro -- Todd Lipcon Software Engineer, Cloudera
Re: java.io.IOException: Function not implemented
I'm using Linux h02 2.6.32.9 #2 SMP Sat Mar 6 19:09:13 BRT 2010 x86_64 GNU/Linux ram...@h02:~/hadoop$ cat /etc/debian_version squeeze/sid Thanks for reply Edson Ramiro On 29 March 2010 16:56, Todd Lipcon t...@cloudera.com wrote: Hi Edson, What operating system are you on? What kernel version? Thanks -Todd On Mon, Mar 29, 2010 at 12:01 PM, Edson Ramiro erlfi...@gmail.com wrote: Hi all, I'm trying to install Hadoop on a cluster, but I'm getting this error. I'm using java version 1.6.0_17 and hadoop-0.20.1+169.56.tar.gz from Cloudera. Its running in a NFS home shared between the nodes and masters. The NameNode works well, but all nodes try to connect and fail. Any Idea ? Thanks in Advance. == logs/hadoop-ramiro-datanode-a05.log == 2010-03-29 15:56:00,168 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 0 time(s). 2010-03-29 15:56:01,172 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 1 time(s). 2010-03-29 15:56:02,176 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 2 time(s). 2010-03-29 15:56:03,180 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 3 time(s). 2010-03-29 15:56:04,184 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 4 time(s). 2010-03-29 15:56:05,188 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 5 time(s). 2010-03-29 15:56:06,192 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 6 time(s). 2010-03-29 15:56:07,196 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 7 time(s). 2010-03-29 15:56:08,200 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 8 time(s). 2010-03-29 15:56:09,204 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 9 time(s). 2010-03-29 15:56:09,204 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Call to lcpad/192.168.1.51:9000 failed on local exception: java.io.IOException: Function not implemented at org.apache.hadoop.ipc.Client.wrapException(Client.java:775) at org.apache.hadoop.ipc.Client.call(Client.java:743) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220) at $Proxy4.getProtocolVersion(Unknown Source) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:346) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:383) at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:314) at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:291) at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:278) at org.apache.hadoop.hdfs.server.datanode.DataNode.init(DataNode.java:225) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1309) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1264) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1272) at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1394) Caused by: java.io.IOException: Function not implemented at sun.nio.ch.EPollArrayWrapper.epollCreate(Native Method) at sun.nio.ch.EPollArrayWrapper.init(EPollArrayWrapper.java:68) at sun.nio.ch.EPollSelectorImpl.init(EPollSelectorImpl.java:52) at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:18) at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.get(SocketIOWithTimeout.java:407) at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:322) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:203) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:407) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:304) at org.apache.hadoop.ipc.Client$Connection.access$1700(Client.java:176) at org.apache.hadoop.ipc.Client.getConnection(Client.java:860) at org.apache.hadoop.ipc.Client.call(Client.java:720) ... 13 more Edson Ramiro -- Todd Lipcon Software Engineer, Cloudera
Re: java.io.IOException: Function not implemented
I'm not involved with Debian community :( ram...@h02:~/hadoop$ cat /proc/sys/fs/epoll/max_user_watches 3373957 and the Java is not the OpenSDK. The version is: ram...@lcpad:/usr/lib/jvm/java-6-sun$ java -version java version 1.6.0_17 Java(TM) SE Runtime Environment (build 1.6.0_17-b04) Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01, mixed mode) Edson Ramiro On 29 March 2010 17:14, Todd Lipcon t...@cloudera.com wrote: Hi Edson, It looks like for some reason your kernel does not have epoll enabled. It's very strange, since your kernel is very recent (in fact, bleeding edge!) Can you check the contents of /proc/sys/fs/epoll/max_user_watches Are you involved with the Debian community? This sounds like a general Java bug. Can you also please verify that you're using the Sun JVM and not OpenJDK (the debian folks like OpenJDK but it has subtle issues with Hadoop) You'll have to add a non-free repository and install sun-java6-jdk -Todd On Mon, Mar 29, 2010 at 1:05 PM, Edson Ramiro erlfi...@gmail.com wrote: I'm using Linux h02 2.6.32.9 #2 SMP Sat Mar 6 19:09:13 BRT 2010 x86_64 GNU/Linux ram...@h02:~/hadoop$ cat /etc/debian_version squeeze/sid Thanks for reply Edson Ramiro On 29 March 2010 16:56, Todd Lipcon t...@cloudera.com wrote: Hi Edson, What operating system are you on? What kernel version? Thanks -Todd On Mon, Mar 29, 2010 at 12:01 PM, Edson Ramiro erlfi...@gmail.com wrote: Hi all, I'm trying to install Hadoop on a cluster, but I'm getting this error. I'm using java version 1.6.0_17 and hadoop-0.20.1+169.56.tar.gz from Cloudera. Its running in a NFS home shared between the nodes and masters. The NameNode works well, but all nodes try to connect and fail. Any Idea ? Thanks in Advance. == logs/hadoop-ramiro-datanode-a05.log == 2010-03-29 15:56:00,168 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 0 time(s). 2010-03-29 15:56:01,172 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 1 time(s). 2010-03-29 15:56:02,176 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 2 time(s). 2010-03-29 15:56:03,180 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 3 time(s). 2010-03-29 15:56:04,184 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 4 time(s). 2010-03-29 15:56:05,188 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 5 time(s). 2010-03-29 15:56:06,192 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 6 time(s). 2010-03-29 15:56:07,196 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 7 time(s). 2010-03-29 15:56:08,200 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 8 time(s). 2010-03-29 15:56:09,204 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 9 time(s). 2010-03-29 15:56:09,204 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Call to lcpad/192.168.1.51:9000 failed on local exception: java.io.IOException: Function not implemented at org.apache.hadoop.ipc.Client.wrapException(Client.java:775) at org.apache.hadoop.ipc.Client.call(Client.java:743) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220) at $Proxy4.getProtocolVersion(Unknown Source) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:346) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:383) at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:314) at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:291) at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:278) at org.apache.hadoop.hdfs.server.datanode.DataNode.init(DataNode.java:225) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1309) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1264) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1272) at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1394) Caused by: java.io.IOException: Function not implemented at sun.nio.ch.EPollArrayWrapper.epollCreate(Native Method
Re: java.io.IOException: Function not implemented
Hey Edson, Unfortunately I'm not sure what's going on here - for whatever reason, the kernel isn't allowing Java NIO to use epoll, and thus the IPC framework from Hadoop isn't working correctly. I don't think this is a hadoop specific bug. Does this issue occur on all of the nodes? -Todd On Mon, Mar 29, 2010 at 2:26 PM, Edson Ramiro erlfi...@gmail.com wrote: I'm not involved with Debian community :( ram...@h02:~/hadoop$ cat /proc/sys/fs/epoll/max_user_watches 3373957 and the Java is not the OpenSDK. The version is: ram...@lcpad:/usr/lib/jvm/java-6-sun$ java -version java version 1.6.0_17 Java(TM) SE Runtime Environment (build 1.6.0_17-b04) Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01, mixed mode) Edson Ramiro On 29 March 2010 17:14, Todd Lipcon t...@cloudera.com wrote: Hi Edson, It looks like for some reason your kernel does not have epoll enabled. It's very strange, since your kernel is very recent (in fact, bleeding edge!) Can you check the contents of /proc/sys/fs/epoll/max_user_watches Are you involved with the Debian community? This sounds like a general Java bug. Can you also please verify that you're using the Sun JVM and not OpenJDK (the debian folks like OpenJDK but it has subtle issues with Hadoop) You'll have to add a non-free repository and install sun-java6-jdk -Todd On Mon, Mar 29, 2010 at 1:05 PM, Edson Ramiro erlfi...@gmail.com wrote: I'm using Linux h02 2.6.32.9 #2 SMP Sat Mar 6 19:09:13 BRT 2010 x86_64 GNU/Linux ram...@h02:~/hadoop$ cat /etc/debian_version squeeze/sid Thanks for reply Edson Ramiro On 29 March 2010 16:56, Todd Lipcon t...@cloudera.com wrote: Hi Edson, What operating system are you on? What kernel version? Thanks -Todd On Mon, Mar 29, 2010 at 12:01 PM, Edson Ramiro erlfi...@gmail.com wrote: Hi all, I'm trying to install Hadoop on a cluster, but I'm getting this error. I'm using java version 1.6.0_17 and hadoop-0.20.1+169.56.tar.gz from Cloudera. Its running in a NFS home shared between the nodes and masters. The NameNode works well, but all nodes try to connect and fail. Any Idea ? Thanks in Advance. == logs/hadoop-ramiro-datanode-a05.log == 2010-03-29 15:56:00,168 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 0 time(s). 2010-03-29 15:56:01,172 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 1 time(s). 2010-03-29 15:56:02,176 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 2 time(s). 2010-03-29 15:56:03,180 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 3 time(s). 2010-03-29 15:56:04,184 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 4 time(s). 2010-03-29 15:56:05,188 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 5 time(s). 2010-03-29 15:56:06,192 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 6 time(s). 2010-03-29 15:56:07,196 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 7 time(s). 2010-03-29 15:56:08,200 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 8 time(s). 2010-03-29 15:56:09,204 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: lcpad/192.168.1.51:9000. Already tried 9 time(s). 2010-03-29 15:56:09,204 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Call to lcpad/192.168.1.51:9000 failed on local exception: java.io.IOException: Function not implemented at org.apache.hadoop.ipc.Client.wrapException(Client.java:775) at org.apache.hadoop.ipc.Client.call(Client.java:743) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220) at $Proxy4.getProtocolVersion(Unknown Source) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:346) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:383) at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:314) at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:291) at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:278) at org.apache.hadoop.hdfs.server.datanode.DataNode.init(DataNode.java:225) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance
