[jira] [Commented] (HDFS-16022) matlab mapreduce v95 demos can't run hadoop-3.2.2 run time

2022-07-12 Thread Masatake Iwasaki (Jira) 


[ 
https://issues.apache.org/jira/browse/HDFS-16022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17565900#comment-17565900
 ] 

Masatake Iwasaki commented on HDFS-16022:
-

update the targets to 3.2.5 for preparing 3.2.4 release.

> matlab mapreduce v95 demos can't run hadoop-3.2.2 run time
> --
>
> Key: HDFS-16022
> URL: https://issues.apache.org/jira/browse/HDFS-16022
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: dfsclient
>Affects Versions: 3.2.2
> Environment: hadoop-3.2.2  + matlab run time+ centos7,  the 
> maxArrivalDelay.ctf file is generated in win10+matlab2018b(V95) by hadoop 
> compiler tools. the airlinesmall.csv upload the HDFS. hadoop can run well by 
> the hadoop-mapreduce-examples-3.2.2.jar wordcount demos, even, jar compiled 
> by the source code in win10+ eclipses env. please help, I have got no idea 
> about this 
>Reporter: cathonxiong
>Priority: Blocker
> Attachments: matlab_errorlog
>
>
>  hadoop \ hadoop \> jar 
> /usr/local/MATLAB/MATLAB_Runtime/v95/toolbox/mlhadoop/jar/a2.2.0/mwmapreduce.jar
>  \> com.mathworks.hadoop.MWMapReduceDriver \> -D 
> mw.mcrroot=/usr/local/MATLAB/MATLAB_Runtime/v95 \> 
> /usr/local/MATLAB/MATLAB_Runtime/v95/maxArrivalDelay.ctf \> 
> hdfs://hadoop.namenode:50070/user/matlab/datasets/airlinesmall.csv \> 
> hdfs://hadoop.namenode:50070/user/matlab/resultsjava.library.path: 
> /usr/local/hadoop-3.2.2/lib/nativeHDFSCTFPath=hdfs://hadoop.namenode:8020/user/root/maxArrivalDelay/maxArrivalDelay.ctfUploading
>  CTF into distributed cache completed.mapred.child.env: 
> MCR_CACHE_ROOT=/tmp,LD_LIBRARY_PATH=/usr/local/MATLAB/MATLAB_Runtime/v95/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/sys/opengl/lib/glnxa64mapred.child.java.opts:
>  
> -Djava.library.path=/usr/local/MATLAB/MATLAB_Runtime/v95/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/sys/opengl/lib/glnxa64New
>  java.library.path: 
> /usr/local/hadoop-3.2.2/lib/native:/usr/local/MATLAB/MATLAB_Runtime/v95/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/sys/opengl/lib/glnxa64Using
>  MATLAB mapper.Set input format class to: ChunkFileRecordReader.Using MATLAB 
> reducer.Set outputformat class to: class 
> org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormatSet map output 
> key class to: class com.mathworks.hadoop.MxArrayWritable2Set map output value 
> class to: class com.mathworks.hadoop.MxArrayWritable2Set reduce output key 
> class to: class com.mathworks.hadoop.MxArrayWritable2Set reduce output value 
> class to: class com.mathworks.hadoop.MxArrayWritable2*** run 
> **2021-05-11 14:58:47,043 INFO client.RMProxy: Connecting to 
> ResourceManager at hadoop.namenode/192.168.0.25:80322021-05-11 14:58:47,139 
> WARN net.NetUtils: Unable to wrap exception of type class 
> org.apache.hadoop.ipc.RpcException: it has no (String) 
> constructorjava.lang.NoSuchMethodException: 
> org.apache.hadoop.ipc.RpcException.(java.lang.String) at 
> java.lang.Class.getConstructor0(Class.java:3082) at 
> java.lang.Class.getConstructor(Class.java:1825) at 
> org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:835) at 
> org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:811) at 
> org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1566) at 
> org.apache.hadoop.ipc.Client.call(Client.java:1508) at 
> org.apache.hadoop.ipc.Client.call(Client.java:1405) at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233)
>  at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118)
>  at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source) at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:910)
>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  at java.lang.reflect.Method.invoke(Method.java:498) at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
>  at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
>  at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
>  at 
>

[jira] [Commented] (HDFS-16022) matlab mapreduce v95 demos can't run hadoop-3.2.2 run time

2021-05-11 Thread cathonxiong (Jira) 


[ 
https://issues.apache.org/jira/browse/HDFS-16022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342568#comment-17342568
 ] 

cathonxiong commented on HDFS-16022:


xiaoqiao，您好




我也是第一次在官网提交问题，已经设置过了这个最大值，
`ipc.maximum.response.length， 问题应该是来自这个容器出错了，
 org.apache.hadoop.ipc.RpcException.(java.lang.String) at 
java.lang.Class.getConstructor0(Class.java:3082) ，还没头绪，还请多指教一下，

熊俊
SWTU



















> matlab mapreduce v95 demos can't run hadoop-3.2.2 run time
> --
>
> Key: HDFS-16022
> URL: https://issues.apache.org/jira/browse/HDFS-16022
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: dfsclient
>Affects Versions: 3.2.2
> Environment: hadoop-3.2.2  + matlab run time+ centos7,  the 
> maxArrivalDelay.ctf file is generated in win10+matlab2018b(V95) by hadoop 
> compiler tools. the airlinesmall.csv upload the HDFS. hadoop can run well by 
> the hadoop-mapreduce-examples-3.2.2.jar wordcount demos, even, jar compiled 
> by the source code in win10+ eclipses env. please help, I have got no idea 
> about this 
>Reporter: cathonxiong
>Priority: Blocker
> Attachments: matlab_errorlog
>
>
>  hadoop \ hadoop \> jar 
> /usr/local/MATLAB/MATLAB_Runtime/v95/toolbox/mlhadoop/jar/a2.2.0/mwmapreduce.jar
>  \> com.mathworks.hadoop.MWMapReduceDriver \> -D 
> mw.mcrroot=/usr/local/MATLAB/MATLAB_Runtime/v95 \> 
> /usr/local/MATLAB/MATLAB_Runtime/v95/maxArrivalDelay.ctf \> 
> hdfs://hadoop.namenode:50070/user/matlab/datasets/airlinesmall.csv \> 
> hdfs://hadoop.namenode:50070/user/matlab/resultsjava.library.path: 
> /usr/local/hadoop-3.2.2/lib/nativeHDFSCTFPath=hdfs://hadoop.namenode:8020/user/root/maxArrivalDelay/maxArrivalDelay.ctfUploading
>  CTF into distributed cache completed.mapred.child.env: 
> MCR_CACHE_ROOT=/tmp,LD_LIBRARY_PATH=/usr/local/MATLAB/MATLAB_Runtime/v95/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/sys/opengl/lib/glnxa64mapred.child.java.opts:
>  
> -Djava.library.path=/usr/local/MATLAB/MATLAB_Runtime/v95/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/sys/opengl/lib/glnxa64New
>  java.library.path: 
> /usr/local/hadoop-3.2.2/lib/native:/usr/local/MATLAB/MATLAB_Runtime/v95/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/sys/opengl/lib/glnxa64Using
>  MATLAB mapper.Set input format class to: ChunkFileRecordReader.Using MATLAB 
> reducer.Set outputformat class to: class 
> org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormatSet map output 
> key class to: class com.mathworks.hadoop.MxArrayWritable2Set map output value 
> class to: class com.mathworks.hadoop.MxArrayWritable2Set reduce output key 
> class to: class com.mathworks.hadoop.MxArrayWritable2Set reduce output value 
> class to: class com.mathworks.hadoop.MxArrayWritable2*** run 
> **2021-05-11 14:58:47,043 INFO client.RMProxy: Connecting to 
> ResourceManager at hadoop.namenode/192.168.0.25:80322021-05-11 14:58:47,139 
> WARN net.NetUtils: Unable to wrap exception of type class 
> org.apache.hadoop.ipc.RpcException: it has no (String) 
> constructorjava.lang.NoSuchMethodException: 
> org.apache.hadoop.ipc.RpcException.(java.lang.String) at 
> java.lang.Class.getConstructor0(Class.java:3082) at 
> java.lang.Class.getConstructor(Class.java:1825) at 
> org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:835) at 
> org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:811) at 
> org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1566) at 
> org.apache.hadoop.ipc.Client.call(Client.java:1508) at 
> org.apache.hadoop.ipc.Client.call(Client.java:1405) at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233)
>  at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118)
>  at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source) at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:910)
>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  at java.lang.reflect.Method.invoke(Method.java:498) at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
>  at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
>  at 
>

[jira] [Commented] (HDFS-16022) matlab mapreduce v95 demos can't run hadoop-3.2.2 run time

2021-05-11 Thread Xiaoqiao He (Jira) 


[ 
https://issues.apache.org/jira/browse/HDFS-16022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342549#comment-17342549
 ] 

Xiaoqiao He commented on HDFS-16022:


Thanks [~cathonsh] for your report, It seems that RPC response exceeds maximum 
data length which can tune configure `ipc.maximum.response.length`.
BTW, I strongly recommend reporting deploy issue via user mail-list: 
u...@hadoop.apache.org
Thanks again.

> matlab mapreduce v95 demos can't run hadoop-3.2.2 run time
> --
>
> Key: HDFS-16022
> URL: https://issues.apache.org/jira/browse/HDFS-16022
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: dfsclient
>Affects Versions: 3.2.2
> Environment: hadoop-3.2.2  + matlab run time+ centos7,  the 
> maxArrivalDelay.ctf file is generated in win10+matlab2018b(V95) by hadoop 
> compiler tools. the airlinesmall.csv upload the HDFS. hadoop can run well by 
> the hadoop-mapreduce-examples-3.2.2.jar wordcount demos, even, jar compiled 
> by the source code in win10+ eclipses env. please help, I have got no idea 
> about this 
>Reporter: cathonxiong
>Priority: Blocker
> Attachments: matlab_errorlog
>
>
>  hadoop \ hadoop \> jar 
> /usr/local/MATLAB/MATLAB_Runtime/v95/toolbox/mlhadoop/jar/a2.2.0/mwmapreduce.jar
>  \> com.mathworks.hadoop.MWMapReduceDriver \> -D 
> mw.mcrroot=/usr/local/MATLAB/MATLAB_Runtime/v95 \> 
> /usr/local/MATLAB/MATLAB_Runtime/v95/maxArrivalDelay.ctf \> 
> hdfs://hadoop.namenode:50070/user/matlab/datasets/airlinesmall.csv \> 
> hdfs://hadoop.namenode:50070/user/matlab/resultsjava.library.path: 
> /usr/local/hadoop-3.2.2/lib/nativeHDFSCTFPath=hdfs://hadoop.namenode:8020/user/root/maxArrivalDelay/maxArrivalDelay.ctfUploading
>  CTF into distributed cache completed.mapred.child.env: 
> MCR_CACHE_ROOT=/tmp,LD_LIBRARY_PATH=/usr/local/MATLAB/MATLAB_Runtime/v95/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/sys/opengl/lib/glnxa64mapred.child.java.opts:
>  
> -Djava.library.path=/usr/local/MATLAB/MATLAB_Runtime/v95/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/sys/opengl/lib/glnxa64New
>  java.library.path: 
> /usr/local/hadoop-3.2.2/lib/native:/usr/local/MATLAB/MATLAB_Runtime/v95/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v95/sys/opengl/lib/glnxa64Using
>  MATLAB mapper.Set input format class to: ChunkFileRecordReader.Using MATLAB 
> reducer.Set outputformat class to: class 
> org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormatSet map output 
> key class to: class com.mathworks.hadoop.MxArrayWritable2Set map output value 
> class to: class com.mathworks.hadoop.MxArrayWritable2Set reduce output key 
> class to: class com.mathworks.hadoop.MxArrayWritable2Set reduce output value 
> class to: class com.mathworks.hadoop.MxArrayWritable2*** run 
> **2021-05-11 14:58:47,043 INFO client.RMProxy: Connecting to 
> ResourceManager at hadoop.namenode/192.168.0.25:80322021-05-11 14:58:47,139 
> WARN net.NetUtils: Unable to wrap exception of type class 
> org.apache.hadoop.ipc.RpcException: it has no (String) 
> constructorjava.lang.NoSuchMethodException: 
> org.apache.hadoop.ipc.RpcException.(java.lang.String) at 
> java.lang.Class.getConstructor0(Class.java:3082) at 
> java.lang.Class.getConstructor(Class.java:1825) at 
> org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:835) at 
> org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:811) at 
> org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1566) at 
> org.apache.hadoop.ipc.Client.call(Client.java:1508) at 
> org.apache.hadoop.ipc.Client.call(Client.java:1405) at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233)
>  at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118)
>  at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source) at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:910)
>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  at java.lang.reflect.Method.invoke(Method.java:498) at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
>  at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
>  at 
>