Re: java.lang.NumberFormatException: null when running Hadoop Mapreduce Wordcount example
Hi Lewis, Thanks for updating me on your progress with the Hadoop MapReduce word count example. It's great to hear that you were able to successfully run it on Hadoop 3.3.4 in pseudo-distributed mode. I appreciate you taking the time to update the Celeborn documentation with your findings. Your contribution will definitely help others who may encounter similar issues. Let me know if you need any further assistance with your other issues. I'm happy to help. Best regards, Ethan Feng Lewis John McGibbney 于2023年10月19日周四 12:26写道： > > Hi Ethan, > > Thanks for the advice! > > As I am in an experimental phase, I decided to try again in > pseudo-distributed mode... > I tried downgrading to Hadoop 3.2.1 (OpenJDK8) but apparently that Hadoop > distribution doesn't run on Apple M1 chip! > > I therefore tried again on Hadoop 3.3.4 and was successfully able to get the > Hadoop MapReduce word count example running. I needed to configure and start > YARN (which I did not do previously). > I will create a pull request such that this is reflected in the Celeborn > documentation. > > I am running into other issues which I will create a new thread for. > > Thank you > lewismc > > On 2023/10/18 05:05:12 Ethan Feng wrote: > > Hi Lewis, > > > > Sorry to hear that you're having trouble running the wordcount example > > with Celeborn. > > > > Based on the information you shared, I would suggest you run MapReduce > > with Celeborn on a Hadoop cluster instead of pseudo-disturb mode. > > Celeborn client in MapReduce needs to write a config file into the > > Hadoop file system. > > > > If that doesn't resolve the issue, please let me know and I'll be > > happy to help you troubleshoot further. > > > > Celeborn has a Slack workspace if you are convenient to join. ( > > https://join.slack.com/t/apachecelebor-kw08030/shared_invite/zt-1ju3hd5j8-4Z5keMdzpcVMspe4UJzF4Q > > ) > > > > Best regards, > > Ethan Feng > >
Re: java.lang.NumberFormatException: null when running Hadoop Mapreduce Wordcount example
Hi Ethan, Thanks for the advice! As I am in an experimental phase, I decided to try again in pseudo-distributed mode... I tried downgrading to Hadoop 3.2.1 (OpenJDK8) but apparently that Hadoop distribution doesn't run on Apple M1 chip! I therefore tried again on Hadoop 3.3.4 and was successfully able to get the Hadoop MapReduce word count example running. I needed to configure and start YARN (which I did not do previously). I will create a pull request such that this is reflected in the Celeborn documentation. I am running into other issues which I will create a new thread for. Thank you lewismc On 2023/10/18 05:05:12 Ethan Feng wrote: > Hi Lewis, > > Sorry to hear that you're having trouble running the wordcount example > with Celeborn. > > Based on the information you shared, I would suggest you run MapReduce > with Celeborn on a Hadoop cluster instead of pseudo-disturb mode. > Celeborn client in MapReduce needs to write a config file into the > Hadoop file system. > > If that doesn't resolve the issue, please let me know and I'll be > happy to help you troubleshoot further. > > Celeborn has a Slack workspace if you are convenient to join. ( > https://join.slack.com/t/apachecelebor-kw08030/shared_invite/zt-1ju3hd5j8-4Z5keMdzpcVMspe4UJzF4Q > ) > > Best regards, > Ethan Feng >
Re: java.lang.NumberFormatException: null when running Hadoop Mapreduce Wordcount example
Hi Lewis, Sorry to hear that you're having trouble running the wordcount example with Celeborn. Based on the information you shared, I would suggest you run MapReduce with Celeborn on a Hadoop cluster instead of pseudo-disturb mode. Celeborn client in MapReduce needs to write a config file into the Hadoop file system. If that doesn't resolve the issue, please let me know and I'll be happy to help you troubleshoot further. Celeborn has a Slack workspace if you are convenient to join. ( https://join.slack.com/t/apachecelebor-kw08030/shared_invite/zt-1ju3hd5j8-4Z5keMdzpcVMspe4UJzF4Q ) Best regards, Ethan Feng lewis john mcgibbney 于2023年10月18日周三 12:19写道： > > Hi user@, > > I cloned Celeborn (0.4.0-Incubating) > 69defcad7f9423c9c24d2d22ead856b4225671c6 today and built it with the > -Pmr profile. > > openjdk version "11.0.20.1" 2023-08-24 > OpenJDK Runtime Environment Homebrew (build 11.0.20.1+0) > OpenJDK 64-Bit Server VM Homebrew (build 11.0.20.1+0, mixed mode) > > Apache Hadoop 3.3.4 running in pseudo-distrib mode. > > I made an attempt to start MapReduce with Celeborn as documented at > https://celeborn.apache.org/docs/latest/#start-mapreduce-with-celeborn > > Everything goes good until it fails when I attempt to run the > wordcount example. The exceptions and stack traces are available at > https://paste.apache.org/3vvy6. > > Is it likely that this has to do with the Hadoop version or is this a > known issue? > > Thanks in advance for any help. > lewismc > > -- > http://home.apache.org/~lewismc/ > http://people.apache.org/keys/committer/lewismc
