Re: java.lang.NumberFormatException: null when running Hadoop Mapreduce Wordcount example

2023-10-18 Thread Lewis John McGibbney
Hi Ethan, Thanks for the advice! As I am in an experimental phase, I decided to try again in pseudo-distributed mode... I tried downgrading to Hadoop 3.2.1 (OpenJDK8) but apparently that Hadoop distribution doesn't run on Apple M1 chip! I therefore tried again on Hadoop 3.3.4 and was

No appenders could be found for logger (org.apache.celeborn.mapreduce.v2.app.MRAppMasterWithCeleborn)

2023-10-18 Thread lewis john mcgibbney
Hi user@, I am making progress in my experiments integrating Nutch 1.20-SNAPSHOT, Hadoop 3.3.4 and Celeborn 0.4.0-SNAPSHOT-incubating! In both the Hadoop work count example and with all of the Nutch MapReduce jobs I run, I see the following output present in the YARN container stderr log output

Re: [NEW FEATURE AVAILABLE] Celeborn support MapReduce engine.

2023-10-18 Thread Lewis John McGibbney
Excellent. Thanks for the heads up :) lewismc On 2023/10/18 03:44:54 Ethan Feng wrote: > Hi Lewis, > > Thanks for reaching out. > > I can confirm that future Celeborn releases will include the "mr" > client jars since Celeborn 0.4.0 and it will start the release process > in a short period. >

java.lang.NumberFormatException: null when running Hadoop Mapreduce Wordcount example

2023-10-17 Thread lewis john mcgibbney
Hi user@, I cloned Celeborn (0.4.0-Incubating) 69defcad7f9423c9c24d2d22ead856b4225671c6 today and built it with the -Pmr profile. openjdk version "11.0.20.1" 2023-08-24 OpenJDK Runtime Environment Homebrew (build 11.0.20.1+0) OpenJDK 64-Bit Server VM Homebrew (build 11.0.20.1+0, mixed mode)

Re: [NEW FEATURE AVAILABLE] Celeborn support MapReduce engine.

2023-10-17 Thread Lewis John McGibbney
Hi Ethan, I'm just picking up Celeborn now and plan on running some experiments with the Apache Nutch (https://nutch.apache.org) project. I downloaded Celeborn 0.3.1-incubating (2023-10-13) from the downloads page and noticed that no Celeborn client jars for MapReduce exist at