Hi Ethan,
Thanks for the advice!
As I am in an experimental phase, I decided to try again in pseudo-distributed
mode...
I tried downgrading to Hadoop 3.2.1 (OpenJDK8) but apparently that Hadoop
distribution doesn't run on Apple M1 chip!
I therefore tried again on Hadoop 3.3.4 and was
Hi user@,
I am making progress in my experiments integrating Nutch
1.20-SNAPSHOT, Hadoop 3.3.4 and Celeborn 0.4.0-SNAPSHOT-incubating!
In both the Hadoop work count example and with all of the Nutch
MapReduce jobs I run, I see the following output present in the YARN
container stderr log output
Excellent. Thanks for the heads up :)
lewismc
On 2023/10/18 03:44:54 Ethan Feng wrote:
> Hi Lewis,
>
> Thanks for reaching out.
>
> I can confirm that future Celeborn releases will include the "mr"
> client jars since Celeborn 0.4.0 and it will start the release process
> in a short period.
>
Hi user@,
I cloned Celeborn (0.4.0-Incubating)
69defcad7f9423c9c24d2d22ead856b4225671c6 today and built it with the
-Pmr profile.
openjdk version "11.0.20.1" 2023-08-24
OpenJDK Runtime Environment Homebrew (build 11.0.20.1+0)
OpenJDK 64-Bit Server VM Homebrew (build 11.0.20.1+0, mixed mode)
Hi Ethan,
I'm just picking up Celeborn now and plan on running some experiments with the
Apache Nutch (https://nutch.apache.org) project.
I downloaded Celeborn 0.3.1-incubating (2023-10-13) from the downloads page and
noticed that no Celeborn client jars for MapReduce exist at