> On Jul 22, 2016, at 5:47 PM, Zheng, Kai <[email protected]> wrote:
>
> For the leveldb thing, wouldn't we have an alternative option in Java for the
> platforms where leveldb isn't supported yet due to whatever reasons. IMO,
> native library would be best to be used for optimization and production for
> performance. For development and pure Java platform, by default pure Java
> approach should still be provided and used. That is to say, if no Hadoop
> native is used, all the functionalities should still work and not break.
Yes and no. I can certainly understand some high-end features being
tied to native libraries, simply because system programming with Java is like
being a touch typist with your nose.
That said, absolutely key functionality should definitely work. Take a
look at the last Linux/ppc64le report that was emailed to these very lists a
few days ago [1]:
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/30/artifact/out/console-report.html
Almost all of those YARN failures are due to MiniYARN trying to
initiate leveldb as part of the service startup but can't because the embedded
shared library is the wrong hardware architecture. Rather than catch the
exception and do something else, the code just blows up in a very dramatic
fashion. That should translate into YARN is completely busted and unusable
without doing some very weird workarounds.
To get us back on topic: the class path isolation work absolutely
cannot make this situation worse. We either need to make sure end users can
replace/modify Hadoop's dependencies if they require native lirbaries or work
harder on making multiplatform stuff better supported. The nightly PowerPC
builds should help tremendously towards this goal. [2]
1 - While I greatly appreciate the OpenPOWER Foundation getting the ASF access
to these boxes -- Mesos and Hadoop are both actively using them -- It'd be
great if they were more reliable so we could get a report every day of the
week. :(
2 - At some point, I'll set up a manually triggered precommit job to test
patches. But until both boxes are online and available on a consistent basis,
it just isn't worth the effort.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
