Thanks for all the push, Andrew!
Looking at the RC. Went through my usual check-list. Here's my summary. Will
cast my final vote after comparing and validating my findings with others.
Verification
- [Check] Successful recompilation from source tar-ball
- [Check] Signature verification
- [Check] Generating dist tarballs from source tar-ball
- [Check] Testing
-- Start NN, DN, RM, NM, JHS, Timeline Service
-- Ran dist-shell example, MR sleep, wordcount, randomwriter, sort, grep, pi
-- Tested CLIs to print nodes, apps etc and also navigated UIs
Issues found during testing
Major
- The previously supported way of being able to use different tar-balls for
different sub-modules is completely broken - common and HDFS tar.gz are
completely empty.
- Cannot enable new UI in YARN because it is under a non-default compilation
flag. It should be on by default.
- One decommissioned node in YARN ResourceManager UI always appears to start
with, even when there are no NodeManagers that are started yet: Info <IP>:-1,
DECOMMISSIONED, null rack. It shows up only in the UI though, not in the CLI
node -list
Minor
- resourcemanager-metrics.out is going into current directory instead of log
directory
- $HADOOP_YARN_HOME/sbin/yarn-daemon.sh start historyserver doesn't even work.
Not just deprecated in favor of timelineserver as was advertised.
- Spurious warnings on CLI
17/11/20 17:07:34 INFO conf.Configuration: resource-types.xml not
found
17/11/20 17:07:34 INFO resource.ResourceUtils: Unable to find
'resource-types.xml'.
Side notes
- When did we stop putting CHANGES files into the source artifacts?
- Even after "mvn install"ing once, shading is repeated again and again for
every new 'mvn install' even though there are no source changes - we should see
how this can be avoided.
- Compatibility notes
-- NM's env list is curtailed unlike in 2.x (For e.g, HADOOP_MAPRED_HOME is
not automatically inherited. Correct behavior)
-- Sleep is moved from hadoop-mapreduce-client-jobclient-3.0.0.jar into
hadoop-mapreduce-client-jobclient-3.0.0-tests.jar
Thanks
+Vinod
> On Nov 14, 2017, at 1:34 PM, Andrew Wang <[email protected]> wrote:
>
> Hi folks,
>
> Thanks as always to the many, many contributors who helped with this
> release. I've created RC0 for Apache Hadoop 3.0.0. The artifacts are
> available here:
>
> http://people.apache.org/~wang/3.0.0-RC0/
>
> This vote will run 5 days, ending on Nov 19th at 1:30pm Pacific.
>
> 3.0.0 GA contains 291 fixed JIRA issues since 3.0.0-beta1. Notable
> additions include the merge of YARN resource types, API-based configuration
> of the CapacityScheduler, and HDFS router-based federation.
>
> I've done my traditional testing with a pseudo cluster and a Pi job. My +1
> to start.
>
> Best,
> Andrew
