Thanks for the spot Sangjin. I think this bug introduced in create-release by HADOOP-14835. The multi-pass maven build generates these dummy client jars during the site build since skipShade is specified.
This might be enough to cancel the RC. Thoughts? Best, Andrew On Mon, Nov 20, 2017 at 7:51 PM, Sangjin Lee <sj...@apache.org> wrote: > I checked the client jars that are supposed to contain shaded > dependencies, and they don't look quite right: > > $ tar -tzvf hadoop-3.0.0.tar.gz | grep hadoop-client-api-3.0.0.jar > -rw-r--r-- 0 andrew andrew 44531 Nov 14 11:53 > hadoop-3.0.0/share/hadoop/client/hadoop-client-api-3.0.0.jar > $ tar -tzvf hadoop-3.0.0.tar.gz | grep hadoop-client-runtime-3.0.0.jar > -rw-r--r-- 0 andrew andrew 45533 Nov 14 11:53 > hadoop-3.0.0/share/hadoop/client/hadoop-client-runtime-3.0.0.jar > $ tar -tzvf hadoop-3.0.0.tar.gz | grep hadoop-client-minicluster-3.0.0.jar > -rw-r--r-- 0 andrew andrew 47015 Nov 14 11:53 > hadoop-3.0.0/share/hadoop/client/hadoop-client-minicluster-3.0.0.jar > > When I look at what's inside those jar, they only seem to include > pom-related files with no class files. Am I missing something? > > When I build from the source with -Pdist, I do get much bigger jars: > total 113760 > -rw-r--r-- 1 sangjinlee 120039211 17055399 Nov 20 17:17 > hadoop-client-api-3.0.0.jar > -rw-r--r-- 1 sangjinlee 120039211 20451447 Nov 20 17:19 > hadoop-client-minicluster-3.0.0.jar > -rw-r--r-- 1 sangjinlee 120039211 20730866 Nov 20 17:18 > hadoop-client-runtime-3.0.0.jar > > Sangjin > > On Mon, Nov 20, 2017 at 5:52 PM, Sangjin Lee <sj...@apache.org> wrote: > >> >> >> On Mon, Nov 20, 2017 at 5:26 PM, Vinod Kumar Vavilapalli < >> vino...@apache.org> wrote: >> >>> Thanks for all the push, Andrew! >>> >>> Looking at the RC. Went through my usual check-list. Here's my summary. >>> Will cast my final vote after comparing and validating my findings with >>> others. >>> >>> Verification >>> >>> - [Check] Successful recompilation from source tar-ball >>> - [Check] Signature verification >>> - [Check] Generating dist tarballs from source tar-ball >>> - [Check] Testing >>> -- Start NN, DN, RM, NM, JHS, Timeline Service >>> -- Ran dist-shell example, MR sleep, wordcount, randomwriter, sort, >>> grep, pi >>> -- Tested CLIs to print nodes, apps etc and also navigated UIs >>> >>> Issues found during testing >>> >>> Major >>> - The previously supported way of being able to use different tar-balls >>> for different sub-modules is completely broken - common and HDFS tar.gz are >>> completely empty. >>> - Cannot enable new UI in YARN because it is under a non-default >>> compilation flag. It should be on by default. >>> - One decommissioned node in YARN ResourceManager UI always appears to >>> start with, even when there are no NodeManagers that are started yet: Info >>> <IP>:-1, DECOMMISSIONED, null rack. It shows up only in the UI though, not >>> in the CLI node -list >>> >>> Minor >>> - resourcemanager-metrics.out is going into current directory instead >>> of log directory >>> - $HADOOP_YARN_HOME/sbin/yarn-daemon.sh start historyserver doesn't >>> even work. Not just deprecated in favor of timelineserver as was advertised. >>> - Spurious warnings on CLI >>> 17/11/20 17:07:34 INFO conf.Configuration: >>> resource-types.xml not found >>> 17/11/20 17:07:34 INFO resource.ResourceUtils: Unable to >>> find 'resource-types.xml'. >>> >>> Side notes >>> >>> - When did we stop putting CHANGES files into the source artifacts? >>> - Even after "mvn install"ing once, shading is repeated again and again >>> for every new 'mvn install' even though there are no source changes - we >>> should see how this can be avoided. >>> - Compatibility notes >>> -- NM's env list is curtailed unlike in 2.x (For e.g, >>> HADOOP_MAPRED_HOME is not automatically inherited. Correct behavior) >>> -- Sleep is moved from hadoop-mapreduce-client-jobclient-3.0.0.jar >>> into hadoop-mapreduce-client-jobclient-3.0.0-tests.jar >>> >> >> Sleep has always been in the jobclient test jar as long as I can >> remember, so it's not new for 3.0. >> >> >>> >>> Thanks >>> +Vinod >>> >>> > On Nov 14, 2017, at 1:34 PM, Andrew Wang <andrew.w...@cloudera.com> >>> wrote: >>> > >>> > Hi folks, >>> > >>> > Thanks as always to the many, many contributors who helped with this >>> > release. I've created RC0 for Apache Hadoop 3.0.0. The artifacts are >>> > available here: >>> > >>> > http://people.apache.org/~wang/3.0.0-RC0/ >>> > >>> > This vote will run 5 days, ending on Nov 19th at 1:30pm Pacific. >>> > >>> > 3.0.0 GA contains 291 fixed JIRA issues since 3.0.0-beta1. Notable >>> > additions include the merge of YARN resource types, API-based >>> configuration >>> > of the CapacityScheduler, and HDFS router-based federation. >>> > >>> > I've done my traditional testing with a pseudo cluster and a Pi job. >>> My +1 >>> > to start. >>> > >>> > Best, >>> > Andrew >>> >>> >> >
