Forgot to update here.
Signatures, digests, etc check out fine.
Checked out tag and build/tested with "-Pspark3.1"
I keep getting the following error:
- metrics/prometheus *** FAILED ***
200 did not equal 404 (ApiBaseResourceSuite.scala:90)
- metrics/json *** FAILED ***
200 did not equal 404 (ApiBaseResourceSuite.scala:96)
In unit-tests.log, I have this [1]
After explicitly ignoring these two tests, I then ran into lint issues with
"web" submodule - and could not get around to debugging the issue (I
installed pnpm - which does not seem to be called out in readme).
Thanks,
Mridul
[1]
24/06/24 01:57:33,545 ERROR [ScalaTest-main-running-DiscoverySuite]
MetricsConfig: Error loading configuration file
file:/home/mridul/work/apache/vote/celeborn/service/target/celeborn-service_2.12-0.5.0-tests.jar!/metrics-api.properties
java.io.FileNotFoundException:
file:/home/mridul/work/apache/vote/celeborn/service/target/celeborn-service_2.12-0.5.0-tests.jar!/metrics-api.properties
(No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at java.io.FileInputStream.<init>(FileInputStream.java:93)
at
org.apache.celeborn.common.metrics.MetricsConfig.loadPropertiesFromFile(MetricsConfig.scala:95)
at
org.apache.celeborn.common.metrics.MetricsConfig.initialize(MetricsConfig.scala:50)
at
org.apache.celeborn.common.metrics.MetricsSystem.<init>(MetricsSystem.scala:53)
at
org.apache.celeborn.common.metrics.MetricsSystem$.createMetricsSystem(MetricsSystem.scala:197)
at
org.apache.celeborn.service.deploy.master.Master.<init>(Master.scala:66)
at
org.apache.celeborn.service.deploy.master.http.api.ApiMasterResourceSuite.beforeAll(ApiMasterResourceSuite.scala:54)
at
org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
at
org.apache.celeborn.server.common.http.ApiBaseResourceSuite.run(ApiBaseResourceSuite.scala:23)
at org.scalatest.Suite.callExecuteOnSuite$1(Suite.scala:1178)
at org.scalatest.Suite.$anonfun$runNestedSuites$1(Suite.scala:1225)
at
scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
at
scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
at
scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198)
at org.scalatest.Suite.runNestedSuites(Suite.scala:1223)
at org.scalatest.Suite.runNestedSuites$(Suite.scala:1156)
at
org.scalatest.tools.DiscoverySuite.runNestedSuites(DiscoverySuite.scala:30)
at org.scalatest.Suite.run(Suite.scala:1111)
at org.scalatest.Suite.run$(Suite.scala:1096)
at org.scalatest.tools.DiscoverySuite.run(DiscoverySuite.scala:30)
at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:47)
at
org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13(Runner.scala:1321)
at
org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13$adapted(Runner.scala:1315)
at scala.collection.immutable.List.foreach(List.scala:392)
at
org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1315)
at
org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24(Runner.scala:992)
at
org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24$adapted(Runner.scala:970)
at
org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1481)
at
org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:970)
at org.scalatest.tools.Runner$.main(Runner.scala:775)
at org.scalatest.tools.Runner.main(Runner.scala)
On Mon, Jun 24, 2024 at 2:22 AM Ethan Feng <ethanf...@apache.org> wrote:
> Thanks for your feedback, I will close this vote
> thread and announce the results soon since 72 hours have passed.
>
>
> Ethan Feng.
>
> kerwin zhang <kerwin.libra...@gmail.com> 于2024年6月24日周一 15:18写道：
> >
> > +1 (binding)
> >
> > I checked
> > - git commit hash is correct.
> > - links are valid.
> > - signatures are good.
> > ```
> > gpg --verify apache-celeborn-0.5.0-bin.tgz.asc
> apache-celeborn-0.5.0-bin.tgz
> > gpg --verify apache-celeborn-0.5.0-source.tgz.asc
> > apache-celeborn-0.5.0-source.tgz
> > ```
> > - checksums are good.
> > ```
> > shasum -a 512 apache-celeborn-0.5.0-bin.tgz
> > shasum -a 512 apache-celeborn-0.5.0-source.tgz
> > ```
> >
> > - build success from source code (macOS).
> >
> > ```
> > ./build/make-distribution.sh -Pspark-3.4
> > ```
> >
> > Thanks,
> > Kerwin Zhang
> >
> > Fu Chen <cfmcgr...@gmail.com> 于2024年6月24日周一 14:46写道：
> > >
> > > +1
> > >
> > > I checked
> > > - download links are valid.
> > > - git commit hash is correct.
> > > - no binary files in the source release.
> > > - build success from source code (JDK 17).
> > > ```
> > > ./build/make-distribution.sh --sbt-enabled -Pspark-3.5
> > > ```
> > >
> > > Cheng Pan <pan3...@gmail.com> 于2024年6月24日周一 14:09写道：
> > > >
> > > > +1 (binding)
> > > >
> > > > - All links are valid.
> > > > - LICENSE/NOTICE looks fine.
> > > > - CI result looks great.
> > > > - No known blockers.
> > > >
> > > > Thanks,
> > > > Cheng Pan
> > > >
> > > >
> > > > > On Jun 19, 2024, at 12:45, Ethan Feng <ethanf...@apache.org>
> wrote:
> > > > >
> > > > > Hello, Celeborn community,
> > > > >
> > > > > This is a call for a vote to release Apache Celeborn
> > > > > 0.5.0-rc3
> > > > >
> > > > > The git tag to be voted upon:
> > > > > https://github.com/apache/celeborn/releases/tag/v0.5.0-rc3
> > > > >
> > > > > Source and binary artifacts can be found at:
> > > > > https://dist.apache.org/repos/dist/dev/celeborn/v0.5.0-rc3
> > > > >
> > > > > The git commit hash:
> > > > > 048ef207359113247bff05dcc203c70021ccfa10
> > > > >
> > > > > The staging repo:
> > > > >
> https://repository.apache.org/content/repositories/orgapacheceleborn-1076/
> > > > >
> > > > > The fingerprint of the PGP key release artifacts is signed with:
> > > > > FCF20BB29C7BEFDF58F998F76392F71F37356FA0
> > > > >
> > > > > My public key to verify signatures can be found in:
> > > > > https://dist.apache.org/repos/dist/release/celeborn/KEYS
> > > > >
> > > > > The vote will be open for at least 72 hours or until the necessary
> > > > > number of votes are reached.
> > > > >
> > > > > Please vote accordingly:
> > > > >
> > > > > [ ] +1 approve
> > > > > [ ] +0 no opinion
> > > > > [ ] -1 disapprove (and the reason)
> > > > >
> > > > > Steps to validate the release:
> > > > > https://www.apache.org/info/verification.html
> > > > >
> > > > > * Download links, checksums, and PGP signatures are valid.
> > > > > * Source code distributions have correct names matching the
> current release.
> > > > > * LICENSE and NOTICE files are correct.
> > > > > * All files have license headers if necessary.
> > > > > * No unlicensed compiled archives bundled in the source archive.
> > > > > * The source tarball matches the git tag.
> > > > > * Build from source is successful.
> > > > >
> > > > > There are additional tests:
> > > > > * Performance test no regression
> > > > > 1 TB TPC-DS, 0.5.0 VS 0.4.1 : 2042(s) VS 2050(s)
> > > > > 1.1 TB pure shuffle, 0.5.0 VS 0.4.1 : 11.8min vs 11.8min
> > > > >
> > > > > * Result correctness test passed
> > > > > 1TB TPC-DS runs concurrently, the results are identical.
> > > > >
> > > > > * Usability test passed
> > > > > Rolling upgrade from version 0.4.1 to 0.5.0 succeed.
> > > > > The metrics system works as expected.
> > > > >
> > > > > * Stability test passed
> > > > > Random worker failures, Celeborn works as expected.
> > > > > Random master failures, Celeborn works as expected.
> > > > > Master meta corrupted, Celeborn works as expected.
> > > > >
> > > > > * Compatibility test passed
> > > > > The Celeborn server version of 0.5.0 works fine with the Celeborn
> client 0.4.1.
> > > > >
> > > > > * Grafana dashboard layout checked
> > > > >
> > > > >
> > > > > Regards,
> > > > > Ethan Feng
> > > >
>
