Verified, RC1 to be good. +1 (binding)
-Subbu Subramaniam PMC, Apache Pinot ________________________________ From: Xiang Fu <xiangfu.1...@gmail.com> Sent: Sunday, September 4, 2022 3:54 AM To: dev@pinot.apache.org <dev@pinot.apache.org> Subject: Re: [VOTE] Apache Pinot 0.11.0 RC0 Hi, I did some research on this, and I can reproduce this on my Ubuntu 18.04.5 LTS. This is due to the h3-core library upgrade from 3.7.0->3.7.2 (https://github.com/apache/pinot/pull/8696<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpinot%2Fpull%2F8696&data=05%7C01%7Cssubramaniam%40linkedin.com%7C49a46726ab8b4a942ce308da8e63e7a2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637978856998221105%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6n9Xez4sQobthf448UwT1tUGTqNHI88onVxdW1dpXLQ%3D&reserved=0>) to support Apple Mac M1. However the h3 library has introduced higher glibc(2.29) requirement from 3.7.1 (See discussion here: https://github.com/uber/h3-java/issues/85<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fuber%2Fh3-java%2Fissues%2F85&data=05%7C01%7Cssubramaniam%40linkedin.com%7C49a46726ab8b4a942ce308da8e63e7a2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637978856998221105%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=hWUjI9P%2BhF%2FXmQ5bWuEb%2BnN2ztOsHfMXgWFm8%2BkGQjE%3D&reserved=0>). This means that there is no way to build a binary which can be running both on Ubuntu 18 & Mac M1, considering it’s nearly impossible to upgrade glibcc on ubuntu 18. Luckily Uber team has recently released h3 4.0.0 which has lower the glibc requirements. I open a new PR: https://github.com/apache/pinot/pull/9335<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpinot%2Fpull%2F9335&data=05%7C01%7Cssubramaniam%40linkedin.com%7C49a46726ab8b4a942ce308da8e63e7a2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637978856998221105%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=y1mH4oyXSUKa2V%2ByQj2dt9JR2oUcDjRgP3gHhX1%2F8BY%3D&reserved=0> to upgrade the h3 lib and will back port to redo RC1 to address this issue. Best, Xiang On Sep 3, 2022, at 11:50 AM, Xiang Fu <xiangfu.1...@gmail.com<mailto:xiangfu.1...@gmail.com>> wrote: Can you try to upgrade the glibc? If it works, I can put a notice for that. On Sat, Sep 3, 2022 at 10:22 Subbu Subramaniam <ssubraman...@linkedin.com.invalid<mailto:ssubraman...@linkedin.com.invalid>> wrote: I am seeing the following exception when the source is compiled with Java 11 on Centos 7.9.2009. Not sure if I need a glibc upgrade? -Subbu ***** Bootstrap tables ***** java.lang.RuntimeException: Caught exception during running - org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.kickoffIngestionJob(IngestionJobLauncher.java:152) at org.apache.pinot.spi.ingestion.batch.IngestionJobLauncher.runIngestionJob(IngestionJobLauncher.java:121) at org.apache.pinot.tools.BootstrapTableTool.bootstrapOfflineTable(BootstrapTableTool.java:188) at org.apache.pinot.tools.BootstrapTableTool.execute(BootstrapTableTool.java:93) at org.apache.pinot.tools.admin.command.QuickstartRunner.bootstrapTable(QuickstartRunner.java:233) at org.apache.pinot.tools.Quickstart.execute(Quickstart.java:85) at org.apache.pinot.tools.admin.command.QuickStartCommand.execute(QuickStartCommand.java:178) at org.apache.pinot.tools.Command.call(Command.java:33) at org.apache.pinot.tools.Command.call(Command.java:29) at picocli.CommandLine.executeUserObject(CommandLine.java:1953) at picocli.CommandLine.access$1300(CommandLine.java:145) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352) at picocli.CommandLine$RunLast.handle(CommandLine.java:2346) at picocli.CommandLine$RunLast.handle(CommandLine.java:2311) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) at picocli.CommandLine.execute(CommandLine.java:2078) at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:165) at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:196) at org.apache.pinot.tools.Quickstart.main(Quickstart.java:146) Caused by: java.lang.RuntimeException: Failed to generate Pinot segment for file - file:/home/XXXXXX/tmp/apache-release/rc-test/apache-pinot-0.11.0-src/pinot-distribution/target/apache-pinot-0.11.0-bin/apache-pinot-0.11.0-bin/examples/batch/starbucksStores/rawdata/data.csv at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.lambda$submitSegmentGenTask$1(SegmentGenerationJobRunner.java:309) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.UnsatisfiedLinkError: /tmp/libh3-java10943907092098384359.so: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by /tmp/libh3-java10943907092098384359.so) at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method) at java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2442) at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2498) at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2694) at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2627) at java.base/java.lang.Runtime.load0(Runtime.java:768) at java.base/java.lang.System.load(System.java:1837) at com.uber.h3core.H3CoreLoader.loadNatives(H3CoreLoader.java:129) at com.uber.h3core.H3CoreLoader.loadNatives(H3CoreLoader.java:96) at com.uber.h3core.H3Core.newInstance(H3Core.java:79) at org.apache.pinot.segment.local.utils.H3Utils.<clinit>(H3Utils.java:51) at org.apache.pinot.segment.local.segment.creator.impl.inv.geospatial.BaseH3IndexCreator.add(BaseH3IndexCreator.java:111) at org.apache.pinot.segment.local.segment.creator.impl.inv.geospatial.OffHeapH3IndexCreator.add(OffHeapH3IndexCreator.java:73) at org.apache.pinot.segment.local.segment.creator.impl.SegmentColumnarIndexCreator.indexRow(SegmentColumnarIndexCreator.java:495) at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.build(SegmentIndexCreationDriverImpl.java:235) at org.apache.pinot.plugin.ingestion.batch.common.SegmentGenerationTaskRunner.run(SegmentGenerationTaskRunner.java:119) at org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.lambda$submitSegmentGenTask$1(SegmentGenerationJobRunner.java:287) ... 5 more ________________________________ From: Xiang Fu <xiangfu.1...@gmail.com<mailto:xiangfu.1...@gmail.com>> Sent: Thursday, September 1, 2022 4:25 PM To: dev@pinot.apache.org<mailto:dev@pinot.apache.org> <dev@pinot.apache.org<mailto:dev@pinot.apache.org>> Subject: [VOTE] Apache Pinot 0.11.0 RC0 Hi Pinot Community, This is a call for a vote to release Apache Pinot 0.11.0. The release candidate: https://dist.apache.org/repos/dist/dev/pinot/apache-pinot-0.11.0-rc0/<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fpinot%2Fapache-pinot-0.11.0-rc0%2F&data=05%7C01%7Cssubramaniam%40linkedin.com%7C49a46726ab8b4a942ce308da8e63e7a2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637978856998221105%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=uM7Ng1l4vp0L606RWf%2BOEujAv05hFtMK4BAN5MI3KC0%3D&reserved=0> Git tag for this release: https://github.com/apache/pinot/tree/release-0.11.0-rc0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpinot%2Ftree%2Frelease-0.11.0-rc0&data=05%7C01%7Cssubramaniam%40linkedin.com%7C49a46726ab8b4a942ce308da8e63e7a2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637978856998221105%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2wnQtGg1GjeU0tAI%2FzEIZTIZbYoYjx0t20b1pYU6i9Y%3D&reserved=0> Git hash for this release: 3d66f604e72eebbc940aeae89dfe5f9d9bbb6fc0 The artifact has been signed with key: 9079294B, which can be found in the following KEYS file: https://dist.apache.org/repos/dist/release/pinot/KEYS<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Frelease%2Fpinot%2FKEYS&data=05%7C01%7Cssubramaniam%40linkedin.com%7C49a46726ab8b4a942ce308da8e63e7a2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637978856998221105%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4V90XLACRsLFGWGAOhgnQvdM3A1yJaO9wufZeUWB6H4%3D&reserved=0> Release notes: https://github.com/apache/pinot/releases/tag/release-0.11.0-rc0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpinot%2Freleases%2Ftag%2Frelease-0.11.0-rc0&data=05%7C01%7Cssubramaniam%40linkedin.com%7C49a46726ab8b4a942ce308da8e63e7a2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637978856998221105%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Y3D5VBttHwwrEKm5h2nVZPmffBUUB9n0c1Yd0dJs1qc%3D&reserved=0> Staging repository: https://repository.apache.org/content/repositories/orgapachepinot-1039<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachepinot-1039&data=05%7C01%7Cssubramaniam%40linkedin.com%7C49a46726ab8b4a942ce308da8e63e7a2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637978856998377824%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=hRsw6%2FkwN0rNWnIpBCoiIDa8bGdVNqBbhHgB4IpiyQE%3D&reserved=0> Documentation on verifying a release candidate: https://cwiki.apache.org/confluence/display/PINOT/Validating+a+release+candidate<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FPINOT%2FValidating%2Ba%2Brelease%2Bcandidate&data=05%7C01%7Cssubramaniam%40linkedin.com%7C49a46726ab8b4a942ce308da8e63e7a2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637978856998377824%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=VTvilEGm3wFZOuQG2PNRUgrbU5KmDOe2rfeOF%2FkXAyg%3D&reserved=0> The vote will be open for at least 72 hours or until a necessary number of votes is reached. Please vote accordingly, [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove with the reason Thanks, Apache Pinot team
