[
https://issues.apache.org/jira/browse/KYLIN-5976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yinghao Lin updated KYLIN-5976:
-------------------------------
Attachment: image-2024-10-08-14-59-22-285.png
> libch.so: cannot allocate memory in static TLS block
> ----------------------------------------------------
>
> Key: KYLIN-5976
> URL: https://issues.apache.org/jira/browse/KYLIN-5976
> Project: Kylin
> Issue Type: Bug
> Components: Release
> Affects Versions: 5.0.0
> Reporter: Alexander
> Priority: Major
> Attachments: check-env.out, image-2024-10-08-14-59-22-285.png
>
>
> Build Kylin with Gluten key - "-withGluten"
> Ensure that build sucessfully added gluten to package
> {code:java}
> // so
> ~/apache-kylin-5.0.0-r5-with-gluten-bin/server/libch.so
> // libs
> ~/apache-kylin-5.0.0-r5-with-gluten-bin/lib/gluten/
> celeborn-client-spark-3-shaded_2.12-0.5.1.jar
> delta-core_2.12-2.3.0.jar
> delta-storage-2.3.0.jar
> gluten-celeborn-clickhouse-1.2.0-SNAPSHOT-jar-with-dependencies.jar
> gluten.jar
> protobuf-java-3.23.4.jar
> //GLUTEN_VERSION
> 1.2.0-kylin-240730{code}
> Try to start Kylin instance without any changes in gluten properties. Use
> default properties.
> Got error in "Checking Spark Availability" step.
> {code:java}
> Checking Spark Availability
> ...................................................[FAIL]
> {code}
> Error description in check-env.out file:
> {code:java}
> [459715] 2024-10-03T20:53:13,051 INFO [main] gluten.GlutenDriverPlugin :
> Gluten build info:
> ==============================================================
> Backend: ClickHouse
> Backend Branch: rebase_ch/20240730
> Backend Revision: f69def8b6a8
> Backend Revision Time: UNKNOWN
> GCC Version: <unknown>
> Gluten Branch: HEAD
> Gluten Build Time: 2024-09-04T08:14:52Z
> Gluten Repo URL: https://github.com/Kyligence/gluten-ext.git
> Gluten Revision: 14934a23ad23034c72229b3ea2a974787f1c3608
> Gluten Revision Time: 2024-07-31 11:18:34 +0800
> Gluten Version: 1.2.0-kylin-240730
> Hadoop Version: 2.7.4
> Java Version: 1.8
> Scala Version: 2.12.15
> Spark Version: 3.3.0-kylin-4.6.27.0-SNAPSHOT
> ==============================================================
> [459715] 2024-10-03T20:53:13,338 INFO [main] vectorized.JniLibLoader : Read
> real path /home/kylin/apache-kylin-5.0.0-r5-with-gluten-bin/server/libch.so
> for libPath /home/kylin/apache-kylin-5.0.0-r5-with-gluten-bin/server/libch.so
> Exception in thread "main" java.lang.UnsatisfiedLinkError:
> /home/kylin/apache-kylin-5.0.0-r5-with-gluten-bin/server/libch.so:
> /home/kylin/apache-kylin-5.0.0-r5-with-gluten-bin/server/libch.so: cannot
> allocate memory in static TLS block
> at java.lang.ClassLoader$NativeLibrary.load(Native Method)
> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
> at java.lang.Runtime.load0(Runtime.java:809)
> at java.lang.System.load(System.java:1086)
> at
> org.apache.gluten.vectorized.JniLibLoader.loadFromPath0(JniLibLoader.java:103)
> at
> org.apache.gluten.vectorized.JniLibLoader.loadFromPath(JniLibLoader.java:117)
> at
> org.apache.gluten.backendsapi.clickhouse.CHListenerApi.initialize(CHListenerApi.scala:70)
> at
> org.apache.gluten.backendsapi.clickhouse.CHListenerApi.onDriverStart(CHListenerApi.scala:45)
> at org.apache.gluten.GlutenDriverPlugin.init(GlutenPlugin.scala:75)
> at
> org.apache.spark.internal.plugin.DriverPluginContainer.$anonfun$driverPlugins$1(PluginContainer.scala:53)
> at
> scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:293)
> at
> scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
> at
> scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
> at scala.collection.TraversableLike.flatMap(TraversableLike.scala:293)
> at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:290)
> at scala.collection.AbstractTraversable.flatMap(Traversable.scala:108)
> at
> org.apache.spark.internal.plugin.DriverPluginContainer.<init>(PluginContainer.scala:46)
> at
> org.apache.spark.internal.plugin.PluginContainer$.apply(PluginContainer.scala:210)
> at
> org.apache.spark.internal.plugin.PluginContainer$.apply(PluginContainer.scala:193)
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:560)
> at
> org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58)
> at
> org.apache.kylin.tool.setup.KapSparkTaskTestCLI.main(KapSparkTaskTestCLI.java:44)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> at
> org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:977)
> at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
> at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
> at
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1065)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1074)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) {code}
> Full log file in attachment.
> Trying to add properties:
> {code:java}
> kylin.engine.spark-conf.spark.executorEnv.LD_PRELOAD=/home/kylin/apache-kylin-5.0.0-r5-with-gluten-bin/spark/libch.so
> kylin.storage.columnar.spark-conf.spark.executorEnv.LD_PRELOAD=/home/kylin/apache-kylin-5.0.0-r5-with-gluten-bin/spark/libch.so
> {code}
> With no success. After startup process this keys and values removed from
> properties file, no way to change it.
>
> p.s. Start without gluten works fine:
> {code:java}
> $KYLIN_HOME/bin/kylin.sh -DwithoutGluten start {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
