[jira] [Commented] (SPARK-7756) Ensure Spark runs clean on IBM Java implementation
[ https://issues.apache.org/jira/browse/SPARK-7756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14602751#comment-14602751 ] Tim Ellison commented on SPARK-7756: Using a single JIRA to gather a disparate set of issues found using IBM's Java is not going to work. I suggest keeping my original SSL failure and a-roberts' RDDOperationScope fix here, but subsequent bugs should have their own JIRA and be tagged with ibmjdk so I can easily find them. Ensure Spark runs clean on IBM Java implementation -- Key: SPARK-7756 URL: https://issues.apache.org/jira/browse/SPARK-7756 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 1.4.0 Reporter: Tim Ellison Assignee: Tim Ellison Priority: Minor Fix For: 1.4.0 Spark should run successfully on the IBM Java implementation. This issue is to gather any minor issues seen running the tests and examples that are attributable to differences in Java vendor. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-7756) Ensure Spark runs clean on IBM Java implementation
Tim Ellison created SPARK-7756: -- Summary: Ensure Spark runs clean on IBM Java implementation Key: SPARK-7756 URL: https://issues.apache.org/jira/browse/SPARK-7756 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 1.5.0 Reporter: Tim Ellison Priority: Minor Spark should run successfully on the IBM Java implementation. This issue is to gather any minor issues seen running the tests and examples that are attributable to differences in Java vendor. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7063) Update lz4 for Java 7 to avoid: when lz4 compression is used, it causes core dump
[ https://issues.apache.org/jira/browse/SPARK-7063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14545368#comment-14545368 ] Tim Ellison commented on SPARK-7063: I can confirm that this failure is no longer seen using LZ4 1.3.0 with IBM Java 7+. Update lz4 for Java 7 to avoid: when lz4 compression is used, it causes core dump - Key: SPARK-7063 URL: https://issues.apache.org/jira/browse/SPARK-7063 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 1.3.1 Environment: IBM JDK Reporter: Jenny MA Priority: Minor this issue is initially noticed by using IBM JDK, below please find the stack track of this issue, caused by violating the rule in critical section. #0 0x00314340f3cb in raise () from /service/pmrs/45638/20/lib64/libpthread.so.0 #1 0x7f795b0323be in j9dump_create () from /service/pmrs/45638/20/opt/ibm/biginsights/jdk/jre/lib/amd64/compressedrefs/libj9prt27.so #2 0x7f795a88ba2a in doSystemDump () from /service/pmrs/45638/20/opt/ibm/biginsights/jdk/jre/lib/amd64/compressedrefs/libj9dmp27.so #3 0x7f795b0405d5 in j9sig_protect () from /service/pmrs/45638/20/opt/ibm/biginsights/jdk/jre/lib/amd64/compressedrefs/libj9prt27.so #4 0x7f795a88a1fd in runDumpFunction () from /service/pmrs/45638/20/opt/ibm/biginsights/jdk/jre/lib/amd64/compressedrefs/libj9dmp27.so #5 0x7f795a88dbab in runDumpAgent () from /service/pmrs/45638/20/opt/ibm/biginsights/jdk/jre/lib/amd64/compressedrefs/libj9dmp27.so #6 0x7f795a8a1c49 in triggerDumpAgents () from /service/pmrs/45638/20/opt/ibm/biginsights/jdk/jre/lib/amd64/compressedrefs/libj9dmp27.so #7 0x7f795a4518fe in doTracePoint () from /service/pmrs/45638/20/opt/ibm/biginsights/jdk/jre/lib/amd64/compressedrefs/libj9trc27.so #8 0x7f795a45210e in j9Trace () from /service/pmrs/45638/20/opt/ibm/biginsights/jdk/jre/lib/amd64/compressedrefs/libj9trc27.so #9 0x7f79590e46e1 in MM_StandardAccessBarrier::jniReleasePrimitiveArrayCritical(J9VMThread*, _jarray*, void*, int) () from /service/pmrs/45638/20/opt/ibm/biginsights/jdk/jre/lib/amd64/compressedrefs/libj9gc27.so #10 0x7f7938bc397c in Java_net_jpountz_lz4_LZ4JNI_LZ4_1compress_1limitedOutput () from /service/pmrs/45638/20/tmp/liblz4-java7155003924599399415.so #11 0x7f795b707149 in VMprJavaSendNative () from /service/pmrs/45638/20/opt/ibm/biginsights/jdk/jre/lib/amd64/compressedrefs/libj9vm27.so #12 0x in ?? () this is an issue introduced by a bug in net.jpountz.lz4.lz4-1.2.0.jar, and fixed in 1.3.0 version. Sun JDK /Open JDK doesn't complain this issue, but this issue will trigger assertion failure when IBM JDK is used. here is the link to the fix https://github.com/jpountz/lz4-java/commit/07229aa2f788229ab4f50379308297f428e3d2d2 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7635) SparkContextSchedulerCreationSuite tests may fail due to unrecognized UnsatisfiedLinkError message.
[ https://issues.apache.org/jira/browse/SPARK-7635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543669#comment-14543669 ] Tim Ellison commented on SPARK-7635: Yep, a happy coincidence! Full disclosure: Matt and I both work for IBM, but we have never spoken. SparkContextSchedulerCreationSuite tests may fail due to unrecognized UnsatisfiedLinkError message. --- Key: SPARK-7635 URL: https://issues.apache.org/jira/browse/SPARK-7635 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 1.3.1 Environment: IBM Java Reporter: Matthew Brandyberry Assignee: Tim Ellison Priority: Minor Labels: test Fix For: 1.5.0 Original Estimate: 1h Remaining Estimate: 1h When mesos is not available, these tests fail due to the difference in the UnsatisfiedLinkError message with IBM Java vs OpenJDK: - mesos fine-grained *** FAILED *** mesos (Not found in java.library.path) did not contain no mesos in (SparkContextSchedulerCreationSuite.scala:162) - mesos coarse-grained *** FAILED *** mesos (Not found in java.library.path) did not contain no mesos in (SparkContextSchedulerCreationSuite.scala:162) - mesos with zookeeper *** FAILED *** mesos (Not found in java.library.path) did not contain no mesos in (SparkContextSchedulerCreationSuite.scala:162) PR to be submitted shortly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
