[GitHub] spark issue #14857: [SPARK-17261][PYSPARK] Using HiveContext after re-creati...
Github user coderfi commented on the issue: https://github.com/apache/spark/pull/14857 Awesome, we ran into this problem as well, and finally had some bandwidth to track down the cause well enough to be able to search for this pull request. Looking forward to test this in our fork. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10542] [PYSPARK] fix serialize namedtup...
Github user coderfi commented on the pull request: https://github.com/apache/spark/pull/8707#issuecomment-140974800 FYI, this works for us @ [NinthDecimal](http://www.ninthdecimal.com). Thanks for the fix, it was a stumper! Python 2.7.5 Spark-1.5 Branch @ [4c4a9ba28d9052fad45caca9a1eba9ef9db309d5](https://github.com/apache/spark/tree/4c4a9ba28d9052fad45caca9a1eba9ef9db309d5) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6908] [SQL] Use isolated Hive client
Github user coderfi commented on the pull request: https://github.com/apache/spark/pull/5876#issuecomment-104503508 Has this been tested with mapr4? When running under python, I am running into an exception Caused by: java.lang.UnsatisfiedLinkError: Native Library /tmp/mapr-root-libMapRClient.4.0.2-mapr.so already loaded in another classloader at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1931) at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890) at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1851) at java.lang.Runtime.load0(Runtime.java:795) at java.lang.System.load(System.java:1062) at com.mapr.fs.shim.LibraryLoader.load(LibraryLoader.java:29) This occurs under the following scenarios: create a spark context + hive context run a hive query stop the spark context create another spark context + hive context again run a hive query I get the Native Library exception From what I can tell, when the ClassLoader is hitting a MAPR class which is attempting to load the native library (I presume, as part of a static initializer). Unfortunately, the JVM prohibits this the second time around. :( I could file a ticket with a full stack trace if you'd like. thanks, Fi --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-6908] [SQL] Use isolated Hive client
Github user coderfi commented on the pull request: https://github.com/apache/spark/pull/5876#issuecomment-104511653 K. Filed as https://issues.apache.org/jira/browse/SPARK-7819 thank you for looking into this! Fi --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: The `hive-0.13.1` maven profile now sets `hive...
GitHub user coderfi opened a pull request: https://github.com/apache/spark/pull/3072 The `hive-0.13.1` maven profile now sets `hive.version=0.13.1a` instead of `hive.version=0.13.1`. e.g. mvn -Phive -Phive=0.13.1 Note: `hive.version=0.13.1a` is the default property value. However, when explicitly specifying the `hive-0.13.1` maven profile, the wrong one would be selected. References: PR #2685, which resolved a package incompatibility issue with Hive-0.13.1 by introducing a special version Hive-0.13.1a You can merge this pull request into a Git repository by running: $ git pull https://github.com/coderfi/spark master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/3072.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3072 commit 7ca4b1effe0a9032dd0478cb6bfca6f1a606e836 Author: fi code...@gmail.com Date: 2014-11-03T08:18:06Z Fixes the `hive-0.13.1` maven profile referencing `hive.version=0.13.1` instead of the Spark compatible `hive.version=0.13.1a` Note: `hive.version=0.13.1a` is the default version. However, when explicitly specifying the `hive-0.13.1` maven profile, the wrong one would be selected. e.g. mvn -Phive -Phive=0.13.1 See PR #2685 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3826][SQL]enable hive-thriftserver to s...
Github user coderfi commented on the pull request: https://github.com/apache/spark/pull/2685#issuecomment-61449633 @liancheng PR #3072 created (all of one line! :) ). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: The `hive-0.13.1` maven profile now sets `hive...
Github user coderfi commented on the pull request: https://github.com/apache/spark/pull/3072#issuecomment-61509091 Hello, I'll file a ticket. Suggestion on the title? Fi On Nov 3, 2014 2:09 AM, Cheng Lian notificati...@github.com wrote: This LGTM besides PR title and the missing JIRA ticket. Thanks! â Reply to this email directly or view it on GitHub https://github.com/apache/spark/pull/3072#issuecomment-61458285. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4211][Build] Fixes hive.version in Mave...
Github user coderfi commented on the pull request: https://github.com/apache/spark/pull/3072#issuecomment-61561386 Glad to be of service. :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
