[GitHub] spark issue #16324: [SPARK-18910][SQL]Resolve faile to use UDF that jar file...
Github user melin commented on the issue: https://github.com/apache/spark/pull/16324 in spark 2.1.0 0: jdbc:hive2://localhost:1> add jar hdfs:///user/datacompute/datacompute-udf-1.0- Error: java.net.MalformedURLException: unknown protocol: hdfs (state=,code=0) 0: jdbc:hive2://localhost:1> --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16324: [SPARK-18910][SQL]Resolve faile to use UDF that jar file...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16324 (gentle ping @henh062326) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16324: [SPARK-18910][SQL]Resolve faile to use UDF that jar file...
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16324 I was saying we need to create a test case for this change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16324: [SPARK-18910][SQL]Resolve faile to use UDF that jar file...
Github user shenh062326 commented on the issue: https://github.com/apache/spark/pull/16324 Iâm sorry, @rxin, I don't understand what you mean. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16324: [SPARK-18910][SQL]Resolve faile to use UDF that jar file...
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16324 This is to allow using jars defined using HDFS-API, not just HDFS right? In that case it sounds like a good idea too ... but we need a test case for it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16324: [SPARK-18910][SQL]Resolve faile to use UDF that jar file...
Github user shenh062326 commented on the issue: https://github.com/apache/spark/pull/16324 Currentlyï¼we can create a UDF with jar in HDFS, but failed to use it. Spark driver won't download the jar from HDFS, it only add the path to the classLoader. If we don't support reading UDF jar from HDFS, we should download the UDF jar. I think support reading UDF jar from HDFS is better. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16324: [SPARK-18910][SQL]Resolve faile to use UDF that jar file...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16324 First, I am not sure whether we should support reading UDF jar from HDFS. Second, if we want to support, the best reviewers are @zsxwing @tdas They added the file [HDFSMetadataLog.scala](https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala) recently --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16324: [SPARK-18910][SQL]Resolve faile to use UDF that jar file...
Github user shenh062326 commented on the issue: https://github.com/apache/spark/pull/16324 Should we download the UDF jar from hdfs. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16324: [SPARK-18910][SQL]Resolve faile to use UDF that jar file...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16324 This might not be the right solution to do it, as explained above. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16324: [SPARK-18910][SQL]Resolve faile to use UDF that jar file...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16324 > This method can only be called once per JVM, so it is typically executed in a static block. This limitation means that if some other part of your programâperhaps a third-party component outside your controlâ sets a URLStreamHandlerFactory, you wonât be able to use this approach for reading data from Hadoop. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16324: [SPARK-18910][SQL]Resolve faile to use UDF that jar file...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16324 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70296/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16324: [SPARK-18910][SQL]Resolve faile to use UDF that jar file...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16324 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16324: [SPARK-18910][SQL]Resolve faile to use UDF that jar file...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16324 **[Test build #70296 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70296/testReport)** for PR 16324 at commit [`a91b08f`](https://github.com/apache/spark/commit/a91b08fcbdd76e02c2a244ea4ffca726339fdba8). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16324: [SPARK-18910][SQL]Resolve faile to use UDF that jar file...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16324 Please add the description in the JIRA to the PR description. FYI, you still can edit the description after you create the PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
