Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-86329564
anyone will test it ? @marmbrus @srowen
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4995#discussion_r27191997
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,18 @@
+package org.apache.spark.sql.hive
+
+import o
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-86384179
yes, it works. I have tested it in the distributed mode with two nodes.
@rxin
---
If your project is set up for it, you can reply to this email and have your
reply a
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-91955538
@DoingDone9, really? In my tests it null pointers when you try to get the
session state on an executor. It seems like they made this change in Hive 13
on purpose becau
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/4995
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enab
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-92159014
my previous test was successful and i will test it again. Thank you
@marmbrus
---
If your project is set up for it, you can reply to this email and have your
reply
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-92189086
Could you tell me how you got this exception? I test with three nodes , and
it works again. Thank you @marmbrus
---
If your project is set up for it, you can reply t
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-92464041
Here is the command I ran:
```
sc.parallelize(1 to 10).map(_ =>
org.apache.hadoop.hive.ql.session.SessionState.get().getCurrentDatabase()).collect()
```
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-92599430
I do not agree that. Because this expression is foldable, then it will be
computed in ConstantFolding of Optimizer. So I will get the name of currentDB
after optimi
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-93587797
Ah, I see. Can you add comments that explain that and reopen this PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on Gi
GitHub user DoingDone9 opened a pull request:
https://github.com/apache/spark/pull/5538
[SPARK-6198][SQL] Support "select current_database()"
to support "select current_database()"
```
The method(evaluate) has changed in UDFCurrentDB, it just throws a
exception.But hiveUdfs
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-93625012
@marmbrus
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-93624977
I have opened a new pr https://github.com/apache/spark/pull/5538
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-93625269
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your pro
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28488519
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Founda
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28498708
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foun
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28535110
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Founda
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-93878012
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-93870998
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
ena
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-93878007
[Test build #30446 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30446/consoleFull)
for PR 5538 at commit
[`def60c3`](https://gith
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-93871493
[Test build #30446 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30446/consoleFull)
for PR 5538 at commit
[`def60c3`](https://githu
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-93899836
[Test build #30459 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30459/consoleFull)
for PR 5538 at commit
[`fad020e`](https://githu
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-93937013
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-93937009
[Test build #30459 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30459/consoleFull)
for PR 5538 at commit
[`fad020e`](https://gith
GitHub user DoingDone9 opened a pull request:
https://github.com/apache/spark/pull/4926
[SPARK-6198][SQL] Support "select current_database()"
The method(evaluate) has changed in UDFCurrentDB, it just throws a
exception.But hiveUdfs call this method and failed.
@Override
publ
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4926#issuecomment-77503622
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your pro
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/4926#issuecomment-77654187
please add a test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feat
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4926#issuecomment-77678092
I have add a test, pleat test it. @marmbrus
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your pr
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4926#discussion_r26013109
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala
---
@@ -179,7 +179,12 @@ private[hive] case class HiveGenericUdf(funcWrapper
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4926#discussion_r26013144
--- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/UDFSuite.scala
---
@@ -32,5 +32,6 @@ class UDFSuite extends QueryTest {
assert(sql
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4926#issuecomment-77808256
HiveQL supports the SELECT clause without FROM. I have test it for several
times. And you can try run SQL like "select 1", it works. @chenghao-intel
---
If your proj
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4926#discussion_r26017842
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala
---
@@ -179,7 +179,12 @@ private[hive] case class HiveGenericUdf(funcWrapper:
H
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/4926#issuecomment-77866817
`SELECT 1` Seems doesn't work in Hive 0.12, probably introduced since Hive
0.13. See:https://issues.apache.org/jira/browse/HIVE-4144
---
If your project is set up
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4926#discussion_r26041674
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala
---
@@ -179,7 +179,12 @@ private[hive] case class HiveGenericUdf(funcWrapper
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4926#issuecomment-7793
yes, my version is 0.13.1. @chenghao-intel
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your pro
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4926#discussion_r26103224
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala
---
@@ -179,7 +179,12 @@ private[hive] case class HiveGenericUdf(funcWrapper:
H
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4926#issuecomment-78182986
could you test it @marmbrus
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
Github user DoingDone9 closed the pull request at:
https://github.com/apache/spark/pull/4926
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
GitHub user DoingDone9 opened a pull request:
https://github.com/apache/spark/pull/4995
[SPARK-6198][SQL] Support "select current_database()"
The method(evaluate) has changed in UDFCurrentDB, it just throws a
exception.But hiveUdfs call this method and failed.
@Override
publ
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-78447220
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your pro
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4926#issuecomment-78447224
I have opened a new pr for this .I create a new UDF and register it
instead of intercepting code.
https://github.com/apache/spark/pull/4995 @chenghao-intel @marmb
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28744774
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Founda
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94609816
Is there a reason you have not added a comment about the lifecycle of this
UDF?
---
If your project is set up for it, you can reply to this email and have your
reply ap
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94617955
sorry, i changed the code then the comment disappear, i will add it again.
---
If your project is set up for it, you can reply to this email and have your
reply appea
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28746101
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foun
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28746146
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foun
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94626030
I am not so sure if this is the correct implementation, as we probably not
able to get the correct `SessionState` object in executors.
Hive seems doesn't provi
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94629707
@chenghao-intel i konw this method that can get dbName, but it can only be
used with CLI. It is necessary to get dbName without cli. And i have explained
that this wi
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94633954
@DoingDone9 Not like Hive, I don't think Spark SQL supports the `local
mode`, the UDF definitely will run within executor(s), or, we can transform it
into an `Exec
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28749336
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94635160
@chenghao-intel your idea is good but âselect current_databaseâ is
syntax of hive. and i want to implemente it . And this UDF do not run within
executor(s),becau
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94640080
@DoingDone9 thanks for the explanation. In thericially, applying the
`Optimizer` rule is optional, and probably not everyone understand how the
constant folding wo
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28751859
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foun
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94760655
[Test build #30668 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30668/consoleFull)
for PR 5538 at commit
[`a81e400`](https://githu
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94772903
[Test build #30671 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30671/consoleFull)
for PR 5538 at commit
[`60e6ee8`](https://githu
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94775676
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94798862
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94798841
[Test build #30668 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30668/consoleFull)
for PR 5538 at commit
[`a81e400`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94807750
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94807716
[Test build #30671 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30671/consoleFull)
for PR 5538 at commit
[`60e6ee8`](https://gith
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28830966
--- Diff:
sql/hive/v0.12.0/src/main/scala/org/apache/spark/sql/hive/Shim12.scala ---
@@ -135,7 +135,13 @@ private[hive] object HiveShim {
Primitiv
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28838875
--- Diff:
sql/hive/v0.12.0/src/main/scala/org/apache/spark/sql/hive/Shim12.scala ---
@@ -135,7 +135,13 @@ private[hive] object HiveShim {
Primit
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28839390
--- Diff:
sql/hive/v0.12.0/src/main/scala/org/apache/spark/sql/hive/Shim12.scala ---
@@ -135,7 +135,13 @@ private[hive] object HiveShim {
Pr
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28839484
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28839705
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28839749
--- Diff:
sql/hive/v0.12.0/src/main/scala/org/apache/spark/sql/hive/Shim12.scala ---
@@ -135,7 +135,13 @@ private[hive] object HiveShim {
Primit
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28839787
--- Diff:
sql/hive/v0.13.1/src/main/scala/org/apache/spark/sql/hive/Shim13.scala ---
@@ -218,7 +218,13 @@ private[hive] object HiveShim {
Ty
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28840547
--- Diff:
sql/hive/v0.12.0/src/main/scala/org/apache/spark/sql/hive/Shim12.scala ---
@@ -135,7 +135,13 @@ private[hive] object HiveShim {
Primit
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28840911
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foun
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28841751
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28841761
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28933263
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foun
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28933363
--- Diff:
sql/hive/v0.13.1/src/main/scala/org/apache/spark/sql/hive/Shim13.scala ---
@@ -218,7 +218,13 @@ private[hive] object HiveShim {
TypeIn
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-96767138
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your pro
Github user DoingDone9 closed the pull request at:
https://github.com/apache/spark/pull/5538
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
75 matches
Mail list logo