Github user scwf closed the pull request at:
https://github.com/apache/spark/pull/5247
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enable
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-117391609
close this, will file a new PR when i have time to fix the test failure
---
If your project is set up for it, you can reply to this email and have your
reply appear on Git
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-102245423
If we can do that in the query planner that sounds reasonable to me. It
would also be nice to add Command to all commands so we can just match on
that and call e
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-102244763
sql("show tables").take(1) still start a spark job on the master branch.
after thinking this more, i think we can convert runnablecommand to
`LocalRelation` and also we
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-101842061
This implementation is still making changes to the query plan in
`DataFrame` that aren't reflected in the `queryExecution`. This is something I
want to avoid. Also, i
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-101106745
ping @marmbrus
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-100416937
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-100416938
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-100416933
[Test build #32289 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32289/consoleFull)
for PR 5247 at commit
[`6402c61`](https://git
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-100407325
[Test build #32289 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32289/consoleFull)
for PR 5247 at commit
[`6402c61`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-100406294
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-100406221
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-99620106
/cc @marmbrus
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
ena
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-98431713
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-98431715
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/316
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-98431711
[Test build #31679 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31679/consoleFull)
for PR 5247 at commit
[`0f623ca`](https://gith
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-98423865
[Test build #31679 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31679/consoleFull)
for PR 5247 at commit
[`0f623ca`](https://githu
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-98423509
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-98423524
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-98423434
Retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-95369504
@marmbrus any more comments?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have th
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-95005505
[Test build #30715 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30715/consoleFull)
for PR 5247 at commit
[`0f623ca`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-95005528
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-94988751
[Test build #30715 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30715/consoleFull)
for PR 5247 at commit
[`0f623ca`](https://githu
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-94987894
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have th
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-94986611
Jenkins failed `org.apache.spark.sql.hive.thriftserver.CliSuite.Simple
commands` but locally test ok.
Retest this please.
---
If your project is set up for it, you can
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-94985437
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-94985427
[Test build #30711 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30711/consoleFull)
for PR 5247 at commit
[`0f623ca`](https://gith
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-94975611
[Test build #30711 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30711/consoleFull)
for PR 5247 at commit
[`0f623ca`](https://githu
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-94975420
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this featu
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-94842568
/cc @marmbrus
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
ena
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-94654202
[Test build #30632 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30632/consoleFull)
for PR 5247 at commit
[`0f623ca`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-94654211
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-94636851
[Test build #30632 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30632/consoleFull)
for PR 5247 at commit
[`0f623ca`](https://githu
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-94619470
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-94619462
[Test build #30619 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30619/consoleFull)
for PR 5247 at commit
[`d92c21d`](https://gith
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-94616559
[Test build #30619 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30619/consoleFull)
for PR 5247 at commit
[`d92c21d`](https://githu
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-94615111
Revert to my first commit version, reconstruct the Logicalplan of command
as LocalRelation in DataFrame since
1 IMO the logicalPlan of DF of command should be LocalRelati
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-93596959
@marmbrus i think we can not directly from RunnableCommand to
LocalTableScan in planner, that will leads to execute command twice as i
described before. So here is other two
Github user scwf commented on a diff in the pull request:
https://github.com/apache/spark/pull/5247#discussion_r28397087
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala ---
@@ -139,11 +139,13 @@ class DataFrame private[sql](
@transient protected[sql] va
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-93218343
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-93217874
[Test build #30317 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30317/consoleFull)
for PR 5247 at commit
[`7f51f7e`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-93217876
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-93217820
Here is a problem need to be fixed: the ddl command will be executed twice
---
If your project is set up for it, you can reply to this email and have your
reply appear on Gi
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-93217483
[Test build #30317 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30317/consoleFull)
for PR 5247 at commit
[`7f51f7e`](https://githu
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-92591857
[Test build #30216 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30216/consoleFull)
for PR 5247 at commit
[`b1232ed`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-92591858
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-92590934
[Test build #30216 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30216/consoleFull)
for PR 5247 at commit
[`b1232ed`](https://githu
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-92573420
yeah, its good to directly from RunnableCommand to LocalTableScan, i am
updating this.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-92551194
You are correct, thanks for clarifying. Query planning was not the right
phrase, but really my point was that ideally the logic in DataFrame would
handle only ensuring
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-92191089
Hi @marmbrus,
first i think this change is not doing query planning because
1 query planning is to convert a logical plan to spark plan (correct me if
i misunderstan
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-90322411
Thanks for working on this! I have one issue with the current
implementation. In particular, it is essentially doing query planning inside
of [[DataFrame]]. If you lo
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-90069855
/cc @liancheng
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
ena
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-88479176
cc @liancheng
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
ena
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-87895135
@liancheng can you look at this ? thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project do
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-87892014
@rxin @liancheng is this ok?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have th
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-87404385
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-87404381
[Test build #29365 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29365/consoleFull)
for PR 5247 at commit
[`a129816`](https://gith
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-87392877
[Test build #29365 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29365/consoleFull)
for PR 5247 at commit
[`a129816`](https://githu
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-87361722
Ok. Get it
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enable
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-87357425
Yes, that's also what I'm going to say. As described in the JIRA ticket
title, using a `LocalRelation` can be simpler.
---
If your project is set up for it, you can re
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-87355369
Can we use LocalRelation?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-87349293
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5247#issuecomment-87349291
[Test build #29355 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29355/consoleFull)
for PR 5247 at commit
[`17358b8`](https://gith
64 matches
Mail list logo
