Github user ethanluoyc commented on the issue:
https://github.com/apache/spark/pull/15279
@HyukjinKwon I am really sorry but please close this since I have not found
the time to complete all the cases.
---
If your project is set up for it, you can reply to this email and have your
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/15279
Hi @ethanluoyc, is it still WIP? I would rather like to propose to close
this if it is inactive.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/15279
shall we use `spark.driver.extraClassPath` in that case?
in python these we could set as options to SparkSession
---
If your project is set up for it, you can reply to this email and have
Github user ethanluoyc commented on the issue:
https://github.com/apache/spark/pull/15279
For example
https://github.com/apache/spark/blob/master/examples/src/main/python/parquet_inputformat.py#L26.
It requires that we pass in jar files, which are not so easy to detect
directly.
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/15279
that looks reasonable. could you point me to the ones with complicated
argument you were referring to?
---
If your project is set up for it, you can reply to this email and have your
reply
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15279
**[Test build #72787 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72787/testReport)**
for PR 15279 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15279
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72787/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15279
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15279
**[Test build #72787 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72787/testReport)**
for PR 15279 at commit
Github user ethanluoyc commented on the issue:
https://github.com/apache/spark/pull/15279
@felixcheung I changed two examples in python. These are the simple ones.
However, some of the examples take really complicated arguments and I am not
sure if doing it in this way would scale.
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/15279
I think that should be reasonable, but let's start with one or two and
discuss. thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user ethanluoyc commented on the issue:
https://github.com/apache/spark/pull/15279
@felixcheung I think we can, although it requires changes to every
examples. If you are fine with that, I can start on some of the examples from
now onwards:D
---
If your project is set up
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/15279
What if we have a bunch of default values when arguments are not set, and
those are the values we could test with? This way the same sample code can run
with and without arguments?
---
If
Github user ethanluoyc commented on the issue:
https://github.com/apache/spark/pull/15279
@felixcheung I actually agree with you on examples being self contained. In
that case, the example authors can just declare the arguments in the examples.
What I think may be
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/15279
@ethanluoyc this seems reasonable although I'm not 100% how this go yet.
`wordcount.py` in the yml file - does it mean 1 yml for all examples? I think
perhaps we need a way for the example to
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15279
**[Test build #71768 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71768/testReport)**
for PR 15279 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15279
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71768/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15279
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15279
**[Test build #71768 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71768/testReport)**
for PR 15279 at commit
Github user ethanluoyc commented on the issue:
https://github.com/apache/spark/pull/15279
@felixcheung I uploaded the yaml file for configuring the arguments passed
into examples.
Basically, what we need to do now is to fill in the yaml with all the
examples. And look for a way
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15279
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70088/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15279
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15279
**[Test build #70088 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70088/consoleFull)**
for PR 15279 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15279
**[Test build #70088 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70088/consoleFull)**
for PR 15279 at commit
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/15279
perhaps a test driver .yml file?
or perhaps it is better to change example to have default input values so
they run standalone if no input is passed?
---
If your project is set up for it,
Github user ethanluoyc commented on the issue:
https://github.com/apache/spark/pull/15279
@felixcheung Last time I was working on this PR, the main challenge is to
work out a nice way to record the examples that require input, and pass those
arguments properly. Do you have any
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/15279
Hi - how are we on this PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/15279
Hi, @ethanluoyc .
Could you add Apache License, too?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15279
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68646/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15279
**[Test build #68646 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68646/consoleFull)**
for PR 15279 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15279
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15279
**[Test build #68646 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68646/consoleFull)**
for PR 15279 at commit
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/15279
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/15279
Can you please change the title to have: "SPARK-12347" -> "[SPARK-12347]"?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
34 matches
Mail list logo
