Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20464
merged to master, thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88039/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20464
**[Test build #88039 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88039/testReport)**
for PR 20464 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1347/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20464
**[Test build #88039 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88039/testReport)**
for PR 20464 at commit
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20464
appveyor tests failed, could you close and reopen this PR to trigger it.
strange, I haven't seen anything like this on appveyor a long time.
```
1. Error: create DataFrame with
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20464
**[Test build #87993 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87993/testReport)**
for PR 20464 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87993/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1305/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20464
**[Test build #87993 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87993/testReport)**
for PR 20464 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/20464
Because 2.3 is released, ping @felixcheung again
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/20464
@felixcheung Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20464
Sorry, I'm a bit occupied with testing 2.3 RC, will get back to this after.
---
-
To unsubscribe, e-mail:
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/20464
ping @felixcheung
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user shivaram commented on the issue:
https://github.com/apache/spark/pull/20464
I think @felixcheung has the most context here, so I'd suggest we wait for
his comments.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86985/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20464
**[Test build #86985 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86985/testReport)**
for PR 20464 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20464
**[Test build #86985 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86985/testReport)**
for PR 20464 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/525/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86924/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20464
**[Test build #86924 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86924/testReport)**
for PR 20464 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20464
**[Test build #86924 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86924/testReport)**
for PR 20464 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/473/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/20464
> One followup question is though, would it be difficult to match the
behaviour with substr in R when the index is 0 or minus? If i understood #20464
(comment) correctly, it sounds better to match
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20464
Just in case, I am testing with:
```R
df <- createDataFrame(list(list(a="abcdef")))
collect(select(df, substr(df$a, 4, 5)))
substr("abcdef", 4, 5)
```
just in
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20464
I was just manually double checking both substr in R and this. It seems
correct; however, I think we should add a note in the doc and release note ...
One followup question is though,
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/20464
@shivaram Thanks for pointing out it. I made change to the description.
Hopefully it is clearer now. Basically I just want to clarify why R's substr
tests are correct previously.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86908/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20464
**[Test build #86908 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86908/testReport)**
for PR 20464 at commit
Github user shivaram commented on the issue:
https://github.com/apache/spark/pull/20464
Thanks for clarifying @viirya. Is the PR description accurate ? I read it
as `..SQL's substr also accepts zero-based starting position` while R uses a
1-based starting position.
---
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/20464
@shivaram This fix is to make it correctly 1-based. Previously SparkR
substr API substracts starting position by 1, so it becomes zero-based.
This fix matches R's substr in above link as I
Github user shivaram commented on the issue:
https://github.com/apache/spark/pull/20464
One thing to keep in mind is what the user's perception of the API is. If R
users are going to use 1-based indexing then this might not be the right fix ?
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/20464
Also @shivaram
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/459/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20464
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/20464
cc @felixcheung @HyukjinKwon
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/20464
One more thing to notice is that the two parameters (starting and ending
positions) of R's substr API is also unaligned with Scala's substr which takes
starting position and substring length.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20464
**[Test build #86908 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86908/testReport)**
for PR 20464 at commit
47 matches
Mail list logo
