Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/19487
crap, my branch got messed up.
I will resubmit the PR - sorry for the mess up.
---
-
To unsubscribe, e-mail:
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/19487
@HyukjinKwon That was exactly the initial solution I tested locally when we
saw the problem with Phoenix.
The reason to expand it was two fold:
a) This change preserves existing behavior
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19487
@mridulm, what do you think about dealing with empty string for now and
other cases later if we can't male sure for other cases for now? I guess the
actual issue found is about empty string
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/19487
> If it does use it, it'll handle an invalid entry in setupJob/setupTask by
throwing an exception there.
This should currently happen and `hasValidPath` does not prevent it.
That is, if
Github user steveloughran commented on the issue:
https://github.com/apache/spark/pull/19487
The more I see of the committer internals, the less confident I am about
understanding any of it.
If your committer isn't writing stuff out, it doesn't need to have any
value of
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/19487
I will change from `test:` to `::invalid::` to explicitly indicate an
invalid path (I picked the first path which gave me a parse error :) ).
On the question of whether `path` constructor
Github user steveloughran commented on the issue:
https://github.com/apache/spark/pull/19487
"" can come in via configuration files; I'd treat that the same as null.
Things which aren't valid URIs though, that's something you want to bounce
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19487
Thanks for cc'ing me @mridulm. Similar concern with ^.
For `null` case, I thought it makes sense because the property is unset.
However, for case of those cases, e.g., `""` and
Github user steveloughran commented on the issue:
https://github.com/apache/spark/pull/19487
Looking a bit more at this. I see it handles """ as well as empty, and also
other forms of invalid URI which Path can't handle today ("multiple colons
except with file:// on windows, etc).
Github user steveloughran commented on the issue:
https://github.com/apache/spark/pull/19487
LGTM. I'm going stick out today a slight roll of my PathOutputCommitter
class which is one layer above FileOutputCommitter : lets people write
committers without output & work paths, yet
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19487
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19487
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82709/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19487
**[Test build #82709 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82709/testReport)**
for PR 19487 at commit
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/19487
+CC @steveloughran : since you were looking at commit.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19487
**[Test build #82709 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82709/testReport)**
for PR 19487 at commit
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/19487
Specifically, this break phoenix output format - where path is set to "".
This used to work in 2.1 and fails with 2.2 and master.
+CC @szhem, @jiangxb1987, @HyukjinKwon
---
16 matches
Mail list logo
