Github user aa8y commented on the issue:
https://github.com/apache/spark/pull/20068
I apologize I haven't had time to work on this. I can close this for now
and reopen it when I have a working fix
Github user aa8y commented on the issue:
https://github.com/apache/spark/pull/20068
I'll work on it in the next week or two. That would involve a PR to the
Univocity CSV parser.
---
-
To unsubscribe, e-mail
Github user aa8y commented on a diff in the pull request:
https://github.com/apache/spark/pull/20068#discussion_r159160508
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVOptions.scala
---
@@ -152,7 +152,11 @@ class CSVOptions
Github user aa8y commented on the issue:
https://github.com/apache/spark/pull/20068
@HyukjinKwon I made code changes based on your suggestions. I also changed
the tests to use the data mentioned in the ticket. However, you're right, the
tests no longer pass. But that is because
Github user aa8y commented on a diff in the pull request:
https://github.com/apache/spark/pull/20068#discussion_r158606107
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVOptions.scala
---
@@ -152,7 +152,7 @@ class CSVOptions
Github user aa8y commented on the issue:
https://github.com/apache/spark/pull/20068
@gatorsmile I've created this PR since #12904 has not been updated in a
while.
---
-
To unsubscribe, e-mail: reviews-unsubscr
GitHub user aa8y opened a pull request:
https://github.com/apache/spark/pull/20068
SPARK-17916: Fix empty string being parsed as null when nullValue is set.
## What changes were proposed in this pull request?
When the option `nullValue` is set, the empty value is also set
Github user aa8y closed the pull request at:
https://github.com/apache/spark/pull/10247
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/10247#issuecomment-217194679
Won't do.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user aa8y commented on a diff in the pull request:
https://github.com/apache/spark/pull/12708#discussion_r61906043
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala ---
@@ -335,6 +358,19 @@ trait Row extends Serializable {
def getAs[T](fieldName
Github user aa8y commented on a diff in the pull request:
https://github.com/apache/spark/pull/12708#discussion_r61793288
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala ---
@@ -335,6 +358,19 @@ trait Row extends Serializable {
def getAs[T](fieldName
Github user aa8y commented on a diff in the pull request:
https://github.com/apache/spark/pull/12708#discussion_r61789596
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala ---
@@ -335,6 +358,19 @@ trait Row extends Serializable {
def getAs[T](fieldName
Github user aa8y commented on a diff in the pull request:
https://github.com/apache/spark/pull/12708#discussion_r61757432
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala ---
@@ -335,6 +358,19 @@ trait Row extends Serializable {
def getAs[T](fieldName
GitHub user aa8y opened a pull request:
https://github.com/apache/spark/pull/12742
[MINOR][MAINTENANCE] Sort the entries in .gitignore.
## What changes were proposed in this pull request?
The contents of `.gitignore` have been sorted to make it more readable. The
actual
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/12587#issuecomment-214869241
PR being closed as it was pointing to my `master`. The new PR is #12708.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user aa8y closed the pull request at:
https://github.com/apache/spark/pull/12587
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/12587#issuecomment-214869110
PR being closed as it was pointing to my `master`. The new PR is #12708.
---
If your project is set up for it, you can reply to this email and have your
reply appear
GitHub user aa8y opened a pull request:
https://github.com/apache/spark/pull/12708
[SPARK-11962] WIP: Added `attempt` and `getOption`.
## What changes were proposed in this pull request?
This PR adds two major methods to Row, `attempt` and `getOption`. The
former returns
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/12653#issuecomment-214860733
Known issue which cannot be fixed until Spark ceases support for Scala 2.10.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user aa8y closed the pull request at:
https://github.com/apache/spark/pull/12653
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/12587#issuecomment-214365652
@srowen I would really appreciate if you could review this WIP PR as well.
I'll close it of course once I get all the feedback. On @holdenk's suggestion I
did send
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/12653#issuecomment-214362282
@srowen, @hvanhovell I'll close this ticket. I am assuming support for
Scala 2.10 will be ceased in Spark 2.0.0. Can this be reopened once the work
for that has been done
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/12653#issuecomment-214010191
I still feel like it is compiling with 2.11 looking at the output on the
console. But it did fail, unable to resolve a library dependency, with the
version set to 2.10
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/12653#issuecomment-214003113
@hvanhovell I am not sure how to do that. Still learning working with SBT.
Any tips?
---
If your project is set up for it, you can reply to this email and have your
GitHub user aa8y opened a pull request:
https://github.com/apache/spark/pull/12653
[MINOR][MAINTENANCE] Deprecation fixes.
## What changes were proposed in this pull request?
Fixed a bunch of deprecation warnings I saw when compiling the current
Spark `master
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/12587#issuecomment-213983438
@hvanhovell The end use case for both `attempt` and `getOption` is to get
values out of a row object without throwing an exception. However, in one of my
previous PRs
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/12587#issuecomment-213164874
@holdenk: I will keep in mind your suggestions in the future. The change in
the .gitignore was that I sorted the contents. I can replace it with what's on
master
GitHub user aa8y opened a pull request:
https://github.com/apache/spark/pull/12587
[SPARK-11962] (Not merge ready) Added functions `attempt` and `getOption`.
## What changes were proposed in this pull request?
Add methods to get `Try` and `Option` types from a `Row
Github user aa8y commented on a diff in the pull request:
https://github.com/apache/spark/pull/10247#discussion_r47579991
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala ---
@@ -325,6 +341,14 @@ trait Row extends Serializable {
def getAs[T](i: Int): T
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/10247#issuecomment-164600717
@marmbrus I agree. I wasn't a fan of `getAsOpt` but couldn't think of a
better name then. I've updated them all to `getOption`. So does this look
better now
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/10247#issuecomment-164604835
Isn't the whole point of `Option` to not throw exceptions? Actually it's to
not return `null`s I guess. But throwing an exception would defeat the purpose
of returning
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/10247#issuecomment-164605606
What would you propose as a solution instead?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/10247#issuecomment-164594278
@jodersky: I incorporated your recommendations and also updated my branch
with the current master. Can you or one of the admins please ask Jenkins to
test this build
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/9943#issuecomment-164592414
Sure. This is an old PR anyway. I did not know the PRs update themselves
with new commits. #10247 is the PR I am working on now. I'll make the
recommended changes
Github user aa8y closed the pull request at:
https://github.com/apache/spark/pull/9943
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/10028#issuecomment-163349969
@rxin You ran a Jenkins build for my previous pull request for this issue.
Could you please do the same for this build request?
---
If your project is set up for it, you
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/10028#issuecomment-163384083
```aallamsetty@aallamsetty-lap ~/c/spark> ./dev/run-tests
Can't install SparkR as R is was not found in PATH
[info] Using build tool sbt with Hadoop prof
GitHub user aa8y opened a pull request:
https://github.com/apache/spark/pull/10247
[SPARK-11962] Added getAsOpt functions to Row and tests for it.
getAsOpt[T] functions have been added to Row and GeneicRowWithSchema to get
the values present in a row object optionally
Github user aa8y closed the pull request at:
https://github.com/apache/spark/pull/10028
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/10028#issuecomment-163522315
@JoshRosen I've created a new pull request, #10247 with a fresh new commit
which should merge. @srowen I would really appreciate it if you could get
Jenkins to test
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/10028#issuecomment-163372735
I remember it passed the style tests last time I checked. I'll run them
again.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/10028#issuecomment-160761838
Anything else you guys need for this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
GitHub user aa8y opened a pull request:
https://github.com/apache/spark/pull/10028
[SPARK-11962] Add optional getAs functions to Row
getAsOpt[T] functions have been added to Row and GeneicRowWithSchema to get
the values present in a row object optionally. Corresponding tests have
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/10027#issuecomment-160388606
I am still learning to work with Git. Looks like I have a couple commits
which are not mine in there. However a compare did not show any code other than
mine, so I filed
GitHub user aa8y opened a pull request:
https://github.com/apache/spark/pull/10027
[SPARK-11962] Add optional getAs functions to Row
`getAsOpt[T]` functions have been added to `Row` and `GeneicRowWithSchema`
to get the values present in a row object optionally. Corresponding tests
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/10027#issuecomment-16038
I just found out this would fail style checks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user aa8y closed the pull request at:
https://github.com/apache/spark/pull/10027
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/9943#issuecomment-159697011
Sorry, I am new to this. Does what you wrote trigger a Jenkins build? If
yes, that's really cool!
---
If your project is set up for it, you can reply to this email
Github user aa8y commented on the pull request:
https://github.com/apache/spark/pull/9943#issuecomment-159731209
Sorry. I ran the tests and I thought they passed on my machine. A few tests
couldn't be run (like docker) but these looked good. I'll take another look
GitHub user aa8y opened a pull request:
https://github.com/apache/spark/pull/9943
[SPARK-11962] Added getAsOpt[T]() methods to enable getting values
optionally.
Adds methods to get values from a row object optionally. I personally have
an implicit class which does this and which I
50 matches
Mail list logo
