GitHub user clarkfitzg opened a pull request:
https://github.com/apache/spark/pull/14783
SPARK-16785 R dapply doesn't return array or raw columns
## What changes were proposed in this pull request?
Fixed bug in `dapplyCollect` by changing the `compute` function of
`worker.R` to explicitly handle raw (binary) vectors.
cc @shivaram
## How was this patch tested?
Unit tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/clarkfitzg/spark SPARK-16785
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14783.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14783
----
commit f8e19202c7bc7698d52a24f9da0d90baf7f8e034
Author: Clark Fitzgerald <clarkfi...@gmail.com>
Date: 2016-08-18T02:35:17Z
test for array and byte columns
commit 13366051459da3533ed873e1ec96e056ac25a3e1
Author: Clark Fitzgerald <clarkfi...@gmail.com>
Date: 2016-08-18T05:40:18Z
R createDataFrame.default uses list of dataframes as rows
commit a0e13ffaf0f9ff0a3f3b9f63a819292d4269856e
Author: Clark Fitzgerald <clarkfi...@gmail.com>
Date: 2016-08-18T23:38:14Z
test is working now
commit 50152339d0f0fa6fd071483326b6b43cd8c26a3d
Author: Clark Fitzgerald <clarkfi...@gmail.com>
Date: 2016-08-19T00:37:09Z
add simple collect() to test
commit d044054da410a94dd97f3321bc0d1ba70c5d5b2d
Author: Clark Fitzgerald <clarkfi...@gmail.com>
Date: 2016-08-19T01:28:35Z
document how tests fail
commit 311b55400b2c7787b74ae18d6fe0854dba29eb5d
Author: Clark Fitzgerald <clarkfi...@gmail.com>
Date: 2016-08-19T02:20:22Z
identified where patch went wrong
commit 2d2654dff670094b7a97d7be593544f65183cd5a
Author: Clark Fitzgerald <clarkfi...@gmail.com>
Date: 2016-08-19T02:48:29Z
back to original code for creating data frame
commit ff1a0d02ea25e134980bf6111ec269f0550b3a80
Author: Clark Fitzgerald <clarkfi...@gmail.com>
Date: 2016-08-19T07:25:12Z
first pass modifying worker.R
commit 1e27ef3f458ac6ccc23cccc2f3624c76e7cd4d0b
Author: Clark Fitzgerald <clarkfi...@gmail.com>
Date: 2016-08-19T07:37:31Z
no change in error message, reverting
commit 25d0ec12436c7f77b79281c348c04ddac64f2156
Author: Clark Fitzgerald <clarkfi...@gmail.com>
Date: 2016-08-19T07:42:51Z
an experiment modifying dapplyCollect directly
commit 77a9822e4cb15c8ee8d48e95597e8e783d2208bb
Author: Clark Fitzgerald <clarkfi...@gmail.com>
Date: 2016-08-24T01:50:49Z
dapplyCollect worked, it just nested things
commit 70b0d44b4f5ea556db63d93e413a0cbbf30c76c2
Author: Clark Fitzgerald <clarkfi...@gmail.com>
Date: 2016-08-24T02:27:30Z
tests pass!
commit b21a21df80a29e2d32c785ef7551cd2a895be8f1
Author: Clark Fitzgerald <clarkfi...@gmail.com>
Date: 2016-08-24T02:47:49Z
put rbind function in utils.R
commit ba87b06f999860f054b25b521258216ec0b264a7
Author: Clark Fitzgerald <clarkfi...@gmail.com>
Date: 2016-08-24T04:23:09Z
rename to rbindRaws and put in utils.R
commit 528fa6e4b57a66f413488e15e8d7f15fbe3ab632
Author: Clark Fitzgerald <clarkfi...@gmail.com>
Date: 2016-08-24T04:40:46Z
just whitespace
commit e0a3894e7fd00285da85b523ff07957f24bb436e
Author: Clark Fitzgerald <clarkfi...@gmail.com>
Date: 2016-08-24T05:16:36Z
syntax error in worker.R
commit 5871257226cc904edac59ec00ca52d5106f84a13
Author: Clark Fitzgerald <clarkfi...@gmail.com>
Date: 2016-08-24T05:32:01Z
satisfy lintr
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
