GitHub user holdenk opened a pull request:
https://github.com/apache/spark/pull/16211
[SPARK-18576][PYTHON] Add basic TaskContext information to PySpark
## What changes were proposed in this pull request?
Adds basic TaskContext information to PySpark.
## How was this patch tested?
New unit tests to `tests.py` & existing unit tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/holdenk/spark SPARK-18576-pyspark-taskcontext
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/16211.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #16211
----
commit b6cbd3b0d2e4c82edc4a426ca7fc37cff18a3123
Author: Holden Karau <hol...@us.ibm.com>
Date: 2016-12-02T17:27:21Z
Start work towards adding a TaskContext for Python users
commit 3deef8e99d018ef07a4b22cdff17d82bbf727218
Author: Holden Karau <hol...@us.ibm.com>
Date: 2016-12-02T17:48:02Z
Start adding tests, add some singleton pattern stuff just for safety (maybe
over kill idk?)
commit 3ee56f435a6b5ec3a89dd3a5d01500113f3657ca
Author: Holden Karau <hol...@us.ibm.com>
Date: 2016-12-04T12:08:00Z
Wait we just use stageId not rddId that's fine
commit 509d8e850c3630bd519ee26f254b1a2002fa796d
Author: Holden Karau <hol...@us.ibm.com>
Date: 2016-12-05T12:34:33Z
Round out the tests
commit b8eb18d4c677c9c5ffab70cab237bb0639635dd2
Author: Holden Karau <hol...@us.ibm.com>
Date: 2016-12-05T12:49:56Z
pep8ify
commit 6b522f237277bc19e2390816f3fdce3a5bafe7a6
Author: Holden Karau <hol...@us.ibm.com>
Date: 2016-12-07T06:09:03Z
Improve the tests a bit
commit 1a76b504a71b1480440898c3a9e97556eb321206
Author: Holden Karau <hol...@us.ibm.com>
Date: 2016-12-08T03:22:39Z
Add some docstrings for the tests
commit c64f39d17badf9fcc4c2e15121f9a0d1ba603dfa
Author: Holden Karau <hol...@us.ibm.com>
Date: 2016-12-08T09:35:45Z
Allow TC to be serialized but return None on the driver program instead
commit ce4d6be115a1b0eef790e3aa9282a14138d69d90
Author: Holden Karau <hol...@us.ibm.com>
Date: 2016-12-08T09:44:23Z
Test direct constructor access
commit a5359cdd571e28cbed55f08a6d81343f18a01d08
Author: Holden Karau <hol...@us.ibm.com>
Date: 2016-12-08T09:52:45Z
Get rid of extra blank line
commit ba1922483301fca87b982dd6e3a56c61e50c3095
Author: Holden Karau <hol...@us.ibm.com>
Date: 2016-12-08T09:54:51Z
Wait blank line is more common
commit 526de0abdec7fdc73ea86f0ce027a20d3bdbd356
Author: Holden Karau <hol...@us.ibm.com>
Date: 2016-12-08T09:57:04Z
Leave out the . for consistency
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
