[GitHub] spark pull request: Set null as default value for TaskContextImpl#...

2015-05-03 Thread tedyu
Github user tedyu commented on the pull request: https://github.com/apache/spark/pull/5874#issuecomment-98571954 Right. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark pull request: Set null as default value for TaskContextImpl#...

2015-05-03 Thread tedyu
Github user tedyu closed the pull request at: https://github.com/apache/spark/pull/5874 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] spark pull request: Set null as default value for TaskContextImpl#...

2015-05-03 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5874#issuecomment-98568420 The focus of my comment was on "from now on"... not existing code. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: Set null as default value for TaskContextImpl#...

2015-05-03 Thread tedyu
Github user tedyu commented on the pull request: https://github.com/apache/spark/pull/5874#issuecomment-98568325 I found two places where taskMemoryManager is used: DAGScheduler#runLocallyWithinThread() Executor#run() taskMemoryManager is constructed in both places.

[GitHub] spark pull request: Set null as default value for TaskContextImpl#...

2015-05-03 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5874#issuecomment-98566402 But this increases the chance of NPE for non-test code added from now on, doesn't it? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: Set null as default value for TaskContextImpl#...

2015-05-03 Thread tedyu
Github user tedyu commented on the pull request: https://github.com/apache/spark/pull/5874#issuecomment-98562614 From what I can tell (e.g. JavaAPISuite), null taskMemoryManager is passed in existing tests. This doesn't seem to increase the chance of NPE. I agree with Josh

[GitHub] spark pull request: Set null as default value for TaskContextImpl#...

2015-05-03 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5874#issuecomment-98562321 @tedyu I'm not sure how useful this change is, as it might make it easier to get null pointer exception? --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: Set null as default value for TaskContextImpl#...

2015-05-03 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5874#issuecomment-98562301 (I was simply adding Ted to the whitelist so he wouldn't need to wait for us to start jenkins) --- If your project is set up for it, you can reply to this email and have yo

[GitHub] spark pull request: Set null as default value for TaskContextImpl#...

2015-05-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/5874#issuecomment-98562192 @rxin I think the motivation for this was to minimize some changes to rest code. If we're going to make this change, shouldn't we also back out the test related change

[GitHub] spark pull request: Set null as default value for TaskContextImpl#...

2015-05-03 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5874#issuecomment-98562152 Jenkins, add to whitelist. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: Set null as default value for TaskContextImpl#...

2015-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5874#issuecomment-98547145 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: Set null as default value for TaskContextImpl#...

2015-05-03 Thread tedyu
GitHub user tedyu opened a pull request: https://github.com/apache/spark/pull/5874 Set null as default value for TaskContextImpl#taskMemoryManager @JoshRosen Please take a look. You can merge this pull request into a Git repository by running: $ git pull https://github.com