Merge pull request #366 from colorant/yarn-dev More yarn code refactor
Try to retrive common code in yarn alpha/stable for client and workerRunnable to reduce duplicated codes. By put them into a trait in common dir and extends with them. Same works could be done for the remaining files in alpha/stable , while the remainning files have much more overlapping codes with different API call here and there within functions, and will need much more close review , aslo it might divide functions into too small trifle ones, thus might not deserve to be done in this way. So just make it run for these two files firstly. Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/cef2af9c Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/cef2af9c Diff: http://git-wip-us.apache.org/repos/asf/incubator-spark/diff/cef2af9c Branch: refs/heads/master Commit: cef2af9c7dceb5679c11adefb05705d897e55406 Parents: 3d9e66d 4c22c55 Author: Thomas Graves <tgra...@apache.org> Authored: Wed Jan 15 10:06:17 2014 -0600 Committer: Thomas Graves <tgra...@apache.org> Committed: Wed Jan 15 10:06:17 2014 -0600 ---------------------------------------------------------------------- .../org/apache/spark/deploy/yarn/Client.scala | 371 +---------------- .../spark/deploy/yarn/WorkerRunnable.scala | 126 +----- .../apache/spark/deploy/yarn/ClientBase.scala | 410 +++++++++++++++++++ .../spark/deploy/yarn/WorkerRunnableUtil.scala | 176 ++++++++ .../org/apache/spark/deploy/yarn/Client.scala | 365 +---------------- .../spark/deploy/yarn/WorkerRunnable.scala | 130 +----- 6 files changed, 627 insertions(+), 951 deletions(-) ----------------------------------------------------------------------
