Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35488/#review91456 --- Ship it! Ship It! - Shwetha GS On June 24, 2015, 7:01 a.m., pavan kumar kolamuri wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35488/ --- (Updated June 24, 2015, 7:01 a.m.) Review request for oozie and Srikanth Sundarrajan. Repository: oozie-git Description --- Spark job launched through spark action is not running as expected in standalone mode and it is not running in yarn mode Diffs - pom.xml ee26366 sharelib/spark/pom.xml 3fe9c71 src/main/assemblies/sharelib.xml ea95c2e Diff: https://reviews.apache.org/r/35488/diff/ Testing --- Basic testing done. Deployed spark job on yarn, standalone cluster and local mode . It is running successfully in all these modes. Thanks, pavan kumar kolamuri
Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode
On June 23, 2015, 7:09 a.m., pavan kumar kolamuri wrote: Thanks shwetha done please check now - pavan kumar --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35488/#review88927 --- On June 24, 2015, 7:01 a.m., pavan kumar kolamuri wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35488/ --- (Updated June 24, 2015, 7:01 a.m.) Review request for oozie and Srikanth Sundarrajan. Repository: oozie-git Description --- Spark job launched through spark action is not running as expected in standalone mode and it is not running in yarn mode Diffs - pom.xml ee26366 sharelib/spark/pom.xml 3fe9c71 src/main/assemblies/sharelib.xml ea95c2e Diff: https://reviews.apache.org/r/35488/diff/ Testing --- Basic testing done. Deployed spark job on yarn, standalone cluster and local mode . It is running successfully in all these modes. Thanks, pavan kumar kolamuri
Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35488/ --- (Updated June 24, 2015, 7:01 a.m.) Review request for oozie and Srikanth Sundarrajan. Changes --- Added dependencies versions and exclusions in parent pom Repository: oozie-git Description --- Spark job launched through spark action is not running as expected in standalone mode and it is not running in yarn mode Diffs (updated) - pom.xml ee26366 sharelib/spark/pom.xml 3fe9c71 src/main/assemblies/sharelib.xml ea95c2e Diff: https://reviews.apache.org/r/35488/diff/ Testing --- Basic testing done. Deployed spark job on yarn, standalone cluster and local mode . It is running successfully in all these modes. Thanks, pavan kumar kolamuri
Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35488/#review88927 --- sharelib/spark/pom.xml (line 107) https://reviews.apache.org/r/35488/#comment141548 Yes i agree with that, it is same across all other actions as well. Refactoring is required for other actions as well , we can raise a jira and fix all this dependency management issues sharelib/spark/pom.xml (line 224) https://reviews.apache.org/r/35488/#comment141544 There is an issue in launching application master of spark uber jar is not used. While launching an application master of spark it is only adding spark jar containing a class SparkYarn Launcher. we tried to find an alternate way , we couldn't able to find. Even for running spark jobs spark assembly jar is always used. - pavan kumar kolamuri On June 16, 2015, 5:09 a.m., pavan kumar kolamuri wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35488/ --- (Updated June 16, 2015, 5:09 a.m.) Review request for oozie and Srikanth Sundarrajan. Repository: oozie-git Description --- Spark job launched through spark action is not running as expected in standalone mode and it is not running in yarn mode Diffs - pom.xml ee26366 sharelib/spark/pom.xml 3fe9c71 src/main/assemblies/sharelib.xml ea95c2e Diff: https://reviews.apache.org/r/35488/diff/ Testing --- Basic testing done. Deployed spark job on yarn, standalone cluster and local mode . It is running successfully in all these modes. Thanks, pavan kumar kolamuri
Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode
On June 23, 2015, 7:09 a.m., pavan kumar kolamuri wrote: sharelib/spark/pom.xml, line 111 https://reviews.apache.org/r/35488/diff/1/?file=985506#file985506line111 Yes i agree with that, it is same across all other actions as well. Refactoring is required for other actions as well , we can raise a jira and fix all this dependency management issues Atleast the versions need to move to dependency management - just like other actions - Shwetha --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35488/#review88927 --- On June 16, 2015, 5:09 a.m., pavan kumar kolamuri wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35488/ --- (Updated June 16, 2015, 5:09 a.m.) Review request for oozie and Srikanth Sundarrajan. Repository: oozie-git Description --- Spark job launched through spark action is not running as expected in standalone mode and it is not running in yarn mode Diffs - pom.xml ee26366 sharelib/spark/pom.xml 3fe9c71 src/main/assemblies/sharelib.xml ea95c2e Diff: https://reviews.apache.org/r/35488/diff/ Testing --- Basic testing done. Deployed spark job on yarn, standalone cluster and local mode . It is running successfully in all these modes. Thanks, pavan kumar kolamuri
Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35488/#review88831 --- Ship it! Ship It! - Jaydeep Vishwakarma On June 16, 2015, 5:09 a.m., pavan kumar kolamuri wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35488/ --- (Updated June 16, 2015, 5:09 a.m.) Review request for oozie and Srikanth Sundarrajan. Repository: oozie-git Description --- Spark job launched through spark action is not running as expected in standalone mode and it is not running in yarn mode Diffs - pom.xml ee26366 sharelib/spark/pom.xml 3fe9c71 src/main/assemblies/sharelib.xml ea95c2e Diff: https://reviews.apache.org/r/35488/diff/ Testing --- Basic testing done. Deployed spark job on yarn, standalone cluster and local mode . It is running successfully in all these modes. Thanks, pavan kumar kolamuri
Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35488/#review88923 --- sharelib/spark/pom.xml (line 107) https://reviews.apache.org/r/35488/#comment141539 can you move the exclusions to parent pom? sharelib/spark/pom.xml (line 224) https://reviews.apache.org/r/35488/#comment141540 why are you creating uber jar? uber jars are difficult to debug - can't figure out what jars and classes are added - Shwetha GS On June 16, 2015, 5:09 a.m., pavan kumar kolamuri wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35488/ --- (Updated June 16, 2015, 5:09 a.m.) Review request for oozie and Srikanth Sundarrajan. Repository: oozie-git Description --- Spark job launched through spark action is not running as expected in standalone mode and it is not running in yarn mode Diffs - pom.xml ee26366 sharelib/spark/pom.xml 3fe9c71 src/main/assemblies/sharelib.xml ea95c2e Diff: https://reviews.apache.org/r/35488/diff/ Testing --- Basic testing done. Deployed spark job on yarn, standalone cluster and local mode . It is running successfully in all these modes. Thanks, pavan kumar kolamuri
Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35488/#review88030 --- Ship it! Ship It! - Srikanth Sundarrajan On June 16, 2015, 5:09 a.m., pavan kumar kolamuri wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35488/ --- (Updated June 16, 2015, 5:09 a.m.) Review request for oozie and Srikanth Sundarrajan. Repository: oozie-git Description --- Spark job launched through spark action is not running as expected in standalone mode and it is not running in yarn mode Diffs - pom.xml ee26366 sharelib/spark/pom.xml 3fe9c71 src/main/assemblies/sharelib.xml ea95c2e Diff: https://reviews.apache.org/r/35488/diff/ Testing --- Basic testing done. Deployed spark job on yarn, standalone cluster and local mode . It is running successfully in all these modes. Thanks, pavan kumar kolamuri
