Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode

[Shwetha GS]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35488/#review91456
---

Ship it!


Ship It!

- Shwetha GS


On June 24, 2015, 7:01 a.m., pavan kumar kolamuri wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35488/
> ---
> 
> (Updated June 24, 2015, 7:01 a.m.)
> 
> 
> Review request for oozie and Srikanth Sundarrajan.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Spark job launched through spark action is not running as expected in 
> standalone mode and  it is not running in yarn mode
> 
> 
> Diffs
> -
> 
>   pom.xml ee26366 
>   sharelib/spark/pom.xml 3fe9c71 
>   src/main/assemblies/sharelib.xml ea95c2e 
> 
> Diff: https://reviews.apache.org/r/35488/diff/
> 
> 
> Testing
> ---
> 
> Basic testing done. Deployed spark job on yarn, standalone cluster and local 
> mode . It is running successfully in all these modes.
> 
> 
> Thanks,
> 
> pavan kumar kolamuri
> 
>

Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode

[pavan kumar kolamuri]

> On June 23, 2015, 7:09 a.m., pavan kumar kolamuri wrote:
> >

Thanks shwetha done please check now


- pavan kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35488/#review88927
---


On June 24, 2015, 7:01 a.m., pavan kumar kolamuri wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35488/
> ---
> 
> (Updated June 24, 2015, 7:01 a.m.)
> 
> 
> Review request for oozie and Srikanth Sundarrajan.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Spark job launched through spark action is not running as expected in 
> standalone mode and  it is not running in yarn mode
> 
> 
> Diffs
> -
> 
>   pom.xml ee26366 
>   sharelib/spark/pom.xml 3fe9c71 
>   src/main/assemblies/sharelib.xml ea95c2e 
> 
> Diff: https://reviews.apache.org/r/35488/diff/
> 
> 
> Testing
> ---
> 
> Basic testing done. Deployed spark job on yarn, standalone cluster and local 
> mode . It is running successfully in all these modes.
> 
> 
> Thanks,
> 
> pavan kumar kolamuri
> 
>

Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode

[pavan kumar kolamuri]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35488/
---

(Updated June 24, 2015, 7:01 a.m.)


Review request for oozie and Srikanth Sundarrajan.


Changes
---

Added dependencies versions and exclusions in parent pom


Repository: oozie-git


Description
---

Spark job launched through spark action is not running as expected in 
standalone mode and  it is not running in yarn mode


Diffs (updated)
-

  pom.xml ee26366 
  sharelib/spark/pom.xml 3fe9c71 
  src/main/assemblies/sharelib.xml ea95c2e 

Diff: https://reviews.apache.org/r/35488/diff/


Testing
---

Basic testing done. Deployed spark job on yarn, standalone cluster and local 
mode . It is running successfully in all these modes.


Thanks,

pavan kumar kolamuri

Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode

[Shwetha GS]

> On June 23, 2015, 7:09 a.m., pavan kumar kolamuri wrote:
> > sharelib/spark/pom.xml, line 111
> > 
> >
> > Yes i agree with that, it is same across all other actions as well. 
> > Refactoring is required for other actions as well , we can raise a jira and 
> > fix all this dependency management issues

Atleast the versions need to move to dependency management - just like other 
actions


- Shwetha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35488/#review88927
---


On June 16, 2015, 5:09 a.m., pavan kumar kolamuri wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35488/
> ---
> 
> (Updated June 16, 2015, 5:09 a.m.)
> 
> 
> Review request for oozie and Srikanth Sundarrajan.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Spark job launched through spark action is not running as expected in 
> standalone mode and  it is not running in yarn mode
> 
> 
> Diffs
> -
> 
>   pom.xml ee26366 
>   sharelib/spark/pom.xml 3fe9c71 
>   src/main/assemblies/sharelib.xml ea95c2e 
> 
> Diff: https://reviews.apache.org/r/35488/diff/
> 
> 
> Testing
> ---
> 
> Basic testing done. Deployed spark job on yarn, standalone cluster and local 
> mode . It is running successfully in all these modes.
> 
> 
> Thanks,
> 
> pavan kumar kolamuri
> 
>

Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode

[pavan kumar kolamuri]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35488/#review88927
---



sharelib/spark/pom.xml (line 107)


Yes i agree with that, it is same across all other actions as well. 
Refactoring is required for other actions as well , we can raise a jira and fix 
all this dependency management issues



sharelib/spark/pom.xml (line 224)


There is an issue in launching application master of spark uber jar is not 
used. While launching an application master of spark it is only adding spark 
jar containing a class SparkYarn Launcher. we tried to find an alternate way , 
we couldn't able to find. Even for running spark jobs spark assembly jar is 
always used.


- pavan kumar kolamuri


On June 16, 2015, 5:09 a.m., pavan kumar kolamuri wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35488/
> ---
> 
> (Updated June 16, 2015, 5:09 a.m.)
> 
> 
> Review request for oozie and Srikanth Sundarrajan.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Spark job launched through spark action is not running as expected in 
> standalone mode and  it is not running in yarn mode
> 
> 
> Diffs
> -
> 
>   pom.xml ee26366 
>   sharelib/spark/pom.xml 3fe9c71 
>   src/main/assemblies/sharelib.xml ea95c2e 
> 
> Diff: https://reviews.apache.org/r/35488/diff/
> 
> 
> Testing
> ---
> 
> Basic testing done. Deployed spark job on yarn, standalone cluster and local 
> mode . It is running successfully in all these modes.
> 
> 
> Thanks,
> 
> pavan kumar kolamuri
> 
>

Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode

[Shwetha GS]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35488/#review88923
---



sharelib/spark/pom.xml (line 107)


can you move the exclusions to parent pom?



sharelib/spark/pom.xml (line 224)


why are you creating uber jar? uber jars are difficult to debug - can't 
figure out what jars and classes are added


- Shwetha GS


On June 16, 2015, 5:09 a.m., pavan kumar kolamuri wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35488/
> ---
> 
> (Updated June 16, 2015, 5:09 a.m.)
> 
> 
> Review request for oozie and Srikanth Sundarrajan.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Spark job launched through spark action is not running as expected in 
> standalone mode and  it is not running in yarn mode
> 
> 
> Diffs
> -
> 
>   pom.xml ee26366 
>   sharelib/spark/pom.xml 3fe9c71 
>   src/main/assemblies/sharelib.xml ea95c2e 
> 
> Diff: https://reviews.apache.org/r/35488/diff/
> 
> 
> Testing
> ---
> 
> Basic testing done. Deployed spark job on yarn, standalone cluster and local 
> mode . It is running successfully in all these modes.
> 
> 
> Thanks,
> 
> pavan kumar kolamuri
> 
>

Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode

[Jaydeep Vishwakarma]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35488/#review88831
---

Ship it!


Ship It!

- Jaydeep Vishwakarma


On June 16, 2015, 5:09 a.m., pavan kumar kolamuri wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35488/
> ---
> 
> (Updated June 16, 2015, 5:09 a.m.)
> 
> 
> Review request for oozie and Srikanth Sundarrajan.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Spark job launched through spark action is not running as expected in 
> standalone mode and  it is not running in yarn mode
> 
> 
> Diffs
> -
> 
>   pom.xml ee26366 
>   sharelib/spark/pom.xml 3fe9c71 
>   src/main/assemblies/sharelib.xml ea95c2e 
> 
> Diff: https://reviews.apache.org/r/35488/diff/
> 
> 
> Testing
> ---
> 
> Basic testing done. Deployed spark job on yarn, standalone cluster and local 
> mode . It is running successfully in all these modes.
> 
> 
> Thanks,
> 
> pavan kumar kolamuri
> 
>

Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode

[Srikanth Sundarrajan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35488/#review88030
---

Ship it!


Ship It!

- Srikanth Sundarrajan


On June 16, 2015, 5:09 a.m., pavan kumar kolamuri wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35488/
> ---
> 
> (Updated June 16, 2015, 5:09 a.m.)
> 
> 
> Review request for oozie and Srikanth Sundarrajan.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Spark job launched through spark action is not running as expected in 
> standalone mode and  it is not running in yarn mode
> 
> 
> Diffs
> -
> 
>   pom.xml ee26366 
>   sharelib/spark/pom.xml 3fe9c71 
>   src/main/assemblies/sharelib.xml ea95c2e 
> 
> Diff: https://reviews.apache.org/r/35488/diff/
> 
> 
> Testing
> ---
> 
> Basic testing done. Deployed spark job on yarn, standalone cluster and local 
> mode . It is running successfully in all these modes.
> 
> 
> Thanks,
> 
> pavan kumar kolamuri
> 
>