[jira] [Commented] (IGNITE-7816) DataFrame examples: wrong folder, hidden on TeamCity

[Denis Magda (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386631#comment-16386631
 ] 

Denis Magda commented on IGNITE-7816:
-

Guys, guess this ticket slips to 2.5 release. Changed the version. Roll it back 
to 2.4 if the changes made their way into the current release.

> DataFrame examples: wrong folder, hidden on TeamCity
> 
>
> Key: IGNITE-7816
> URL: https://issues.apache.org/jira/browse/IGNITE-7816
> Project: Ignite
>  Issue Type: Sub-task
>  Components: documentation, examples
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Trivial
> Fix For: 2.5
>
>
> 1. Spark DataFrame examples are in 
> {{examples/src/main/spark/org/apache/ignite/examples/spark}} folder. Should 
> be moved to 
> {{examples/src/main/scala/org/apache/ignite/scalar/examples/spark}}.
> 2. Spark DataFrame examples not tested on Team City under Ignite Examples. 
> https://issues.apache.org/jira/browse/IGNITE-7655?focusedCommentId=16377875=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16377875



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7816) DataFrame examples: wrong folder, hidden on TeamCity

[Nikolay Izhikov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381790#comment-16381790
 ] 

Nikolay Izhikov commented on IGNITE-7816:
-

[~vveider]

>  Where to look at dependencies?

1. {{spark}} - 
https://github.com/apache/ignite/blob/master/modules/spark/pom.xml#L144
2. {{spark_2.10}} - 
https://github.com/apache/ignite/blob/master/modules/spark-2.10/pom.xml#L150
3. spark examples - 
https://github.com/apache/ignite/pull/3590/files#diff-08740066c64337d38cccd84991ac0912R155

The long list of dependencies with `test` scope is introduced because of 
flatten plugin.
We don't need it without flatten plugin.

> DataFrame examples: wrong folder, hidden on TeamCity
> 
>
> Key: IGNITE-7816
> URL: https://issues.apache.org/jira/browse/IGNITE-7816
> Project: Ignite
>  Issue Type: Sub-task
>  Components: documentation, examples
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Trivial
> Fix For: 2.4
>
>
> 1. Spark DataFrame examples are in 
> {{examples/src/main/spark/org/apache/ignite/examples/spark}} folder. Should 
> be moved to 
> {{examples/src/main/scala/org/apache/ignite/scalar/examples/spark}}.
> 2. Spark DataFrame examples not tested on Team City under Ignite Examples. 
> https://issues.apache.org/jira/browse/IGNITE-7655?focusedCommentId=16377875=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16377875



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7816) DataFrame examples: wrong folder, hidden on TeamCity

[Dmitriy Pavlov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381689#comment-16381689
 ] 

Dmitriy Pavlov commented on IGNITE-7816:


[~vveider] it is about examples, flatten plugin forces us to specify all 
transitive dependencies. You can look to proposed change in examples pom: 
https://github.com/apache/ignite/pull/3590

> DataFrame examples: wrong folder, hidden on TeamCity
> 
>
> Key: IGNITE-7816
> URL: https://issues.apache.org/jira/browse/IGNITE-7816
> Project: Ignite
>  Issue Type: Sub-task
>  Components: documentation, examples
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Trivial
> Fix For: 2.4
>
>
> 1. Spark DataFrame examples are in 
> {{examples/src/main/spark/org/apache/ignite/examples/spark}} folder. Should 
> be moved to 
> {{examples/src/main/scala/org/apache/ignite/scalar/examples/spark}}.
> 2. Spark DataFrame examples not tested on Team City under Ignite Examples. 
> https://issues.apache.org/jira/browse/IGNITE-7655?focusedCommentId=16377875=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16377875



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7816) DataFrame examples: wrong folder, hidden on TeamCity

[Peter Ivanov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381684#comment-16381684
 ] 

Peter Ivanov commented on IGNITE-7816:
--

What module we are talking about? Where to look at dependencies?

> DataFrame examples: wrong folder, hidden on TeamCity
> 
>
> Key: IGNITE-7816
> URL: https://issues.apache.org/jira/browse/IGNITE-7816
> Project: Ignite
>  Issue Type: Sub-task
>  Components: documentation, examples
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Trivial
> Fix For: 2.4
>
>
> 1. Spark DataFrame examples are in 
> {{examples/src/main/spark/org/apache/ignite/examples/spark}} folder. Should 
> be moved to 
> {{examples/src/main/scala/org/apache/ignite/scalar/examples/spark}}.
> 2. Spark DataFrame examples not tested on Team City under Ignite Examples. 
> https://issues.apache.org/jira/browse/IGNITE-7655?focusedCommentId=16377875=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16377875



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7816) DataFrame examples: wrong folder, hidden on TeamCity

[Nikolay Izhikov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381039#comment-16381039
 ] 

Nikolay Izhikov commented on IGNITE-7816:
-

[~dmagda], [~dpavlov]

>  can't we just import ignite-spark instead of listing all the jars this module

Short answer - no we can't because of flatten plugin from parent pom.

I raised discussion of this issue on dev-lis.
Please, see my mail:

http://apache-ignite-developers.2346864.n4.nabble.com/Maven-Issues-with-flatten-plugin-td27537.html

> DataFrame examples: wrong folder, hidden on TeamCity
> 
>
> Key: IGNITE-7816
> URL: https://issues.apache.org/jira/browse/IGNITE-7816
> Project: Ignite
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Trivial
> Fix For: 2.4
>
>
> 1. Spark DataFrame examples are in 
> {{examples/src/main/spark/org/apache/ignite/examples/spark}} folder. Should 
> be moved to 
> {{examples/src/main/scala/org/apache/ignite/scalar/examples/spark}}.
> 2. Spark DataFrame examples not tested on Team City under Ignite Examples. 
> https://issues.apache.org/jira/browse/IGNITE-7655?focusedCommentId=16377875=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16377875



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7816) DataFrame examples: wrong folder, hidden on TeamCity

[Dmitriy Pavlov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381014#comment-16381014
 ] 

Dmitriy Pavlov commented on IGNITE-7816:


Hi [~dmagda], [~NIzhikov], I've noticed before we need always list all 
dependencies. It seems transitive dependencies are disabled somehow in our 
poms. May be [~vveider] know where and how it can bypassed. Unfortunately, I am 
not maven guru, I prefer gradle :)

I can expects user will need to download tons of jars from maven central to see 
Ignite's examples. [~NIzhikov], is it enabled by default (e.g. under maven by 
profile)?

Dependency hell is always possible, but as far as I can see, spark version here 
is taken from variable, from parent poms. So if some conflict occur, then we 
will need to fix it anyway.

I suggest to create distrubution of Ignite from sorces and test examples 
manually for the first time before merge.

> DataFrame examples: wrong folder, hidden on TeamCity
> 
>
> Key: IGNITE-7816
> URL: https://issues.apache.org/jira/browse/IGNITE-7816
> Project: Ignite
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Trivial
> Fix For: 2.4
>
>
> 1. Spark DataFrame examples are in 
> {{examples/src/main/spark/org/apache/ignite/examples/spark}} folder. Should 
> be moved to 
> {{examples/src/main/scala/org/apache/ignite/scalar/examples/spark}}.
> 2. Spark DataFrame examples not tested on Team City under Ignite Examples. 
> https://issues.apache.org/jira/browse/IGNITE-7655?focusedCommentId=16377875=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16377875



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7816) DataFrame examples: wrong folder, hidden on TeamCity

[Denis Magda (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16380968#comment-16380968
 ] 

Denis Magda commented on IGNITE-7816:
-

[~NIzhikov] , can't we just import {{ignite-spark}} instead of listing all the 
jars this module and your examples depend on one? Also how to deal with the 
situation when different scala or Spark versions are used, wouldn't it cause 
any problems with the examples? [~dpavlov] , please step in.

> DataFrame examples: wrong folder, hidden on TeamCity
> 
>
> Key: IGNITE-7816
> URL: https://issues.apache.org/jira/browse/IGNITE-7816
> Project: Ignite
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Trivial
> Fix For: 2.4
>
>
> 1. Spark DataFrame examples are in 
> {{examples/src/main/spark/org/apache/ignite/examples/spark}} folder. Should 
> be moved to 
> {{examples/src/main/scala/org/apache/ignite/scalar/examples/spark}}.
> 2. Spark DataFrame examples not tested on Team City under Ignite Examples. 
> https://issues.apache.org/jira/browse/IGNITE-7655?focusedCommentId=16377875=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16377875



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7816) DataFrame examples: wrong folder, hidden on TeamCity

[Nikolay Izhikov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16380758#comment-16380758
 ] 

Nikolay Izhikov commented on IGNITE-7816:
-

[~dmagda] 

I've closed my PR by mistake.
So I created a new one - https://github.com/apache/ignite/pull/3590
I fixed pom.xml and test suites to make Spark examples runnable on TC.
Please, take a look.

> DataFrame examples: wrong folder, hidden on TeamCity
> 
>
> Key: IGNITE-7816
> URL: https://issues.apache.org/jira/browse/IGNITE-7816
> Project: Ignite
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Trivial
> Fix For: 2.4
>
>
> 1. Spark DataFrame examples are in 
> {{examples/src/main/spark/org/apache/ignite/examples/spark}} folder. Should 
> be moved to 
> {{examples/src/main/scala/org/apache/ignite/scalar/examples/spark}}.
> 2. Spark DataFrame examples not tested on Team City under Ignite Examples. 
> https://issues.apache.org/jira/browse/IGNITE-7655?focusedCommentId=16377875=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16377875



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7816) DataFrame examples: wrong folder, hidden on TeamCity

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16380737#comment-16380737
 ] 

ASF GitHub Bot commented on IGNITE-7816:


GitHub user nizhikov opened a pull request:

https://github.com/apache/ignite/pull/3590

IGNITE-7816: Update profile to execute spark examples on TC



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nizhikov/ignite IGNITE-7816

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3590.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3590






> DataFrame examples: wrong folder, hidden on TeamCity
> 
>
> Key: IGNITE-7816
> URL: https://issues.apache.org/jira/browse/IGNITE-7816
> Project: Ignite
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Trivial
> Fix For: 2.4
>
>
> 1. Spark DataFrame examples are in 
> {{examples/src/main/spark/org/apache/ignite/examples/spark}} folder. Should 
> be moved to 
> {{examples/src/main/scala/org/apache/ignite/scalar/examples/spark}}.
> 2. Spark DataFrame examples not tested on Team City under Ignite Examples. 
> https://issues.apache.org/jira/browse/IGNITE-7655?focusedCommentId=16377875=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16377875



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7816) DataFrame examples: wrong folder, hidden on TeamCity

[Nikolay Izhikov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379910#comment-16379910
 ] 

Nikolay Izhikov commented on IGNITE-7816:
-

According to Valentin Kulichenk mail scalar folder is not related to Spark 
integration.

http://apache-ignite-developers.2346864.n4.nabble.com/TeamCity-DataFrame-Examples-tp27452p27484.html

> DataFrame examples: wrong folder, hidden on TeamCity
> 
>
> Key: IGNITE-7816
> URL: https://issues.apache.org/jira/browse/IGNITE-7816
> Project: Ignite
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Trivial
> Fix For: 2.4
>
>
> 1. Spark DataFrame examples are in 
> {{examples/src/main/spark/org/apache/ignite/examples/spark}} folder. Should 
> be moved to 
> {{examples/src/main/scala/org/apache/ignite/scalar/examples/spark}}.
> 2. Spark DataFrame examples not tested on Team City under Ignite Examples. 
> https://issues.apache.org/jira/browse/IGNITE-7655?focusedCommentId=16377875=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16377875



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7816) DataFrame examples: wrong folder, hidden on TeamCity

[Denis Magda (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379762#comment-16379762
 ] 

Denis Magda commented on IGNITE-7816:
-

[~NIzhikov] , looks good to me but I'm not sure the community will allow 
merging the changes into 2.4. Please bring up this question on @dev.

> DataFrame examples: wrong folder, hidden on TeamCity
> 
>
> Key: IGNITE-7816
> URL: https://issues.apache.org/jira/browse/IGNITE-7816
> Project: Ignite
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Trivial
> Fix For: 2.4
>
>
> 1. Spark DataFrame examples are in 
> {{examples/src/main/spark/org/apache/ignite/examples/spark}} folder. Should 
> be moved to 
> {{examples/src/main/scala/org/apache/ignite/scalar/examples/spark}}.
> 2. Spark DataFrame examples not tested on Team City under Ignite Examples. 
> https://issues.apache.org/jira/browse/IGNITE-7655?focusedCommentId=16377875=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16377875



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7816) DataFrame examples: wrong folder, hidden on TeamCity

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379188#comment-16379188
 ] 

ASF GitHub Bot commented on IGNITE-7816:


Github user nizhikov closed the pull request at:

https://github.com/apache/ignite/pull/3573


> DataFrame examples: wrong folder, hidden on TeamCity
> 
>
> Key: IGNITE-7816
> URL: https://issues.apache.org/jira/browse/IGNITE-7816
> Project: Ignite
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Trivial
> Fix For: 2.4
>
>
> 1. Spark DataFrame examples are in 
> {{examples/src/main/spark/org/apache/ignite/examples/spark}} folder. Should 
> be moved to 
> {{examples/src/main/scala/org/apache/ignite/scalar/examples/spark}}.
> 2. Spark DataFrame examples not tested on Team City under Ignite Examples. 
> https://issues.apache.org/jira/browse/IGNITE-7655?focusedCommentId=16377875=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16377875



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7816) DataFrame examples: wrong folder, hidden on TeamCity

[Nikolay Izhikov (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378394#comment-16378394
 ] 

Nikolay Izhikov commented on IGNITE-7816:
-

[~dmagda] I've moved DataFrame examples to 
{{org.apache.ignite.scalar.examples.spark}} package. 
Please, take a look at my changes.

> DataFrame examples: wrong folder, hidden on TeamCity
> 
>
> Key: IGNITE-7816
> URL: https://issues.apache.org/jira/browse/IGNITE-7816
> Project: Ignite
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Trivial
> Fix For: 2.4
>
>
> 1. Spark DataFrame examples are in 
> {{examples/src/main/spark/org/apache/ignite/examples/spark}} folder. Should 
> be moved to 
> {{examples/src/main/scala/org/apache/ignite/scalar/examples/spark}}.
> 2. Spark DataFrame examples not tested on Team City under Ignite Examples. 
> https://issues.apache.org/jira/browse/IGNITE-7655?focusedCommentId=16377875=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16377875



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-7816) DataFrame examples: wrong folder, hidden on TeamCity

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378355#comment-16378355
 ] 

ASF GitHub Bot commented on IGNITE-7816:


GitHub user nizhikov opened a pull request:

https://github.com/apache/ignite/pull/3573

IGNITE-7816: DataFrame examples fixed.

 moved to `org.apache.ignite.scalar.examples.spark` package. Minor examples 
improvements.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nizhikov/ignite IGNITE-7816

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3573.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3573


commit 7c89b4b2d1bc03c0790862826d83777ba31fe0df
Author: Nikolay Izhikov 
Date:   2018-02-27T10:26:39Z

IGNITE-7816: DataFrame examples moved to 
`org.apache.ignite.scalar.examples.spark` package. Minor examples improvements.




> DataFrame examples: wrong folder, hidden on TeamCity
> 
>
> Key: IGNITE-7816
> URL: https://issues.apache.org/jira/browse/IGNITE-7816
> Project: Ignite
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Trivial
> Fix For: 2.4
>
>
> 1. Spark DataFrame examples are in 
> {{examples/src/main/spark/org/apache/ignite/examples/spark}} folder. Should 
> be moved to 
> {{examples/src/main/scala/org/apache/ignite/scalar/examples/spark}}.
> 2. Spark DataFrame examples not tested on Team City under Ignite Examples. 
> https://issues.apache.org/jira/browse/IGNITE-7655?focusedCommentId=16377875=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16377875



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)