[jira] [Commented] (SPARK-22202) Release tgz content differences for python and R
[ https://issues.apache.org/jira/browse/SPARK-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194141#comment-16194141 ] Felix Cheung commented on SPARK-22202: -- [~holden.ka...@gmail.com] actually, I think for R we would go the other way - we would want to include what's in hadoop2.6 only in all other release profiles (ie. run *this* then create tgz) so I think the approaches are potentially opposite for R and python. > Release tgz content differences for python and R > > > Key: SPARK-22202 > URL: https://issues.apache.org/jira/browse/SPARK-22202 > Project: Spark > Issue Type: Bug > Components: PySpark, SparkR >Affects Versions: 2.1.2, 2.2.1, 2.3.0 >Reporter: Felix Cheung >Priority: Minor > > As a follow up to SPARK-22167, currently we are running different > profiles/steps in make-release.sh for hadoop2.7 vs hadoop2.6 (and others), we > should consider if these differences are significant and whether they should > be addressed. > A couple of things: > - R.../doc directory is not in any release jar except hadoop 2.6 > - python/dist, python.egg-info are not in any release jar except hadoop 2.7 > - R DESCRIPTION has a few additions > I've checked to confirm these are the same in 2.1.1 release so this isn't a > regression. > {code} > spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/doc: > sparkr-vignettes.Rmd > sparkr-vignettes.R > sparkr-vignettes.html > index.html > Only in spark-2.1.2-bin-hadoop2.7/python: dist > Only in spark-2.1.2-bin-hadoop2.7/python/pyspark: python > Only in spark-2.1.2-bin-hadoop2.7/python: pyspark.egg-info > diff -r spark-2.1.2-bin-hadoop2.7/R/lib/SparkR/DESCRIPTION > spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/DESCRIPTION > 25a26,27 > > NeedsCompilation: no > > Packaged: 2017-10-03 00:42:30 UTC; holden > 31c33 > < Built: R 3.4.1; ; 2017-10-02 23:18:21 UTC; unix > --- > > Built: R 3.4.1; ; 2017-10-03 00:45:27 UTC; unix > Only in spark-2.1.2-bin-hadoop2.6/R/lib/SparkR: doc > diff -r spark-2.1.2-bin-hadoop2.7/R/lib/SparkR/html/00Index.html > spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/html/00Index.html > 16a17 > > User guides, package vignettes and other > > documentation. > Only in spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/Meta: vignette.rds > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-22202) Release tgz content differences for python and R
[ https://issues.apache.org/jira/browse/SPARK-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193247#comment-16193247 ] holdenk commented on SPARK-22202: - [~felixcheung] for Python I think it would not be bad to be consistent, but I'd probably put it at a trivial rather than major level personally. The fix could be the same for both (e.g. create tgz's _then_ run python/r packaging) so I think keeping it together is fine. > Release tgz content differences for python and R > > > Key: SPARK-22202 > URL: https://issues.apache.org/jira/browse/SPARK-22202 > Project: Spark > Issue Type: Bug > Components: PySpark, SparkR >Affects Versions: 2.1.2, 2.2.1, 2.3.0 >Reporter: Felix Cheung > > As a follow up to SPARK-22167, currently we are running different > profiles/steps in make-release.sh for hadoop2.7 vs hadoop2.6 (and others), we > should consider if these differences are significant and whether they should > be addressed. > A couple of things: > - R.../doc directory is not in any release jar except hadoop 2.6 > - python/dist, python.egg-info are not in any release jar except hadoop 2.7 > - R DESCRIPTION has a few additions > I've checked to confirm these are the same in 2.1.1 release so this isn't a > regression. > {code} > spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/doc: > sparkr-vignettes.Rmd > sparkr-vignettes.R > sparkr-vignettes.html > index.html > Only in spark-2.1.2-bin-hadoop2.7/python: dist > Only in spark-2.1.2-bin-hadoop2.7/python/pyspark: python > Only in spark-2.1.2-bin-hadoop2.7/python: pyspark.egg-info > diff -r spark-2.1.2-bin-hadoop2.7/R/lib/SparkR/DESCRIPTION > spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/DESCRIPTION > 25a26,27 > > NeedsCompilation: no > > Packaged: 2017-10-03 00:42:30 UTC; holden > 31c33 > < Built: R 3.4.1; ; 2017-10-02 23:18:21 UTC; unix > --- > > Built: R 3.4.1; ; 2017-10-03 00:45:27 UTC; unix > Only in spark-2.1.2-bin-hadoop2.6/R/lib/SparkR: doc > diff -r spark-2.1.2-bin-hadoop2.7/R/lib/SparkR/html/00Index.html > spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/html/00Index.html > 16a17 > > User guides, package vignettes and other > > documentation. > Only in spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/Meta: vignette.rds > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-22202) Release tgz content differences for python and R
[ https://issues.apache.org/jira/browse/SPARK-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193239#comment-16193239 ] Felix Cheung commented on SPARK-22202: -- [~holden.ka...@gmail.com] would you be concerned with the python differences? if not, I'll turn this into just for R. > Release tgz content differences for python and R > > > Key: SPARK-22202 > URL: https://issues.apache.org/jira/browse/SPARK-22202 > Project: Spark > Issue Type: Bug > Components: PySpark, SparkR >Affects Versions: 2.1.2, 2.2.1, 2.3.0 >Reporter: Felix Cheung > > As a follow up to SPARK-22167, currently we are running different > profiles/steps in make-release.sh for hadoop2.7 vs hadoop2.6 (and others), we > should consider if these differences are significant and whether they should > be addressed. > A couple of things: > - R.../doc directory is not in any release jar except hadoop 2.6 > - python/dist, python.egg-info are not in any release jar except hadoop 2.7 > - R DESCRIPTION has a few additions > I've checked to confirm these are the same in 2.1.1 release so this isn't a > regression. > {code} > spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/doc: > sparkr-vignettes.Rmd > sparkr-vignettes.R > sparkr-vignettes.html > index.html > Only in spark-2.1.2-bin-hadoop2.7/python: dist > Only in spark-2.1.2-bin-hadoop2.7/python/pyspark: python > Only in spark-2.1.2-bin-hadoop2.7/python: pyspark.egg-info > diff -r spark-2.1.2-bin-hadoop2.7/R/lib/SparkR/DESCRIPTION > spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/DESCRIPTION > 25a26,27 > > NeedsCompilation: no > > Packaged: 2017-10-03 00:42:30 UTC; holden > 31c33 > < Built: R 3.4.1; ; 2017-10-02 23:18:21 UTC; unix > --- > > Built: R 3.4.1; ; 2017-10-03 00:45:27 UTC; unix > Only in spark-2.1.2-bin-hadoop2.6/R/lib/SparkR: doc > diff -r spark-2.1.2-bin-hadoop2.7/R/lib/SparkR/html/00Index.html > spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/html/00Index.html > 16a17 > > User guides, package vignettes and other > > documentation. > Only in spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/Meta: vignette.rds > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-22202) Release tgz content differences for python and R
[ https://issues.apache.org/jira/browse/SPARK-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193238#comment-16193238 ] Felix Cheung commented on SPARK-22202: -- Yes, exactly. > Release tgz content differences for python and R > > > Key: SPARK-22202 > URL: https://issues.apache.org/jira/browse/SPARK-22202 > Project: Spark > Issue Type: Bug > Components: PySpark, SparkR >Affects Versions: 2.1.2, 2.2.1, 2.3.0 >Reporter: Felix Cheung > > As a follow up to SPARK-22167, currently we are running different > profiles/steps in make-release.sh for hadoop2.7 vs hadoop2.6 (and others), we > should consider if these differences are significant and whether they should > be addressed. > A couple of things: > - R.../doc directory is not in any release jar except hadoop 2.6 > - python/dist, python.egg-info are not in any release jar except hadoop 2.7 > - R DESCRIPTION has a few additions > I've checked to confirm these are the same in 2.1.1 release so this isn't a > regression. > {code} > spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/doc: > sparkr-vignettes.Rmd > sparkr-vignettes.R > sparkr-vignettes.html > index.html > Only in spark-2.1.2-bin-hadoop2.7/python: dist > Only in spark-2.1.2-bin-hadoop2.7/python/pyspark: python > Only in spark-2.1.2-bin-hadoop2.7/python: pyspark.egg-info > diff -r spark-2.1.2-bin-hadoop2.7/R/lib/SparkR/DESCRIPTION > spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/DESCRIPTION > 25a26,27 > > NeedsCompilation: no > > Packaged: 2017-10-03 00:42:30 UTC; holden > 31c33 > < Built: R 3.4.1; ; 2017-10-02 23:18:21 UTC; unix > --- > > Built: R 3.4.1; ; 2017-10-03 00:45:27 UTC; unix > Only in spark-2.1.2-bin-hadoop2.6/R/lib/SparkR: doc > diff -r spark-2.1.2-bin-hadoop2.7/R/lib/SparkR/html/00Index.html > spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/html/00Index.html > 16a17 > > User guides, package vignettes and other > > documentation. > Only in spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/Meta: vignette.rds > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-22202) Release tgz content differences for python and R
[ https://issues.apache.org/jira/browse/SPARK-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193111#comment-16193111 ] Shivaram Venkataraman commented on SPARK-22202: --- I think the differences happen because we build the CRAN package from one of the Hadoop versions ? > Release tgz content differences for python and R > > > Key: SPARK-22202 > URL: https://issues.apache.org/jira/browse/SPARK-22202 > Project: Spark > Issue Type: Bug > Components: PySpark, SparkR >Affects Versions: 2.1.2, 2.2.1, 2.3.0 >Reporter: Felix Cheung > > As a follow up to SPARK-22167, currently we are running different > profiles/steps in make-release.sh for hadoop2.7 vs hadoop2.6 (and others), we > should consider if these differences are significant and whether they should > be addressed. > A couple of things: > - R.../doc directory is not in any release jar except hadoop 2.6 > - python/dist, python.egg-info are not in any release jar except hadoop 2.7 > - R DESCRIPTION has a few additions > I've checked to confirm these are the same in 2.1.1 release so this isn't a > regression. > {code} > spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/doc: > sparkr-vignettes.Rmd > sparkr-vignettes.R > sparkr-vignettes.html > index.html > Only in spark-2.1.2-bin-hadoop2.7/python: dist > Only in spark-2.1.2-bin-hadoop2.7/python/pyspark: python > Only in spark-2.1.2-bin-hadoop2.7/python: pyspark.egg-info > diff -r spark-2.1.2-bin-hadoop2.7/R/lib/SparkR/DESCRIPTION > spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/DESCRIPTION > 25a26,27 > > NeedsCompilation: no > > Packaged: 2017-10-03 00:42:30 UTC; holden > 31c33 > < Built: R 3.4.1; ; 2017-10-02 23:18:21 UTC; unix > --- > > Built: R 3.4.1; ; 2017-10-03 00:45:27 UTC; unix > Only in spark-2.1.2-bin-hadoop2.6/R/lib/SparkR: doc > diff -r spark-2.1.2-bin-hadoop2.7/R/lib/SparkR/html/00Index.html > spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/html/00Index.html > 16a17 > > User guides, package vignettes and other > > documentation. > Only in spark-2.1.2-bin-hadoop2.6/R/lib/SparkR/Meta: vignette.rds > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org