[jira] [Commented] (SPARK-19511) insert into table does not work on second session of beeline

2017-07-26 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102804#comment-16102804 ] xinzhang commented on SPARK-19511: -- [~chenerlu] hi it always appear . which scene does i

[jira] [Commented] (SPARK-21538) Attribute resolution inconsistency in Dataset API

2017-07-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102782#comment-16102782 ] Xiao Li commented on SPARK-21538: - https://github.com/apache/spark/pull/18740 > Attribut

[jira] [Commented] (SPARK-11083) insert overwrite table failed when beeline reconnect

2017-07-26 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102775#comment-16102775 ] xinzhang commented on SPARK-11083: -- reappeared in Spark 2.1.0. any one working on this

[jira] [Commented] (SPARK-21543) Should not count executor initialize failed towards task failures

2017-07-26 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102767#comment-16102767 ] zhoukang commented on SPARK-21543: -- I have created a pr https://github.com/apache/spark/

[jira] [Commented] (SPARK-21544) Test jar of some module should not install or deploy twice

2017-07-26 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102766#comment-16102766 ] zhoukang commented on SPARK-21544: -- I have create a pr: https://github.com/apache/spark/

[jira] [Updated] (SPARK-21545) pyspark2

2017-07-26 Thread gumpcheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gumpcheng updated SPARK-21545: -- Description: I install spark2.2 following the official steps with CDH5.12. Info on Cloudera Manager is

[jira] [Updated] (SPARK-21545) pyspark2

2017-07-26 Thread gumpcheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gumpcheng updated SPARK-21545: -- Description: I install spark2.2 following the official steps with CDH5.12. Info on Cloudera Manager is

[jira] [Created] (SPARK-21545) pyspark2

2017-07-26 Thread gumpcheng (JIRA)
gumpcheng created SPARK-21545: - Summary: pyspark2 Key: SPARK-21545 URL: https://issues.apache.org/jira/browse/SPARK-21545 Project: Spark Issue Type: Bug Components: PySpark Affects

[jira] [Resolved] (SPARK-21530) Update description of spark.shuffle.maxChunksBeingTransferred

2017-07-26 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21530. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18735 [https://githu

[jira] [Assigned] (SPARK-21530) Update description of spark.shuffle.maxChunksBeingTransferred

2017-07-26 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21530: --- Assignee: jin xing > Update description of spark.shuffle.maxChunksBeingTransferred > ---

[jira] [Closed] (SPARK-21400) Spark shouldn't ignore user defined output committer in append mode

2017-07-26 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski closed SPARK-21400. - Resolution: Fixed Fix Version/s: 2.3.0 > Spark shouldn't ignore user defined outpu

[jira] [Created] (SPARK-21544) Test jar of some module should not install or deploy twice

2017-07-26 Thread zhoukang (JIRA)
zhoukang created SPARK-21544: Summary: Test jar of some module should not install or deploy twice Key: SPARK-21544 URL: https://issues.apache.org/jira/browse/SPARK-21544 Project: Spark Issue Type

[jira] [Commented] (SPARK-21400) Spark shouldn't ignore user defined output committer in append mode

2017-07-26 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102638#comment-16102638 ] Robert Kruszewski commented on SPARK-21400: --- Fixed in [https://github.com/apach

[jira] [Created] (SPARK-21543) Should not count executor initialize failed towards task failures

2017-07-26 Thread zhoukang (JIRA)
zhoukang created SPARK-21543: Summary: Should not count executor initialize failed towards task failures Key: SPARK-21543 URL: https://issues.apache.org/jira/browse/SPARK-21543 Project: Spark Is

[jira] [Comment Edited] (SPARK-21539) Job should not be aborted when dynamic allocation is enabled or spark.executor.instances larger then current allocated number by yarn

2017-07-26 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101686#comment-16101686 ] zhoukang edited comment on SPARK-21539 at 7/27/17 2:06 AM: --- I w

[jira] [Updated] (SPARK-21539) Job should not be aborted when dynamic allocation is enabled or spark.executor.instances larger then current allocated number by yarn

2017-07-26 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang updated SPARK-21539: - Affects Version/s: (was: 2.2.0) > Job should not be aborted when dynamic allocation is enabled or >

[jira] [Reopened] (SPARK-21539) Job should not be aborted when dynamic allocation is enabled or spark.executor.instances larger then current allocated number by yarn

2017-07-26 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang reopened SPARK-21539: -- > Job should not be aborted when dynamic allocation is enabled or > spark.executor.instances larger then c

[jira] [Commented] (SPARK-21533) "configure(...)" method not called when using Hive Generic UDFs

2017-07-26 Thread Feng Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102598#comment-16102598 ] Feng Zhu commented on SPARK-21533: -- Could you post any examples? > "configure(...)" met

[jira] [Closed] (SPARK-21539) Job should not be aborted when dynamic allocation is enabled or spark.executor.instances larger then current allocated number by yarn

2017-07-26 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang closed SPARK-21539. Resolution: Not A Problem > Job should not be aborted when dynamic allocation is enabled or > spark.execut

[jira] [Resolved] (SPARK-21540) add spark.sql.functions.map_keys and spark.sql.functions.map_values

2017-07-26 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21540. -- Resolution: Duplicate I guess this was added in SPARK-19975. > add spark.sql.functions.map_key

[jira] [Updated] (SPARK-21542) Helper functions for custom Python Persistence

2017-07-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-21542: -- Description: Currently, there is no way to easily persist Json-serializable parameters

[jira] [Updated] (SPARK-21542) Helper functions for custom Python Persistence

2017-07-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-21542: -- Component/s: ML > Helper functions for custom Python Persistence >

[jira] [Updated] (SPARK-21542) Helper functions for custom Python Persistence

2017-07-26 Thread Ajay Saini (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Saini updated SPARK-21542: --- Component/s: (was: ML) > Helper functions for custom Python Persistence > ---

[jira] [Created] (SPARK-21542) Helper functions for custom Python Persistence

2017-07-26 Thread Ajay Saini (JIRA)
Ajay Saini created SPARK-21542: -- Summary: Helper functions for custom Python Persistence Key: SPARK-21542 URL: https://issues.apache.org/jira/browse/SPARK-21542 Project: Spark Issue Type: New Fe

[jira] [Commented] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-07-26 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102420#comment-16102420 ] Bryan Cutler commented on SPARK-21190: -- Hi [~icexelloss], yes I think there is defin

[jira] [Updated] (SPARK-21541) Spark Logs show incorrect job status for a job that does not create SparkContext

2017-07-26 Thread Parth Gandhi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Gandhi updated SPARK-21541: - Description: If you run a spark job without creating the SparkSession or SparkContext, the spark

[jira] [Commented] (SPARK-21541) Spark Logs show incorrect job status for a job that does not create SparkContext

2017-07-26 Thread Parth Gandhi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102308#comment-16102308 ] Parth Gandhi commented on SPARK-21541: -- Currently working on the fix, will file a pu

[jira] [Created] (SPARK-21541) Spark Logs show incorrect job status for a job that does not create SparkContext

2017-07-26 Thread Parth Gandhi (JIRA)
Parth Gandhi created SPARK-21541: Summary: Spark Logs show incorrect job status for a job that does not create SparkContext Key: SPARK-21541 URL: https://issues.apache.org/jira/browse/SPARK-21541 Proj

[jira] [Commented] (SPARK-20418) multi-label classification support

2017-07-26 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102280#comment-16102280 ] Weichen Xu commented on SPARK-20418: I will work on this. > multi-label classificati

[jira] [Commented] (SPARK-11215) Add multiple columns support to StringIndexer

2017-07-26 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102274#comment-16102274 ] Weichen Xu commented on SPARK-11215: I will take over this feature and create a PR so

[jira] [Comment Edited] (SPARK-21535) Reduce memory requirement for CrossValidator and TrainValidationSplit

2017-07-26 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16100860#comment-16100860 ] yuhao yang edited comment on SPARK-21535 at 7/26/17 6:30 PM: -

[jira] [Commented] (SPARK-21087) CrossValidator, TrainValidationSplit should preserve all models after fitting: Scala

2017-07-26 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102048#comment-16102048 ] Weichen Xu commented on SPARK-21087: I will work on it. > CrossValidator, TrainValid

[jira] [Commented] (SPARK-17025) Cannot persist PySpark ML Pipeline model that includes custom Transformer

2017-07-26 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101994#comment-16101994 ] Weichen Xu commented on SPARK-17025: Because currently, scala calling python will be

[jira] [Comment Edited] (SPARK-17025) Cannot persist PySpark ML Pipeline model that includes custom Transformer

2017-07-26 Thread Ajay Saini (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101984#comment-16101984 ] Ajay Saini edited comment on SPARK-17025 at 7/26/17 5:40 PM: -

[jira] [Commented] (SPARK-17025) Cannot persist PySpark ML Pipeline model that includes custom Transformer

2017-07-26 Thread Ajay Saini (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101984#comment-16101984 ] Ajay Saini commented on SPARK-17025: I'm currently working on a solution to this tha

[jira] [Resolved] (SPARK-21485) API Documentation for Spark SQL functions

2017-07-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-21485. - Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.3.0 > API Documentation

[jira] [Commented] (SPARK-6809) Make numPartitions optional in pairRDD APIs

2017-07-26 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101900#comment-16101900 ] Shivaram Venkataraman commented on SPARK-6809: -- Yeah I dont this JIRA is appl

[jira] [Resolved] (SPARK-6809) Make numPartitions optional in pairRDD APIs

2017-07-26 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-6809. -- Resolution: Not A Problem > Make numPartitions optional in pairRDD APIs > --

[jira] [Created] (SPARK-21540) add spark.sql.functions.map_keys and spark.sql.functions.map_values

2017-07-26 Thread yu peng (JIRA)
yu peng created SPARK-21540: --- Summary: add spark.sql.functions.map_keys and spark.sql.functions.map_values Key: SPARK-21540 URL: https://issues.apache.org/jira/browse/SPARK-21540 Project: Spark Is

[jira] [Updated] (SPARK-21245) Resolve code duplication for classification/regression summarizers

2017-07-26 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hendrickson updated SPARK-21245: - Labels: starter (was: ) Priority: Minor (was: Major) > Resolve code duplication f

[jira] [Commented] (SPARK-21535) Reduce memory requirement for CrossValidator and TrainValidationSplit

2017-07-26 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101870#comment-16101870 ] yuhao yang commented on SPARK-21535: The basic idea is that we should release the dri

[jira] [Resolved] (SPARK-12957) Derive and propagate data constrains in logical plan

2017-07-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12957. - Resolution: Fixed Fix Version/s: 2.0.0 > Derive and propagate data constrains in logical p

[jira] [Updated] (SPARK-21538) Attribute resolution inconsistency in Dataset API

2017-07-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-21538: Affects Version/s: (was: 3.0.0) 2.3.0 > Attribute resolution inconsistency in Da

[jira] [Updated] (SPARK-21538) Attribute resolution inconsistency in Dataset API

2017-07-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-21538: Issue Type: Improvement (was: Story) > Attribute resolution inconsistency in Dataset API > ---

[jira] [Commented] (SPARK-21539) Job should not be aborted when dynamic allocation is enabled or spark.executor.instances larger then current allocated number by yarn

2017-07-26 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101686#comment-16101686 ] zhoukang commented on SPARK-21539: -- I am working on this. > Job should not be aborted w

[jira] [Updated] (SPARK-21539) Job should not be aborted when dynamic allocation is enabled or spark.executor.instances larger then current allocated number by yarn

2017-07-26 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang updated SPARK-21539: - Description: For spark on yarn. Right now, when TaskSet can not run on any node or host.Which means blac

[jira] [Created] (SPARK-21539) Job should not be aborted when dynamic allocation is enabled or spark.executor.instances larger then current allocated number by yarn

2017-07-26 Thread zhoukang (JIRA)
zhoukang created SPARK-21539: Summary: Job should not be aborted when dynamic allocation is enabled or spark.executor.instances larger then current allocated number by yarn Key: SPARK-21539 URL: https://issues.apache.

[jira] [Comment Edited] (SPARK-9776) Another instance of Derby may have already booted the database

2017-07-26 Thread Quincy HSIEH (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101597#comment-16101597 ] Quincy HSIEH edited comment on SPARK-9776 at 7/26/17 12:10 PM: -

[jira] [Comment Edited] (SPARK-9776) Another instance of Derby may have already booted the database

2017-07-26 Thread Quincy HSIEH (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101597#comment-16101597 ] Quincy HSIEH edited comment on SPARK-9776 at 7/26/17 12:09 PM: -

[jira] [Comment Edited] (SPARK-9776) Another instance of Derby may have already booted the database

2017-07-26 Thread Quincy HSIEH (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101597#comment-16101597 ] Quincy HSIEH edited comment on SPARK-9776 at 7/26/17 12:08 PM: -

[jira] [Comment Edited] (SPARK-9776) Another instance of Derby may have already booted the database

2017-07-26 Thread Quincy HSIEH (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101597#comment-16101597 ] Quincy HSIEH edited comment on SPARK-9776 at 7/26/17 12:07 PM: -

[jira] [Comment Edited] (SPARK-9776) Another instance of Derby may have already booted the database

2017-07-26 Thread Quincy HSIEH (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101597#comment-16101597 ] Quincy HSIEH edited comment on SPARK-9776 at 7/26/17 12:06 PM: -

[jira] [Commented] (SPARK-9776) Another instance of Derby may have already booted the database

2017-07-26 Thread Quincy HSIEH (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101597#comment-16101597 ] Quincy HSIEH commented on SPARK-9776: - Hi, In case that someone who has the same issu

[jira] [Comment Edited] (SPARK-9776) Another instance of Derby may have already booted the database

2017-07-26 Thread Quincy HSIEH (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101597#comment-16101597 ] Quincy HSIEH edited comment on SPARK-9776 at 7/26/17 12:05 PM: -

[jira] [Resolved] (SPARK-20988) Convert logistic regression to new aggregator framework

2017-07-26 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-20988. Resolution: Fixed Fix Version/s: 2.3.0 > Convert logistic regression to new aggregat

[jira] [Assigned] (SPARK-20988) Convert logistic regression to new aggregator framework

2017-07-26 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath reassigned SPARK-20988: -- Assignee: Seth Hendrickson > Convert logistic regression to new aggregator framework >

[jira] [Created] (SPARK-21538) Attribute resolution inconsistency in Dataset API

2017-07-26 Thread Adrian Ionescu (JIRA)
Adrian Ionescu created SPARK-21538: -- Summary: Attribute resolution inconsistency in Dataset API Key: SPARK-21538 URL: https://issues.apache.org/jira/browse/SPARK-21538 Project: Spark Issue T

[jira] [Updated] (SPARK-21537) toPandas() should handle nested columns (as a Pandas MultiIndex)

2017-07-26 Thread Eric O. LEBIGOT (EOL) (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric O. LEBIGOT (EOL) updated SPARK-21537: -- Description: The conversion of a *PySpark dataframe with nested columns* to Pan

[jira] [Created] (SPARK-21537) toPandas() should handle nested columns (as a Pandas MultiIndex)

2017-07-26 Thread Eric O. LEBIGOT (EOL) (JIRA)
Eric O. LEBIGOT (EOL) created SPARK-21537: - Summary: toPandas() should handle nested columns (as a Pandas MultiIndex) Key: SPARK-21537 URL: https://issues.apache.org/jira/browse/SPARK-21537 Pr

[jira] [Commented] (SPARK-21476) RandomForest classification model not using broadcast in transform

2017-07-26 Thread Peng Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101476#comment-16101476 ] Peng Meng commented on SPARK-21476: --- Hi [~sagraw], could you please test copy pasted th

[jira] [Comment Edited] (SPARK-21476) RandomForest classification model not using broadcast in transform

2017-07-26 Thread Peng Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101476#comment-16101476 ] Peng Meng edited comment on SPARK-21476 at 7/26/17 10:06 AM: -

[jira] [Commented] (SPARK-6809) Make numPartitions optional in pairRDD APIs

2017-07-26 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101459#comment-16101459 ] Hyukjin Kwon commented on SPARK-6809: - Hi [~davies], I can't find the APIs of pairRDD.

[jira] [Commented] (SPARK-21535) Reduce memory requirement for CrossValidator and TrainValidationSplit

2017-07-26 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101452#comment-16101452 ] Nick Pentreath commented on SPARK-21535: Parallel CV is in progress: https://gith

[jira] [Resolved] (SPARK-21524) ValidatorParamsSuiteHelpers generates wrong temp files

2017-07-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21524. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18728 [https://github.co

[jira] [Assigned] (SPARK-21524) ValidatorParamsSuiteHelpers generates wrong temp files

2017-07-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-21524: - Assignee: yuhao yang > ValidatorParamsSuiteHelpers generates wrong temp files >

[jira] [Resolved] (SPARK-11046) Pass schema from R to JVM using JSON format

2017-07-26 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11046. -- Resolution: Not A Problem I lately touched some codes around here - SPARK-20493. I assume we ar

[jira] [Commented] (SPARK-21476) RandomForest classification model not using broadcast in transform

2017-07-26 Thread Saurabh Agrawal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101443#comment-16101443 ] Saurabh Agrawal commented on SPARK-21476: - [~peng.m...@intel.com] My streaming ap

[jira] [Commented] (SPARK-21536) Remove the workaroud to allow dots in field names in R's createDataFame

2017-07-26 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101418#comment-16101418 ] Hyukjin Kwon commented on SPARK-21536: -- BTW, my try was - https://github.com/apache/

[jira] [Commented] (SPARK-21536) Remove the workaroud to allow dots in field names in R's createDataFame

2017-07-26 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101410#comment-16101410 ] Hyukjin Kwon commented on SPARK-21536: -- I just realised what I found while fixing th

[jira] [Comment Edited] (SPARK-16784) Configurable log4j settings

2017-07-26 Thread HanCheol Cho (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099477#comment-16099477 ] HanCheol Cho edited comment on SPARK-16784 at 7/26/17 8:27 AM:

[jira] [Comment Edited] (SPARK-16784) Configurable log4j settings

2017-07-26 Thread HanCheol Cho (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099477#comment-16099477 ] HanCheol Cho edited comment on SPARK-16784 at 7/26/17 8:26 AM:

[jira] [Commented] (SPARK-21476) RandomForest classification model not using broadcast in transform

2017-07-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101298#comment-16101298 ] Sean Owen commented on SPARK-21476: --- [~sagraw] first someone would have to propose a fi

[jira] [Resolved] (SPARK-21412) Reset BufferHolder while initialize an UnsafeRowWriter

2017-07-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21412. --- Resolution: Not A Problem > Reset BufferHolder while initialize an UnsafeRowWriter >

[jira] [Comment Edited] (SPARK-21495) DIGEST-MD5: Out of order sequencing of messages from server

2017-07-26 Thread Xin Yu Pan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16097885#comment-16097885 ] Xin Yu Pan edited comment on SPARK-21495 at 7/26/17 7:47 AM: -

[jira] [Commented] (SPARK-21476) RandomForest classification model not using broadcast in transform

2017-07-26 Thread Peng Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101253#comment-16101253 ] Peng Meng commented on SPARK-21476: --- Not each transform uses broadcast, do you have som