[jira] [Resolved] (SPARK-23708) Comment of ShutdownHookManager.addShutdownHook is error

2018-03-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-23708. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20845

[jira] [Assigned] (SPARK-23708) Comment of ShutdownHookManager.addShutdownHook is error

2018-03-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-23708: --- Assignee: zhoukang > Comment of ShutdownHookManager.addShutdownHook is error >

[jira] [Created] (SPARK-23734) InvalidSchemaException While Saving ALSModel

2018-03-18 Thread Stanley Poon (JIRA)
Stanley Poon created SPARK-23734: Summary: InvalidSchemaException While Saving ALSModel Key: SPARK-23734 URL: https://issues.apache.org/jira/browse/SPARK-23734 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-23732) Broken link to scala source code in Spark Scala api Scaladoc

2018-03-18 Thread Yogesh Tewari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404290#comment-16404290 ] Yogesh Tewari commented on SPARK-23732: --- Adding sourcepath to the code seems to fix the issue.

[jira] [Resolved] (SPARK-23665) Add adaptive algorithm to select query result collect method

2018-03-18 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang resolved SPARK-23665. -- Resolution: Won't Fix > Add adaptive algorithm to select query result collect method >

[jira] [Updated] (SPARK-23732) Broken link to scala source code in Spark Scala api Scaladoc

2018-03-18 Thread Yogesh Tewari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogesh Tewari updated SPARK-23732: -- Description: Scala source code link in Spark api scaladoc is broken. Turns out instead of the

[jira] [Updated] (SPARK-23732) Broken link to scala source code in Spark Scala api Scaladoc

2018-03-18 Thread Yogesh Tewari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogesh Tewari updated SPARK-23732: -- Description: Scala source code link in Spark api scaladoc is broken. Turns out instead of the

[jira] [Updated] (SPARK-23733) Broken link to java source code in Spark Scala api Scaladoc

2018-03-18 Thread Yogesh Tewari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogesh Tewari updated SPARK-23733: -- Description: Java source code link in Spark api scaladoc is broken. The relative path

[jira] [Updated] (SPARK-23733) Broken link to java source code in Spark Scala api Scaladoc

2018-03-18 Thread Yogesh Tewari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogesh Tewari updated SPARK-23733: -- Affects Version/s: (was: 2.3.0) > Broken link to java source code in Spark Scala api

[jira] [Updated] (SPARK-23733) Broken link to java source code in Spark Scala api Scaladoc

2018-03-18 Thread Yogesh Tewari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogesh Tewari updated SPARK-23733: -- Description: Java source code link in Spark api scaladoc is broken. The relative path

[jira] [Updated] (SPARK-23733) Broken link to java source code in Spark Scala api Scaladoc

2018-03-18 Thread Yogesh Tewari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogesh Tewari updated SPARK-23733: -- Affects Version/s: (was: 2.3.1) 1.6.3 2.0.2

[jira] [Created] (SPARK-23733) Broken link to java source code in Spark Scala api Scaladoc

2018-03-18 Thread Yogesh Tewari (JIRA)
Yogesh Tewari created SPARK-23733: - Summary: Broken link to java source code in Spark Scala api Scaladoc Key: SPARK-23733 URL: https://issues.apache.org/jira/browse/SPARK-23733 Project: Spark

[jira] [Updated] (SPARK-23732) Broken link to scala source code in Spark Scala api Scaladoc

2018-03-18 Thread Yogesh Tewari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogesh Tewari updated SPARK-23732: -- Description: Scala source code link in Spark api scaladoc is broken. Turns out instead of the

[jira] [Created] (SPARK-23732) Broken link to scala source code in Spark Scala api Scaladoc

2018-03-18 Thread Yogesh Tewari (JIRA)
Yogesh Tewari created SPARK-23732: - Summary: Broken link to scala source code in Spark Scala api Scaladoc Key: SPARK-23732 URL: https://issues.apache.org/jira/browse/SPARK-23732 Project: Spark

[jira] [Comment Edited] (SPARK-23650) Slow SparkR udf (dapply)

2018-03-18 Thread Deepansh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404213#comment-16404213 ] Deepansh edited comment on SPARK-23650 at 3/18/18 10:16 PM: I tried reading

[jira] [Updated] (SPARK-23650) Slow SparkR udf (dapply)

2018-03-18 Thread Deepansh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepansh updated SPARK-23650: - Attachment: read_model_in_udf.txt > Slow SparkR udf (dapply) > > >

[jira] [Commented] (SPARK-23650) Slow SparkR udf (dapply)

2018-03-18 Thread Deepansh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404213#comment-16404213 ] Deepansh commented on SPARK-23650: -- I tried reading the model in UDF, but for every new stream, the

[jira] [Commented] (SPARK-23650) Slow SparkR udf (dapply)

2018-03-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404198#comment-16404198 ] Felix Cheung commented on SPARK-23650: -- Is there a reason for the broadcast? Could you instead

[jira] [Commented] (SPARK-23650) Slow SparkR udf (dapply)

2018-03-18 Thread Deepansh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404183#comment-16404183 ] Deepansh commented on SPARK-23650: -- Is there any other way to implement my use case with minimum(ms)

[jira] [Commented] (SPARK-23645) pandas_udf can not be called with keyword arguments

2018-03-18 Thread Stu (Michael Stewart) (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404126#comment-16404126 ] Stu (Michael Stewart) commented on SPARK-23645: --- {quote}Sounds a good to do if the change

[jira] [Commented] (SPARK-23731) FileSourceScanExec throws NullPointerException in subexpression elimination

2018-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404067#comment-16404067 ] Apache Spark commented on SPARK-23731: -- User 'jaceklaskowski' has created a pull request for this

[jira] [Assigned] (SPARK-23731) FileSourceScanExec throws NullPointerException in subexpression elimination

2018-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23731: Assignee: Apache Spark > FileSourceScanExec throws NullPointerException in subexpression

[jira] [Assigned] (SPARK-23731) FileSourceScanExec throws NullPointerException in subexpression elimination

2018-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23731: Assignee: (was: Apache Spark) > FileSourceScanExec throws NullPointerException in

[jira] [Commented] (SPARK-23731) FileSourceScanExec throws NullPointerException in subexpression elimination

2018-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404062#comment-16404062 ] Apache Spark commented on SPARK-23731: -- User 'jaceklaskowski' has created a pull request for this

[jira] [Created] (SPARK-23731) FileSourceScanExec throws NullPointerException in subexpression elimination

2018-03-18 Thread Jacek Laskowski (JIRA)
Jacek Laskowski created SPARK-23731: --- Summary: FileSourceScanExec throws NullPointerException in subexpression elimination Key: SPARK-23731 URL: https://issues.apache.org/jira/browse/SPARK-23731

[jira] [Assigned] (SPARK-23712) Investigate replacing Code Generated UnsafeRowJoiner with an Interpreted version

2018-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23712: Assignee: Herman van Hovell (was: Apache Spark) > Investigate replacing Code Generated

[jira] [Assigned] (SPARK-23712) Investigate replacing Code Generated UnsafeRowJoiner with an Interpreted version

2018-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23712: Assignee: Apache Spark (was: Herman van Hovell) > Investigate replacing Code Generated

[jira] [Commented] (SPARK-23712) Investigate replacing Code Generated UnsafeRowJoiner with an Interpreted version

2018-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404001#comment-16404001 ] Apache Spark commented on SPARK-23712: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Commented] (SPARK-23646) pyspark DataFrameWriter ignores customized settings?

2018-03-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403985#comment-16403985 ] Hyukjin Kwon commented on SPARK-23646: -- Thanks for logging here. It should be helpful if anyone

[jira] [Resolved] (SPARK-23706) spark.conf.get(value, default=None) should produce None in PySpark

2018-03-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23706. -- Resolution: Fixed Fix Version/s: 2.4.0 2.3.1 Fixed in

[jira] [Created] (SPARK-23730) Save and expose "in bag" tracking for random forest model

2018-03-18 Thread Julian King (JIRA)
Julian King created SPARK-23730: --- Summary: Save and expose "in bag" tracking for random forest model Key: SPARK-23730 URL: https://issues.apache.org/jira/browse/SPARK-23730 Project: Spark

[jira] [Commented] (SPARK-23710) Upgrade Hive to 2.3.2

2018-03-18 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403886#comment-16403886 ] Yuming Wang commented on SPARK-23710: - Without {{-Phive}} works fine, because {{$SPARK_HOME/jars}} 

[jira] [Assigned] (SPARK-23729) Glob resolution breaks remote naming of files/archives

2018-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23729: Assignee: (was: Apache Spark) > Glob resolution breaks remote naming of

[jira] [Assigned] (SPARK-23729) Glob resolution breaks remote naming of files/archives

2018-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23729: Assignee: Apache Spark > Glob resolution breaks remote naming of files/archives >

[jira] [Commented] (SPARK-23729) Glob resolution breaks remote naming of files/archives

2018-03-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403879#comment-16403879 ] Apache Spark commented on SPARK-23729: -- User 'misutoth' has created a pull request for this issue:

[jira] [Commented] (SPARK-23729) Glob resolution breaks remote naming of files/archives

2018-03-18 Thread Mihaly Toth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403876#comment-16403876 ] Mihaly Toth commented on SPARK-23729: - Already working on this. Will submit a PR shortly. > Glob

[jira] [Created] (SPARK-23729) Glob resolution breaks remote naming of files/archives

2018-03-18 Thread Mihaly Toth (JIRA)
Mihaly Toth created SPARK-23729: --- Summary: Glob resolution breaks remote naming of files/archives Key: SPARK-23729 URL: https://issues.apache.org/jira/browse/SPARK-23729 Project: Spark Issue