[jira] [Comment Edited] (SPARK-18589) persist() resolves "java.lang.RuntimeException: Invalid PythonUDF (...), requires attributes from more than one child"

[ https://issues.apache.org/jira/browse/SPARK-18589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761741#comment-15761741 ] Franklyn Dsouza edited comment on SPARK-18589 at 12/19/16 5:22 PM: --- The

[jira] [Comment Edited] (SPARK-18589) persist() resolves "java.lang.RuntimeException: Invalid PythonUDF (...), requires attributes from more than one child"

[ https://issues.apache.org/jira/browse/SPARK-18589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761741#comment-15761741 ] Franklyn Dsouza edited comment on SPARK-18589 at 12/19/16 5:24 PM: --- The

[jira] [Comment Edited] (SPARK-18589) persist() resolves "java.lang.RuntimeException: Invalid PythonUDF (...), requires attributes from more than one child"

[ https://issues.apache.org/jira/browse/SPARK-18589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761741#comment-15761741 ] Franklyn Dsouza edited comment on SPARK-18589 at 12/19/16 5:21 PM: --- The

[jira] [Comment Edited] (SPARK-18589) persist() resolves "java.lang.RuntimeException: Invalid PythonUDF (...), requires attributes from more than one child"

[ https://issues.apache.org/jira/browse/SPARK-18589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761741#comment-15761741 ] Franklyn Dsouza edited comment on SPARK-18589 at 12/19/16 5:20 PM: --- The

[jira] [Commented] (SPARK-18589) persist() resolves "java.lang.RuntimeException: Invalid PythonUDF (...), requires attributes from more than one child"

[ https://issues.apache.org/jira/browse/SPARK-18589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761741#comment-15761741 ] Franklyn Dsouza commented on SPARK-18589: - The sequence of steps that causes this are: {code}

[jira] [Assigned] (SPARK-18837) Very long stage descriptions do not wrap in the UI

[ https://issues.apache.org/jira/browse/SPARK-18837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18837: Assignee: (was: Apache Spark) > Very long stage descriptions do not wrap in the UI >

[jira] [Assigned] (SPARK-18837) Very long stage descriptions do not wrap in the UI

[ https://issues.apache.org/jira/browse/SPARK-18837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18837: Assignee: Apache Spark > Very long stage descriptions do not wrap in the UI >

[jira] [Commented] (SPARK-18837) Very long stage descriptions do not wrap in the UI

[ https://issues.apache.org/jira/browse/SPARK-18837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761643#comment-15761643 ] Apache Spark commented on SPARK-18837: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-18648) spark-shell --jars option does not add jars to classpath on windows

[ https://issues.apache.org/jira/browse/SPARK-18648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761619#comment-15761619 ] Michel Lemay commented on SPARK-18648: -- I tried that approach before (spark.driver.extraClassPath)

[jira] [Created] (SPARK-18926) run-example SparkPi terminates with error message

Alex DeCastro created SPARK-18926: - Summary: run-example SparkPi terminates with error message Key: SPARK-18926 URL: https://issues.apache.org/jira/browse/SPARK-18926 Project: Spark Issue

[jira] [Commented] (SPARK-18837) Very long stage descriptions do not wrap in the UI

[ https://issues.apache.org/jira/browse/SPARK-18837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761556#comment-15761556 ] Kousuke Saruta commented on SPARK-18837: I've found the cause. The width of "td" elements in a

[jira] [Commented] (SPARK-18917) Dataframe - Time Out Issues / Taking long time in append mode on object stores

[ https://issues.apache.org/jira/browse/SPARK-18917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761444#comment-15761444 ] Anbu Cheeralan commented on SPARK-18917: Thanks [~dongjoon]. This is my first PR. will not do

[jira] [Updated] (SPARK-17772) Add helper testing methods for instance weighting

[ https://issues.apache.org/jira/browse/SPARK-17772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-17772: Shepherd: Yanbo Liang > Add helper testing methods for instance weighting >

[jira] [Commented] (SPARK-18608) Spark ML algorithms that check RDD cache level for internal caching double-cache data

[ https://issues.apache.org/jira/browse/SPARK-18608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761233#comment-15761233 ] zakaria hili commented on SPARK-18608: -- Hi, Can I join this discussion? Could you please explain to

[jira] [Commented] (SPARK-18832) Spark SQL: Incorrect error message on calling registered UDF.

[ https://issues.apache.org/jira/browse/SPARK-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761156#comment-15761156 ] Lokesh Yadav commented on SPARK-18832: -- Sorry for the late response. Yes the Hive UDTF works as

[jira] [Commented] (SPARK-6062) HashMap.merged - Null Pointer Exception

[ https://issues.apache.org/jira/browse/SPARK-6062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761109#comment-15761109 ] Seth Bromberger commented on SPARK-6062: This appears to be a known bug in scala's hashmaps: See

[jira] [Commented] (SPARK-18916) Possible bug in Pregel / mergeMsg with hashmaps

[ https://issues.apache.org/jira/browse/SPARK-18916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761104#comment-15761104 ] Seth Bromberger commented on SPARK-18916: - It looks like it so far. Thanks. > Possible bug in

[jira] [Commented] (SPARK-18903) uiWebUrl is not accessible to SparkR

[ https://issues.apache.org/jira/browse/SPARK-18903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761059#comment-15761059 ] Diogo Munaro Vieira commented on SPARK-18903: - Thank you guys! > uiWebUrl is not accessible

[jira] [Updated] (SPARK-18925) Reduce memory usage of mapWithState

[ https://issues.apache.org/jira/browse/SPARK-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Pchelko updated SPARK-18925: - Description: With default settings mapWithState leads to storing up to 10 copies of

[jira] [Updated] (SPARK-18925) Reduce memory usage of mapWithState

[ https://issues.apache.org/jira/browse/SPARK-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Pchelko updated SPARK-18925: - Description: With default settings mapWithState leads to storing up to 10 copies of

[jira] [Issue Comment Deleted] (SPARK-8605) Exclude files in StreamingContext. textFileStream(directory)

[ https://issues.apache.org/jira/browse/SPARK-8605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Wang updated SPARK-8605: - Comment: was deleted (was: How about change FileInputDStream.defaultFilter(path) to fix this bug?) >

[jira] [Comment Edited] (SPARK-8605) Exclude files in StreamingContext. textFileStream(directory)

[ https://issues.apache.org/jira/browse/SPARK-8605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15760867#comment-15760867 ] Adam Wang edited comment on SPARK-8605 at 12/19/16 11:01 AM: - How about change

[jira] [Commented] (SPARK-8605) Exclude files in StreamingContext. textFileStream(directory)

[ https://issues.apache.org/jira/browse/SPARK-8605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15760867#comment-15760867 ] Adam Wang commented on SPARK-8605: -- How about change FileInputDStream.defaultFilter(path) to fix this

[jira] [Commented] (SPARK-8605) Exclude files in StreamingContext. textFileStream(directory)

[ https://issues.apache.org/jira/browse/SPARK-8605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15760858#comment-15760858 ] Adam Wang commented on SPARK-8605: -- How about change FileInputDStream.defaultFilter(path) to fix this

[jira] [Issue Comment Deleted] (SPARK-8605) Exclude files in StreamingContext. textFileStream(directory)

[ https://issues.apache.org/jira/browse/SPARK-8605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Wang updated SPARK-8605: - Comment: was deleted (was: How about change defaultFilter() to fix this bug?) > Exclude files in

[jira] [Commented] (SPARK-8605) Exclude files in StreamingContext. textFileStream(directory)

[ https://issues.apache.org/jira/browse/SPARK-8605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15760854#comment-15760854 ] Adam Wang commented on SPARK-8605: -- How about change defaultFilter() to fix this bug? > Exclude files in

[jira] [Updated] (SPARK-18925) Reduce memory usage of mapWithState

[ https://issues.apache.org/jira/browse/SPARK-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Pchelko updated SPARK-18925: - Attachment: (was: 0001-override-clearMetadata-for-InternalMapWithStateDStre.patch)

[jira] [Updated] (SPARK-18925) Reduce memory usage of mapWithState

[ https://issues.apache.org/jira/browse/SPARK-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Pchelko updated SPARK-18925: - Attachment: 0001-override-clearMetadata-for-InternalMapWithStateDStre.patch > Reduce

[jira] [Updated] (SPARK-18925) Reduce memory usage of mapWithState

[ https://issues.apache.org/jira/browse/SPARK-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Pchelko updated SPARK-18925: - Affects Version/s: 2.0.0 2.0.1 2.0.2

[jira] [Created] (SPARK-18925) Reduce memory usage of mapWithState

Vladimir Pchelko created SPARK-18925: Summary: Reduce memory usage of mapWithState Key: SPARK-18925 URL: https://issues.apache.org/jira/browse/SPARK-18925 Project: Spark Issue Type:

[jira] [Updated] (SPARK-18885) unify CREATE TABLE syntax for data source and hive serde tables

[ https://issues.apache.org/jira/browse/SPARK-18885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-18885: Attachment: CREATE-TABLE.pdf > unify CREATE TABLE syntax for data source and hive serde tables >

[jira] [Updated] (SPARK-18700) getCached in HiveMetastoreCatalog not thread safe cause driver OOM

[ https://issues.apache.org/jira/browse/SPARK-18700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Yuanjian updated SPARK-18700: Affects Version/s: 2.1.1 > getCached in HiveMetastoreCatalog not thread safe cause driver OOM >

[jira] [Commented] (SPARK-8582) Optimize checkpointing to avoid computing an RDD twice

[ https://issues.apache.org/jira/browse/SPARK-8582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15760693#comment-15760693 ] Lev Katzav commented on SPARK-8582: --- Any update on this? what are the plans for spark 2? thanks >

[jira] [Resolved] (SPARK-18880) Cannot use imported class as parameter of case class in spark-shell

[ https://issues.apache.org/jira/browse/SPARK-18880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18880. --- Resolution: Duplicate > Cannot use imported class as parameter of case class in spark-shell >

[jira] [Commented] (SPARK-18916) Possible bug in Pregel / mergeMsg with hashmaps

[ https://issues.apache.org/jira/browse/SPARK-18916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15760630#comment-15760630 ] Sean Owen commented on SPARK-18916: --- OK, is this then just a Scala problem to be fixed in 2.11.9 /

[jira] [Resolved] (SPARK-18716) Restrict the disk usage of spark event log.

[ https://issues.apache.org/jira/browse/SPARK-18716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18716. --- Resolution: Won't Fix FWIW I tend to agree that deleting big files first is problematic. > Restrict

<    1   2