[jira] [Commented] (SPARK-9594) Failed to get broadcast_33_piece0 while using Accumulators in UDF

2015-08-04 Thread Poorvi Lashkary (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653440#comment-14653440 ] Poorvi Lashkary commented on SPARK-9594: Use case: I need to create auto increment

[jira] [Comment Edited] (SPARK-9594) Failed to get broadcast_33_piece0 while using Accumulators in UDF

2015-08-04 Thread Poorvi Lashkary (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653440#comment-14653440 ] Poorvi Lashkary edited comment on SPARK-9594 at 8/4/15 10:42 AM:

[jira] [Comment Edited] (SPARK-9594) Failed to get broadcast_33_piece0 while using Accumulators in UDF

2015-08-04 Thread Poorvi Lashkary (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653440#comment-14653440 ] Poorvi Lashkary edited comment on SPARK-9594 at 8/4/15 10:42 AM:

[jira] [Comment Edited] (SPARK-9594) Failed to get broadcast_33_piece0 while using Accumulators in UDF

2015-08-04 Thread Poorvi Lashkary (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653440#comment-14653440 ] Poorvi Lashkary edited comment on SPARK-9594 at 8/4/15 10:41 AM:

[jira] [Updated] (SPARK-9359) Support IntervalType for Parquet

2015-08-04 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-9359: -- Assignee: Liang-Chi Hsieh Support IntervalType for Parquet

[jira] [Resolved] (SPARK-9534) Enable javac lint for scalac parity; fix a lot of build warnings, 1.5.0 edition

2015-08-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-9534. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7862

[jira] [Updated] (SPARK-9359) Support IntervalType for Parquet

2015-08-04 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-9359: -- Shepherd: Cheng Lian Support IntervalType for Parquet

[jira] [Updated] (SPARK-9592) First and Last aggregates are calculating the values for entire DataFrame partition not on GroupedData partition.

2015-08-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9592: - Affects Version/s: (was: 1.5.0) Target Version/s: (was: 1.4.0) Fix Version/s: (was:

[jira] [Updated] (SPARK-9594) Failed to get broadcast_33_piece0 while using Accumulators in UDF

2015-08-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9594: - Target Version/s: (was: 1.3.1) Priority: Minor (was: Major) Component/s: SQL

[jira] [Updated] (SPARK-9574) Review the contents of uber JARs spark-streaming-XXX-assembly

2015-08-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9574: - Component/s: Streaming Issue Type: Task (was: Bug) Review the contents of uber JARs

[jira] [Updated] (SPARK-9573) Forward exceptions in batch jobs to the awaitTermination thread

2015-08-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9573: - Component/s: Streaming Forward exceptions in batch jobs to the awaitTermination thread

[jira] [Created] (SPARK-9596) We better not load Hadoop classes again

2015-08-04 Thread Tao Wang (JIRA)
Tao Wang created SPARK-9596: --- Summary: We better not load Hadoop classes again Key: SPARK-9596 URL: https://issues.apache.org/jira/browse/SPARK-9596 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-9587) Spark Web UI not displaying while changing another network

2015-08-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653573#comment-14653573 ] Sean Owen commented on SPARK-9587: -- OK, I think you may be asking the same question as in

[jira] [Created] (SPARK-9595) Adding API to SparkConf for kryo serializers registration

2015-08-04 Thread John Chen (JIRA)
John Chen created SPARK-9595: Summary: Adding API to SparkConf for kryo serializers registration Key: SPARK-9595 URL: https://issues.apache.org/jira/browse/SPARK-9595 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-9596) Avoid reloading Hadoop classes like UserGroupInformation

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9596: --- Assignee: Apache Spark Avoid reloading Hadoop classes like UserGroupInformation

[jira] [Updated] (SPARK-9596) Avoid reloading Hadoop classes like UserGroupInformation

2015-08-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9596: - Summary: Avoid reloading Hadoop classes like UserGroupInformation (was: We better not load Hadoop

[jira] [Commented] (SPARK-9119) In some cases, we may save wrong decimal values to parquet

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653312#comment-14653312 ] Apache Spark commented on SPARK-9119: - User 'davies' has created a pull request for

[jira] [Assigned] (SPARK-9119) In some cases, we may save wrong decimal values to parquet

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9119: --- Assignee: Apache Spark (was: Davies Liu) In some cases, we may save wrong decimal values

[jira] [Assigned] (SPARK-9591) Job failed for exception during getting Broadcast variable

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9591: --- Assignee: (was: Apache Spark) Job failed for exception during getting Broadcast

[jira] [Commented] (SPARK-9593) Hive ShimLoader loads wrong Hadoop shims when Spark is compiled against Hadoop 2.0.0-mr1-cdh4.1.1

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653401#comment-14653401 ] Apache Spark commented on SPARK-9593: - User 'liancheng' has created a pull request for

[jira] [Assigned] (SPARK-9593) Hive ShimLoader loads wrong Hadoop shims when Spark is compiled against Hadoop 2.0.0-mr1-cdh4.1.1

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9593: --- Assignee: Cheng Lian (was: Apache Spark) Hive ShimLoader loads wrong Hadoop shims when

[jira] [Created] (SPARK-9594) Failed to get broadcast_33_piece0 while using Accumulators in UDF

2015-08-04 Thread Poorvi Lashkary (JIRA)
Poorvi Lashkary created SPARK-9594: -- Summary: Failed to get broadcast_33_piece0 while using Accumulators in UDF Key: SPARK-9594 URL: https://issues.apache.org/jira/browse/SPARK-9594 Project: Spark

[jira] [Assigned] (SPARK-9533) Add missing methods in Word2Vec ML (Python API)

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9533: --- Assignee: Apache Spark Add missing methods in Word2Vec ML (Python API)

[jira] [Closed] (SPARK-6283) Add a CassandraInputDStream to stream from a C* table

2015-08-04 Thread Helena Edelson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Helena Edelson closed SPARK-6283. - Resolution: Done I've written this but sadly DataStax has decided to close source it. Add a

[jira] [Created] (SPARK-9598) do not expose generic getter in internal row

2015-08-04 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-9598: -- Summary: do not expose generic getter in internal row Key: SPARK-9598 URL: https://issues.apache.org/jira/browse/SPARK-9598 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-9598) do not expose generic getter in internal row

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9598: --- Assignee: (was: Apache Spark) do not expose generic getter in internal row

[jira] [Assigned] (SPARK-9598) do not expose generic getter in internal row

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9598: --- Assignee: Apache Spark do not expose generic getter in internal row

[jira] [Commented] (SPARK-9598) do not expose generic getter in internal row

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653691#comment-14653691 ] Apache Spark commented on SPARK-9598: - User 'cloud-fan' has created a pull request for

[jira] [Resolved] (SPARK-3190) Creation of large graph( 2.15 B nodes) seems to be broken:possible overflow somewhere

2015-08-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3190. Resolution: Fixed Fix Version/s: 1.5.0 1.4.2 1.3.2

[jira] [Resolved] (SPARK-9555) Cannot use spark-csv in spark-shell

2015-08-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-9555. -- Resolution: Not A Problem I think this is a question about usage of that library then, which is not

[jira] [Created] (SPARK-9589) Flaky test: HiveCompatibilitySuite.groupby8

2015-08-04 Thread Davies Liu (JIRA)
Davies Liu created SPARK-9589: - Summary: Flaky test: HiveCompatibilitySuite.groupby8 Key: SPARK-9589 URL: https://issues.apache.org/jira/browse/SPARK-9589 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-9548) BytesToBytesMap could have a destructive iterator

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9548: --- Assignee: Apache Spark BytesToBytesMap could have a destructive iterator

[jira] [Created] (SPARK-9588) spark sql cache: partition level cache eviction

2015-08-04 Thread Shenghu Yang (JIRA)
Shenghu Yang created SPARK-9588: --- Summary: spark sql cache: partition level cache eviction Key: SPARK-9588 URL: https://issues.apache.org/jira/browse/SPARK-9588 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-9548) BytesToBytesMap could have a destructive iterator

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9548: --- Assignee: (was: Apache Spark) BytesToBytesMap could have a destructive iterator

[jira] [Commented] (SPARK-9587) Spark Web UI not displaying while changing another network

2015-08-04 Thread Kaveen Raajan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653178#comment-14653178 ] Kaveen Raajan commented on SPARK-9587: -- I found *SPARK_PUBLIC_DNS* in spark

[jira] [Updated] (SPARK-9533) Add missing methods in Word2Vec ML (Python API)

2015-08-04 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Kumar updated SPARK-9533: --- Component/s: PySpark Add missing methods in Word2Vec ML (Python API)

[jira] [Updated] (SPARK-9588) spark sql cache: partition level cache eviction

2015-08-04 Thread Shenghu Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shenghu Yang updated SPARK-9588: Description: In spark 1.4, we can only do 'cache table table_name'. However, if we have table

[jira] [Commented] (SPARK-9548) BytesToBytesMap could have a destructive iterator

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653173#comment-14653173 ] Apache Spark commented on SPARK-9548: - User 'viirya' has created a pull request for

[jira] [Created] (SPARK-9590) support metaq to streaming

2015-08-04 Thread zhouxiaoke (JIRA)
zhouxiaoke created SPARK-9590: - Summary: support metaq to streaming Key: SPARK-9590 URL: https://issues.apache.org/jira/browse/SPARK-9590 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-9588) spark sql cache: partition level cache eviction

2015-08-04 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653172#comment-14653172 ] Davies Liu commented on SPARK-9588: --- cc [~lian cheng], Does the partition improvement in

[jira] [Updated] (SPARK-9588) spark sql cache: partition level cache eviction

2015-08-04 Thread Shenghu Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shenghu Yang updated SPARK-9588: Description: In spark 1.4, we can only do 'cache table table_name'. However, if we have table

[jira] [Updated] (SPARK-9588) spark sql cache: partition level cache eviction

2015-08-04 Thread Shenghu Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shenghu Yang updated SPARK-9588: Description: In spark 1.4, we can only do 'cache table table_name'. However, if we have table

[jira] [Commented] (SPARK-9587) Spark Web UI not displaying while changing another network

2015-08-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653121#comment-14653121 ] Sean Owen commented on SPARK-9587: -- By not working you mean a host isn't resolving where

[jira] [Comment Edited] (SPARK-9587) Spark Web UI not displaying while changing another network

2015-08-04 Thread Kaveen Raajan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653178#comment-14653178 ] Kaveen Raajan edited comment on SPARK-9587 at 8/4/15 7:08 AM: --

[jira] [Comment Edited] (SPARK-9587) Spark Web UI not displaying while changing another network

2015-08-04 Thread Kaveen Raajan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653178#comment-14653178 ] Kaveen Raajan edited comment on SPARK-9587 at 8/4/15 7:08 AM: --

[jira] [Assigned] (SPARK-6591) Python data source load options should auto convert common types into strings

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6591: --- Assignee: Yijie Shen (was: Apache Spark) Python data source load options should auto

[jira] [Assigned] (SPARK-6591) Python data source load options should auto convert common types into strings

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6591: --- Assignee: Apache Spark (was: Yijie Shen) Python data source load options should auto

[jira] [Assigned] (SPARK-9592) First and Last aggregates are calculating the values for entire DataFrame partition not on GroupedData partition.

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9592: --- Assignee: (was: Apache Spark) First and Last aggregates are calculating the values for

[jira] [Assigned] (SPARK-9592) First and Last aggregates are calculating the values for entire DataFrame partition not on GroupedData partition.

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9592: --- Assignee: Apache Spark First and Last aggregates are calculating the values for entire

[jira] [Commented] (SPARK-6591) Python data source load options should auto convert common types into strings

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653352#comment-14653352 ] Apache Spark commented on SPARK-6591: - User 'yjshen' has created a pull request for

[jira] [Assigned] (SPARK-9591) Job failed for exception during getting Broadcast variable

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9591: --- Assignee: Apache Spark Job failed for exception during getting Broadcast variable

[jira] [Commented] (SPARK-9591) Job failed for exception during getting Broadcast variable

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653362#comment-14653362 ] Apache Spark commented on SPARK-9591: - User 'jeanlyn' has created a pull request for

[jira] [Commented] (SPARK-9578) Stemmer feature transformer

2015-08-04 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653360#comment-14653360 ] yuhao yang commented on SPARK-9578: --- Share a great link

[jira] [Created] (SPARK-9593) Hive ShimLoader loads wrong Hadoop shims when Spark is compiled against Hadoop 2.0.0-mr1-cdh4.1.1

2015-08-04 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-9593: - Summary: Hive ShimLoader loads wrong Hadoop shims when Spark is compiled against Hadoop 2.0.0-mr1-cdh4.1.1 Key: SPARK-9593 URL: https://issues.apache.org/jira/browse/SPARK-9593

[jira] [Commented] (SPARK-7160) Support converting DataFrames to typed RDDs.

2015-08-04 Thread Ray Ortigas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653713#comment-14653713 ] Ray Ortigas commented on SPARK-7160: Thanks for trying to resolve the conflicts,

[jira] [Comment Edited] (SPARK-7160) Support converting DataFrames to typed RDDs.

2015-08-04 Thread Ray Ortigas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653713#comment-14653713 ] Ray Ortigas edited comment on SPARK-7160 at 8/4/15 2:28 PM:

[jira] [Updated] (SPARK-9599) Dynamic partitioning based on key-distribution

2015-08-04 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltán Zvara updated SPARK-9599: Summary: Dynamic partitioning based on key-distribution (was: Dynamically partitioning based on

[jira] [Created] (SPARK-9599) Dynamically partitioning based on key-distribution

2015-08-04 Thread JIRA
Zoltán Zvara created SPARK-9599: --- Summary: Dynamically partitioning based on key-distribution Key: SPARK-9599 URL: https://issues.apache.org/jira/browse/SPARK-9599 Project: Spark Issue Type:

[jira] [Commented] (SPARK-9599) Dynamic partitioning based on key-distribution

2015-08-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653723#comment-14653723 ] Sean Owen commented on SPARK-9599: -- For example, in the case of groupByKey, how would

[jira] [Comment Edited] (SPARK-9599) Dynamic partitioning based on key-distribution

2015-08-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653723#comment-14653723 ] Sean Owen edited comment on SPARK-9599 at 8/4/15 2:33 PM: -- For

[jira] [Commented] (SPARK-9599) Dynamic partitioning based on key-distribution

2015-08-04 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653746#comment-14653746 ] Zoltán Zvara commented on SPARK-9599: - What I can think of is a new, guided

[jira] [Commented] (SPARK-5774) Support save RDD append to file

2015-08-04 Thread Murtaza Kanchwala (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653287#comment-14653287 ] Murtaza Kanchwala commented on SPARK-5774: --

[jira] [Created] (SPARK-9591) Job failed for exception during getting Broadcast variable

2015-08-04 Thread jeanlyn (JIRA)
jeanlyn created SPARK-9591: -- Summary: Job failed for exception during getting Broadcast variable Key: SPARK-9591 URL: https://issues.apache.org/jira/browse/SPARK-9591 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-9592) First and Last aggregates are calculating the values for entire DataFrame partition not on GroupedData partition.

2015-08-04 Thread gaurav (JIRA)
gaurav created SPARK-9592: - Summary: First and Last aggregates are calculating the values for entire DataFrame partition not on GroupedData partition. Key: SPARK-9592 URL: https://issues.apache.org/jira/browse/SPARK-9592

[jira] [Commented] (SPARK-5774) Support save RDD append to file

2015-08-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653374#comment-14653374 ] Sean Owen commented on SPARK-5774: -- Appending is not even necessarily possible in the

[jira] [Assigned] (SPARK-9119) In some cases, we may save wrong decimal values to parquet

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9119: --- Assignee: Davies Liu (was: Apache Spark) In some cases, we may save wrong decimal values

[jira] [Commented] (SPARK-8359) Spark SQL Decimal type precision loss on multiplication

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653313#comment-14653313 ] Apache Spark commented on SPARK-8359: - User 'davies' has created a pull request for

[jira] [Commented] (SPARK-9533) Add missing methods in Word2Vec ML (Python API)

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653403#comment-14653403 ] Apache Spark commented on SPARK-9533: - User 'MechCoder' has created a pull request for

[jira] [Assigned] (SPARK-9533) Add missing methods in Word2Vec ML (Python API)

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9533: --- Assignee: (was: Apache Spark) Add missing methods in Word2Vec ML (Python API)

[jira] [Assigned] (SPARK-9593) Hive ShimLoader loads wrong Hadoop shims when Spark is compiled against Hadoop 2.0.0-mr1-cdh4.1.1

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9593: --- Assignee: Apache Spark (was: Cheng Lian) Hive ShimLoader loads wrong Hadoop shims when

[jira] [Commented] (SPARK-9593) Hive ShimLoader loads wrong Hadoop shims when Spark is compiled against Hadoop 2.0.0-mr1-cdh4.1.1

2015-08-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653419#comment-14653419 ] Sean Owen commented on SPARK-9593: -- This isn't quite right: the method that the shim

[jira] [Updated] (SPARK-2016) rdd in-memory storage UI becomes unresponsive when the number of RDD partitions is large

2015-08-04 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2016: -- Assignee: Carson Wang rdd in-memory storage UI becomes unresponsive when the number of RDD

[jira] [Updated] (SPARK-2016) rdd in-memory storage UI becomes unresponsive when the number of RDD partitions is large

2015-08-04 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2016: -- Fix Version/s: 1.5.0 rdd in-memory storage UI becomes unresponsive when the number of RDD

[jira] [Resolved] (SPARK-2016) rdd in-memory storage UI becomes unresponsive when the number of RDD partitions is large

2015-08-04 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-2016. --- Resolution: Fixed rdd in-memory storage UI becomes unresponsive when the number of RDD

[jira] [Created] (SPARK-9597) Spark Streaming + MQTT Integration Guide

2015-08-04 Thread Prabeesh K (JIRA)
Prabeesh K created SPARK-9597: - Summary: Spark Streaming + MQTT Integration Guide Key: SPARK-9597 URL: https://issues.apache.org/jira/browse/SPARK-9597 Project: Spark Issue Type: Documentation

[jira] [Updated] (SPARK-2016) rdd in-memory storage UI becomes unresponsive when the number of RDD partitions is large

2015-08-04 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2016: -- Affects Version/s: 1.5.0 rdd in-memory storage UI becomes unresponsive when the number of RDD

[jira] [Updated] (SPARK-9597) Spark Streaming + MQTT Integration Guide

2015-08-04 Thread Prabeesh K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabeesh K updated SPARK-9597: -- Description: Add Spark Streaming + Spark Streaming + MQTT Integration Guide

[jira] [Updated] (SPARK-9597) Spark Streaming + MQTT Integration Guide

2015-08-04 Thread Prabeesh K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabeesh K updated SPARK-9597: -- Description: Add Spark Streaming + MQTT Integration Guide like [Spark Streaming + Flume Integration

[jira] [Updated] (SPARK-9597) Add Spark Streaming + MQTT Integration Guide

2015-08-04 Thread Prabeesh K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabeesh K updated SPARK-9597: -- Summary: Add Spark Streaming + MQTT Integration Guide (was: Spark Streaming + MQTT Integration Guide)

[jira] [Updated] (SPARK-9583) build/mvn script should not print debug messages to stdout

2015-08-04 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-9583: -- Assignee: Marcelo Vanzin build/mvn script should not print debug messages to stdout

[jira] [Updated] (SPARK-9583) build/mvn script should not print debug messages to stdout

2015-08-04 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-9583: -- Fix Version/s: 1.5.0 build/mvn script should not print debug messages to stdout

[jira] [Resolved] (SPARK-9583) build/mvn script should not print debug messages to stdout

2015-08-04 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-9583. --- Resolution: Fixed build/mvn script should not print debug messages to stdout

[jira] [Updated] (SPARK-8131) Improve Database support

2015-08-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8131: - Assignee: Cheng Lian Improve Database support Key:

[jira] [Updated] (SPARK-9255) SQL codegen fails with value is not a member of TimestampType.this.InternalType

2015-08-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9255: - Assignee: Reynold Xin SQL codegen fails with value is not a member of TimestampType.this.InternalType

[jira] [Updated] (SPARK-9588) spark sql cache: partition level cache eviction

2015-08-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9588: - Component/s: SQL spark sql cache: partition level cache eviction

[jira] [Closed] (SPARK-9587) Spark Web UI not displaying while changing another network

2015-08-04 Thread Kaveen Raajan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaveen Raajan closed SPARK-9587. Resolution: Not A Problem This was working if I *set SPARK_LOCAL_HOSTNAME={COMPUTERNAME}*. This

[jira] [Commented] (SPARK-9596) Avoid reloading Hadoop classes like UserGroupInformation

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653576#comment-14653576 ] Apache Spark commented on SPARK-9596: - User 'WangTaoTheTonic' has created a pull

[jira] [Assigned] (SPARK-9596) Avoid reloading Hadoop classes like UserGroupInformation

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9596: --- Assignee: (was: Apache Spark) Avoid reloading Hadoop classes like UserGroupInformation

[jira] [Created] (SPARK-9600) DataFrameWriter.saveAsTable always writes data to /user/hive/warehouse

2015-08-04 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-9600: - Summary: DataFrameWriter.saveAsTable always writes data to /user/hive/warehouse Key: SPARK-9600 URL: https://issues.apache.org/jira/browse/SPARK-9600 Project: Spark

[jira] [Created] (SPARK-9601) Join example fix in streaming-programming-guide.md

2015-08-04 Thread Jayant Shekhar (JIRA)
Jayant Shekhar created SPARK-9601: - Summary: Join example fix in streaming-programming-guide.md Key: SPARK-9601 URL: https://issues.apache.org/jira/browse/SPARK-9601 Project: Spark Issue

[jira] [Commented] (SPARK-9478) Add class weights to Random Forest

2015-08-04 Thread Patrick Crenshaw (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653799#comment-14653799 ] Patrick Crenshaw commented on SPARK-9478: - If I work on this, should I wait until

[jira] [Updated] (SPARK-9601) Join example fix in streaming-programming-guide.md

2015-08-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9601: - Priority: Trivial (was: Major) Looks good -- are you making a PR? Join example fix in

[jira] [Created] (SPARK-9626) Add python api for base64, crc32, pmod, factorial and conv functions

2015-08-04 Thread zhichao-li (JIRA)
zhichao-li created SPARK-9626: - Summary: Add python api for base64, crc32, pmod, factorial and conv functions Key: SPARK-9626 URL: https://issues.apache.org/jira/browse/SPARK-9626 Project: Spark

[jira] [Closed] (SPARK-9625) SparkILoop creates sql context continuously, thousands of times

2015-08-04 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov closed SPARK-9625. -- Resolution: Won't Fix SparkILoop creates sql context continuously, thousands of times

[jira] [Commented] (SPARK-9625) SparkILoop creates sql context continuously, thousands of times

2015-08-04 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654806#comment-14654806 ] Simeon Simeonov commented on SPARK-9625: This is a spark shell-specific issue

[jira] [Closed] (SPARK-9626) Add python api for base64, crc32, pmod, factorial and conv functions

2015-08-04 Thread zhichao-li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhichao-li closed SPARK-9626. - Resolution: Duplicate duplicated with SPARK-9513 Add python api for base64, crc32, pmod, factorial and

[jira] [Commented] (SPARK-8542) PMML export for Decision Trees

2015-08-04 Thread Jasmine George (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654819#comment-14654819 ] Jasmine George commented on SPARK-8542: --- Hi Joseph, I have the pull request ready.

[jira] [Assigned] (SPARK-9493) Chain logistic regression with isotonic regression under the pipeline API

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9493: --- Assignee: Xiangrui Meng (was: Apache Spark) Chain logistic regression with isotonic

[jira] [Assigned] (SPARK-9493) Chain logistic regression with isotonic regression under the pipeline API

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9493: --- Assignee: Apache Spark (was: Xiangrui Meng) Chain logistic regression with isotonic

[jira] [Commented] (SPARK-9493) Chain logistic regression with isotonic regression under the pipeline API

2015-08-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654833#comment-14654833 ] Apache Spark commented on SPARK-9493: - User 'mengxr' has created a pull request for

  1   2   3   >