[jira] [Commented] (SPARK-18609) [SQL] column mixup with CROSS JOIN

2016-12-06 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15728042#comment-15728042 ] Song Jun commented on SPARK-18609: -- I'm working on this~ > [SQL] column mixup with CROSS JOIN >

[jira] [Resolved] (SPARK-18763) What algorithm is used in spark decision tree (is ID3, C4.5 or CART)?

2016-12-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18763. --- Resolution: Invalid Fix Version/s: (was: 1.6.0) > What algorithm is used in spark

[jira] [Commented] (SPARK-18763) What algorithm is used in spark decision tree (is ID3, C4.5 or CART)?

2016-12-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15728037#comment-15728037 ] Sean Owen commented on SPARK-18763: --- Please ask questions on the mailing list. > What algorithm is

[jira] [Assigned] (SPARK-18764) Add a warning log when skipping a corrupted file

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18764: Assignee: Apache Spark (was: Shixiong Zhu) > Add a warning log when skipping a corrupted

[jira] [Assigned] (SPARK-18764) Add a warning log when skipping a corrupted file

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18764: Assignee: Shixiong Zhu (was: Apache Spark) > Add a warning log when skipping a corrupted

[jira] [Commented] (SPARK-18764) Add a warning log when skipping a corrupted file

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15728021#comment-15728021 ] Apache Spark commented on SPARK-18764: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Updated] (SPARK-18764) Add a warning log when skipping a corrupted file

2016-12-06 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18764: - Affects Version/s: 2.1.0 > Add a warning log when skipping a corrupted file >

[jira] [Created] (SPARK-18764) Add a warning log when skipping a corrupted file

2016-12-06 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-18764: Summary: Add a warning log when skipping a corrupted file Key: SPARK-18764 URL: https://issues.apache.org/jira/browse/SPARK-18764 Project: Spark Issue Type:

[jira] [Created] (SPARK-18763) What algorithm is used in spark decision tree (is ID3, C4.5 or CART)?

2016-12-06 Thread lklong (JIRA)
lklong created SPARK-18763: -- Summary: What algorithm is used in spark decision tree (is ID3, C4.5 or CART)? Key: SPARK-18763 URL: https://issues.apache.org/jira/browse/SPARK-18763 Project: Spark

[jira] [Closed] (SPARK-18759) when use spark streaming with sparksql, lots of temp directories are created.

2016-12-06 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh closed SPARK-18759. --- Resolution: Duplicate duplicate to SPARK-18703 > when use spark streaming with sparksql,

[jira] [Assigned] (SPARK-18762) Web UI should be http:4040 instead of https:4040

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18762: Assignee: Apache Spark > Web UI should be http:4040 instead of https:4040 >

[jira] [Commented] (SPARK-18762) Web UI should be http:4040 instead of https:4040

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727990#comment-15727990 ] Apache Spark commented on SPARK-18762: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18762) Web UI should be http:4040 instead of https:4040

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18762: Assignee: (was: Apache Spark) > Web UI should be http:4040 instead of https:4040 >

[jira] [Commented] (SPARK-18762) Web UI should be http:4040 instead of https:4040

2016-12-06 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727986#comment-15727986 ] Kousuke Saruta commented on SPARK-18762: Yeah of course. > Web UI should be http:4040 instead of

[jira] [Commented] (SPARK-18761) Uncancellable / unkillable tasks may starve jobs of resoures

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727983#comment-15727983 ] Apache Spark commented on SPARK-18761: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-18762) Web UI should be http:4040 instead of https:4040

2016-12-06 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727981#comment-15727981 ] Xiangrui Meng commented on SPARK-18762: --- Thanks! Please make sure spark history server still works

[jira] [Commented] (SPARK-18756) Memory leak in Spark streaming

2016-12-06 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727975#comment-15727975 ] Liang-Chi Hsieh commented on SPARK-18756: - As we already upgrade to 4.0.42.Final, this should not

[jira] [Commented] (SPARK-18756) Memory leak in Spark streaming

2016-12-06 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727972#comment-15727972 ] Liang-Chi Hsieh commented on SPARK-18756: - I believe this bug is fixed by

[jira] [Issue Comment Deleted] (SPARK-18713) using SparkR build step wise regression model (glm)

2016-12-06 Thread Prasann modi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasann modi updated SPARK-18713: - Comment: was deleted (was: Can u add step wise regression function into upcoming Spark version.)

[jira] [Reopened] (SPARK-18713) using SparkR build step wise regression model (glm)

2016-12-06 Thread Prasann modi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasann modi reopened SPARK-18713: -- Can you add step wise regression function into upcoming Spark version. > using SparkR build step

[jira] [Commented] (SPARK-18762) Web UI should be http:4040 instead of https:4040

2016-12-06 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727963#comment-15727963 ] Kousuke Saruta commented on SPARK-18762: [~mengxr] Ah... O.K, I'll submit a PR to revert it. >

[jira] [Commented] (SPARK-18759) when use spark streaming with sparksql, lots of temp directories are created.

2016-12-06 Thread Albert Cheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727958#comment-15727958 ] Albert Cheng commented on SPARK-18759: -- [~viirya] is right, this issue is duplicate to SPARK-18703.

[jira] [Commented] (SPARK-18713) using SparkR build step wise regression model (glm)

2016-12-06 Thread Prasann modi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727956#comment-15727956 ] Prasann modi commented on SPARK-18713: -- Can u add step wise regression function into upcoming Spark

[jira] [Updated] (SPARK-18762) Web UI should be http:4040 instead of https:4040

2016-12-06 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-18762: -- Description: When SSL is enabled, the Spark shell shows: {code} Spark context Web UI

[jira] [Comment Edited] (SPARK-18762) Web UI should be http:4040 instead of https:4040

2016-12-06 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727929#comment-15727929 ] Xiangrui Meng edited comment on SPARK-18762 at 12/7/16 6:56 AM: cc

[jira] [Commented] (SPARK-18762) Web UI should be http:4040 instead of https:4040

2016-12-06 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727929#comment-15727929 ] Xiangrui Meng commented on SPARK-18762: --- cc [~hayashidac] [~sarutak] > Web UI should be http:4040

[jira] [Updated] (SPARK-18762) Web UI should be http:4040 instead of https:4040

2016-12-06 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-18762: -- Description: When SSL is enabled, the Spark shell shows: {code} Spark context Web UI

[jira] [Updated] (SPARK-18762) Web UI should be http:4040 instead of https:4040

2016-12-06 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-18762: -- Priority: Blocker (was: Critical) > Web UI should be http:4040 instead of https:4040 >

[jira] [Updated] (SPARK-18762) Web UI should be http:4040 instead of https:4040

2016-12-06 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-18762: -- Priority: Critical (was: Major) > Web UI should be http:4040 instead of https:4040 >

[jira] [Created] (SPARK-18762) Web UI should be http:4040 instead of https:4040

2016-12-06 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-18762: - Summary: Web UI should be http:4040 instead of https:4040 Key: SPARK-18762 URL: https://issues.apache.org/jira/browse/SPARK-18762 Project: Spark Issue

[jira] [Commented] (SPARK-18759) when use spark streaming with sparksql, lots of temp directories are created.

2016-12-06 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727899#comment-15727899 ] Liang-Chi Hsieh commented on SPARK-18759: - I think this is duplicate to SPARK-18703. > when use

[jira] [Assigned] (SPARK-18761) Uncancellable / unkillable tasks may starve jobs of resoures

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18761: Assignee: Apache Spark (was: Josh Rosen) > Uncancellable / unkillable tasks may starve

[jira] [Assigned] (SPARK-18761) Uncancellable / unkillable tasks may starve jobs of resoures

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18761: Assignee: Josh Rosen (was: Apache Spark) > Uncancellable / unkillable tasks may starve

[jira] [Commented] (SPARK-18761) Uncancellable / unkillable tasks may starve jobs of resoures

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727896#comment-15727896 ] Apache Spark commented on SPARK-18761: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Created] (SPARK-18761) Uncancellable / unkillable tasks may starve jobs of resoures

2016-12-06 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-18761: -- Summary: Uncancellable / unkillable tasks may starve jobs of resoures Key: SPARK-18761 URL: https://issues.apache.org/jira/browse/SPARK-18761 Project: Spark

[jira] [Comment Edited] (SPARK-18709) Automatic null conversion bug (instead of throwing error) when creating a Spark Datarame with incompatible types for fields.

2016-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727763#comment-15727763 ] Dongjoon Hyun edited comment on SPARK-18709 at 12/7/16 5:32 AM: Hi,

[jira] [Commented] (SPARK-18709) Automatic null conversion bug (instead of throwing error) when creating a Spark Datarame with incompatible types for fields.

2016-12-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727763#comment-15727763 ] Dongjoon Hyun commented on SPARK-18709: --- @srowen . The type verification was introduced by

[jira] [Assigned] (SPARK-18760) Provide consistent format output for all file formats

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18760: Assignee: Reynold Xin (was: Apache Spark) > Provide consistent format output for all

[jira] [Commented] (SPARK-18760) Provide consistent format output for all file formats

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727746#comment-15727746 ] Apache Spark commented on SPARK-18760: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18760) Provide consistent format output for all file formats

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18760: Assignee: Apache Spark (was: Reynold Xin) > Provide consistent format output for all

[jira] [Created] (SPARK-18760) Provide consistent format output for all file formats

2016-12-06 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-18760: --- Summary: Provide consistent format output for all file formats Key: SPARK-18760 URL: https://issues.apache.org/jira/browse/SPARK-18760 Project: Spark Issue

[jira] [Closed] (SPARK-11482) Maven repo in IsolatedClientLoader should be configurable.

2016-12-06 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-11482. --- Resolution: Later > Maven repo in IsolatedClientLoader should be configurable. >

[jira] [Closed] (SPARK-7263) Add new shuffle manager which stores shuffle blocks in Parquet

2016-12-06 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-7263. -- Resolution: Later > Add new shuffle manager which stores shuffle blocks in Parquet >

[jira] [Closed] (SPARK-8398) Consistently expose Hadoop Configuration/JobConf parameters for Hadoop input/output formats

2016-12-06 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-8398. -- Resolution: Later > Consistently expose Hadoop Configuration/JobConf parameters for Hadoop >

[jira] [Updated] (SPARK-18678) Skewed reservoir sampling in SamplingUtils

2016-12-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18678: -- Summary: Skewed reservoir sampling in SamplingUtils (was: Skewed feature subsampling in Random

[jira] [Resolved] (SPARK-16948) Use metastore schema instead of inferring schema for ORC in HiveMetastoreCatalog

2016-12-06 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16948. - Resolution: Fixed Assignee: Eric Liang Fix Version/s: 2.1.0 > Use metastore

[jira] [Created] (SPARK-18759) when use spark streaming with sparksql, lots of temp directories are created.

2016-12-06 Thread Albert Cheng (JIRA)
Albert Cheng created SPARK-18759: Summary: when use spark streaming with sparksql, lots of temp directories are created. Key: SPARK-18759 URL: https://issues.apache.org/jira/browse/SPARK-18759

[jira] [Commented] (SPARK-18758) StreamingQueryListener events from a StreamingQuery should be sent only to the listeners in the same session as the query

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727580#comment-15727580 ] Apache Spark commented on SPARK-18758: -- User 'tdas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18758) StreamingQueryListener events from a StreamingQuery should be sent only to the listeners in the same session as the query

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18758: Assignee: (was: Apache Spark) > StreamingQueryListener events from a StreamingQuery

[jira] [Assigned] (SPARK-18758) StreamingQueryListener events from a StreamingQuery should be sent only to the listeners in the same session as the query

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18758: Assignee: Apache Spark > StreamingQueryListener events from a StreamingQuery should be

[jira] [Updated] (SPARK-18758) StreamingQueryListener events from a StreamingQuery should be sent only to the listeners in the same session as the query

2016-12-06 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-18758: -- Description: Listeners added with `sparkSession.streams.addListener(l)` are added to a

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

2016-12-06 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727542#comment-15727542 ] Liang-Chi Hsieh commented on SPARK-18539: - [~lian cheng], in Parquet's code, looks like a null

[jira] [Created] (SPARK-18758) StreamingQueryListener events from a StreamingQuery should be sent only to the listeners in the same session as the query

2016-12-06 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-18758: - Summary: StreamingQueryListener events from a StreamingQuery should be sent only to the listeners in the same session as the query Key: SPARK-18758 URL:

[jira] [Updated] (SPARK-18757) Models in Pyspark support column setters

2016-12-06 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-18757: - Description: Recently, I found three places in which column setters are missing: KMeansModel,

[jira] [Commented] (SPARK-18753) Inconsistent behavior after writing to parquet files

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727496#comment-15727496 ] Apache Spark commented on SPARK-18753: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-18753) Inconsistent behavior after writing to parquet files

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18753: Assignee: Apache Spark > Inconsistent behavior after writing to parquet files >

[jira] [Updated] (SPARK-18757) Models in Pyspark support column setters

2016-12-06 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-18757: - Description: Recently, I found three places in which column setters are missing: KMeansModel,

[jira] [Assigned] (SPARK-18753) Inconsistent behavior after writing to parquet files

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18753: Assignee: (was: Apache Spark) > Inconsistent behavior after writing to parquet files

[jira] [Updated] (SPARK-18757) Models in Pyspark support column setters

2016-12-06 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-18757: - Description: Recently, I found three places in which column setters are missing: KMeansModel,

[jira] [Updated] (SPARK-18757) Models in Pyspark support column setters

2016-12-06 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-18757: - Description: Recently, I found three places in which column setters are missing: KMeansModel,

[jira] [Created] (SPARK-18757) Models in Pyspark support column setters

2016-12-06 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-18757: Summary: Models in Pyspark support column setters Key: SPARK-18757 URL: https://issues.apache.org/jira/browse/SPARK-18757 Project: Spark Issue Type:

[jira] [Commented] (SPARK-18085) Better History Server scalability for many / large applications

2016-12-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727475#comment-15727475 ] Marcelo Vanzin commented on SPARK-18085: I'm not trying to flame you. I'm trying to point out

[jira] [Commented] (SPARK-18756) Memory leak in Spark streaming

2016-12-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727466#comment-15727466 ] Sean Owen commented on SPARK-18756: --- CC [~zsxwing] is this related to the netty byte buffer stuff

[jira] [Commented] (SPARK-18085) Better History Server scalability for many / large applications

2016-12-06 Thread Dmitry Buzolin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727446#comment-15727446 ] Dmitry Buzolin commented on SPARK-18085: I posted my comments not to start the endless flame on

[jira] [Updated] (SPARK-18756) Memory leak in Spark streaming

2016-12-06 Thread Udit Mehrotra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra updated SPARK-18756: -- Description: We have a Spark streaming application, that processes data from Kinesis. In our

[jira] [Created] (SPARK-18756) Memory leak in Spark streaming

2016-12-06 Thread Udit Mehrotra (JIRA)
Udit Mehrotra created SPARK-18756: - Summary: Memory leak in Spark streaming Key: SPARK-18756 URL: https://issues.apache.org/jira/browse/SPARK-18756 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-18739) Models in pyspark.classification and regression support setXXXCol methods

2016-12-06 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-18739: - Summary: Models in pyspark.classification and regression support setXXXCol methods (was: Models

[jira] [Commented] (SPARK-18736) CreateMap allows non-unique keys

2016-12-06 Thread Shuai Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727398#comment-15727398 ] Shuai Lin commented on SPARK-18736: --- Ok, sounds good to me. > CreateMap allows non-unique keys >

[jira] [Updated] (SPARK-18755) Add Randomized Grid Search to Spark ML

2016-12-06 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang updated SPARK-18755: --- Description: Randomized Grid Search implements a randomized search over parameters, where each

[jira] [Updated] (SPARK-18755) Add Randomized Grid Search to Spark ML

2016-12-06 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang updated SPARK-18755: --- Description: Randomized Grid Search implements a randomized search over parameters, where each

[jira] [Commented] (SPARK-18671) Add tests to ensure stability of that all Structured Streaming log formats

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727365#comment-15727365 ] Apache Spark commented on SPARK-18671: -- User 'tdas' has created a pull request for this issue:

[jira] [Created] (SPARK-18755) Add Randomized Grid Search to Spark ML

2016-12-06 Thread yuhao yang (JIRA)
yuhao yang created SPARK-18755: -- Summary: Add Randomized Grid Search to Spark ML Key: SPARK-18755 URL: https://issues.apache.org/jira/browse/SPARK-18755 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-18754) Rename recentProgresses to recentProgress

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18754: Assignee: Michael Armbrust (was: Apache Spark) > Rename recentProgresses to

[jira] [Assigned] (SPARK-18754) Rename recentProgresses to recentProgress

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18754: Assignee: Apache Spark (was: Michael Armbrust) > Rename recentProgresses to

[jira] [Commented] (SPARK-18754) Rename recentProgresses to recentProgress

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727318#comment-15727318 ] Apache Spark commented on SPARK-18754: -- User 'marmbrus' has created a pull request for this issue:

[jira] [Updated] (SPARK-18754) Rename recentProgresses to recentProgress

2016-12-06 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-18754: - Target Version/s: 2.1.0 > Rename recentProgresses to recentProgress >

[jira] [Created] (SPARK-18754) Rename recentProgresses to recentProgress

2016-12-06 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-18754: Summary: Rename recentProgresses to recentProgress Key: SPARK-18754 URL: https://issues.apache.org/jira/browse/SPARK-18754 Project: Spark Issue

[jira] [Assigned] (SPARK-18697) Upgrade sbt plugins

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18697: Assignee: Apache Spark (was: Weiqing Yang) > Upgrade sbt plugins > --- >

[jira] [Assigned] (SPARK-18697) Upgrade sbt plugins

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18697: Assignee: Weiqing Yang (was: Apache Spark) > Upgrade sbt plugins > --- >

[jira] [Updated] (SPARK-18697) Upgrade sbt plugins

2016-12-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18697: -- Fix Version/s: (was: 2.2.0) > Upgrade sbt plugins > --- > > Key:

[jira] [Resolved] (SPARK-18734) Represent timestamp in StreamingQueryProgress as formatted string instead of millis

2016-12-06 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-18734. -- Resolution: Fixed Fix Version/s: 2.1.0 > Represent timestamp in StreamingQueryProgress

[jira] [Reopened] (SPARK-18697) Upgrade sbt plugins

2016-12-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-18697: --- I had to revert this because it didn't work with Scala 2.10 > Upgrade sbt plugins > ---

[jira] [Assigned] (SPARK-18752) "isSrcLocal" parameter to Hive loadTable / loadPartition should come from user

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18752: Assignee: Apache Spark > "isSrcLocal" parameter to Hive loadTable / loadPartition should

[jira] [Assigned] (SPARK-18752) "isSrcLocal" parameter to Hive loadTable / loadPartition should come from user

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18752: Assignee: (was: Apache Spark) > "isSrcLocal" parameter to Hive loadTable /

[jira] [Commented] (SPARK-18752) "isSrcLocal" parameter to Hive loadTable / loadPartition should come from user

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727203#comment-15727203 ] Apache Spark commented on SPARK-18752: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-18753) Inconsistent behavior after writing to parquet files

2016-12-06 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727199#comment-15727199 ] Shixiong Zhu commented on SPARK-18753: -- cc [~liancheng] > Inconsistent behavior after writing to

[jira] [Updated] (SPARK-18753) Inconsistent behavior after writing to parquet files

2016-12-06 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18753: - Description: Found an inconsistent behavior when using parquet. {code} scala> val ds =

[jira] [Updated] (SPARK-18753) Inconsistent behavior after writing to parquet files

2016-12-06 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18753: - Description: Found an inconsistent behavior when using parquet. {code} scala> val ds =

[jira] [Updated] (SPARK-18753) Inconsistent behavior after writing to parquet files

2016-12-06 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18753: - Description: Found an inconsistent behavior when using parquet. {code} scala> val ds =

[jira] [Created] (SPARK-18753) Inconsistent behavior after writing to parquet files

2016-12-06 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-18753: Summary: Inconsistent behavior after writing to parquet files Key: SPARK-18753 URL: https://issues.apache.org/jira/browse/SPARK-18753 Project: Spark Issue

[jira] [Resolved] (SPARK-18662) Move cluster managers into their own sub-directory

2016-12-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-18662. Resolution: Fixed Assignee: Anirudh Ramanathan Fix Version/s: 2.2.0 > Move

[jira] [Reopened] (SPARK-17838) Strict type checking for arguments with a better messages across APIs.

2016-12-06 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reopened SPARK-17838: -- Assignee: (was: Hyukjin Kwon) Re-open as per discussion in PR. > Strict type checking

[jira] [Resolved] (SPARK-18171) Show correct framework address in mesos master web ui when the advertised address is used

2016-12-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18171. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 15684

[jira] [Updated] (SPARK-18171) Show correct framework address in mesos master web ui when the advertised address is used

2016-12-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18171: -- Assignee: Shuai Lin > Show correct framework address in mesos master web ui when the advertised >

[jira] [Created] (SPARK-18752) "isSrcLocal" parameter to Hive loadTable / loadPartition should come from user

2016-12-06 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-18752: -- Summary: "isSrcLocal" parameter to Hive loadTable / loadPartition should come from user Key: SPARK-18752 URL: https://issues.apache.org/jira/browse/SPARK-18752

[jira] [Closed] (SPARK-18741) Reuse/Explicitly clean-up SparkContext in Streaming tests

2016-12-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-18741. - Resolution: Not A Problem > Reuse/Explicitly clean-up SparkContext in Streaming tests >

[jira] [Commented] (SPARK-18728) Consider using Algebird's Aggregator instead of org.apache.spark.sql.expressions.Aggregator

2016-12-06 Thread Alex Levenson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726986#comment-15726986 ] Alex Levenson commented on SPARK-18728: --- I think my comment above lists some concrete benefits.

[jira] [Assigned] (SPARK-18751) Deadlock when SparkContext.stop is called in Utils.tryOrStopSparkContext

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18751: Assignee: Shixiong Zhu (was: Apache Spark) > Deadlock when SparkContext.stop is called

[jira] [Commented] (SPARK-18751) Deadlock when SparkContext.stop is called in Utils.tryOrStopSparkContext

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726963#comment-15726963 ] Apache Spark commented on SPARK-18751: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18751) Deadlock when SparkContext.stop is called in Utils.tryOrStopSparkContext

2016-12-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18751: Assignee: Apache Spark (was: Shixiong Zhu) > Deadlock when SparkContext.stop is called

  1   2   3   >