[jira] [Assigned] (SPARK-16929) Speculation-related synchronization bottleneck in checkSpeculatableTasks

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16929: Assignee: Apache Spark > Speculation-related synchronization bottleneck in

[jira] [Assigned] (SPARK-16929) Speculation-related synchronization bottleneck in checkSpeculatableTasks

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16929: Assignee: (was: Apache Spark) > Speculation-related synchronization bottleneck in

[jira] [Commented] (SPARK-16929) Speculation-related synchronization bottleneck in checkSpeculatableTasks

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859175#comment-15859175 ] Apache Spark commented on SPARK-16929: -- User 'jinxing64' has created a pull request for this issue:

[jira] [Commented] (SPARK-19493) Remove Java 7 support

2017-02-08 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859166#comment-15859166 ] Liang-Chi Hsieh commented on SPARK-19493: - +1 > Remove Java 7 support > - >

[jira] [Assigned] (SPARK-19529) TransportClientFactory.createClient() shouldn't call awaitUninterruptibly()

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19529: Assignee: Josh Rosen (was: Apache Spark) > TransportClientFactory.createClient()

[jira] [Commented] (SPARK-19529) TransportClientFactory.createClient() shouldn't call awaitUninterruptibly()

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859160#comment-15859160 ] Apache Spark commented on SPARK-19529: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19529) TransportClientFactory.createClient() shouldn't call awaitUninterruptibly()

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19529: Assignee: Apache Spark (was: Josh Rosen) > TransportClientFactory.createClient()

[jira] [Assigned] (SPARK-19530) Use guava weigher for code cache eviction

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19530: Assignee: Apache Spark > Use guava weigher for code cache eviction >

[jira] [Assigned] (SPARK-19530) Use guava weigher for code cache eviction

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19530: Assignee: (was: Apache Spark) > Use guava weigher for code cache eviction >

[jira] [Commented] (SPARK-19530) Use guava weigher for code cache eviction

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859157#comment-15859157 ] Apache Spark commented on SPARK-19530: -- User 'viirya' has created a pull request for this issue:

[jira] [Created] (SPARK-19530) Use guava weigher for code cache eviction

2017-02-08 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-19530: --- Summary: Use guava weigher for code cache eviction Key: SPARK-19530 URL: https://issues.apache.org/jira/browse/SPARK-19530 Project: Spark Issue Type:

[jira] [Created] (SPARK-19529) TransportClientFactory.createClient() shouldn't call awaitUninterruptibly()

2017-02-08 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-19529: -- Summary: TransportClientFactory.createClient() shouldn't call awaitUninterruptibly() Key: SPARK-19529 URL: https://issues.apache.org/jira/browse/SPARK-19529 Project:

[jira] [Commented] (SPARK-19473) Several DataFrame Methods still fail with dot in column names

2017-02-08 Thread Thomas Sebastian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859135#comment-15859135 ] Thomas Sebastian commented on SPARK-19473: -- As mentioned in the PR:

[jira] [Commented] (SPARK-19496) to_date with format has weird behavior

2017-02-08 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859091#comment-15859091 ] Xiao Li commented on SPARK-19496: - [~cloud_fan] Yeah, null looks better. Illegal inputs should not be

[jira] [Commented] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

2017-02-08 Thread Shirish (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859086#comment-15859086 ] Shirish commented on SPARK-12837: - Any workaround guys till we fix this? > Spark driver requires large

[jira] [Commented] (SPARK-18113) Sending AskPermissionToCommitOutput failed, driver enter into task deadloop

2017-02-08 Thread jin xing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859015#comment-15859015 ] jin xing commented on SPARK-18113: -- [~xukun] Can you reproduce the bug with steps above? When

[jira] [Commented] (SPARK-18924) Improve collect/createDataFrame performance in SparkR

2017-02-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858963#comment-15858963 ] Xiangrui Meng commented on SPARK-18924: --- I'm going to work on this one. So removed myself from

[jira] [Updated] (SPARK-18924) Improve collect/createDataFrame performance in SparkR

2017-02-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-18924: -- Shepherd: (was: Xiangrui Meng) > Improve collect/createDataFrame performance in SparkR >

[jira] [Assigned] (SPARK-18924) Improve collect/createDataFrame performance in SparkR

2017-02-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-18924: - Assignee: Xiangrui Meng > Improve collect/createDataFrame performance in SparkR >

[jira] [Created] (SPARK-19528) external shuffle service would close while still have request from executor when dynamic allocation is enabled

2017-02-08 Thread KaiXu (JIRA)
KaiXu created SPARK-19528: - Summary: external shuffle service would close while still have request from executor when dynamic allocation is enabled Key: SPARK-19528 URL: https://issues.apache.org/jira/browse/SPARK-19528

[jira] [Comment Edited] (SPARK-19523) Spark streaming+ insert into table leaves bunch of trash in table directory

2017-02-08 Thread Egor Pahomov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858913#comment-15858913 ] Egor Pahomov edited comment on SPARK-19523 at 2/9/17 2:52 AM: -- @zsxwing, I

[jira] [Commented] (SPARK-19523) Spark streaming+ insert into table leaves bunch of trash in table directory

2017-02-08 Thread Egor Pahomov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858913#comment-15858913 ] Egor Pahomov commented on SPARK-19523: -- @zsxwing, I haven't found the way to that. inside foreachRDD

[jira] [Commented] (SPARK-19524) newFilesOnly does not work according to docs.

2017-02-08 Thread Egor Pahomov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858912#comment-15858912 ] Egor Pahomov commented on SPARK-19524: -- [~uncleGen], sorry, I haven't understood. > newFilesOnly

[jira] [Commented] (SPARK-19524) newFilesOnly does not work according to docs.

2017-02-08 Thread Egor Pahomov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858910#comment-15858910 ] Egor Pahomov commented on SPARK-19524: -- [~srowen], based on documentation, which says "newFilesOnly

[jira] [Assigned] (SPARK-19527) Approximate Size of Intersection of Bloom Filters

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19527: Assignee: Apache Spark > Approximate Size of Intersection of Bloom Filters >

[jira] [Assigned] (SPARK-19527) Approximate Size of Intersection of Bloom Filters

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19527: Assignee: (was: Apache Spark) > Approximate Size of Intersection of Bloom Filters >

[jira] [Commented] (SPARK-19527) Approximate Size of Intersection of Bloom Filters

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858881#comment-15858881 ] Apache Spark commented on SPARK-19527: -- User 'Bcpoole' has created a pull request for this issue:

[jira] [Updated] (SPARK-19527) Approximate Size of Intersection of Bloom Filters

2017-02-08 Thread Brandon Poole (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Poole updated SPARK-19527: -- Summary: Approximate Size of Intersection of Bloom Filters (was: Approximate size of

[jira] [Created] (SPARK-19527) Approximate size of Intersection of Bloom Filters

2017-02-08 Thread Brandon Poole (JIRA)
Brandon Poole created SPARK-19527: - Summary: Approximate size of Intersection of Bloom Filters Key: SPARK-19527 URL: https://issues.apache.org/jira/browse/SPARK-19527 Project: Spark Issue

[jira] [Created] (SPARK-19526) Spark should rise an exception when it tries to read a Hive view but it doesn't have read access on the corresponding table(s)

2017-02-08 Thread Reza Safi (JIRA)
Reza Safi created SPARK-19526: - Summary: Spark should rise an exception when it tries to read a Hive view but it doesn't have read access on the corresponding table(s) Key: SPARK-19526 URL:

[jira] [Commented] (SPARK-19524) newFilesOnly does not work according to docs.

2017-02-08 Thread Genmao Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858819#comment-15858819 ] Genmao Yu commented on SPARK-19524: --- Current implementation will clear the old time-to-files mappings

[jira] [Commented] (SPARK-19525) Enable Compression of Spark Streaming Checkpoints

2017-02-08 Thread Aaditya Ramesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858810#comment-15858810 ] Aaditya Ramesh commented on SPARK-19525: We have a patch that works for an older version. I am

[jira] [Assigned] (SPARK-19520) WAL should not be encrypted

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19520: Assignee: Apache Spark > WAL should not be encrypted > --- > >

[jira] [Commented] (SPARK-19520) WAL should not be encrypted

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858797#comment-15858797 ] Apache Spark commented on SPARK-19520: -- User 'vanzin' has created a pull request for this issue:

[jira] [Created] (SPARK-19525) Enable Compression of Spark Streaming Checkpoints

2017-02-08 Thread Aaditya Ramesh (JIRA)
Aaditya Ramesh created SPARK-19525: -- Summary: Enable Compression of Spark Streaming Checkpoints Key: SPARK-19525 URL: https://issues.apache.org/jira/browse/SPARK-19525 Project: Spark Issue

[jira] [Assigned] (SPARK-19520) WAL should not be encrypted

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19520: Assignee: (was: Apache Spark) > WAL should not be encrypted >

[jira] [Commented] (SPARK-19282) RandomForestRegressionModel should expose getMaxDepth

2017-02-08 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858744#comment-15858744 ] Xin Ren commented on SPARK-19282: - I just got approved by my company to work on this one resuming my

[jira] [Commented] (SPARK-19523) Spark streaming+ insert into table leaves bunch of trash in table directory

2017-02-08 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858716#comment-15858716 ] Shixiong Zhu commented on SPARK-19523: -- These files are temp files created by HiveContext. You can

[jira] [Updated] (SPARK-19523) Spark streaming+ insert into table leaves bunch of trash in table directory

2017-02-08 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19523: - Component/s: DStreams > Spark streaming+ insert into table leaves bunch of trash in table

[jira] [Updated] (SPARK-19523) Spark streaming+ insert into table leaves bunch of trash in table directory

2017-02-08 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19523: - Component/s: (was: Structured Streaming) > Spark streaming+ insert into table leaves bunch

[jira] [Updated] (SPARK-19524) newFilesOnly does not work according to docs.

2017-02-08 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19524: - Component/s: (was: Structured Streaming) DStreams > newFilesOnly does not

[jira] [Updated] (SPARK-19522) --executor-memory flag doesn't work in local-cluster mode

2017-02-08 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-19522: -- Description: {code} bin/spark-shell --master local-cluster[2,1,2048] {code} doesn't do what you think

[jira] [Commented] (SPARK-19524) newFilesOnly does not work according to docs.

2017-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858667#comment-15858667 ] Sean Owen commented on SPARK-19524: --- I don't understand what you're reporting. Summarize expected,

[jira] [Assigned] (SPARK-18613) spark.ml LDA classes should not expose spark.mllib in APIs

2017-02-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-18613: - Assignee: Sue Ann Hong > spark.ml LDA classes should not expose spark.mllib in

[jira] [Created] (SPARK-19524) newFilesOnly does not work according to docs.

2017-02-08 Thread Egor Pahomov (JIRA)
Egor Pahomov created SPARK-19524: Summary: newFilesOnly does not work according to docs. Key: SPARK-19524 URL: https://issues.apache.org/jira/browse/SPARK-19524 Project: Spark Issue Type:

[jira] [Updated] (SPARK-19523) Spark streaming+ insert into table leaves bunch of trash in table directory

2017-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19523: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) > Spark streaming+ insert into

[jira] [Created] (SPARK-19523) Spark streaming+ insert into table leaves bunch of trash in table directory

2017-02-08 Thread Egor Pahomov (JIRA)
Egor Pahomov created SPARK-19523: Summary: Spark streaming+ insert into table leaves bunch of trash in table directory Key: SPARK-19523 URL: https://issues.apache.org/jira/browse/SPARK-19523 Project:

[jira] [Assigned] (SPARK-18613) spark.ml LDA classes should not expose spark.mllib in APIs

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18613: Assignee: (was: Apache Spark) > spark.ml LDA classes should not expose spark.mllib in

[jira] [Assigned] (SPARK-18613) spark.ml LDA classes should not expose spark.mllib in APIs

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18613: Assignee: Apache Spark > spark.ml LDA classes should not expose spark.mllib in APIs >

[jira] [Commented] (SPARK-18613) spark.ml LDA classes should not expose spark.mllib in APIs

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858645#comment-15858645 ] Apache Spark commented on SPARK-18613: -- User 'sueann' has created a pull request for this issue:

[jira] [Created] (SPARK-19522) --executor-memory flag doesn't work in local-cluster mode

2017-02-08 Thread Andrew Or (JIRA)
Andrew Or created SPARK-19522: - Summary: --executor-memory flag doesn't work in local-cluster mode Key: SPARK-19522 URL: https://issues.apache.org/jira/browse/SPARK-19522 Project: Spark Issue

[jira] [Issue Comment Deleted] (SPARK-19521) Error with embedded line break (multi-line record) in csv file.

2017-02-08 Thread Ruslan Korniichuk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruslan Korniichuk updated SPARK-19521: -- Comment: was deleted (was: :: def create_json(src, dst, logger=None): """Create

[jira] [Commented] (SPARK-19521) Error with embedded line break (multi-line record) in csv file.

2017-02-08 Thread Ruslan Korniichuk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858632#comment-15858632 ] Ruslan Korniichuk commented on SPARK-19521: --- df = sql_context.read.csv( path=src,

[jira] [Commented] (SPARK-19521) Error with embedded line break (multi-line record) in csv file.

2017-02-08 Thread Ruslan Korniichuk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858631#comment-15858631 ] Ruslan Korniichuk commented on SPARK-19521: --- :: def create_json(src, dst, logger=None):

[jira] [Issue Comment Deleted] (SPARK-19521) Error with embedded line break (multi-line record) in csv file.

2017-02-08 Thread Ruslan Korniichuk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruslan Korniichuk updated SPARK-19521: -- Comment: was deleted (was: def create_json(src, dst, logger=None): """Create json

[jira] [Commented] (SPARK-19521) Error with embedded line break (multi-line record) in csv file.

2017-02-08 Thread Ruslan Korniichuk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858628#comment-15858628 ] Ruslan Korniichuk commented on SPARK-19521: --- def create_json(src, dst, logger=None):

[jira] [Updated] (SPARK-19521) Error with embedded line break (multi-line record) in csv file.

2017-02-08 Thread Ruslan Korniichuk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruslan Korniichuk updated SPARK-19521: -- Description: Input csv file: id,name,amount,isActive,Remark 1,Barney &

[jira] [Created] (SPARK-19521) Error with embedded line break (multi-line record) in csv file.

2017-02-08 Thread Ruslan Korniichuk (JIRA)
Ruslan Korniichuk created SPARK-19521: - Summary: Error with embedded line break (multi-line record) in csv file. Key: SPARK-19521 URL: https://issues.apache.org/jira/browse/SPARK-19521 Project:

[jira] [Created] (SPARK-19520) WAL should not be encrypted

2017-02-08 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-19520: -- Summary: WAL should not be encrypted Key: SPARK-19520 URL: https://issues.apache.org/jira/browse/SPARK-19520 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-19489) Stable serialization format for external & native code integration

2017-02-08 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858577#comment-15858577 ] Wes McKinney commented on SPARK-19489: -- I'm really glad to see this is becoming a priority in 2017.

[jira] [Updated] (SPARK-19519) Groupby for multiple columns not working

2017-02-08 Thread Faisal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Faisal updated SPARK-19519: --- Description: Please look at the below join between multiple dataframes, then while applying groupby

[jira] [Updated] (SPARK-19519) Groupby for multiple columns not working

2017-02-08 Thread Faisal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Faisal updated SPARK-19519: --- Description: Please look at the below join between multiple dataframes, then while applying groupby

[jira] [Comment Edited] (SPARK-19507) pyspark.sql.types._verify_type() exceptions too broad to debug collections or nested data

2017-02-08 Thread David Gingrich (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858565#comment-15858565 ] David Gingrich edited comment on SPARK-19507 at 2/8/17 9:37 PM: I'd

[jira] [Resolved] (SPARK-18072) empty/null Timestamp field

2017-02-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18072. --- Resolution: Cannot Reproduce > empty/null Timestamp field > -- > >

[jira] [Updated] (SPARK-19519) Groupby for multiple columns not working

2017-02-08 Thread Faisal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Faisal updated SPARK-19519: --- Description: DataFrame joinModCtypeAsgns = modCtypeAsgnsDf.as("mod")

[jira] [Created] (SPARK-19519) Groupby for multiple columns not working

2017-02-08 Thread Faisal (JIRA)
Faisal created SPARK-19519: -- Summary: Groupby for multiple columns not working Key: SPARK-19519 URL: https://issues.apache.org/jira/browse/SPARK-19519 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-19507) pyspark.sql.types._verify_type() exceptions too broad to debug collections or nested data

2017-02-08 Thread David Gingrich (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858565#comment-15858565 ] David Gingrich commented on SPARK-19507: I'd advocate that it be made public, but I was saving

[jira] [Updated] (SPARK-19517) KafkaSource fails to initialize partition offsets

2017-02-08 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19517: - Target Version/s: 2.1.1, 2.2.0 > KafkaSource fails to initialize partition offsets >

[jira] [Updated] (SPARK-19517) KafkaSource fails to initialize partition offsets

2017-02-08 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19517: - Priority: Blocker (was: Critical) > KafkaSource fails to initialize partition offsets >

[jira] [Assigned] (SPARK-17714) ClassCircularityError is thrown when using org.apache.spark.util.Utils.classForName 

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17714: Assignee: (was: Apache Spark) > ClassCircularityError is thrown when using >

[jira] [Assigned] (SPARK-17714) ClassCircularityError is thrown when using org.apache.spark.util.Utils.classForName 

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17714: Assignee: Apache Spark > ClassCircularityError is thrown when using >

[jira] [Commented] (SPARK-17714) ClassCircularityError is thrown when using org.apache.spark.util.Utils.classForName 

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858482#comment-15858482 ] Apache Spark commented on SPARK-17714: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Updated] (SPARK-19413) Basic mapGroupsWithState API

2017-02-08 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19413: - Fix Version/s: 2.1.1 > Basic mapGroupsWithState API > > >

[jira] [Commented] (SPARK-17139) Add model summary for MultinomialLogisticRegression

2017-02-08 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858438#comment-15858438 ] Seth Hendrickson commented on SPARK-17139: -- Seems like a reasonable way to solve a messy problem

[jira] [Commented] (SPARK-17139) Add model summary for MultinomialLogisticRegression

2017-02-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858373#comment-15858373 ] Joseph K. Bradley commented on SPARK-17139: --- @sethah Yep, that looks like what I had in mind.

[jira] [Resolved] (SPARK-19400) GLM fails for intercept only model

2017-02-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-19400. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16740

[jira] [Assigned] (SPARK-19400) GLM fails for intercept only model

2017-02-08 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-19400: - Assignee: Wayne Zhang > GLM fails for intercept only model >

[jira] [Commented] (SPARK-19464) Remove support for Hadoop 2.5 and earlier

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858330#comment-15858330 ] Apache Spark commented on SPARK-19464: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-19473) Several DataFrame Methods still fail with dot in column names

2017-02-08 Thread Jayadevan M (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858239#comment-15858239 ] Jayadevan M commented on SPARK-19473: - I think this is a duplicate jira -

[jira] [Commented] (SPARK-19518) IGNORE NULLS in first_value / last_value should be supported in SQL statements

2017-02-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858163#comment-15858163 ] Herman van Hovell commented on SPARK-19518: --- Spark SQL has its own parser as of Spark 2.0. We

[jira] [Commented] (SPARK-19489) Stable serialization format for external & native code integration

2017-02-08 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858160#comment-15858160 ] Takeshi Yamamuro commented on SPARK-19489: -- Aha, I see. > Stable serialization format for

[jira] [Commented] (SPARK-19489) Stable serialization format for external & native code integration

2017-02-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858156#comment-15858156 ] Reynold Xin commented on SPARK-19489: - The ticket doesn't dictate either. > Stable serialization

[jira] [Created] (SPARK-19518) IGNORE NULLS in first_value / last_value should be supported in SQL statements

2017-02-08 Thread Ferenc Erdelyi (JIRA)
Ferenc Erdelyi created SPARK-19518: -- Summary: IGNORE NULLS in first_value / last_value should be supported in SQL statements Key: SPARK-19518 URL: https://issues.apache.org/jira/browse/SPARK-19518

[jira] [Assigned] (SPARK-19517) KafkaSource fails to initialize partition offsets

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19517: Assignee: (was: Apache Spark) > KafkaSource fails to initialize partition offsets >

[jira] [Assigned] (SPARK-19517) KafkaSource fails to initialize partition offsets

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19517: Assignee: Apache Spark > KafkaSource fails to initialize partition offsets >

[jira] [Commented] (SPARK-19517) KafkaSource fails to initialize partition offsets

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858144#comment-15858144 ] Apache Spark commented on SPARK-19517: -- User 'vitillo' has created a pull request for this issue:

[jira] [Updated] (SPARK-19517) KafkaSource fails to initialize partition offsets

2017-02-08 Thread Roberto Agostino Vitillo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roberto Agostino Vitillo updated SPARK-19517: - Description: A Kafka source with many partitions can cause the

[jira] [Created] (SPARK-19517) KafkaSource fails to initialize partition offsets

2017-02-08 Thread Roberto Agostino Vitillo (JIRA)
Roberto Agostino Vitillo created SPARK-19517: Summary: KafkaSource fails to initialize partition offsets Key: SPARK-19517 URL: https://issues.apache.org/jira/browse/SPARK-19517 Project:

[jira] [Updated] (SPARK-19161) Improving UDF Docstrings

2017-02-08 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-19161: --- Priority: Minor (was: Major) > Improving UDF Docstrings >

[jira] [Updated] (SPARK-19162) UserDefinedFunction constructor should verify that func is callable

2017-02-08 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-19162: --- Priority: Minor (was: Major) > UserDefinedFunction constructor should verify that

[jira] [Updated] (SPARK-19165) UserDefinedFunction should verify call arguments and provide readable exception in case of mismatch

2017-02-08 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-19165: --- Priority: Minor (was: Major) > UserDefinedFunction should verify call arguments and

[jira] [Updated] (SPARK-19161) Improving UDF Docstrings

2017-02-08 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-19161: --- Priority: Major (was: Minor) > Improving UDF Docstrings >

[jira] [Commented] (SPARK-19489) Stable serialization format for external & native code integration

2017-02-08 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858129#comment-15858129 ] Takeshi Yamamuro commented on SPARK-19489: -- This ticket means we need to make new formats for

[jira] [Assigned] (SPARK-19516) update public doc to use SparkSession instead of SparkContext

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19516: Assignee: Apache Spark (was: Wenchen Fan) > update public doc to use SparkSession

[jira] [Assigned] (SPARK-19516) update public doc to use SparkSession instead of SparkContext

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19516: Assignee: Wenchen Fan (was: Apache Spark) > update public doc to use SparkSession

[jira] [Commented] (SPARK-19516) update public doc to use SparkSession instead of SparkContext

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858114#comment-15858114 ] Apache Spark commented on SPARK-19516: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19516) update public doc to use SparkSession instead of SparkContext

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19516: Assignee: Wenchen Fan (was: Apache Spark) > update public doc to use SparkSession

[jira] [Assigned] (SPARK-19516) update public doc to use SparkSession instead of SparkContext

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19516: Assignee: Apache Spark (was: Wenchen Fan) > update public doc to use SparkSession

[jira] [Created] (SPARK-19516) update public doc to use SparkSession instead of SparkContext

2017-02-08 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-19516: --- Summary: update public doc to use SparkSession instead of SparkContext Key: SPARK-19516 URL: https://issues.apache.org/jira/browse/SPARK-19516 Project: Spark

[jira] [Commented] (SPARK-13931) Resolve stage hanging up problem in a particular case

2017-02-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858103#comment-15858103 ] Apache Spark commented on SPARK-13931: -- User 'GavinGavinNo1' has created a pull request for this

  1   2   >