[jira] [Created] (SPARK-12145) Command "Set Role [ADMIN|NONE|ALL]" doesn't work in SQL based authorization

2015-12-04 Thread Ferdinand Xu (JIRA)
Ferdinand Xu created SPARK-12145: Summary: Command "Set Role [ADMIN|NONE|ALL]" doesn't work in SQL based authorization Key: SPARK-12145 URL: https://issues.apache.org/jira/browse/SPARK-12145 Project:

[jira] [Commented] (SPARK-9838) Support Poisson family in SparkR:::glm

2015-12-04 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041258#comment-15041258 ] Xusen Yin commented on SPARK-9838: -- Hi [~mengxr], can I work on this? > Support Poisson family in

[jira] [Commented] (SPARK-10123) Cannot set "--deploy-mode" in default configuration

2015-12-04 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041260#comment-15041260 ] Saisai Shao commented on SPARK-10123: - Hi [~vanzin], would you mind letting me take a crack on this?

[jira] [Assigned] (SPARK-12145) Command "Set Role [ADMIN|NONE|ALL]" doesn't work in SQL based authorization

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12145: Assignee: (was: Apache Spark) > Command "Set Role [ADMIN|NONE|ALL]" doesn't work in

[jira] [Commented] (SPARK-12145) Command "Set Role [ADMIN|NONE|ALL]" doesn't work in SQL based authorization

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041254#comment-15041254 ] Apache Spark commented on SPARK-12145: -- User 'winningsix' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12145) Command "Set Role [ADMIN|NONE|ALL]" doesn't work in SQL based authorization

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12145: Assignee: Apache Spark > Command "Set Role [ADMIN|NONE|ALL]" doesn't work in SQL based

[jira] [Created] (SPARK-12146) SparkR jsonFile should support multiple input files

2015-12-04 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-12146: --- Summary: SparkR jsonFile should support multiple input files Key: SPARK-12146 URL: https://issues.apache.org/jira/browse/SPARK-12146 Project: Spark Issue

[jira] [Commented] (SPARK-12103) KafkaUtils createStream with multiple topics -- does not work as expected

2015-12-04 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041301#comment-15041301 ] Saisai Shao commented on SPARK-12103: - I think I had a proposal of message handler (receiver

[jira] [Updated] (SPARK-12146) SparkR jsonFile should support multiple input files

2015-12-04 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-12146: Description: This bug is easy to reproduce, jsonFile did not support character vector as

[jira] [Commented] (SPARK-12137) Spark Streaming State Recovery limitations

2015-12-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041500#comment-15041500 ] Sean Owen commented on SPARK-12137: --- The checkpoint mechanism works for recovering after an app fails

[jira] [Commented] (SPARK-11868) wrong results returned from dataframe create from Rows without consistent schma on pyspark

2015-12-04 Thread Yuval Tanny (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041497#comment-15041497 ] Yuval Tanny commented on SPARK-11868: - No. I think there should be a decision on the desired

[jira] [Updated] (SPARK-12146) SparkR jsonFile should support multiple input files

2015-12-04 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-12146: Description: > path <- c("/path/to/dir1","/path/to/dir2") > raw.terror<-jsonFile(sqlContext,path)

[jira] [Resolved] (SPARK-12122) Recovered streaming context can sometimes run a batch twice

2015-12-04 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-12122. --- Resolution: Fixed Fix Version/s: 1.6.0 > Recovered streaming context can sometimes

[jira] [Assigned] (SPARK-12146) SparkR jsonFile should support multiple input files

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12146: Assignee: (was: Apache Spark) > SparkR jsonFile should support multiple input files >

[jira] [Commented] (SPARK-12146) SparkR jsonFile should support multiple input files

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041320#comment-15041320 ] Apache Spark commented on SPARK-12146: -- User 'yanboliang' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12146) SparkR jsonFile should support multiple input files

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12146: Assignee: Apache Spark > SparkR jsonFile should support multiple input files >

[jira] [Created] (SPARK-12147) Off heap storage and dynamicAllocation operation

2015-12-04 Thread Rares Mirica (JIRA)
Rares Mirica created SPARK-12147: Summary: Off heap storage and dynamicAllocation operation Key: SPARK-12147 URL: https://issues.apache.org/jira/browse/SPARK-12147 Project: Spark Issue Type:

[jira] [Commented] (SPARK-2489) Unsupported parquet datatype optional fixed_len_byte_array

2015-12-04 Thread Jorge Sanchez (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041626#comment-15041626 ] Jorge Sanchez commented on SPARK-2489: -- This functionality would be very helpful. Is there any way we

[jira] [Commented] (SPARK-11707) StreamCorruptedException if authentication is enabled

2015-12-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041623#comment-15041623 ] Sean Owen commented on SPARK-11707: --- [~jlewandowski] any chance you can try this with the latest code?

[jira] [Updated] (SPARK-12147) Off heap storage and dynamicAllocation operation

2015-12-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12147: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) (Make the title more

[jira] [Updated] (SPARK-12147) Off heap storage and dynamicAllocation operation

2015-12-04 Thread Rares Mirica (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rares Mirica updated SPARK-12147: - Attachment: spark-defaults.conf > Off heap storage and dynamicAllocation operation >

[jira] [Commented] (SPARK-12147) Off heap storage and dynamicAllocation operation

2015-12-04 Thread Rares Mirica (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041679#comment-15041679 ] Rares Mirica commented on SPARK-12147: -- Sorry, I wasn't specific enough about the use-case and how

[jira] [Assigned] (SPARK-12026) ChiSqTest gets slower and slower over time when number of features is large

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12026: Assignee: Apache Spark > ChiSqTest gets slower and slower over time when number of

[jira] [Assigned] (SPARK-12026) ChiSqTest gets slower and slower over time when number of features is large

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12026: Assignee: (was: Apache Spark) > ChiSqTest gets slower and slower over time when

[jira] [Resolved] (SPARK-7999) Graph complement function in GraphX

2015-12-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7999. -- Resolution: Won't Fix > Graph complement function in GraphX > --- > >

[jira] [Resolved] (SPARK-7894) Graph Union Operator

2015-12-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7894. -- Resolution: Won't Fix > Graph Union Operator > > > Key: SPARK-7894

[jira] [Resolved] (SPARK-7893) Complex Operators between Graphs

2015-12-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7893. -- Resolution: Won't Fix > Complex Operators between Graphs > > >

[jira] [Commented] (SPARK-12026) ChiSqTest gets slower and slower over time when number of features is large

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041620#comment-15041620 ] Apache Spark commented on SPARK-12026: -- User 'hhbyyh' has created a pull request for this issue:

[jira] [Commented] (SPARK-12147) Off heap storage and dynamicAllocation operation

2015-12-04 Thread Rares Mirica (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041789#comment-15041789 ] Rares Mirica commented on SPARK-12147: -- Yes, I am talking about the executor stopping as part of

[jira] [Commented] (SPARK-12140) Support Streaming UI in HistoryServer

2015-12-04 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041820#comment-15041820 ] Marcelo Vanzin commented on SPARK-12140: You don't need to ask permission to work on things. >

[jira] [Commented] (SPARK-10123) Cannot set "--deploy-mode" in default configuration

2015-12-04 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041817#comment-15041817 ] Marcelo Vanzin commented on SPARK-10123: Bug is not assigned to anyone, so no need to ask

[jira] [Comment Edited] (SPARK-8513) _temporary may be left undeleted when a write job committed with FileOutputCommitter fails due to a race condition

2015-12-04 Thread Jiri Syrovy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041742#comment-15041742 ] Jiri Syrovy edited comment on SPARK-8513 at 12/4/15 4:47 PM: - The same thing

[jira] [Created] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame

2015-12-04 Thread Michael Lawrence (JIRA)
Michael Lawrence created SPARK-12148: Summary: SparkR: rename DataFrame to SparkDataFrame Key: SPARK-12148 URL: https://issues.apache.org/jira/browse/SPARK-12148 Project: Spark Issue

[jira] [Commented] (SPARK-12147) Off heap storage and dynamicAllocation operation

2015-12-04 Thread Rares Mirica (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041683#comment-15041683 ] Rares Mirica commented on SPARK-12147: -- I would also like to object on setting this as minor, this

[jira] [Commented] (SPARK-8513) _temporary may be left undeleted when a write job committed with FileOutputCommitter fails due to a race condition

2015-12-04 Thread Jiri Syrovy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041742#comment-15041742 ] Jiri Syrovy commented on SPARK-8513: The same thing happens when writing partitioned files with

[jira] [Commented] (SPARK-12147) Off heap storage and dynamicAllocation operation

2015-12-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041740#comment-15041740 ] Sean Owen commented on SPARK-12147: --- To be honest, the priority doesn't matter much at all. It won't

[jira] [Comment Edited] (SPARK-8513) _temporary may be left undeleted when a write job committed with FileOutputCommitter fails due to a race condition

2015-12-04 Thread Jiri Syrovy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041742#comment-15041742 ] Jiri Syrovy edited comment on SPARK-8513 at 12/4/15 4:45 PM: - The same thing

[jira] [Commented] (SPARK-12144) Implement DataFrameReader and DataFrameWriter API in SparkR

2015-12-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042103#comment-15042103 ] Felix Cheung commented on SPARK-12144: -- +1 [~shivaram] The style {code}

[jira] [Commented] (SPARK-12038) Add support for pickle protocol to graphite sink

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042124#comment-15042124 ] Apache Spark commented on SPARK-12038: -- User 'alberskib' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12038) Add support for pickle protocol to graphite sink

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12038: Assignee: (was: Apache Spark) > Add support for pickle protocol to graphite sink >

[jira] [Assigned] (SPARK-12038) Add support for pickle protocol to graphite sink

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12038: Assignee: Apache Spark > Add support for pickle protocol to graphite sink >

[jira] [Assigned] (SPARK-11944) Python API for mllib.clustering.BisectingKMeans

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11944: Assignee: Apache Spark > Python API for mllib.clustering.BisectingKMeans >

[jira] [Assigned] (SPARK-11944) Python API for mllib.clustering.BisectingKMeans

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11944: Assignee: (was: Apache Spark) > Python API for mllib.clustering.BisectingKMeans >

[jira] [Commented] (SPARK-11944) Python API for mllib.clustering.BisectingKMeans

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042339#comment-15042339 ] Apache Spark commented on SPARK-11944: -- User 'holdenk' has created a pull request for this issue:

[jira] [Commented] (SPARK-12152) Speed up Scalastyle by only running one SBT command instead of four

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042381#comment-15042381 ] Apache Spark commented on SPARK-12152: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12152) Speed up Scalastyle by only running one SBT command instead of four

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12152: Assignee: Apache Spark (was: Josh Rosen) > Speed up Scalastyle by only running one SBT

[jira] [Comment Edited] (SPARK-1832) Executor UI improvement suggestions

2015-12-04 Thread Alexander Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041936#comment-15041936 ] Alexander Bozarth edited comment on SPARK-1832 at 12/4/15 8:49 PM: --- I've

[jira] [Resolved] (SPARK-6990) Add Java linting script

2015-12-04 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-6990. --- Resolution: Fixed Fix Version/s: 2.0.0 Fixed by https://github.com/apache/spark/pull/9867 >

[jira] [Assigned] (SPARK-12152) Speed up Scalastyle by only running one SBT command instead of four

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12152: Assignee: Josh Rosen (was: Apache Spark) > Speed up Scalastyle by only running one SBT

[jira] [Assigned] (SPARK-12150) numPartitions argument to sqlContext.range() should be optional

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12150: Assignee: (was: Apache Spark) > numPartitions argument to sqlContext.range() should

[jira] [Commented] (SPARK-12150) numPartitions argument to sqlContext.range() should be optional

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042194#comment-15042194 ] Apache Spark commented on SPARK-12150: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12150) numPartitions argument to sqlContext.range() should be optional

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12150: Assignee: Apache Spark > numPartitions argument to sqlContext.range() should be optional

[jira] [Updated] (SPARK-11255) R Test build should run on R 3.1.1

2015-12-04 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-11255: --- Assignee: shane knapp > R Test build should run on R 3.1.1 > -- > >

[jira] [Created] (SPARK-12152) Speed up Scalastyle by only running one SBT command instead of four

2015-12-04 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-12152: -- Summary: Speed up Scalastyle by only running one SBT command instead of four Key: SPARK-12152 URL: https://issues.apache.org/jira/browse/SPARK-12152 Project: Spark

[jira] [Resolved] (SPARK-12058) Fix KinesisStreamTests in python/pyspark/streaming/tests.py and enable it

2015-12-04 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-12058. --- Resolution: Fixed Fix Version/s: 1.6.0 > Fix KinesisStreamTests in

[jira] [Updated] (SPARK-12138) Escape \u in the generated comments.

2015-12-04 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-12138: - Summary: Escape \u in the generated comments. (was: [SPARK-11352][SQL] Escape \u in the generated

[jira] [Commented] (SPARK-11823) HiveThriftBinaryServerSuite tests timing out, leaves hanging processes

2015-12-04 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042260#comment-15042260 ] shane knapp commented on SPARK-11823: - it looks like it's this particular test that's hanging (and

[jira] [Updated] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame

2015-12-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-12148: - Component/s: SparkR > SparkR: rename DataFrame to SparkDataFrame >

[jira] [Created] (SPARK-12151) Improve PySpark MLLib prediction performance when using pickled vectors

2015-12-04 Thread holdenk (JIRA)
holdenk created SPARK-12151: --- Summary: Improve PySpark MLLib prediction performance when using pickled vectors Key: SPARK-12151 URL: https://issues.apache.org/jira/browse/SPARK-12151 Project: Spark

[jira] [Commented] (SPARK-12150) numPartitions argument to sqlContext.range() should be optional

2015-12-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042156#comment-15042156 ] Xiao Li commented on SPARK-12150: - Yeah, you are right. Will deliver a PR soon. Thanks! > numPartitions

[jira] [Commented] (SPARK-10931) PySpark ML Models should contain Param values

2015-12-04 Thread Evan Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042482#comment-15042482 ] Evan Chen commented on SPARK-10931: --- Hey Joseph, Just wanted to give a status update. I should be

[jira] [Resolved] (SPARK-12142) Can't request executor when container allocator is not ready

2015-12-04 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-12142. Resolution: Fixed Assignee: meiyoula Fix Version/s: 2.0.0 > Can't request

[jira] [Commented] (SPARK-12138) Escape \u in the generated comments.

2015-12-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042492#comment-15042492 ] Xiao Li commented on SPARK-12138: - If nobody takes it, I can make a try. [~yhuai] Could you explain how

[jira] [Resolved] (SPARK-12080) Kryo - Support multiple user registrators

2015-12-04 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-12080. Resolution: Fixed Assignee: Rotem Fix Version/s: 2.0.0 > Kryo - Support

[jira] [Resolved] (SPARK-12084) Fix codes that uses ByteBuffer.array incorrectly

2015-12-04 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-12084. Resolution: Fixed Fix Version/s: 2.0.0 > Fix codes that uses ByteBuffer.array

[jira] [Commented] (SPARK-11081) Make spark-core pull in Jersey and javax.ws.rs dependencies separately for easier overriding

2015-12-04 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042516#comment-15042516 ] Matt Cheah commented on SPARK-11081: Upgrading to Jersey 2 definitely sounds more reasonable. Perhaps

[jira] [Commented] (SPARK-11081) Make spark-core pull in Jersey and javax.ws.rs dependencies separately for easier overriding

2015-12-04 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042519#comment-15042519 ] Marcelo Vanzin commented on SPARK-11081: Opening a new one or re-purposing this one, either is

[jira] [Resolved] (SPARK-11081) Make spark-core pull in Jersey and javax.ws.rs dependencies separately for easier overriding

2015-12-04 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah resolved SPARK-11081. Resolution: Not A Problem Ok, I filed SPARK-12154 to track it. > Make spark-core pull in Jersey

[jira] [Resolved] (SPARK-12112) Upgrade to SBT 0.13.9

2015-12-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12112. - Resolution: Fixed Fix Version/s: 2.0.0 > Upgrade to SBT 0.13.9 > - >

[jira] [Commented] (SPARK-12138) Escape \u in the generated comments.

2015-12-04 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042539#comment-15042539 ] Yin Huai commented on SPARK-12138: -- Yeah. It will be great if you can work on it. To reproduce it, you

[jira] [Commented] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

2015-12-04 Thread Justin Bailey (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042438#comment-15042438 ] Justin Bailey commented on SPARK-8118: -- Can this issue be re-opened? It's really terrible how much

[jira] [Comment Edited] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

2015-12-04 Thread Justin Bailey (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042438#comment-15042438 ] Justin Bailey edited comment on SPARK-8118 at 12/4/15 11:48 PM: Can this

[jira] [Created] (SPARK-12153) Word2Vec uses a fixed length for sentences which is not reasonable for reality, and similarity functions and fields are not accessible

2015-12-04 Thread YongGang Cao (JIRA)
YongGang Cao created SPARK-12153: Summary: Word2Vec uses a fixed length for sentences which is not reasonable for reality, and similarity functions and fields are not accessible Key: SPARK-12153 URL:

[jira] [Commented] (SPARK-4036) Add Conditional Random Fields (CRF) algorithm to Spark MLlib

2015-12-04 Thread Andrew Philpot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042462#comment-15042462 ] Andrew Philpot commented on SPARK-4036: --- Hi, what is the maturity of this code? Are you interested

[jira] [Updated] (SPARK-12153) Word2Vec uses a fixed length for sentences which is not reasonable for reality, and similarity functions and fields are not accessible

2015-12-04 Thread YongGang Cao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YongGang Cao updated SPARK-12153: - Description: sentence boundary matters for sliding window, we shouldn't train model from a

[jira] [Commented] (SPARK-11046) Pass schema from R to JVM using JSON format

2015-12-04 Thread Nakul Jindal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042513#comment-15042513 ] Nakul Jindal commented on SPARK-11046: -- Hi, I am trying to look into this. When you say that SparkR

[jira] [Created] (SPARK-12154) Upgrade to Jersey 2

2015-12-04 Thread Matt Cheah (JIRA)
Matt Cheah created SPARK-12154: -- Summary: Upgrade to Jersey 2 Key: SPARK-12154 URL: https://issues.apache.org/jira/browse/SPARK-12154 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-12138) Escape \u in the generated comments.

2015-12-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042541#comment-15042541 ] Xiao Li commented on SPARK-12138: - Sure. Will try it tonight or tomorrow. Thanks! : ) > Escape \u in the

[jira] [Commented] (SPARK-10123) Cannot set "--deploy-mode" in default configuration

2015-12-04 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042598#comment-15042598 ] Saisai Shao commented on SPARK-10123: - Just confirm if it is on your plan, in case duplicated :). >

[jira] [Commented] (SPARK-12138) Escape \u in the generated comments.

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042612#comment-15042612 ] Apache Spark commented on SPARK-12138: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12138) Escape \u in the generated comments.

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12138: Assignee: Apache Spark > Escape \u in the generated comments. >

[jira] [Assigned] (SPARK-12138) Escape \u in the generated comments.

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12138: Assignee: (was: Apache Spark) > Escape \u in the generated comments. >

[jira] [Commented] (SPARK-12155) Execution OOM after a relative large dataset cached in the cluster.

2015-12-04 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042618#comment-15042618 ] Yin Huai commented on SPARK-12155: -- We can take a look at task {{136}}. {code} 15/12/05 02:51:33 INFO

[jira] [Created] (SPARK-12155) Execution OOM after a relative large dataset cached in the cluster.

2015-12-04 Thread Yin Huai (JIRA)
Yin Huai created SPARK-12155: Summary: Execution OOM after a relative large dataset cached in the cluster. Key: SPARK-12155 URL: https://issues.apache.org/jira/browse/SPARK-12155 Project: Spark

[jira] [Commented] (SPARK-12155) Execution OOM after a relative large dataset cached in the cluster.

2015-12-04 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042616#comment-15042616 ] Yin Huai commented on SPARK-12155: -- I have some new logs. {code} 15/12/05 02:51:33 INFO MemoryConsumer:

[jira] [Commented] (SPARK-12155) Execution OOM after a relative large dataset cached in the cluster.

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042582#comment-15042582 ] Apache Spark commented on SPARK-12155: -- User 'yhuai' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12155) Execution OOM after a relative large dataset cached in the cluster.

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12155: Assignee: (was: Apache Spark) > Execution OOM after a relative large dataset cached

[jira] [Assigned] (SPARK-12155) Execution OOM after a relative large dataset cached in the cluster.

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12155: Assignee: Apache Spark > Execution OOM after a relative large dataset cached in the

[jira] [Commented] (SPARK-12149) Executor UI improvement suggestions - Color UI

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042605#comment-15042605 ] Apache Spark commented on SPARK-12149: -- User 'ajbozarth' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12149) Executor UI improvement suggestions - Color UI

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12149: Assignee: Apache Spark > Executor UI improvement suggestions - Color UI >

[jira] [Assigned] (SPARK-12149) Executor UI improvement suggestions - Color UI

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12149: Assignee: (was: Apache Spark) > Executor UI improvement suggestions - Color UI >

[jira] [Commented] (SPARK-12140) Support Streaming UI in HistoryServer

2015-12-04 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-12140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042681#comment-15042681 ] Jean-Baptiste Onofré commented on SPARK-12140: -- It's just to know if you didn't already

[jira] [Assigned] (SPARK-12102) Cast a non-nullable struct field to a nullable field during analysis

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12102: Assignee: (was: Apache Spark) > Cast a non-nullable struct field to a nullable field

[jira] [Commented] (SPARK-12102) Cast a non-nullable struct field to a nullable field during analysis

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042717#comment-15042717 ] Apache Spark commented on SPARK-12102: -- User 'dilipbiswal' has created a pull request for this

[jira] [Assigned] (SPARK-12102) Cast a non-nullable struct field to a nullable field during analysis

2015-12-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12102: Assignee: Apache Spark > Cast a non-nullable struct field to a nullable field during

[jira] [Created] (SPARK-12149) Executor UI improvement suggestions - Color UI

2015-12-04 Thread Alexander Bozarth (JIRA)
Alexander Bozarth created SPARK-12149: - Summary: Executor UI improvement suggestions - Color UI Key: SPARK-12149 URL: https://issues.apache.org/jira/browse/SPARK-12149 Project: Spark

[jira] [Commented] (SPARK-10509) Excessive param boiler plate code

2015-12-04 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041937#comment-15041937 ] holdenk commented on SPARK-10509: - So I'm thinking a pretty simple option would be just having a reset

[jira] [Commented] (SPARK-1832) Executor UI improvement suggestions

2015-12-04 Thread Alexander Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041936#comment-15041936 ] Alexander Bozarth commented on SPARK-1832: -- I've split off a subtask for the Color improvements

[jira] [Commented] (SPARK-12128) Multiplication on decimals in dataframe returns null

2015-12-04 Thread kevin yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041954#comment-15041954 ] kevin yu commented on SPARK-12128: -- Hello Philip: Thanks for reporting this problem, this looks like bug

[jira] [Updated] (SPARK-12089) java.lang.NegativeArraySizeException when growing BufferHolder

2015-12-04 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-12089: - Fix Version/s: (was: 2.0.0) > java.lang.NegativeArraySizeException when growing BufferHolder >

  1   2   >