[jira] [Commented] (SPARK-21349) Make TASK_SIZE_TO_WARN_KB configurable

2017-08-29 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146624#comment-16146624 ] Shivaram Venkataraman commented on SPARK-21349: --- Thanks for checking. In that case I dont

[jira] [Commented] (SPARK-20462) Spark-Kinesis Direct Connector

2017-08-29 Thread Gaurav Shah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146614#comment-16146614 ] Gaurav Shah commented on SPARK-20462: - related blog post:

[jira] [Commented] (SPARK-21349) Make TASK_SIZE_TO_WARN_KB configurable

2017-08-29 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146599#comment-16146599 ] Dongjoon Hyun commented on SPARK-21349: --- Yes. For the fewer values like 24*365*1, the warning does

[jira] [Commented] (SPARK-21856) Update Python API for MultilayerPerceptronClassifierModel

2017-08-29 Thread Chunsheng Ji (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146584#comment-16146584 ] Chunsheng Ji commented on SPARK-21856: -- I am working on it. > Update Python API for

[jira] [Commented] (SPARK-21349) Make TASK_SIZE_TO_WARN_KB configurable

2017-08-29 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146585#comment-16146585 ] Shivaram Venkataraman commented on SPARK-21349: --- I think this might be that we create a

[jira] [Commented] (SPARK-21854) Python interface for MLOR summary

2017-08-29 Thread Ming Jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146579#comment-16146579 ] Ming Jiang commented on SPARK-21854: I can work on this, thanks! > Python interface for MLOR summary

[jira] [Issue Comment Deleted] (SPARK-21856) Update Python API for MultilayerPerceptronClassifierModel

2017-08-29 Thread Ming Jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Jiang updated SPARK-21856: --- Comment: was deleted (was: I can work on it, thanks!) > Update Python API for

[jira] [Commented] (SPARK-21349) Make TASK_SIZE_TO_WARN_KB configurable

2017-08-29 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146569#comment-16146569 ] Dongjoon Hyun commented on SPARK-21349: --- Hi, [~jiangxb] and all. I hit this issue again in another

[jira] [Assigned] (SPARK-20886) HadoopMapReduceCommitProtocol to fail with message if FileOutputCommitter.getWorkPath==null

2017-08-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-20886: Assignee: Steve Loughran This was fixed in a way to allow {{null}} case. >

[jira] [Resolved] (SPARK-20886) HadoopMapReduceCommitProtocol to fail with message if FileOutputCommitter.getWorkPath==null

2017-08-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-20886. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18111

[jira] [Resolved] (SPARK-21845) Make codegen fallback of expressions configurable

2017-08-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21845. - Resolution: Fixed Fix Version/s: 2.3.0 > Make codegen fallback of expressions configurable >

[jira] [Created] (SPARK-21872) Is job duration value of Spark Jobs page on Web UI correct?

2017-08-29 Thread iamhumanbeing (JIRA)
iamhumanbeing created SPARK-21872: - Summary: Is job duration value of Spark Jobs page on Web UI correct? Key: SPARK-21872 URL: https://issues.apache.org/jira/browse/SPARK-21872 Project: Spark

[jira] [Commented] (SPARK-17139) Add model summary for MultinomialLogisticRegression

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146494#comment-16146494 ] Apache Spark commented on SPARK-17139: -- User 'WeichenXu123' has created a pull request for this

[jira] [Assigned] (SPARK-20711) MultivariateOnlineSummarizer incorrect min/max for NaN value

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20711: Assignee: (was: Apache Spark) > MultivariateOnlineSummarizer incorrect min/max for

[jira] [Assigned] (SPARK-21871) Check actual bytecode size when compiling generated code

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21871: Assignee: Apache Spark > Check actual bytecode size when compiling generated code >

[jira] [Commented] (SPARK-21871) Check actual bytecode size when compiling generated code

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146493#comment-16146493 ] Apache Spark commented on SPARK-21871: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20711) MultivariateOnlineSummarizer incorrect min/max for NaN value

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20711: Assignee: Apache Spark > MultivariateOnlineSummarizer incorrect min/max for NaN value >

[jira] [Assigned] (SPARK-21871) Check actual bytecode size when compiling generated code

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21871: Assignee: (was: Apache Spark) > Check actual bytecode size when compiling generated

[jira] [Commented] (SPARK-20711) MultivariateOnlineSummarizer incorrect min/max for NaN value

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146492#comment-16146492 ] Apache Spark commented on SPARK-20711: -- User 'zhengruifeng' has created a pull request for this

[jira] [Updated] (SPARK-20711) MultivariateOnlineSummarizer incorrect min/max for NaN value

2017-08-29 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-20711: - Summary: MultivariateOnlineSummarizer incorrect min/max for NaN value (was:

[jira] [Created] (SPARK-21871) Check actual bytecode size when compiling generated code

2017-08-29 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-21871: Summary: Check actual bytecode size when compiling generated code Key: SPARK-21871 URL: https://issues.apache.org/jira/browse/SPARK-21871 Project: Spark

[jira] [Commented] (SPARK-20711) MultivariateOnlineSummarizer incorrect min/max for identical NaN feature

2017-08-29 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146480#comment-16146480 ] zhengruifeng commented on SPARK-20711: -- [~WeichenXu123] I notice that you have just fixed a bug in

[jira] [Reopened] (SPARK-20711) MultivariateOnlineSummarizer incorrect min/max for identical NaN feature

2017-08-29 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng reopened SPARK-20711: -- > MultivariateOnlineSummarizer incorrect min/max for identical NaN feature >

[jira] [Updated] (SPARK-21862) Add overflow check in PCA

2017-08-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-21862: -- Shepherd: Joseph K. Bradley > Add overflow check in PCA > - >

[jira] [Assigned] (SPARK-21862) Add overflow check in PCA

2017-08-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-21862: - Assignee: Weichen Xu > Add overflow check in PCA > - >

[jira] [Assigned] (SPARK-21870) Split codegen'd aggregation code into small functions for the HotSpot

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21870: Assignee: (was: Apache Spark) > Split codegen'd aggregation code into small functions

[jira] [Assigned] (SPARK-21870) Split codegen'd aggregation code into small functions for the HotSpot

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21870: Assignee: Apache Spark > Split codegen'd aggregation code into small functions for the

[jira] [Commented] (SPARK-21870) Split codegen'd aggregation code into small functions for the HotSpot

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146424#comment-16146424 ] Apache Spark commented on SPARK-21870: -- User 'maropu' has created a pull request for this issue:

[jira] [Updated] (SPARK-21870) Split codegen'd aggregation code into small functions for the HotSpot

2017-08-29 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-21870: - Description: In SPARK-21603, we got performance regression if the HotSpot didn't compile

[jira] [Updated] (SPARK-21870) Split codegen'd aggregation code into small functions for the HotSpot

2017-08-29 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-21870: - Description: In SPARK-21603, we got performance regression if the HotSpot didn't compile

[jira] [Created] (SPARK-21870) Split codegen'd aggregation code into small functions for the HotSpot

2017-08-29 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-21870: Summary: Split codegen'd aggregation code into small functions for the HotSpot Key: SPARK-21870 URL: https://issues.apache.org/jira/browse/SPARK-21870

[jira] [Updated] (SPARK-18278) SPIP: Support native submission of spark jobs to a kubernetes cluster

2017-08-29 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-18278: -- Labels: SPIP (was: ) > SPIP: Support native submission of spark jobs to a kubernetes cluster

[jira] [Updated] (SPARK-21866) SPIP: Image support in Spark

2017-08-29 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-21866: -- Labels: SPIP (was: ) > SPIP: Image support in Spark > > >

[jira] [Commented] (SPARK-21869) A cached Kafka producer should not be closed if any task is using it.

2017-08-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146347#comment-16146347 ] Shixiong Zhu commented on SPARK-21869: -- [~scrapco...@gmail.com] do you want to take this task? > A

[jira] [Created] (SPARK-21869) A cached Kafka producer should not be closed if any task is using it.

2017-08-29 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-21869: Summary: A cached Kafka producer should not be closed if any task is using it. Key: SPARK-21869 URL: https://issues.apache.org/jira/browse/SPARK-21869 Project: Spark

[jira] [Commented] (SPARK-21864) Spark 2.0.1 - SaveMode.Overwrite does not work while saving data to memsql

2017-08-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146287#comment-16146287 ] Sean Owen commented on SPARK-21864: --- My first guess is that it's a MemSQL problem, if you're not able

[jira] [Comment Edited] (SPARK-19307) SPARK-17387 caused ignorance of conf object passed to SparkContext:

2017-08-29 Thread Charlie Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146280#comment-16146280 ] Charlie Tsai edited comment on SPARK-19307 at 8/29/17 10:48 PM: Hi, I am

[jira] [Resolved] (SPARK-21849) Make the serializer function more robust

2017-08-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21849. --- Resolution: Not A Problem > Make the serializer function more robust >

[jira] [Assigned] (SPARK-21813) [core] Modify TaskMemoryManager.MAXIMUM_PAGE_SIZE_BYTES comments

2017-08-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-21813: - Assignee: he.qiao > [core] Modify TaskMemoryManager.MAXIMUM_PAGE_SIZE_BYTES comments >

[jira] [Resolved] (SPARK-21813) [core] Modify TaskMemoryManager.MAXIMUM_PAGE_SIZE_BYTES comments

2017-08-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21813. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19025

[jira] [Comment Edited] (SPARK-19307) SPARK-17387 caused ignorance of conf object passed to SparkContext:

2017-08-29 Thread Charlie Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146280#comment-16146280 ] Charlie Tsai edited comment on SPARK-19307 at 8/29/17 10:43 PM: Hi, I am

[jira] [Comment Edited] (SPARK-19307) SPARK-17387 caused ignorance of conf object passed to SparkContext:

2017-08-29 Thread Charlie Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146280#comment-16146280 ] Charlie Tsai edited comment on SPARK-19307 at 8/29/17 10:43 PM: Hi, I am

[jira] [Commented] (SPARK-19307) SPARK-17387 caused ignorance of conf object passed to SparkContext:

2017-08-29 Thread Charlie Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146280#comment-16146280 ] Charlie Tsai commented on SPARK-19307: -- Hi, I am using 2.2.0 but find that command line {{--conf}}

[jira] [Commented] (SPARK-21834) Incorrect executor request in case of dynamic allocation

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146235#comment-16146235 ] Apache Spark commented on SPARK-21834: -- User 'sitalkedia' has created a pull request for this issue:

[jira] [Commented] (SPARK-21797) spark cannot read partitioned data in S3 that are partly in glacier

2017-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146193#comment-16146193 ] Steve Loughran commented on SPARK-21797: No> That's a shame. I only came across the option when I

[jira] [Resolved] (SPARK-21728) Allow SparkSubmit to use logging

2017-08-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-21728. Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.3.0 > Allow

[jira] [Resolved] (SPARK-21868) Spark job fails on java 9 NumberFormatException for input string ea

2017-08-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21868. --- Resolution: Not A Problem Java 9 is not supported > Spark job fails on java 9 NumberFormatException

[jira] [Updated] (SPARK-21868) Spark job fails on java 9 NumberFormatException for input string ea

2017-08-29 Thread rahul sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] rahul sharma updated SPARK-21868: -- Description: I have a sample spark job which I am successfully able to run on java 8 but when

[jira] [Created] (SPARK-21868) Spark job fails on java 9 NumberFormatException for input string ea

2017-08-29 Thread rahul sharma (JIRA)
rahul sharma created SPARK-21868: - Summary: Spark job fails on java 9 NumberFormatException for input string ea Key: SPARK-21868 URL: https://issues.apache.org/jira/browse/SPARK-21868 Project: Spark

[jira] [Commented] (SPARK-21867) Support async spilling in UnsafeShuffleWriter

2017-08-29 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146060#comment-16146060 ] Sital Kedia commented on SPARK-21867: - cc - [~rxin], [~joshrosen], [~sameer] - What do you think of

[jira] [Created] (SPARK-21867) Support async spilling in UnsafeShuffleWriter

2017-08-29 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-21867: --- Summary: Support async spilling in UnsafeShuffleWriter Key: SPARK-21867 URL: https://issues.apache.org/jira/browse/SPARK-21867 Project: Spark Issue Type:

[jira] [Updated] (SPARK-21714) SparkSubmit in Yarn Client mode downloads remote files and then reuploads them again

2017-08-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-21714: --- Fix Version/s: (was: 2.2.1) > SparkSubmit in Yarn Client mode downloads remote files and

[jira] [Comment Edited] (SPARK-18350) Support session local timezone

2017-08-29 Thread Vinayak (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136567#comment-16136567 ] Vinayak edited comment on SPARK-18350 at 8/29/17 7:30 PM: -- [~ueshin] I have

[jira] [Commented] (SPARK-9213) Improve regular expression performance (via joni)

2017-08-29 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145831#comment-16145831 ] Mridul Muralidharan commented on SPARK-9213: [~rxin] Curious what happened to this effort -

[jira] [Commented] (SPARK-21866) SPIP: Image support in Spark

2017-08-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145808#comment-16145808 ] Sean Owen commented on SPARK-21866: --- Why would this need to be part of Spark? I assume it's

[jira] [Updated] (SPARK-21714) SparkSubmit in Yarn Client mode downloads remote files and then reuploads them again

2017-08-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-21714: --- Fix Version/s: 2.2.1 > SparkSubmit in Yarn Client mode downloads remote files and then

[jira] [Updated] (SPARK-21866) SPIP: Image support in Spark

2017-08-29 Thread Timothy Hunter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Hunter updated SPARK-21866: --- Attachment: SPIP - Image support for Apache Spark.pdf > SPIP: Image support in Spark >

[jira] [Created] (SPARK-21866) SPIP: Image support in Spark

2017-08-29 Thread Timothy Hunter (JIRA)
Timothy Hunter created SPARK-21866: -- Summary: SPIP: Image support in Spark Key: SPARK-21866 URL: https://issues.apache.org/jira/browse/SPARK-21866 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-21797) spark cannot read partitioned data in S3 that are partly in glacier

2017-08-29 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145716#comment-16145716 ] Boris Clémençon commented on SPARK-21797: -- FYI, the flag

[jira] [Commented] (SPARK-21097) Dynamic allocation will preserve cached data

2017-08-29 Thread Brad (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145708#comment-16145708 ] Brad commented on SPARK-21097: -- Here is a document with some of my benchmark results. I am working on adding

[jira] [Assigned] (SPARK-21801) SparkR unit test randomly fail on trees

2017-08-29 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-21801: Assignee: Felix Cheung > SparkR unit test randomly fail on trees >

[jira] [Resolved] (SPARK-21801) SparkR unit test randomly fail on trees

2017-08-29 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-21801. -- Resolution: Fixed Fix Version/s: 2.3.0 > SparkR unit test randomly fail on trees >

[jira] [Commented] (SPARK-21857) Exception in thread "main" java.lang.ExceptionInInitializerError

2017-08-29 Thread Nagamanoj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145683#comment-16145683 ] Nagamanoj commented on SPARK-21857: --- Thank you very much Sean Owen... I reverted to Java 8 and now it

[jira] [Commented] (SPARK-21865) remove Partitioning.compatibleWith

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145675#comment-16145675 ] Apache Spark commented on SPARK-21865: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-21865) remove Partitioning.compatibleWith

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21865: Assignee: Wenchen Fan (was: Apache Spark) > remove Partitioning.compatibleWith >

[jira] [Assigned] (SPARK-21865) remove Partitioning.compatibleWith

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21865: Assignee: Apache Spark (was: Wenchen Fan) > remove Partitioning.compatibleWith >

[jira] [Assigned] (SPARK-21822) When insert Hive Table is finished, it is better to clean out the tmpLocation dir

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21822: Assignee: Apache Spark > When insert Hive Table is finished, it is better to clean out

[jira] [Assigned] (SPARK-21822) When insert Hive Table is finished, it is better to clean out the tmpLocation dir

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21822: Assignee: (was: Apache Spark) > When insert Hive Table is finished, it is better to

[jira] [Assigned] (SPARK-21849) Make the serializer function more robust

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21849: Assignee: (was: Apache Spark) > Make the serializer function more robust >

[jira] [Assigned] (SPARK-21806) BinaryClassificationMetrics pr(): first point (0.0, 1.0) is misleading

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21806: Assignee: (was: Apache Spark) > BinaryClassificationMetrics pr(): first point (0.0,

[jira] [Assigned] (SPARK-20628) Keep track of nodes which are going to be shut down & avoid scheduling new tasks

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20628: Assignee: (was: Apache Spark) > Keep track of nodes which are going to be shut down &

[jira] [Assigned] (SPARK-21845) Make codegen fallback of expressions configurable

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21845: Assignee: Apache Spark (was: Xiao Li) > Make codegen fallback of expressions

[jira] [Assigned] (SPARK-21097) Dynamic allocation will preserve cached data

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21097: Assignee: Apache Spark > Dynamic allocation will preserve cached data >

[jira] [Assigned] (SPARK-21811) Inconsistency when finding the widest common type of a combination of DateType, StringType, and NumericType

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21811: Assignee: (was: Apache Spark) > Inconsistency when finding the widest common type of

[jira] [Assigned] (SPARK-21097) Dynamic allocation will preserve cached data

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21097: Assignee: (was: Apache Spark) > Dynamic allocation will preserve cached data >

[jira] [Assigned] (SPARK-21849) Make the serializer function more robust

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21849: Assignee: Apache Spark > Make the serializer function more robust >

[jira] [Assigned] (SPARK-21811) Inconsistency when finding the widest common type of a combination of DateType, StringType, and NumericType

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21811: Assignee: Apache Spark > Inconsistency when finding the widest common type of a

[jira] [Assigned] (SPARK-20628) Keep track of nodes which are going to be shut down & avoid scheduling new tasks

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20628: Assignee: Apache Spark > Keep track of nodes which are going to be shut down & avoid

[jira] [Assigned] (SPARK-21806) BinaryClassificationMetrics pr(): first point (0.0, 1.0) is misleading

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21806: Assignee: Apache Spark > BinaryClassificationMetrics pr(): first point (0.0, 1.0) is

[jira] [Assigned] (SPARK-21845) Make codegen fallback of expressions configurable

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21845: Assignee: Xiao Li (was: Apache Spark) > Make codegen fallback of expressions

[jira] [Assigned] (SPARK-21469) Add doc and example for FeatureHasher

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21469: Assignee: Apache Spark > Add doc and example for FeatureHasher >

[jira] [Assigned] (SPARK-21808) Add R interface of binarizer

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21808: Assignee: (was: Apache Spark) > Add R interface of binarizer >

[jira] [Assigned] (SPARK-21808) Add R interface of binarizer

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21808: Assignee: Apache Spark > Add R interface of binarizer > > >

[jira] [Assigned] (SPARK-21469) Add doc and example for FeatureHasher

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21469: Assignee: (was: Apache Spark) > Add doc and example for FeatureHasher >

[jira] [Assigned] (SPARK-21787) Support for pushing down filters for date types in ORC

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21787: Assignee: (was: Apache Spark) > Support for pushing down filters for date types in

[jira] [Assigned] (SPARK-21728) Allow SparkSubmit to use logging

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21728: Assignee: Apache Spark > Allow SparkSubmit to use logging >

[jira] [Assigned] (SPARK-21784) Add ALTER TABLE ADD CONSTRANT DDL to support defining primary key and foreign keys

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21784: Assignee: (was: Apache Spark) > Add ALTER TABLE ADD CONSTRANT DDL to support defining

[jira] [Assigned] (SPARK-21728) Allow SparkSubmit to use logging

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21728: Assignee: (was: Apache Spark) > Allow SparkSubmit to use logging >

[jira] [Assigned] (SPARK-21779) Simpler Dataset.sample API in Python

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21779: Assignee: Apache Spark > Simpler Dataset.sample API in Python >

[jira] [Assigned] (SPARK-21787) Support for pushing down filters for date types in ORC

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21787: Assignee: Apache Spark > Support for pushing down filters for date types in ORC >

[jira] [Assigned] (SPARK-21774) The rule PromoteStrings cast string to a wrong data type

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21774: Assignee: Apache Spark > The rule PromoteStrings cast string to a wrong data type >

[jira] [Assigned] (SPARK-21783) Turn on ORC filter push-down by default

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21783: Assignee: Apache Spark > Turn on ORC filter push-down by default >

[jira] [Assigned] (SPARK-21624) Optimize communication cost of RF/GBT/DT

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21624: Assignee: (was: Apache Spark) > Optimize communication cost of RF/GBT/DT >

[jira] [Assigned] (SPARK-19256) Hive bucketing support

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19256: Assignee: Apache Spark > Hive bucketing support > -- > >

[jira] [Assigned] (SPARK-21720) Filter predicate with many conditions throw stackoverflow error

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21720: Assignee: Apache Spark > Filter predicate with many conditions throw stackoverflow error

[jira] [Assigned] (SPARK-21791) ORC should support column names with dot

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21791: Assignee: Apache Spark > ORC should support column names with dot >

[jira] [Assigned] (SPARK-21779) Simpler Dataset.sample API in Python

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21779: Assignee: (was: Apache Spark) > Simpler Dataset.sample API in Python >

[jira] [Assigned] (SPARK-20589) Allow limiting task concurrency per stage

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20589: Assignee: Apache Spark > Allow limiting task concurrency per stage >

[jira] [Assigned] (SPARK-21685) Params isSet in scala Transformer triggered by _setDefault in pyspark

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21685: Assignee: Apache Spark > Params isSet in scala Transformer triggered by _setDefault in

[jira] [Assigned] (SPARK-21771) SparkSQLEnv creates a useless meta hive client

2017-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21771: Assignee: Apache Spark > SparkSQLEnv creates a useless meta hive client >

  1   2   3   >