[jira] [Created] (SPARK-21336) Revise rand comparison in BatchEvalPythonExecSuite

2017-07-06 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-21336: -- Summary: Revise rand comparison in BatchEvalPythonExecSuite Key: SPARK-21336 URL: https://issues.apache.org/jira/browse/SPARK-21336 Project: Spark Issue

[jira] [Commented] (SPARK-21331) java.lang.NullPointerException for certain methods in classes of MLlib

2017-07-06 Thread Anirban Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077623#comment-16077623 ] Anirban Das commented on SPARK-21331: - The code works fine on spark-shell as you saw. But the problem

[jira] [Commented] (SPARK-21331) java.lang.NullPointerException for certain methods in classes of MLlib

2017-07-06 Thread 颜发才
[ https://issues.apache.org/jira/browse/SPARK-21331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077620#comment-16077620 ] Yan Facai (颜发才) commented on SPARK-21331: - [~anirband] How about using this code? {code} val

[jira] [Commented] (SPARK-18085) Better History Server scalability for many / large applications

2017-07-06 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077606#comment-16077606 ] Reynold Xin commented on SPARK-18085: - [~vanzin] seems like this should have a SPIP? Looks super

[jira] [Comment Edited] (SPARK-21331) java.lang.NullPointerException for certain methods in classes of MLlib

2017-07-06 Thread 颜发才
[ https://issues.apache.org/jira/browse/SPARK-21331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077605#comment-16077605 ] Yan Facai (颜发才) edited comment on SPARK-21331 at 7/7/17 5:21 AM: - Hi, I

[jira] [Commented] (SPARK-21331) java.lang.NullPointerException for certain methods in classes of MLlib

2017-07-06 Thread 颜发才
[ https://issues.apache.org/jira/browse/SPARK-21331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077605#comment-16077605 ] Yan Facai (颜发才) commented on SPARK-21331: - Hi, I run the code in description on mac, spark-2.1.1.

[jira] [Commented] (SPARK-1691) Support quoted arguments inside of spark-submit

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077600#comment-16077600 ] Apache Spark commented on SPARK-1691: - User 'pwendell' has created a pull request for this issue:

[jira] [Resolved] (SPARK-21217) Support ColumnVector.Array.toArray()

2017-07-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21217. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18425

[jira] [Assigned] (SPARK-21217) Support ColumnVector.Array.toArray()

2017-07-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21217: --- Assignee: Kazuaki Ishizaki > Support ColumnVector.Array.toArray() >

[jira] [Resolved] (SPARK-21327) ArrayConstructor should handle an array of typecode 'l' as long rather than int in Python 2.

2017-07-06 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-21327. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18553

[jira] [Assigned] (SPARK-21327) ArrayConstructor should handle an array of typecode 'l' as long rather than int in Python 2.

2017-07-06 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-21327: - Assignee: Takuya Ueshin > ArrayConstructor should handle an array of typecode 'l' as

[jira] [Updated] (SPARK-20566) ColumnVector should support `appendFloats` for array

2017-07-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-20566: Issue Type: Improvement (was: Bug) > ColumnVector should support `appendFloats` for array >

[jira] [Resolved] (SPARK-21066) LibSVM load just one input file

2017-07-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21066. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18556

[jira] [Assigned] (SPARK-21066) LibSVM load just one input file

2017-07-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21066: --- Assignee: Hyukjin Kwon > LibSVM load just one input file > ---

[jira] [Resolved] (SPARK-21326) Use TextFileFormat in implementation of LibSVMFileFormat

2017-07-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21326. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18556

[jira] [Assigned] (SPARK-21326) Use TextFileFormat in implementation of LibSVMFileFormat

2017-07-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21326: --- Assignee: Hyukjin Kwon > Use TextFileFormat in implementation of LibSVMFileFormat >

[jira] [Assigned] (SPARK-21335) support un-aliased subquery

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21335: Assignee: Apache Spark (was: Wenchen Fan) > support un-aliased subquery >

[jira] [Commented] (SPARK-21335) support un-aliased subquery

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077540#comment-16077540 ] Apache Spark commented on SPARK-21335: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-21335) support un-aliased subquery

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21335: Assignee: Wenchen Fan (was: Apache Spark) > support un-aliased subquery >

[jira] [Reopened] (SPARK-21107) Pyspark: ISO-8859-1 column names inconsistently converted to UTF-8

2017-07-06 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-21107: -- Oh, wait, it describes ISO-8859-1 specific issue. Yea, we have a strong assumption about UTF-8 in

[jira] [Updated] (SPARK-21335) support un-aliased subquery

2017-07-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-21335: Description: un-aliased subquery is supported by Spark SQL for a long time. Its semantic was not

[jira] [Created] (SPARK-21335) support un-aliased subquery

2017-07-06 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-21335: --- Summary: support un-aliased subquery Key: SPARK-21335 URL: https://issues.apache.org/jira/browse/SPARK-21335 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-21107) Pyspark: ISO-8859-1 column names inconsistently converted to UTF-8

2017-07-06 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21107. -- Resolution: Invalid I can't follow what it describes. {code} >>> u'L\xc3\xa0' == u"Là" False

[jira] [Updated] (SPARK-21334) Fix metrics for external shuffle service

2017-07-06 Thread Raajay Viswanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raajay Viswanathan updated SPARK-21334: --- Labels: external-shuffle-service (was: shuffle) > Fix metrics for external shuffle

[jira] [Created] (SPARK-21334) Fix metrics for external shuffle service

2017-07-06 Thread Raajay Viswanathan (JIRA)
Raajay Viswanathan created SPARK-21334: -- Summary: Fix metrics for external shuffle service Key: SPARK-21334 URL: https://issues.apache.org/jira/browse/SPARK-21334 Project: Spark Issue

[jira] [Commented] (SPARK-6679) java.lang.ClassNotFoundException on Mesos fine grained mode and input replication

2017-07-06 Thread Anwar HUda (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077500#comment-16077500 ] Anwar HUda commented on SPARK-6679: --- I am also facing the same issue > java.lang.ClassNotFoundException

[jira] [Commented] (SPARK-20703) Add an operator for writing data out

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077494#comment-16077494 ] Apache Spark commented on SPARK-20703: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-07-06 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077484#comment-16077484 ] Reynold Xin commented on SPARK-21190: - [~bryanc] Sorry I don't think it makes sense to not introduce

[jira] [Resolved] (SPARK-21328) Corpus creation using sparkR for textmining

2017-07-06 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21328. -- Resolution: Invalid I am resolving this per the type. Questions should go to mailing list. >

[jira] [Resolved] (SPARK-21329) Make EventTimeWatermarkExec explicitly UnaryExecNode

2017-07-06 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-21329. -- Resolution: Fixed Assignee: Jacek Laskowski Fix Version/s: 2.3.0 > Make

[jira] [Commented] (SPARK-20566) ColumnVector should support `appendFloats` for array

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077433#comment-16077433 ] Apache Spark commented on SPARK-20566: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Resolved] (SPARK-20946) Do not update conf for existing SparkContext in SparkSession.getOrCreate

2017-07-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20946. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18536

[jira] [Resolved] (SPARK-21267) Improvements to the Structured Streaming programming guide

2017-07-06 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-21267. -- Resolution: Fixed Fix Version/s: 2.3.0 2.2.1 > Improvements to the

[jira] [Commented] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-07-06 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077390#comment-16077390 ] Bryan Cutler commented on SPARK-21190: -- This is a great discussion so far and I would love to see

[jira] [Assigned] (SPARK-21333) joinWith documents and analysis allow invalid join types

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21333: Assignee: Apache Spark > joinWith documents and analysis allow invalid join types >

[jira] [Assigned] (SPARK-21333) joinWith documents and analysis allow invalid join types

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21333: Assignee: (was: Apache Spark) > joinWith documents and analysis allow invalid join

[jira] [Commented] (SPARK-21333) joinWith documents and analysis allow invalid join types

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077381#comment-16077381 ] Apache Spark commented on SPARK-21333: -- User 'coreywoodfield' has created a pull request for this

[jira] [Created] (SPARK-21333) joinWith documents and analysis allow invalid join types

2017-07-06 Thread Corey Woodfield (JIRA)
Corey Woodfield created SPARK-21333: --- Summary: joinWith documents and analysis allow invalid join types Key: SPARK-21333 URL: https://issues.apache.org/jira/browse/SPARK-21333 Project: Spark

[jira] [Created] (SPARK-21332) Incorrect result type inferred for some decimal expressions

2017-07-06 Thread Alexander Shkapsky (JIRA)
Alexander Shkapsky created SPARK-21332: -- Summary: Incorrect result type inferred for some decimal expressions Key: SPARK-21332 URL: https://issues.apache.org/jira/browse/SPARK-21332 Project:

[jira] [Updated] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-07-06 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-21190: Description: *Background and Motivation* Python is one of the most popular programming languages

[jira] [Created] (SPARK-21331) java.lang.NullPointerException for certain methods in classes of MLlib

2017-07-06 Thread Anirban Das (JIRA)
Anirban Das created SPARK-21331: --- Summary: java.lang.NullPointerException for certain methods in classes of MLlib Key: SPARK-21331 URL: https://issues.apache.org/jira/browse/SPARK-21331 Project: Spark

[jira] [Resolved] (SPARK-21323) Rename sql.catalyst.plans.logical.statsEstimation.Range to ValueInterval

2017-07-06 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-21323. - Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 2.3.0 > Rename

[jira] [Updated] (SPARK-18724) Add TuningSummary for TrainValidationSplit and CountVectorizer

2017-07-06 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang updated SPARK-18724: --- Summary: Add TuningSummary for TrainValidationSplit and CountVectorizer (was: Add TuningSummary for

[jira] [Assigned] (SPARK-21204) RuntimeException with Set and Case Class in Spark 2.1.1

2017-07-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21204: --- Assignee: Liang-Chi Hsieh > RuntimeException with Set and Case Class in Spark 2.1.1 >

[jira] [Resolved] (SPARK-21204) RuntimeException with Set and Case Class in Spark 2.1.1

2017-07-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21204. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18416

[jira] [Assigned] (SPARK-21228) InSet incorrect handling of structs

2017-07-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21228: --- Assignee: Bogdan Raducanu > InSet incorrect handling of structs >

[jira] [Resolved] (SPARK-21228) InSet incorrect handling of structs

2017-07-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21228. - Resolution: Fixed Fix Version/s: 2.3.0 > InSet incorrect handling of structs >

[jira] [Resolved] (SPARK-21182) Structured streaming on Spark-shell on windows

2017-07-06 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21182. -- Resolution: Cannot Reproduce I am resolving this as I can't reproduce this and the reporter

[jira] [Comment Edited] (SPARK-21227) Unicode in Json field causes AnalysisException when selecting from Dataframe

2017-07-06 Thread Seydou Dia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076897#comment-16076897 ] Seydou Dia edited comment on SPARK-21227 at 7/6/17 5:02 PM: Hi [~srowen], I

[jira] [Commented] (SPARK-21316) Dataset Union output is not consistent with the column sequence

2017-07-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076900#comment-16076900 ] Dongjoon Hyun commented on SPARK-21316: --- [~skp33]. It's not `specified in alphabetical order`. As

[jira] [Comment Edited] (SPARK-21227) Unicode in Json field causes AnalysisException when selecting from Dataframe

2017-07-06 Thread Seydou Dia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076897#comment-16076897 ] Seydou Dia edited comment on SPARK-21227 at 7/6/17 5:02 PM: Hi [~srowen], I

[jira] [Comment Edited] (SPARK-21227) Unicode in Json field causes AnalysisException when selecting from Dataframe

2017-07-06 Thread Seydou Dia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076897#comment-16076897 ] Seydou Dia edited comment on SPARK-21227 at 7/6/17 5:01 PM: Hi [~srowen], I

[jira] [Comment Edited] (SPARK-21227) Unicode in Json field causes AnalysisException when selecting from Dataframe

2017-07-06 Thread Seydou Dia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076897#comment-16076897 ] Seydou Dia edited comment on SPARK-21227 at 7/6/17 4:59 PM: Hi [~srowen], I

[jira] [Comment Edited] (SPARK-21227) Unicode in Json field causes AnalysisException when selecting from Dataframe

2017-07-06 Thread Seydou Dia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076897#comment-16076897 ] Seydou Dia edited comment on SPARK-21227 at 7/6/17 4:59 PM: Hi [~srowen], I

[jira] [Commented] (SPARK-21227) Unicode in Json field causes AnalysisException when selecting from Dataframe

2017-07-06 Thread Seydou Dia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076897#comment-16076897 ] Seydou Dia commented on SPARK-21227: Hi [~srowen], I had time to dig further the issue and come up

[jira] [Comment Edited] (SPARK-21227) Unicode in Json field causes AnalysisException when selecting from Dataframe

2017-07-06 Thread Seydou Dia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076897#comment-16076897 ] Seydou Dia edited comment on SPARK-21227 at 7/6/17 4:58 PM: Hi [~srowen], I

[jira] [Assigned] (SPARK-21326) Use TextFileFormat in implementation of LibSVMFileFormat

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21326: Assignee: Apache Spark > Use TextFileFormat in implementation of LibSVMFileFormat >

[jira] [Commented] (SPARK-21326) Use TextFileFormat in implementation of LibSVMFileFormat

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076884#comment-16076884 ] Apache Spark commented on SPARK-21326: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-21066) LibSVM load just one input file

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076885#comment-16076885 ] Apache Spark commented on SPARK-21066: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-21326) Use TextFileFormat in implementation of LibSVMFileFormat

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21326: Assignee: (was: Apache Spark) > Use TextFileFormat in implementation of

[jira] [Updated] (SPARK-21321) Spark very verbose on shutdown confusing users

2017-07-06 Thread Jong Yoon Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jong Yoon Lee updated SPARK-21321: -- Description: On shutdown, spark can be very verbose and can spit out errors that causes the

[jira] [Commented] (SPARK-19680) Offsets out of range with no configured reset policy for partitions

2017-07-06 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076646#comment-16076646 ] Cody Koeninger commented on SPARK-19680: Direct Stream can take a mapping from topicpartition to

[jira] [Comment Edited] (SPARK-19606) Support constraints in spark-dispatcher

2017-07-06 Thread Pascal GILLET (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076510#comment-16076510 ] Pascal GILLET edited comment on SPARK-19606 at 7/6/17 2:13 PM: --- +1 Need to

[jira] [Commented] (SPARK-19606) Support constraints in spark-dispatcher

2017-07-06 Thread Pascal GILLET (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076510#comment-16076510 ] Pascal GILLET commented on SPARK-19606: --- +1 Need to run Spark drivers and executors on 2 exclusive

[jira] [Commented] (SPARK-21326) Use TextFileFormat in implementation of LibSVMFileFormat

2017-07-06 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076477#comment-16076477 ] Nick Pentreath commented on SPARK-21326: I think SPARK-21066 for multiple input files would be

[jira] [Commented] (SPARK-18859) Catalyst codegen does not mark column as nullable when it should. Causes NPE

2017-07-06 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076476#comment-16076476 ] Takeshi Yamamuro commented on SPARK-18859: -- we better asking qualified guys cc: [~smilegator]

[jira] [Commented] (SPARK-21280) org.apache.spark.util.sketch.BloomFilter not bean compliant

2017-07-06 Thread Eran Moscovici (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076466#comment-16076466 ] Eran Moscovici commented on SPARK-21280: Thanks for your responses. Unfortunately for our

[jira] [Created] (SPARK-21330) Bad partitioning does not allow to read a JDBC table with extreme values on the partition column

2017-07-06 Thread Stefano Parmesan (JIRA)
Stefano Parmesan created SPARK-21330: Summary: Bad partitioning does not allow to read a JDBC table with extreme values on the partition column Key: SPARK-21330 URL:

[jira] [Commented] (SPARK-18859) Catalyst codegen does not mark column as nullable when it should. Causes NPE

2017-07-06 Thread Anton Okolnychyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076463#comment-16076463 ] Anton Okolnychyi commented on SPARK-18859: -- [~maropu] I think the approached suggested by you is

[jira] [Comment Edited] (SPARK-19341) Bucketing support for Structured Streaming

2017-07-06 Thread Fei Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058590#comment-16058590 ] Fei Shao edited comment on SPARK-19341 at 7/6/17 11:57 AM: --- [~tanejagagan]

[jira] [Resolved] (SPARK-20950) add a new config to diskWriteBufferSize which is hard coded before

2017-07-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20950. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18174

[jira] [Assigned] (SPARK-20950) add a new config to diskWriteBufferSize which is hard coded before

2017-07-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20950: --- Assignee: caoxuewen > add a new config to diskWriteBufferSize which is hard coded before >

[jira] [Assigned] (SPARK-21329) Make EventTimeWatermarkExec explicitly UnaryExecNode

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21329: Assignee: (was: Apache Spark) > Make EventTimeWatermarkExec explicitly UnaryExecNode

[jira] [Assigned] (SPARK-21329) Make EventTimeWatermarkExec explicitly UnaryExecNode

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21329: Assignee: Apache Spark > Make EventTimeWatermarkExec explicitly UnaryExecNode >

[jira] [Commented] (SPARK-21329) Make EventTimeWatermarkExec explicitly UnaryExecNode

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076383#comment-16076383 ] Apache Spark commented on SPARK-21329: -- User 'jaceklaskowski' has created a pull request for this

[jira] [Created] (SPARK-21329) Make EventTimeWatermarkExec explicitly UnaryExecNode

2017-07-06 Thread Jacek Laskowski (JIRA)
Jacek Laskowski created SPARK-21329: --- Summary: Make EventTimeWatermarkExec explicitly UnaryExecNode Key: SPARK-21329 URL: https://issues.apache.org/jira/browse/SPARK-21329 Project: Spark

[jira] [Commented] (SPARK-21316) Dataset Union output is not consistent with the column sequence

2017-07-06 Thread Kaushal Prajapati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076337#comment-16076337 ] Kaushal Prajapati commented on SPARK-21316: --- [~dongjoon] It works when the column names are

[jira] [Assigned] (SPARK-21306) OneVsRest Conceals Columns That May Be Relevant To Underlying Classifier

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21306: Assignee: (was: Apache Spark) > OneVsRest Conceals Columns That May Be Relevant To

[jira] [Assigned] (SPARK-21306) OneVsRest Conceals Columns That May Be Relevant To Underlying Classifier

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21306: Assignee: Apache Spark > OneVsRest Conceals Columns That May Be Relevant To Underlying

[jira] [Commented] (SPARK-21306) OneVsRest Conceals Columns That May Be Relevant To Underlying Classifier

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076274#comment-16076274 ] Apache Spark commented on SPARK-21306: -- User 'facaiy' has created a pull request for this issue:

[jira] [Updated] (SPARK-21326) Use TextFileFormat in implementation of LibSVMFileFormat

2017-07-06 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-21326: - Description: This is related with SPARK-19918 and SPARK-18362. There are three points here: -

[jira] [Comment Edited] (SPARK-21325) The shell of 'spark-submit' about '--jars' and '--fils', jars and files can be placed on local and hdfs.

2017-07-06 Thread guoxiaolongzte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076244#comment-16076244 ] guoxiaolongzte edited comment on SPARK-21325 at 7/6/17 9:36 AM: This

[jira] [Resolved] (SPARK-21325) The shell of 'spark-submit' about '--jars' and '--fils', jars and files can be placed on local and hdfs.

2017-07-06 Thread guoxiaolongzte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolongzte resolved SPARK-21325. Resolution: Fixed This question has been modified by jerryshao. [~jerryshao] > The shell

[jira] [Updated] (SPARK-21325) The shell of 'spark-submit' about '--jars' and '--fils', jars and files can be placed on local and hdfs.

2017-07-06 Thread guoxiaolongzte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolongzte updated SPARK-21325: --- Description: 1.My submit way: spark-submit --class cn.gxl.TestSql{color:red} --jars

[jira] [Updated] (SPARK-21325) The shell of 'spark-submit' about '--jars' and '--fils', jars and files can be placed on local and hdfs.

2017-07-06 Thread guoxiaolongzte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolongzte updated SPARK-21325: --- Description: 1.My submit way: spark-submit --class cn.gxl.TestSql{color:red} --jars

[jira] [Assigned] (SPARK-21327) ArrayConstructor should handle an array of typecode 'l' as long rather than int in Python 2.

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21327: Assignee: Apache Spark > ArrayConstructor should handle an array of typecode 'l' as long

[jira] [Assigned] (SPARK-21327) ArrayConstructor should handle an array of typecode 'l' as long rather than int in Python 2.

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21327: Assignee: (was: Apache Spark) > ArrayConstructor should handle an array of typecode

[jira] [Updated] (SPARK-21325) The shell of 'spark-submit' about '--jars' and '--fils', jars and files can be placed on local and hdfs.

2017-07-06 Thread guoxiaolongzte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolongzte updated SPARK-21325: --- Description: 1.My submit way: spark-submit --class cn.gxl.TestSql{color:red} --jars

[jira] [Created] (SPARK-21328) Corpus creation using sparkR for textmining

2017-07-06 Thread Joe (JIRA)
Joe created SPARK-21328: --- Summary: Corpus creation using sparkR for textmining Key: SPARK-21328 URL: https://issues.apache.org/jira/browse/SPARK-21328 Project: Spark Issue Type: Question

[jira] [Commented] (SPARK-21327) ArrayConstructor should handle an array of typecode 'l' as long rather than int in Python 2.

2017-07-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076226#comment-16076226 ] Apache Spark commented on SPARK-21327: -- User 'ueshin' has created a pull request for this issue:

[jira] [Updated] (SPARK-21325) The shell of 'spark-submit' about '--jars' and '--fils', jars and files can be placed on local and hdfs.

2017-07-06 Thread guoxiaolongzte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolongzte updated SPARK-21325: --- Description: 1.My submit way: spark-submit --class cn.gxl.TestSql --jars

[jira] [Reopened] (SPARK-21325) The shell of 'spark-submit' about '--jars' and '--fils', jars and files can be placed on local and hdfs.

2017-07-06 Thread guoxiaolongzte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolongzte reopened SPARK-21325: 1.My submit way: spark-submit --class cn.gxl.TestSql -{color:red}-jars

[jira] [Created] (SPARK-21327) ArrayConstructor should handle an array of typecode 'l' as long rather than int in Python 2.

2017-07-06 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-21327: - Summary: ArrayConstructor should handle an array of typecode 'l' as long rather than int in Python 2. Key: SPARK-21327 URL: https://issues.apache.org/jira/browse/SPARK-21327

[jira] [Commented] (SPARK-19680) Offsets out of range with no configured reset policy for partitions

2017-07-06 Thread Simon Kitching (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076214#comment-16076214 ] Simon Kitching commented on SPARK-19680: I seem to be getting this problem too. Environment:

[jira] [Issue Comment Deleted] (SPARK-21325) The shell of 'spark-submit' about '--jars' and '--fils', jars and files can be placed on local and hdfs.

2017-07-06 Thread guoxiaolongzte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolongzte updated SPARK-21325: --- Comment: was deleted (was: .I'm going to add details. What about it now?) > The shell of

[jira] [Commented] (SPARK-21325) The shell of 'spark-submit' about '--jars' and '--fils', jars and files can be placed on local and hdfs.

2017-07-06 Thread guoxiaolongzte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076202#comment-16076202 ] guoxiaolongzte commented on SPARK-21325: .I'm going to add details. What about it now? > The

[jira] [Commented] (SPARK-21326) Use TextFileFormat in implementation of LibSVMFileFormat

2017-07-06 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076195#comment-16076195 ] Hyukjin Kwon commented on SPARK-21326: -- Let me open a PR in coming few days. > Use TextFileFormat

[jira] [Created] (SPARK-21326) Use TextFileFormat in implementation of LibSVMFileFormat

2017-07-06 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-21326: Summary: Use TextFileFormat in implementation of LibSVMFileFormat Key: SPARK-21326 URL: https://issues.apache.org/jira/browse/SPARK-21326 Project: Spark

[jira] [Resolved] (SPARK-21325) The shell of 'spark-submit' about '--jars' and '--fils', jars and files can be placed on local and hdfs.

2017-07-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21325. --- Resolution: Invalid I'm preemptively resolving these types of JIRAs, as there is no detail at all

[jira] [Commented] (SPARK-21306) OneVsRest Conceals Columns That May Be Relevant To Underlying Classifier

2017-07-06 Thread Cathal Garvey (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076172#comment-16076172 ] Cathal Garvey commented on SPARK-21306: --- Hi [~mlnick] & [~facai] - Passing-through the weights

  1   2   >