[jira] [Commented] (SPARK-25632) KafkaRDDSuite: compacted topic 2 min 5 sec.

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641397#comment-16641397 ] Apache Spark commented on SPARK-25632: -- User 'dilipbiswal' has created a pull reque

[jira] [Commented] (SPARK-25631) KafkaRDDSuite: basic usage 2 min 4 sec

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641394#comment-16641394 ] Apache Spark commented on SPARK-25631: -- User 'dilipbiswal' has created a pull reque

[jira] [Commented] (SPARK-25632) KafkaRDDSuite: compacted topic 2 min 5 sec.

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641396#comment-16641396 ] Apache Spark commented on SPARK-25632: -- User 'dilipbiswal' has created a pull reque

[jira] [Assigned] (SPARK-25632) KafkaRDDSuite: compacted topic 2 min 5 sec.

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25632: Assignee: (was: Apache Spark) > KafkaRDDSuite: compacted topic 2 min 5 sec. > ---

[jira] [Assigned] (SPARK-25631) KafkaRDDSuite: basic usage 2 min 4 sec

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25631: Assignee: (was: Apache Spark) > KafkaRDDSuite: basic usage2 min 4 sec > -

[jira] [Assigned] (SPARK-25631) KafkaRDDSuite: basic usage 2 min 4 sec

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25631: Assignee: Apache Spark > KafkaRDDSuite: basic usage2 min 4 sec >

[jira] [Assigned] (SPARK-25632) KafkaRDDSuite: compacted topic 2 min 5 sec.

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25632: Assignee: Apache Spark > KafkaRDDSuite: compacted topic 2 min 5 sec. > --

[jira] [Commented] (SPARK-25631) KafkaRDDSuite: basic usage 2 min 4 sec

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641395#comment-16641395 ] Apache Spark commented on SPARK-25631: -- User 'dilipbiswal' has created a pull reque

[jira] [Comment Edited] (SPARK-25588) SchemaParseException: Can't redefine: list when reading from Parquet

2018-10-07 Thread Michael Heuer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16638226#comment-16638226 ] Michael Heuer edited comment on SPARK-25588 at 10/8/18 6:16 AM: --

[jira] [Updated] (SPARK-25677) Configuring zstd compression in JDBC throwing IllegalArgumentException Exception

2018-10-07 Thread ABHISHEK KUMAR GUPTA (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ABHISHEK KUMAR GUPTA updated SPARK-25677: - Description: To check the Event Log compression size with different compression

[jira] [Assigned] (SPARK-25677) [Spark Compression] spark.io.compression.codec = org.apache.spark.io.ZstdCompressionCodec throwing IllegalArgumentException Exception

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25677: Assignee: (was: Apache Spark) > [Spark Compression] spark.io.compression.codec = > o

[jira] [Commented] (SPARK-25677) [Spark Compression] spark.io.compression.codec = org.apache.spark.io.ZstdCompressionCodec throwing IllegalArgumentException Exception

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641392#comment-16641392 ] Apache Spark commented on SPARK-25677: -- User 'shivusondur' has created a pull reque

[jira] [Commented] (SPARK-25677) [Spark Compression] spark.io.compression.codec = org.apache.spark.io.ZstdCompressionCodec throwing IllegalArgumentException Exception

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641391#comment-16641391 ] Apache Spark commented on SPARK-25677: -- User 'shivusondur' has created a pull reque

[jira] [Assigned] (SPARK-25677) [Spark Compression] spark.io.compression.codec = org.apache.spark.io.ZstdCompressionCodec throwing IllegalArgumentException Exception

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25677: Assignee: Apache Spark > [Spark Compression] spark.io.compression.codec = > org.apache.s

[jira] [Updated] (SPARK-25591) PySpark Accumulators with multiple PythonUDFs

2018-10-07 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25591: - Target Version/s: 2.4.0 Labels: data-loss (was: ) Priority: Critical

[jira] [Commented] (SPARK-25649) CatalystTypeConverter throws exception for ScalaesRow type when converting from ArrayConverter

2018-10-07 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641379#comment-16641379 ] Hyukjin Kwon commented on SPARK-25649: -- [~mauliksoneji], have you tried ^? Let me l

[jira] [Resolved] (SPARK-25649) CatalystTypeConverter throws exception for ScalaesRow type when converting from ArrayConverter

2018-10-07 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25649. -- Resolution: Invalid > CatalystTypeConverter throws exception for ScalaesRow type when converti

[jira] [Commented] (SPARK-25648) Spark 2.3.1 reads orc format files with native and hive, and return different results

2018-10-07 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641377#comment-16641377 ] Hyukjin Kwon commented on SPARK-25648: -- {quote} There is some results lost with the

[jira] [Commented] (SPARK-25677) [Spark Compression] spark.io.compression.codec = org.apache.spark.io.ZstdCompressionCodec throwing IllegalArgumentException Exception

2018-10-07 Thread shivusondur (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641376#comment-16641376 ] shivusondur commented on SPARK-25677: - i am working on this issue > [Spark Compress

[jira] [Created] (SPARK-25677) [Spark Compression] spark.io.compression.codec = org.apache.spark.io.ZstdCompressionCodec throwing IllegalArgumentException Exception

2018-10-07 Thread ABHISHEK KUMAR GUPTA (JIRA)
ABHISHEK KUMAR GUPTA created SPARK-25677: Summary: [Spark Compression] spark.io.compression.codec = org.apache.spark.io.ZstdCompressionCodec throwing IllegalArgumentException Exception Key: SPARK-25677 UR

[jira] [Commented] (SPARK-25599) Stateful aggregation in PySpark

2018-10-07 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641373#comment-16641373 ] Hyukjin Kwon commented on SPARK-25599: -- Are you proposong UDAF for Python side? The

[jira] [Resolved] (SPARK-25651) spark-shell gets wrong version of spark on windows

2018-10-07 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25651. -- Resolution: Not A Problem > spark-shell gets wrong version of spark on windows > -

[jira] [Commented] (SPARK-25651) spark-shell gets wrong version of spark on windows

2018-10-07 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641372#comment-16641372 ] Hyukjin Kwon commented on SPARK-25651: -- You should set {{SPARK_HOME}} correctly. {{

[jira] [Commented] (SPARK-25587) NPE in Dataset when reading from Parquet as Product

2018-10-07 Thread Michael Heuer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641371#comment-16641371 ] Michael Heuer commented on SPARK-25587: --- [~hyukjin.kwon], I agree that this isn't

[jira] [Commented] (SPARK-25652) Wrong datetime conversion between Java and Python

2018-10-07 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641369#comment-16641369 ] Hyukjin Kwon commented on SPARK-25652: -- Mind describing what's expected output and

[jira] [Resolved] (SPARK-25580) com.mongodb.spark.exceptions.MongoTypeConversionException: Cannot cast STRING into a DoubleType

2018-10-07 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25580. -- Resolution: Invalid That looks an issue at MongoDB - https://github.com/mongodb/mongo-spark/b

[jira] [Commented] (SPARK-25587) NPE in Dataset when reading from Parquet as Product

2018-10-07 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641366#comment-16641366 ] Hyukjin Kwon commented on SPARK-25587: -- [~heuermh], mind fixing the JIRA accordingl

[jira] [Created] (SPARK-25676) Refactor BenchmarkWideTable to use main method

2018-10-07 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-25676: --- Summary: Refactor BenchmarkWideTable to use main method Key: SPARK-25676 URL: https://issues.apache.org/jira/browse/SPARK-25676 Project: Spark Issue Type: Sub-

[jira] [Updated] (SPARK-25591) PySpark Accumulators with multiple PythonUDFs

2018-10-07 Thread Abdeali Kothari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abdeali Kothari updated SPARK-25591: Description: When having multiple Python UDFs - the last Python UDF's accumulator is the

[jira] [Commented] (SPARK-25675) [Spark Job History] Job UI page does not show pagination with one page

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641345#comment-16641345 ] Apache Spark commented on SPARK-25675: -- User 'shivusondur' has created a pull reque

[jira] [Commented] (SPARK-25675) [Spark Job History] Job UI page does not show pagination with one page

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641344#comment-16641344 ] Apache Spark commented on SPARK-25675: -- User 'shivusondur' has created a pull reque

[jira] [Assigned] (SPARK-25675) [Spark Job History] Job UI page does not show pagination with one page

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25675: Assignee: (was: Apache Spark) > [Spark Job History] Job UI page does not show paginat

[jira] [Assigned] (SPARK-25675) [Spark Job History] Job UI page does not show pagination with one page

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25675: Assignee: Apache Spark > [Spark Job History] Job UI page does not show pagination with on

[jira] [Commented] (SPARK-25675) [Spark Job History] Job UI page does not show pagination with one page

2018-10-07 Thread shivusondur (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641340#comment-16641340 ] shivusondur commented on SPARK-25675: - I am working on this issue > [Spark Job Hist

[jira] [Created] (SPARK-25675) [Spark Job History] Job UI page does not show pagination with one page

2018-10-07 Thread ABHISHEK KUMAR GUPTA (JIRA)
ABHISHEK KUMAR GUPTA created SPARK-25675: Summary: [Spark Job History] Job UI page does not show pagination with one page Key: SPARK-25675 URL: https://issues.apache.org/jira/browse/SPARK-25675

[jira] [Commented] (SPARK-24630) SPIP: Support SQLStreaming in Spark

2018-10-07 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641327#comment-16641327 ] Jungtaek Lim commented on SPARK-24630: -- [~Jackey Lee] For DDL it would be better t

[jira] [Commented] (SPARK-25625) LogisticRegressionSuite.binary logistic regression with intercept with ElasticNet regularization - 33 sec

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641307#comment-16641307 ] Apache Spark commented on SPARK-25625: -- User 'shahidki31' has created a pull reques

[jira] [Assigned] (SPARK-25625) LogisticRegressionSuite.binary logistic regression with intercept with ElasticNet regularization - 33 sec

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25625: Assignee: Apache Spark > LogisticRegressionSuite.binary logistic regression with intercep

[jira] [Commented] (SPARK-25625) LogisticRegressionSuite.binary logistic regression with intercept with ElasticNet regularization - 33 sec

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641308#comment-16641308 ] Apache Spark commented on SPARK-25625: -- User 'shahidki31' has created a pull reques

[jira] [Assigned] (SPARK-25625) LogisticRegressionSuite.binary logistic regression with intercept with ElasticNet regularization - 33 sec

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25625: Assignee: (was: Apache Spark) > LogisticRegressionSuite.binary logistic regression wi

[jira] [Commented] (SPARK-25624) LogisticRegressionSuite.multinomial logistic regression with intercept with elasticnet regularization 56 seconds

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641306#comment-16641306 ] Apache Spark commented on SPARK-25624: -- User 'shahidki31' has created a pull reques

[jira] [Resolved] (SPARK-19224) [PYSPARK] Python tests organization

2018-10-07 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-19224. -- Resolution: Duplicate > [PYSPARK] Python tests organization >

[jira] [Commented] (SPARK-25674) If the records are incremented by more than 1 at a time,the number of bytes might rarely ever get updated

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641295#comment-16641295 ] Apache Spark commented on SPARK-25674: -- User '10110346' has created a pull request

[jira] [Commented] (SPARK-25344) Break large tests.py files into smaller files

2018-10-07 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641294#comment-16641294 ] Hyukjin Kwon commented on SPARK-25344: -- [~irashid], would you mind if I try to take

[jira] [Assigned] (SPARK-25674) If the records are incremented by more than 1 at a time,the number of bytes might rarely ever get updated

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25674: Assignee: (was: Apache Spark) > If the records are incremented by more than 1 at a ti

[jira] [Assigned] (SPARK-25674) If the records are incremented by more than 1 at a time,the number of bytes might rarely ever get updated

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25674: Assignee: Apache Spark > If the records are incremented by more than 1 at a time,the numb

[jira] [Created] (SPARK-25674) If the records are incremented by more than 1 at a time,the number of bytes might rarely ever get updated

2018-10-07 Thread liuxian (JIRA)
liuxian created SPARK-25674: --- Summary: If the records are incremented by more than 1 at a time,the number of bytes might rarely ever get updated Key: SPARK-25674 URL: https://issues.apache.org/jira/browse/SPARK-25674

[jira] [Assigned] (SPARK-25673) Remove Travis CI which enables Java lint check

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25673: Assignee: (was: Apache Spark) > Remove Travis CI which enables Java lint check >

[jira] [Assigned] (SPARK-25673) Remove Travis CI which enables Java lint check

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25673: Assignee: Apache Spark > Remove Travis CI which enables Java lint check > ---

[jira] [Commented] (SPARK-25673) Remove Travis CI which enables Java lint check

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641284#comment-16641284 ] Apache Spark commented on SPARK-25673: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-25673) Remove Travis CI which enables Java lint check

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641285#comment-16641285 ] Apache Spark commented on SPARK-25673: -- User 'HyukjinKwon' has created a pull reque

[jira] [Created] (SPARK-25673) Remove Travis CI which enables Java lint check

2018-10-07 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-25673: Summary: Remove Travis CI which enables Java lint check Key: SPARK-25673 URL: https://issues.apache.org/jira/browse/SPARK-25673 Project: Spark Issue Type: Te

[jira] [Commented] (SPARK-24630) SPIP: Support SQLStreaming in Spark

2018-10-07 Thread Jackey Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641258#comment-16641258 ] Jackey Lee commented on SPARK-24630: SQLStreaming is another interfaces for StructSt

[jira] [Updated] (SPARK-25552) Upgrade from Spark 1.6.3 to 2.3.0 seems to make jobs use about 50% more memory

2018-10-07 Thread Nuno Azevedo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nuno Azevedo updated SPARK-25552: - Description: After upgrading from Spark 1.6.3 to 2.3.0 our jobs started to need about 50% more

[jira] [Updated] (SPARK-25661) Refactor AvroWriteBenchmark to use main method

2018-10-07 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25661: Summary: Refactor AvroWriteBenchmark to use main method (was: Refactor BuiltInDataSourceWriteBenc

[jira] [Updated] (SPARK-25663) Refactor BuiltInDataSourceWriteBenchmark and DataSourceWriteBenchmark to use main method

2018-10-07 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25663: Summary: Refactor BuiltInDataSourceWriteBenchmark and DataSourceWriteBenchmark to use main method

[jira] [Updated] (SPARK-25661) Refactor BuiltInDataSourceWriteBenchmark and AvroWriteBenchmark to use main method

2018-10-07 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25661: Summary: Refactor BuiltInDataSourceWriteBenchmark and AvroWriteBenchmark to use main method (was:

[jira] [Updated] (SPARK-25663) Refactor and DataSourceWriteBenchmark to use main method

2018-10-07 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25663: Summary: Refactor and DataSourceWriteBenchmark to use main method (was: Refactor BuiltInDataSourc

[jira] [Updated] (SPARK-25663) Refactor BuiltInDataSourceWriteBenchmark and DataSourceWriteBenchmark to use main method

2018-10-07 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25663: Summary: Refactor BuiltInDataSourceWriteBenchmark and DataSourceWriteBenchmark to use main method

[jira] [Commented] (SPARK-25672) Inferring schema from CSV string literal

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641201#comment-16641201 ] Apache Spark commented on SPARK-25672: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-25672) Inferring schema from CSV string literal

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25672: Assignee: Apache Spark > Inferring schema from CSV string literal > -

[jira] [Assigned] (SPARK-25672) Inferring schema from CSV string literal

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25672: Assignee: (was: Apache Spark) > Inferring schema from CSV string literal > --

[jira] [Commented] (SPARK-25672) Inferring schema from CSV string literal

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641200#comment-16641200 ] Apache Spark commented on SPARK-25672: -- User 'MaxGekk' has created a pull request f

[jira] [Created] (SPARK-25672) Inferring schema from CSV string literal

2018-10-07 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-25672: -- Summary: Inferring schema from CSV string literal Key: SPARK-25672 URL: https://issues.apache.org/jira/browse/SPARK-25672 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-25062) Clean up BlockLocations in FileStatus objects

2018-10-07 Thread Peter Toth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641152#comment-16641152 ] Peter Toth commented on SPARK-25062: Thanks [~dongjoon]. :) > Clean up BlockLocatio

[jira] [Commented] (SPARK-25576) Fix lint failure in 2.2

2018-10-07 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641151#comment-16641151 ] Dongjoon Hyun commented on SPARK-25576: --- [~samdvr] .I added you as Spark Contribut

[jira] [Assigned] (SPARK-25576) Fix lint failure in 2.2

2018-10-07 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25576: - Assignee: Sam Davarnia > Fix lint failure in 2.2 > --- > >

[jira] [Commented] (SPARK-25062) Clean up BlockLocations in FileStatus objects

2018-10-07 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641150#comment-16641150 ] Dongjoon Hyun commented on SPARK-25062: --- Finally, I added you to Spark Contributor

[jira] [Assigned] (SPARK-25062) Clean up BlockLocations in FileStatus objects

2018-10-07 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25062: - Assignee: Peter Toth > Clean up BlockLocations in FileStatus objects >

[jira] [Commented] (SPARK-25062) Clean up BlockLocations in FileStatus objects

2018-10-07 Thread Peter Toth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641146#comment-16641146 ] Peter Toth commented on SPARK-25062: [~dongjoon], do I need to set anything in my pr

[jira] [Commented] (SPARK-14681) Provide label/impurity stats for spark.ml decision tree nodes

2018-10-07 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641138#comment-16641138 ] Dongjoon Hyun commented on SPARK-14681: --- This is reverted on `master` branch, too.

[jira] [Assigned] (SPARK-25657) Refactor HashBenchmark to use main method

2018-10-07 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25657: - Assignee: Yuming Wang > Refactor HashBenchmark to use main method > ---

[jira] [Resolved] (SPARK-25657) Refactor HashBenchmark to use main method

2018-10-07 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25657. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22651 [https://

[jira] [Assigned] (SPARK-25658) Refactor HashByteArrayBenchmark to use main method

2018-10-07 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25658: - Assignee: Yuming Wang > Refactor HashByteArrayBenchmark to use main method > --

[jira] [Resolved] (SPARK-25658) Refactor HashByteArrayBenchmark to use main method

2018-10-07 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25658. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22652 [https://

[jira] [Commented] (SPARK-25062) Clean up BlockLocations in FileStatus objects

2018-10-07 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641131#comment-16641131 ] Dongjoon Hyun commented on SPARK-25062: --- This is done by [~petertoth], but current

[jira] [Updated] (SPARK-25062) Clean up BlockLocations in FileStatus objects

2018-10-07 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25062: -- Attachment: petertoth.png > Clean up BlockLocations in FileStatus objects > --

[jira] [Assigned] (SPARK-25461) PySpark Pandas UDF outputs incorrect results when input columns contain None

2018-10-07 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25461: Assignee: Liang-Chi Hsieh > PySpark Pandas UDF outputs incorrect results when input colum

[jira] [Resolved] (SPARK-25461) PySpark Pandas UDF outputs incorrect results when input columns contain None

2018-10-07 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25461. -- Resolution: Fixed Fix Version/s: 3.0.0 Fixed in https://github.com/apache/spark/pull/22

[jira] [Commented] (SPARK-25662) Refactor DataSourceReadBenchmark to use main method

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641093#comment-16641093 ] Apache Spark commented on SPARK-25662: -- User 'peter-toth' has created a pull reques

[jira] [Assigned] (SPARK-25662) Refactor DataSourceReadBenchmark to use main method

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25662: Assignee: (was: Apache Spark) > Refactor DataSourceReadBenchmark to use main method >

[jira] [Assigned] (SPARK-25662) Refactor DataSourceReadBenchmark to use main method

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25662: Assignee: Apache Spark > Refactor DataSourceReadBenchmark to use main method > --

[jira] [Assigned] (SPARK-25539) Update lz4-java to get speed improvement

2018-10-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25539: - Assignee: Yuming Wang > Update lz4-java to get speed improvement >

[jira] [Updated] (SPARK-25539) Update lz4-java to get speed improvement

2018-10-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25539: -- Priority: Minor (was: Major) > Update lz4-java to get speed improvement > ---

[jira] [Resolved] (SPARK-25539) Update lz4-java to get speed improvement

2018-10-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25539. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22551 [https://github.c

[jira] [Commented] (SPARK-25490) Refactor KryoBenchmark

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641057#comment-16641057 ] Apache Spark commented on SPARK-25490: -- User 'gengliangwang' has created a pull req

[jira] [Commented] (SPARK-25490) Refactor KryoBenchmark

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641063#comment-16641063 ] Apache Spark commented on SPARK-25490: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-25490) Refactor KryoBenchmark

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25490: Assignee: Apache Spark > Refactor KryoBenchmark > -- > >

[jira] [Assigned] (SPARK-25490) Refactor KryoBenchmark

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25490: Assignee: (was: Apache Spark) > Refactor KryoBenchmark > -- > >

[jira] [Commented] (SPARK-20415) SPARK job hangs while writing DataFrame to HDFS

2018-10-07 Thread Yan Zhitao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641054#comment-16641054 ] Yan Zhitao commented on SPARK-20415: I have similar issue but the thread dump has mi

[jira] [Commented] (SPARK-25490) Refactor KryoBenchmark

2018-10-07 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641044#comment-16641044 ] Gengliang Wang commented on SPARK-25490: I am working on this. > Refactor KryoB

[jira] [Assigned] (SPARK-25627) ContinuousStressSuite - 8 mins 13 sec

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25627: Assignee: (was: Apache Spark) > ContinuousStressSuite - 8 mins 13 sec > -

[jira] [Assigned] (SPARK-25627) ContinuousStressSuite - 8 mins 13 sec

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25627: Assignee: Apache Spark > ContinuousStressSuite - 8 mins 13 sec >

[jira] [Commented] (SPARK-25627) ContinuousStressSuite - 8 mins 13 sec

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641003#comment-16641003 ] Apache Spark commented on SPARK-25627: -- User 'viirya' has created a pull request fo

[jira] [Commented] (SPARK-25664) Refactor JoinBenchmark to use main method

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641000#comment-16641000 ] Apache Spark commented on SPARK-25664: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-25664) Refactor JoinBenchmark to use main method

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25664: Assignee: (was: Apache Spark) > Refactor JoinBenchmark to use main method > -

[jira] [Assigned] (SPARK-25664) Refactor JoinBenchmark to use main method

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25664: Assignee: Apache Spark > Refactor JoinBenchmark to use main method >

[jira] [Commented] (SPARK-25664) Refactor JoinBenchmark to use main method

2018-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16640999#comment-16640999 ] Apache Spark commented on SPARK-25664: -- User 'wangyum' has created a pull request f

[jira] [Updated] (SPARK-25466) Documentation does not specify how to set Kafka consumer cache capacity for SS

2018-10-07 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-25466: --- Summary: Documentation does not specify how to set Kafka consumer cache capacity for SS (was: Docum