[jira] [Resolved] (SPARK-25734) Literal should have a value corresponding to dataType

2018-10-16 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25734. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22724 [https://gith

[jira] [Assigned] (SPARK-25734) Literal should have a value corresponding to dataType

2018-10-16 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25734: --- Assignee: Takeshi Yamamuro > Literal should have a value corresponding to dataType > --

[jira] [Created] (SPARK-25754) Change CDN for MathJax

2018-10-16 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-25754: -- Summary: Change CDN for MathJax Key: SPARK-25754 URL: https://issues.apache.org/jira/browse/SPARK-25754 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-24424) Support ANSI-SQL compliant syntax for GROUPING SET

2018-10-16 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24424: Description: Currently, our Group By clause follows Hive [https://cwiki.apache.org/confluence/display/Hiv

[jira] [Updated] (SPARK-24433) R Bindings for K8S

2018-10-16 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24433: Summary: R Bindings for K8S (was: Add Spark R support) > R Bindings for K8S > -- > >

[jira] [Updated] (SPARK-24215) Implement eager evaluation for DataFrame APIs

2018-10-16 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24215: Summary: Implement eager evaluation for DataFrame APIs (was: Implement __repr__ and _repr_html_ for data

[jira] [Commented] (SPARK-24472) Orc RecordReaderFactory throws IndexOutOfBoundsException

2018-10-16 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652969#comment-16652969 ] Dongjoon Hyun commented on SPARK-24472: --- Thank you, [~desmoon]. We know that, but

[jira] [Commented] (SPARK-10816) EventTime based sessionization

2018-10-16 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652925#comment-16652925 ] Jungtaek Lim commented on SPARK-10816: -- I've been thinking about requirements on st

[jira] [Commented] (SPARK-25332) Instead of broadcast hash join ,Sort merge join has selected when restart spark-shell/spark-JDBC for hive provider

2018-10-16 Thread Sujith (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652914#comment-16652914 ] Sujith commented on SPARK-25332: [~Bjangir] i think you are right, there is a bug while

[jira] [Commented] (SPARK-24472) Orc RecordReaderFactory throws IndexOutOfBoundsException

2018-10-16 Thread StephenZou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652885#comment-16652885 ] StephenZou commented on SPARK-24472: Hive already fix this in HIVE-11981 HIVE-12625

[jira] [Commented] (SPARK-10816) EventTime based sessionization

2018-10-16 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652878#comment-16652878 ] Jungtaek Lim commented on SPARK-10816: -- Update on test results for both data patter

[jira] [Resolved] (SPARK-25393) Parsing CSV strings in a column

2018-10-16 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25393. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22379 [https://gi

[jira] [Assigned] (SPARK-25393) Parsing CSV strings in a column

2018-10-16 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25393: Assignee: Maxim Gekk > Parsing CSV strings in a column > ---

[jira] [Assigned] (SPARK-25753) binaryFiles broken for small files

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25753: Assignee: Apache Spark > binaryFiles broken for small files > ---

[jira] [Assigned] (SPARK-25753) binaryFiles broken for small files

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25753: Assignee: (was: Apache Spark) > binaryFiles broken for small files >

[jira] [Commented] (SPARK-25753) binaryFiles broken for small files

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652801#comment-16652801 ] Apache Spark commented on SPARK-25753: -- User '10110346' has created a pull request

[jira] [Updated] (SPARK-25753) binaryFiles broken for small files

2018-10-16 Thread liuxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-25753: Description: _{{StreamFileInputFormat}}_ and {{_WholeTextFileInputFormat_(https://issues.apache.org/jira/

[jira] [Created] (SPARK-25753) binaryFiles broken for small files

2018-10-16 Thread liuxian (JIRA)
liuxian created SPARK-25753: --- Summary: binaryFiles broken for small files Key: SPARK-25753 URL: https://issues.apache.org/jira/browse/SPARK-25753 Project: Spark Issue Type: Bug Components

[jira] [Commented] (SPARK-25733) The method toLocalIterator() with dataframe doesn't work

2018-10-16 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652746#comment-16652746 ] Bryan Cutler commented on SPARK-25733: -- Is this a duplicate of SPARK-23961? > The

[jira] [Assigned] (SPARK-24787) Events being dropped at an alarming rate due to hsync being slow for eventLogging

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24787: Assignee: (was: Apache Spark) > Events being dropped at an alarming rate due to hsync

[jira] [Commented] (SPARK-24787) Events being dropped at an alarming rate due to hsync being slow for eventLogging

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652691#comment-16652691 ] Apache Spark commented on SPARK-24787: -- User 'devaraj-kavali' has created a pull re

[jira] [Commented] (SPARK-24787) Events being dropped at an alarming rate due to hsync being slow for eventLogging

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652690#comment-16652690 ] Apache Spark commented on SPARK-24787: -- User 'devaraj-kavali' has created a pull re

[jira] [Assigned] (SPARK-24787) Events being dropped at an alarming rate due to hsync being slow for eventLogging

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24787: Assignee: Apache Spark > Events being dropped at an alarming rate due to hsync being slow

[jira] [Commented] (SPARK-24787) Events being dropped at an alarming rate due to hsync being slow for eventLogging

2018-10-16 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652687#comment-16652687 ] Devaraj K commented on SPARK-24787: --- It seems here the overhead is coming due the forc

[jira] [Resolved] (SPARK-25693) Fix the multiple "manager" classes in org.apache.spark.deploy.security

2018-10-16 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25693. Resolution: Duplicate I started looking at this while waiting for reviews of SPARK-23781,

[jira] [Resolved] (SPARK-25632) KafkaRDDSuite: compacted topic 2 min 5 sec.

2018-10-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25632. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22670 [https://github.c

[jira] [Assigned] (SPARK-25632) KafkaRDDSuite: compacted topic 2 min 5 sec.

2018-10-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25632: - Assignee: Dilip Biswal > KafkaRDDSuite: compacted topic 2 min 5 sec. >

[jira] [Resolved] (SPARK-25631) KafkaRDDSuite: basic usage 2 min 4 sec

2018-10-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25631. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22670 [https://github.c

[jira] [Assigned] (SPARK-25631) KafkaRDDSuite: basic usage 2 min 4 sec

2018-10-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25631: - Assignee: Dilip Biswal > KafkaRDDSuite: basic usage2 min 4 sec > --

[jira] [Assigned] (SPARK-25394) Expose App status metrics as Source

2018-10-16 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25394: -- Assignee: Stavros Kontopoulos > Expose App status metrics as Source > ---

[jira] [Resolved] (SPARK-25394) Expose App status metrics as Source

2018-10-16 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25394. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22381 [https:

[jira] [Created] (SPARK-25752) Add trait to easily whitelist logical operators that produce named output from CleanupAliases

2018-10-16 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-25752: - Summary: Add trait to easily whitelist logical operators that produce named output from CleanupAliases Key: SPARK-25752 URL: https://issues.apache.org/jira/browse/SPARK-25752

[jira] [Commented] (SPARK-18673) Dataframes doesn't work on Hadoop 3.x; Hive rejects Hadoop version

2018-10-16 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652449#comment-16652449 ] t oo commented on SPARK-18673: -- bump > Dataframes doesn't work on Hadoop 3.x; Hive rejects

[jira] [Updated] (SPARK-25750) Integration Testing for Kerberos Support for Spark on Kubernetes

2018-10-16 Thread Ilan Filonenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilan Filonenko updated SPARK-25750: --- Summary: Integration Testing for Kerberos Support for Spark on Kubernetes (was: Secure HDFS

[jira] [Created] (SPARK-25751) Unit Testing for Kerberos Support for Spark on Kubernetes

2018-10-16 Thread Ilan Filonenko (JIRA)
Ilan Filonenko created SPARK-25751: -- Summary: Unit Testing for Kerberos Support for Spark on Kubernetes Key: SPARK-25751 URL: https://issues.apache.org/jira/browse/SPARK-25751 Project: Spark

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

2018-10-16 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652440#comment-16652440 ] t oo commented on SPARK-20202: -- bump > Remove references to org.spark-project.hive > -

[jira] [Assigned] (SPARK-25750) Secure HDFS Integration Testing

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25750: Assignee: (was: Apache Spark) > Secure HDFS Integration Testing > ---

[jira] [Commented] (SPARK-25750) Secure HDFS Integration Testing

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652400#comment-16652400 ] Apache Spark commented on SPARK-25750: -- User 'ifilonenko' has created a pull reques

[jira] [Assigned] (SPARK-25750) Secure HDFS Integration Testing

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25750: Assignee: Apache Spark > Secure HDFS Integration Testing > --

[jira] [Created] (SPARK-25750) Secure HDFS Integration Testing

2018-10-16 Thread Ilan Filonenko (JIRA)
Ilan Filonenko created SPARK-25750: -- Summary: Secure HDFS Integration Testing Key: SPARK-25750 URL: https://issues.apache.org/jira/browse/SPARK-25750 Project: Spark Issue Type: Test

[jira] [Commented] (SPARK-20327) Add CLI support for YARN custom resources, like GPUs

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652395#comment-16652395 ] Apache Spark commented on SPARK-20327: -- User 'vanzin' has created a pull request fo

[jira] [Commented] (SPARK-20327) Add CLI support for YARN custom resources, like GPUs

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652393#comment-16652393 ] Apache Spark commented on SPARK-20327: -- User 'vanzin' has created a pull request fo

[jira] [Resolved] (SPARK-25748) Upgrade net.razorvine:pyrolite version

2018-10-16 Thread Ankur Gupta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Gupta resolved SPARK-25748. - Resolution: Invalid Yeah, it seems this jar was tagged incorrectly in the internal security scan

[jira] [Updated] (SPARK-25749) Exception thrown while reading avro file with large schema

2018-10-16 Thread Raj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raj updated SPARK-25749: Description: Hi, We are migrating our jobs from Spark 2.2.0 to Spark 2.3.1. One of the job reads avro source that

[jira] [Updated] (SPARK-25749) Exception thrown while reading avro file with large schema

2018-10-16 Thread Raj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raj updated SPARK-25749: Attachment: exception > Exception thrown while reading avro file with large schema > -

[jira] [Updated] (SPARK-25749) Exception thrown while reading avro file with large schema

2018-10-16 Thread Raj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raj updated SPARK-25749: Attachment: build.sbt > Exception thrown while reading avro file with large schema > -

[jira] [Updated] (SPARK-25749) Exception thrown while reading avro file with large schema

2018-10-16 Thread Raj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raj updated SPARK-25749: Attachment: MainCC.scala EncoderExample.scala > Exception thrown while reading avro file with larg

[jira] [Created] (SPARK-25749) Exception thrown while reading avro file with large schema

2018-10-16 Thread Raj (JIRA)
Raj created SPARK-25749: --- Summary: Exception thrown while reading avro file with large schema Key: SPARK-25749 URL: https://issues.apache.org/jira/browse/SPARK-25749 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-25748) Upgrade net.razorvine:pyrolite version

2018-10-16 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652109#comment-16652109 ] Marcelo Vanzin commented on SPARK-25748: That CVE seems completely unrelated to

[jira] [Reopened] (SPARK-17875) Remove unneeded direct dependence on Netty 3.x

2018-10-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-17875: --- > Remove unneeded direct dependence on Netty 3.x > -- > >

[jira] [Created] (SPARK-25748) Upgrade net.razorvine:pyrolite version

2018-10-16 Thread Ankur Gupta (JIRA)
Ankur Gupta created SPARK-25748: --- Summary: Upgrade net.razorvine:pyrolite version Key: SPARK-25748 URL: https://issues.apache.org/jira/browse/SPARK-25748 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-25742) Is there a way to pass the Azure blob storage credentials to the spark for k8s init-container?

2018-10-16 Thread Yinan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652064#comment-16652064 ] Yinan Li commented on SPARK-25742: -- The k8s secrets you add through the {{spark.kuberne

[jira] [Commented] (SPARK-25643) Performance issues querying wide rows

2018-10-16 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652039#comment-16652039 ] Ruslan Dautkhanov commented on SPARK-25643: --- [~viirya] we confirm this problem

[jira] [Commented] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

2018-10-16 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652029#comment-16652029 ] Marcelo Vanzin commented on SPARK-25732: bq. livy server would not know which da

[jira] [Assigned] (SPARK-25747) remove ColumnarBatchScan.needsUnsafeRowConversion

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25747: Assignee: Wenchen Fan (was: Apache Spark) > remove ColumnarBatchScan.needsUnsafeRowConve

[jira] [Commented] (SPARK-25747) remove ColumnarBatchScan.needsUnsafeRowConversion

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652007#comment-16652007 ] Apache Spark commented on SPARK-25747: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-25747) remove ColumnarBatchScan.needsUnsafeRowConversion

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25747: Assignee: Apache Spark (was: Wenchen Fan) > remove ColumnarBatchScan.needsUnsafeRowConve

[jira] [Created] (SPARK-25747) remove ColumnarBatchScan.needsUnsafeRowConversion

2018-10-16 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-25747: --- Summary: remove ColumnarBatchScan.needsUnsafeRowConversion Key: SPARK-25747 URL: https://issues.apache.org/jira/browse/SPARK-25747 Project: Spark Issue Type: I

[jira] [Commented] (SPARK-25746) Refactoring ExpressionEncoder

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651927#comment-16651927 ] Apache Spark commented on SPARK-25746: -- User 'viirya' has created a pull request fo

[jira] [Assigned] (SPARK-25746) Refactoring ExpressionEncoder

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25746: Assignee: Apache Spark > Refactoring ExpressionEncoder > - >

[jira] [Assigned] (SPARK-25746) Refactoring ExpressionEncoder

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25746: Assignee: (was: Apache Spark) > Refactoring ExpressionEncoder > -

[jira] [Created] (SPARK-25746) Refactoring ExpressionEncoder

2018-10-16 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-25746: --- Summary: Refactoring ExpressionEncoder Key: SPARK-25746 URL: https://issues.apache.org/jira/browse/SPARK-25746 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

2018-10-16 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651860#comment-16651860 ] Marco Gaido commented on SPARK-25732: - [~tgraves] yes, exactly it is what I am refer

[jira] [Comment Edited] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

2018-10-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651840#comment-16651840 ] Thomas Graves edited comment on SPARK-25732 at 10/16/18 2:53 PM: -

[jira] [Comment Edited] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

2018-10-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651840#comment-16651840 ] Thomas Graves edited comment on SPARK-25732 at 10/16/18 2:49 PM: -

[jira] [Commented] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

2018-10-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651840#comment-16651840 ] Thomas Graves commented on SPARK-25732: --- sorry just realized I misread the second

[jira] [Commented] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

2018-10-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651827#comment-16651827 ] Thomas Graves commented on SPARK-25732: --- yeah I understand the concern, we don't w

[jira] [Commented] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

2018-10-16 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651800#comment-16651800 ] Marco Gaido commented on SPARK-25732: - [~tgraves] I think they can be reused, the po

[jira] [Updated] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

2018-10-16 Thread Brian Jones (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Jones updated SPARK-25739: Flags: Patch,Important (was: Important) > Double quote coming in as empty value even when emptyVa

[jira] [Updated] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

2018-10-16 Thread Brian Jones (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Jones updated SPARK-25739: Flags: Important (was: Patch,Important) > Double quote coming in as empty value even when emptyVa

[jira] [Assigned] (SPARK-25705) Remove Kafka 0.8 support in Spark 3.0.0

2018-10-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25705: - Assignee: Sean Owen > Remove Kafka 0.8 support in Spark 3.0.0 > ---

[jira] [Resolved] (SPARK-25705) Remove Kafka 0.8 support in Spark 3.0.0

2018-10-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25705. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22703 [https://github.c

[jira] [Commented] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

2018-10-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651657#comment-16651657 ] Thomas Graves commented on SPARK-25732: --- So like Marcelo mentioned can't you re-us

[jira] [Commented] (SPARK-25745) docker-image-tool.sh ignores errors from Docker

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651631#comment-16651631 ] Apache Spark commented on SPARK-25745: -- User 'rvesse' has created a pull request fo

[jira] [Assigned] (SPARK-25745) docker-image-tool.sh ignores errors from Docker

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25745: Assignee: (was: Apache Spark) > docker-image-tool.sh ignores errors from Docker > ---

[jira] [Assigned] (SPARK-25745) docker-image-tool.sh ignores errors from Docker

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25745: Assignee: Apache Spark > docker-image-tool.sh ignores errors from Docker > --

[jira] [Commented] (SPARK-25745) docker-image-tool.sh ignores errors from Docker

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651630#comment-16651630 ] Apache Spark commented on SPARK-25745: -- User 'rvesse' has created a pull request fo

[jira] [Commented] (SPARK-24499) Documentation improvement of Spark core and SQL

2018-10-16 Thread Yuanjian Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651628#comment-16651628 ] Yuanjian Li commented on SPARK-24499: - {quote} Is this just about splitting up the d

[jira] [Comment Edited] (SPARK-24902) Add integration tests for PVs

2018-10-16 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651621#comment-16651621 ] Stavros Kontopoulos edited comment on SPARK-24902 at 10/16/18 12:50 PM: --

[jira] [Comment Edited] (SPARK-24902) Add integration tests for PVs

2018-10-16 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651621#comment-16651621 ] Stavros Kontopoulos edited comment on SPARK-24902 at 10/16/18 12:50 PM: --

[jira] [Commented] (SPARK-24902) Add integration tests for PVs

2018-10-16 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651621#comment-16651621 ] Stavros Kontopoulos commented on SPARK-24902: - This is blocked by this issue

[jira] [Created] (SPARK-25745) docker-image-tool.sh ignores errors from Docker

2018-10-16 Thread Rob Vesse (JIRA)
Rob Vesse created SPARK-25745: - Summary: docker-image-tool.sh ignores errors from Docker Key: SPARK-25745 URL: https://issues.apache.org/jira/browse/SPARK-25745 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-25579) Use quoted attribute names if needed in pushed ORC predicates

2018-10-16 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25579. -- Resolution: Fixed Fix Version/s: 2.4.0 3.0.0 Issue resolved by pull

[jira] [Commented] (SPARK-24499) Documentation improvement of Spark core and SQL

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651598#comment-16651598 ] Apache Spark commented on SPARK-24499: -- User 'xuanyuanking' has created a pull requ

[jira] [Assigned] (SPARK-24499) Documentation improvement of Spark core and SQL

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24499: Assignee: (was: Apache Spark) > Documentation improvement of Spark core and SQL > ---

[jira] [Assigned] (SPARK-24499) Documentation improvement of Spark core and SQL

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24499: Assignee: Apache Spark > Documentation improvement of Spark core and SQL > --

[jira] [Updated] (SPARK-24882) data source v2 API improvement

2018-10-16 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-24882: Fix Version/s: (was: 2.4.0) 3.0.0 > data source v2 API improvement > --

[jira] [Updated] (SPARK-24882) data source v2 API improvement

2018-10-16 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-24882: Issue Type: Sub-task (was: Improvement) Parent: SPARK-22386 > data source v2 API improvem

[jira] [Commented] (SPARK-18832) Spark SQL: Thriftserver unable to run a registered Hive UDTF

2018-10-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651565#comment-16651565 ] Yuming Wang commented on SPARK-18832: - [~roadster11x] I can't reproduce. > Spark SQ

[jira] [Commented] (SPARK-21402) Java encoders - switch fields on collectAsList

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651550#comment-16651550 ] Apache Spark commented on SPARK-21402: -- User 'vofque' has created a pull request fo

[jira] [Commented] (SPARK-21402) Java encoders - switch fields on collectAsList

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651546#comment-16651546 ] Apache Spark commented on SPARK-21402: -- User 'vofque' has created a pull request fo

[jira] [Created] (SPARK-25744) Allow kubernetes integration tests to be run against a real cluster.

2018-10-16 Thread Prashant Sharma (JIRA)
Prashant Sharma created SPARK-25744: --- Summary: Allow kubernetes integration tests to be run against a real cluster. Key: SPARK-25744 URL: https://issues.apache.org/jira/browse/SPARK-25744 Project: S

[jira] [Updated] (SPARK-25743) New executors are not launched for kubernetes spark thrift on deleting existing executors

2018-10-16 Thread neenu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] neenu updated SPARK-25743: -- Attachment: driver.log > New executors are not launched for kubernetes spark thrift on deleting > existing ex

[jira] [Updated] (SPARK-25743) New executors are not launched for kubernetes spark thrift on deleting existing executors

2018-10-16 Thread neenu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] neenu updated SPARK-25743: -- Attachment: query_0_correct.sql > New executors are not launched for kubernetes spark thrift on deleting > ex

[jira] [Commented] (SPARK-25743) New executors are not launched for kubernetes spark thrift on deleting existing executors

2018-10-16 Thread neenu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651514#comment-16651514 ] neenu commented on SPARK-25743: --- Also attached the tpcds query list executed [^query_0_cor

[jira] [Created] (SPARK-25743) New executors are not launched for kubernetes spark thrift on deleting existing executors

2018-10-16 Thread neenu (JIRA)
neenu created SPARK-25743: - Summary: New executors are not launched for kubernetes spark thrift on deleting existing executors Key: SPARK-25743 URL: https://issues.apache.org/jira/browse/SPARK-25743 Project:

[jira] [Created] (SPARK-25742) Is there a way to pass the Azure blob storage credentials to the spark for k8s init-container?

2018-10-16 Thread Oscar Bonilla (JIRA)
Oscar Bonilla created SPARK-25742: - Summary: Is there a way to pass the Azure blob storage credentials to the spark for k8s init-container? Key: SPARK-25742 URL: https://issues.apache.org/jira/browse/SPARK-25742

[jira] [Commented] (SPARK-25728) SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code

2018-10-16 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651480#comment-16651480 ] Marco Gaido commented on SPARK-25728: - Thanks [~kiszk]. I will check it ASAP, thanks

[jira] [Assigned] (SPARK-25741) Long URLs are not rendered properly in web UI

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25741: Assignee: (was: Apache Spark) > Long URLs are not rendered properly in web UI > -

[jira] [Commented] (SPARK-25741) Long URLs are not rendered properly in web UI

2018-10-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651437#comment-16651437 ] Apache Spark commented on SPARK-25741: -- User 'gengliangwang' has created a pull req

  1   2   >