[jira] [Created] (SPARK-25729) It is better to replace `minPartitions` with `defaultParallelism` , when `minPartitions` is less than `defaultParallelism`

liuxian created SPARK-25729: --- Summary: It is better to replace `minPartitions` with `defaultParallelism` , when `minPartitions` is less than `defaultParallelism` Key: SPARK-25729 URL: https://issues.apache.org/jira/brow

[jira] [Commented] (SPARK-25729) It is better to replace `minPartitions` with `defaultParallelism` , when `minPartitions` is less than `defaultParallelism`

[ https://issues.apache.org/jira/browse/SPARK-25729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649827#comment-16649827 ] Apache Spark commented on SPARK-25729: -- User '10110346' has created a pull request

[jira] [Assigned] (SPARK-25729) It is better to replace `minPartitions` with `defaultParallelism` , when `minPartitions` is less than `defaultParallelism`

[ https://issues.apache.org/jira/browse/SPARK-25729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25729: Assignee: Apache Spark > It is better to replace `minPartitions` with `defaultParallelism

[jira] [Assigned] (SPARK-25729) It is better to replace `minPartitions` with `defaultParallelism` , when `minPartitions` is less than `defaultParallelism`

[ https://issues.apache.org/jira/browse/SPARK-25729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25729: Assignee: (was: Apache Spark) > It is better to replace `minPartitions` with `default

[jira] [Created] (SPARK-25730) Kubernetes scheduler tries to read pod details that it just deleted

Mike Kaplinskiy created SPARK-25730: --- Summary: Kubernetes scheduler tries to read pod details that it just deleted Key: SPARK-25730 URL: https://issues.apache.org/jira/browse/SPARK-25730 Project: Sp

[jira] [Assigned] (SPARK-25730) Kubernetes scheduler tries to read pod details that it just deleted

[ https://issues.apache.org/jira/browse/SPARK-25730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25730: Assignee: (was: Apache Spark) > Kubernetes scheduler tries to read pod details that i

[jira] [Commented] (SPARK-25730) Kubernetes scheduler tries to read pod details that it just deleted

[ https://issues.apache.org/jira/browse/SPARK-25730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649833#comment-16649833 ] Apache Spark commented on SPARK-25730: -- User 'mikekap' has created a pull request f

[jira] [Assigned] (SPARK-25730) Kubernetes scheduler tries to read pod details that it just deleted

[ https://issues.apache.org/jira/browse/SPARK-25730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25730: Assignee: Apache Spark > Kubernetes scheduler tries to read pod details that it just dele

[jira] [Reopened] (SPARK-24213) Power Iteration Clustering in the SparkML throws exception, when the ID is IntType

[ https://issues.apache.org/jira/browse/SPARK-24213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid reopened SPARK-24213: > Power Iteration Clustering in the SparkML throws exception, when the ID is > IntType >

[jira] [Resolved] (SPARK-24213) Power Iteration Clustering in the SparkML throws exception, when the ID is IntType

[ https://issues.apache.org/jira/browse/SPARK-24213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid resolved SPARK-24213. Resolution: Done Target Version/s: (was: 2.4.0) > Power Iteration Clustering in the SparkML th

[jira] [Resolved] (SPARK-24213) Power Iteration Clustering in the SparkML throws exception, when the ID is IntType

[ https://issues.apache.org/jira/browse/SPARK-24213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid resolved SPARK-24213. Resolution: Done Fix Version/s: (was: 2.4.0) > Power Iteration Clustering in the SparkML throws

[jira] [Reopened] (SPARK-24213) Power Iteration Clustering in the SparkML throws exception, when the ID is IntType

[ https://issues.apache.org/jira/browse/SPARK-24213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid reopened SPARK-24213: > Power Iteration Clustering in the SparkML throws exception, when the ID is > IntType >

[jira] [Reopened] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid reopened SPARK-24217: > Power Iteration Clustering is not displaying cluster indices corresponding to > some vertices. > --

[jira] [Resolved] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid resolved SPARK-24217. Resolution: Done Fix Version/s: (was: 2.4.0) Target Version/s: (was: 2.4.0) > Power

[jira] [Created] (SPARK-25731) Spark Structured Streaming Support for Kafka 2.0

Chandan created SPARK-25731: --- Summary: Spark Structured Streaming Support for Kafka 2.0 Key: SPARK-25731 URL: https://issues.apache.org/jira/browse/SPARK-25731 Project: Spark Issue Type: Improvemen

[jira] [Updated] (SPARK-25731) Spark Structured Streaming Support for Kafka 2.0

[ https://issues.apache.org/jira/browse/SPARK-25731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-25731: Description: [https://github.com/apache/spark/tree/master/external] As far as I can see, This doesn't h

[jira] [Updated] (SPARK-25731) Spark Structured Streaming Support for Kafka 2.0

[ https://issues.apache.org/jira/browse/SPARK-25731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-25731: Description: [https://github.com/apache/spark/tree/master/external] As far as I can see, This doesn't h

[jira] [Created] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

Marco Gaido created SPARK-25732: --- Summary: Allow specifying a keytab/principal for proxy user for token renewal Key: SPARK-25732 URL: https://issues.apache.org/jira/browse/SPARK-25732 Project: Spark

[jira] [Commented] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

[ https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649917#comment-16649917 ] Marco Gaido commented on SPARK-25732: - cc [~vanzin] [~tgraves] [~jerryshao] [~mridul

[jira] [Updated] (SPARK-25731) Spark Structured Streaming Support for Kafka 2.0

[ https://issues.apache.org/jira/browse/SPARK-25731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-25731: Labels: beginner features (was: ) > Spark Structured Streaming Support for Kafka 2.0 > --

[jira] [Created] (SPARK-25733) The method toLocalIterator() with dataframe doesn't work

Bihui Jin created SPARK-25733: - Summary: The method toLocalIterator() with dataframe doesn't work Key: SPARK-25733 URL: https://issues.apache.org/jira/browse/SPARK-25733 Project: Spark Issue Type

[jira] [Updated] (SPARK-25733) The method toLocalIterator() with dataframe doesn't work

[ https://issues.apache.org/jira/browse/SPARK-25733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bihui Jin updated SPARK-25733: -- Description: {color:#FF}The dataset which I used attached.{color}   First I loaded a dataframe f

[jira] [Updated] (SPARK-25733) The method toLocalIterator() with dataframe doesn't work

[ https://issues.apache.org/jira/browse/SPARK-25733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bihui Jin updated SPARK-25733: -- Attachment: report_dataset.zip.002 > The method toLocalIterator() with dataframe doesn't work > --

[jira] [Updated] (SPARK-25733) The method toLocalIterator() with dataframe doesn't work

[ https://issues.apache.org/jira/browse/SPARK-25733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bihui Jin updated SPARK-25733: -- Attachment: report_dataset.zip.001 > The method toLocalIterator() with dataframe doesn't work > --

[jira] [Commented] (SPARK-24630) SPIP: Support SQLStreaming in Spark

[ https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649972#comment-16649972 ] shijinkui commented on SPARK-24630: --- I prefer without stream keyword. Because in the f

[jira] [Updated] (SPARK-25733) The method toLocalIterator() with dataframe doesn't work

[ https://issues.apache.org/jira/browse/SPARK-25733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bihui Jin updated SPARK-25733: -- Environment: Spark in standalone mode, and 48 cores are available. spark-defaults.conf as blew: spark

[jira] [Commented] (SPARK-25527) Job stuck waiting for last stage to start

[ https://issues.apache.org/jira/browse/SPARK-25527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649990#comment-16649990 ] Ran Haim commented on SPARK-25527: -- Any update? > Job stuck waiting for last stage to

[jira] [Updated] (SPARK-25723) spark sql External DataSource

[ https://issues.apache.org/jira/browse/SPARK-25723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huanghuai updated SPARK-25723: -- Description: (was: {color:#33}*spark.read()*{color} {color:#33}*.format("com.myself.dataso

[jira] [Updated] (SPARK-25723) spark sql External DataSource

[ https://issues.apache.org/jira/browse/SPARK-25723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huanghuai updated SPARK-25723: -- Attachment: QQ图片20181015182502.jpg > spark sql External DataSource > - > >

[jira] [Updated] (SPARK-25723) spark sql External DataSource question

[ https://issues.apache.org/jira/browse/SPARK-25723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huanghuai updated SPARK-25723: -- Description: trait PrunedFilteredScan { def buildScan(requiredColumns: Array[String], filters: Array[

[jira] [Reopened] (SPARK-25723) spark sql External DataSource question

[ https://issues.apache.org/jira/browse/SPARK-25723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huanghuai reopened SPARK-25723: --- reproduce > spark sql External DataSource question > -- > >

[jira] [Created] (SPARK-25734) Literal should have a value corresponding to dataType

Takeshi Yamamuro created SPARK-25734: Summary: Literal should have a value corresponding to dataType Key: SPARK-25734 URL: https://issues.apache.org/jira/browse/SPARK-25734 Project: Spark

[jira] [Assigned] (SPARK-25734) Literal should have a value corresponding to dataType

[ https://issues.apache.org/jira/browse/SPARK-25734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25734: Assignee: (was: Apache Spark) > Literal should have a value corresponding to dataType

[jira] [Commented] (SPARK-25734) Literal should have a value corresponding to dataType

[ https://issues.apache.org/jira/browse/SPARK-25734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650059#comment-16650059 ] Apache Spark commented on SPARK-25734: -- User 'maropu' has created a pull request fo

[jira] [Assigned] (SPARK-25734) Literal should have a value corresponding to dataType

[ https://issues.apache.org/jira/browse/SPARK-25734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25734: Assignee: Apache Spark > Literal should have a value corresponding to dataType >

[jira] [Commented] (SPARK-24610) wholeTextFiles broken for small files

[ https://issues.apache.org/jira/browse/SPARK-24610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650106#comment-16650106 ] Apache Spark commented on SPARK-24610: -- User '10110346' has created a pull request

[jira] [Commented] (SPARK-24610) wholeTextFiles broken for small files

[ https://issues.apache.org/jira/browse/SPARK-24610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650109#comment-16650109 ] Apache Spark commented on SPARK-24610: -- User '10110346' has created a pull request

[jira] [Commented] (SPARK-25727) makeCopy failed in InMemoryRelation

[ https://issues.apache.org/jira/browse/SPARK-25727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650153#comment-16650153 ] Apache Spark commented on SPARK-25727: -- User 'mgaido91' has created a pull request

[jira] [Updated] (SPARK-25723) spark sql External DataSource question

[ https://issues.apache.org/jira/browse/SPARK-25723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huanghuai updated SPARK-25723: -- Priority: Minor (was: Major) > spark sql External DataSource question > -

[jira] [Created] (SPARK-25735) Improve start-thriftserver.sh: print clean usage and exit with code 1

Gengliang Wang created SPARK-25735: -- Summary: Improve start-thriftserver.sh: print clean usage and exit with code 1 Key: SPARK-25735 URL: https://issues.apache.org/jira/browse/SPARK-25735 Project: Sp

[jira] [Assigned] (SPARK-25735) Improve start-thriftserver.sh: print clean usage and exit with code 1

[ https://issues.apache.org/jira/browse/SPARK-25735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25735: Assignee: Apache Spark > Improve start-thriftserver.sh: print clean usage and exit with c

[jira] [Commented] (SPARK-25735) Improve start-thriftserver.sh: print clean usage and exit with code 1

[ https://issues.apache.org/jira/browse/SPARK-25735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650244#comment-16650244 ] Apache Spark commented on SPARK-25735: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-25735) Improve start-thriftserver.sh: print clean usage and exit with code 1

[ https://issues.apache.org/jira/browse/SPARK-25735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25735: Assignee: (was: Apache Spark) > Improve start-thriftserver.sh: print clean usage and

[jira] [Commented] (SPARK-25735) Improve start-thriftserver.sh: print clean usage and exit with code 1

[ https://issues.apache.org/jira/browse/SPARK-25735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650247#comment-16650247 ] Apache Spark commented on SPARK-25735: -- User 'gengliangwang' has created a pull req

[jira] [Commented] (SPARK-13478) Fetching delegation tokens for Hive fails when using proxy users

[ https://issues.apache.org/jira/browse/SPARK-13478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650302#comment-16650302 ] Sunayan Saikia commented on SPARK-13478: [~vanzin] As we know 'spark-submit' com

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650303#comment-16650303 ] Victor Tso commented on SPARK-20144: I looked at the PR and liked what I saw. I woul

[jira] [Commented] (SPARK-25369) Replace Java shim functional interfaces like java.api.Function with Java 8 equivalents

[ https://issues.apache.org/jira/browse/SPARK-25369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650356#comment-16650356 ] Sean Owen commented on SPARK-25369: --- Example: {code:java} @FunctionalInterface public

[jira] [Created] (SPARK-25736) add tests to verify the behavior of multi-column count

Wenchen Fan created SPARK-25736: --- Summary: add tests to verify the behavior of multi-column count Key: SPARK-25736 URL: https://issues.apache.org/jira/browse/SPARK-25736 Project: Spark Issue Ty

[jira] [Assigned] (SPARK-25736) add tests to verify the behavior of multi-column count

[ https://issues.apache.org/jira/browse/SPARK-25736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25736: Assignee: Wenchen Fan (was: Apache Spark) > add tests to verify the behavior of multi-co

[jira] [Commented] (SPARK-25736) add tests to verify the behavior of multi-column count

[ https://issues.apache.org/jira/browse/SPARK-25736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650375#comment-16650375 ] Apache Spark commented on SPARK-25736: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-25736) add tests to verify the behavior of multi-column count

[ https://issues.apache.org/jira/browse/SPARK-25736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25736: Assignee: Apache Spark (was: Wenchen Fan) > add tests to verify the behavior of multi-co

[jira] [Created] (SPARK-25737) Remove JavaSparkContextVarargsWorkaround and standardize union() methods

Sean Owen created SPARK-25737: - Summary: Remove JavaSparkContextVarargsWorkaround and standardize union() methods Key: SPARK-25737 URL: https://issues.apache.org/jira/browse/SPARK-25737 Project: Spark

[jira] [Updated] (SPARK-25737) Remove JavaSparkContextVarargsWorkaround and standardize union() methods

[ https://issues.apache.org/jira/browse/SPARK-25737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25737: -- Target Version/s: 3.0.0 > Remove JavaSparkContextVarargsWorkaround and standardize union() methods > -

[jira] [Updated] (SPARK-25737) Remove JavaSparkContextVarargsWorkaround and standardize union() methods

[ https://issues.apache.org/jira/browse/SPARK-25737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25737: -- Environment: (was: In ancient times in 2013, JavaSparkContext got a superclass JavaSparkContextVar

[jira] [Resolved] (SPARK-24154) AccumulatorV2 loses type information during serialization

[ https://issues.apache.org/jira/browse/SPARK-24154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24154. --- Resolution: Won't Fix > AccumulatorV2 loses type information during serialization >

[jira] [Updated] (SPARK-16775) Remove deprecated accumulator v1 APIs

[ https://issues.apache.org/jira/browse/SPARK-16775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16775: -- Affects Version/s: (was: 2.1.0) 3.0.0 Target Version/s: 3.0.0

[jira] [Commented] (SPARK-25737) Remove JavaSparkContextVarargsWorkaround and standardize union() methods

[ https://issues.apache.org/jira/browse/SPARK-25737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650425#comment-16650425 ] Apache Spark commented on SPARK-25737: -- User 'srowen' has created a pull request fo

[jira] [Assigned] (SPARK-25737) Remove JavaSparkContextVarargsWorkaround and standardize union() methods

[ https://issues.apache.org/jira/browse/SPARK-25737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25737: Assignee: Apache Spark (was: Sean Owen) > Remove JavaSparkContextVarargsWorkaround and s

[jira] [Assigned] (SPARK-25737) Remove JavaSparkContextVarargsWorkaround and standardize union() methods

[ https://issues.apache.org/jira/browse/SPARK-25737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25737: Assignee: Sean Owen (was: Apache Spark) > Remove JavaSparkContextVarargsWorkaround and s

[jira] [Commented] (SPARK-13478) Fetching delegation tokens for Hive fails when using proxy users

[ https://issues.apache.org/jira/browse/SPARK-13478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650451#comment-16650451 ] Marcelo Vanzin commented on SPARK-13478: You don't need a keytab to log in to ke

[jira] [Commented] (SPARK-16775) Remove deprecated accumulator v1 APIs

[ https://issues.apache.org/jira/browse/SPARK-16775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650472#comment-16650472 ] Apache Spark commented on SPARK-16775: -- User 'srowen' has created a pull request fo

[jira] [Assigned] (SPARK-16775) Remove deprecated accumulator v1 APIs

[ https://issues.apache.org/jira/browse/SPARK-16775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16775: Assignee: Apache Spark > Remove deprecated accumulator v1 APIs >

[jira] [Assigned] (SPARK-16775) Remove deprecated accumulator v1 APIs

[ https://issues.apache.org/jira/browse/SPARK-16775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16775: Assignee: (was: Apache Spark) > Remove deprecated accumulator v1 APIs > -

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650492#comment-16650492 ] Daniel Darabos commented on SPARK-20144: Thanks Victor! I've expanded the test w

[jira] [Commented] (SPARK-18278) SPIP: Support native submission of spark jobs to a kubernetes cluster

[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650519#comment-16650519 ] Oleg Frenkel commented on SPARK-18278: -- When is this fork expected to make its way

[jira] [Commented] (SPARK-18278) SPIP: Support native submission of spark jobs to a kubernetes cluster

[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650524#comment-16650524 ] Matt Cheah commented on SPARK-18278: The fork is no longer being maintained, because

[jira] [Commented] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

[ https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650535#comment-16650535 ] Marcelo Vanzin commented on SPARK-25732: I'd have preferred a system where Livy

[jira] [Commented] (SPARK-25674) If the records are incremented by more than 1 at a time,the number of bytes might rarely ever get updated

[ https://issues.apache.org/jira/browse/SPARK-25674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650541#comment-16650541 ] Apache Spark commented on SPARK-25674: -- User 'gatorsmile' has created a pull reques

[jira] [Commented] (SPARK-25674) If the records are incremented by more than 1 at a time,the number of bytes might rarely ever get updated

[ https://issues.apache.org/jira/browse/SPARK-25674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650539#comment-16650539 ] Apache Spark commented on SPARK-25674: -- User 'gatorsmile' has created a pull reques

[jira] [Updated] (SPARK-25547) Pluggable jdbc connection factory

[ https://issues.apache.org/jira/browse/SPARK-25547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25547: Target Version/s: 3.0.0 > Pluggable jdbc connection factory > - > >

[jira] [Commented] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

[ https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650560#comment-16650560 ] Thomas Graves commented on SPARK-25732: --- I would much rather see Spark start to pu

[jira] [Commented] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

[ https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650591#comment-16650591 ] Marcelo Vanzin commented on SPARK-25732: In fact, do you even need proxy user +

[jira] [Commented] (SPARK-25044) Address translation of LMF closure primitive args to Object in Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-25044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650608#comment-16650608 ] Apache Spark commented on SPARK-25044: -- User 'maryannxue' has created a pull reques

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650642#comment-16650642 ] Dongjoon Hyun commented on SPARK-20144: --- For me, I don't think that PR resolve thi

[jira] [Resolved] (SPARK-25424) Window duration and slide duration with negative values should fail fast

[ https://issues.apache.org/jira/browse/SPARK-25424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghav Kumar Gautam resolved SPARK-25424. - Resolution: Not A Bug > Window duration and slide duration with negative values

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650704#comment-16650704 ] Victor Tso commented on SPARK-20144: It should, because by convention the parquet fi

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650722#comment-16650722 ] Daniel Darabos commented on SPARK-20144: Yeah, I'm not too happy about the alpha

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650779#comment-16650779 ] Dongjoon Hyun commented on SPARK-20144: --- [~silvermast] and [~darabos].  1. The pr

[jira] [Created] (SPARK-25738) LOAD DATA INPATH doesn't work if hdfs conf includes port

Imran Rashid created SPARK-25738: Summary: LOAD DATA INPATH doesn't work if hdfs conf includes port Key: SPARK-25738 URL: https://issues.apache.org/jira/browse/SPARK-25738 Project: Spark Issu

[jira] [Updated] (SPARK-25738) LOAD DATA INPATH doesn't work if hdfs conf includes port

[ https://issues.apache.org/jira/browse/SPARK-25738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-25738: - Priority: Blocker (was: Critical) > LOAD DATA INPATH doesn't work if hdfs conf includes port >

[jira] [Commented] (SPARK-25738) LOAD DATA INPATH doesn't work if hdfs conf includes port

[ https://issues.apache.org/jira/browse/SPARK-25738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650796#comment-16650796 ] Shixiong Zhu commented on SPARK-25738: -- Marked as a blocker since this is a regress

[jira] [Commented] (SPARK-25738) LOAD DATA INPATH doesn't work if hdfs conf includes port

[ https://issues.apache.org/jira/browse/SPARK-25738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650799#comment-16650799 ] Imran Rashid commented on SPARK-25738: -- the fix is pretty trivial, I'm posting a pr

[jira] [Commented] (SPARK-25738) LOAD DATA INPATH doesn't work if hdfs conf includes port

[ https://issues.apache.org/jira/browse/SPARK-25738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650807#comment-16650807 ] Apache Spark commented on SPARK-25738: -- User 'squito' has created a pull request fo

[jira] [Assigned] (SPARK-25738) LOAD DATA INPATH doesn't work if hdfs conf includes port

[ https://issues.apache.org/jira/browse/SPARK-25738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25738: Assignee: Apache Spark > LOAD DATA INPATH doesn't work if hdfs conf includes port > -

[jira] [Assigned] (SPARK-25738) LOAD DATA INPATH doesn't work if hdfs conf includes port

[ https://issues.apache.org/jira/browse/SPARK-25738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25738: Assignee: (was: Apache Spark) > LOAD DATA INPATH doesn't work if hdfs conf includes p

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650817#comment-16650817 ] Daniel Darabos commented on SPARK-20144: Thanks, those are good questions. # Th

[jira] [Created] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

Brian Jones created SPARK-25739: --- Summary: Double quote coming in as empty value even when emptyValue set as null Key: SPARK-25739 URL: https://issues.apache.org/jira/browse/SPARK-25739 Project: Spark

[jira] [Updated] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

[ https://issues.apache.org/jira/browse/SPARK-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Jones updated SPARK-25739: Environment:   Databricks - 4.2 (includes Apache Spark 2.3.1, Scala 2.11)  was:  Example code

[jira] [Updated] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

[ https://issues.apache.org/jira/browse/SPARK-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Jones updated SPARK-25739: Description:  Example code -  {code:java} val df = List((1,""),(2,"hello"),(3,"hi"),(4,null)).toDF

[jira] [Updated] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

[ https://issues.apache.org/jira/browse/SPARK-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Jones updated SPARK-25739: Environment:  Databricks - 4.2 (includes Apache Spark 2.3.1, Scala 2.11)  (was:   Databricks - 4

[jira] [Updated] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

[ https://issues.apache.org/jira/browse/SPARK-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Jones updated SPARK-25739: Description:  Example code -  {code:java} val df = List((1,""),(2,"hello"),(3,"hi"),(4,null)).toDF

[jira] [Updated] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

[ https://issues.apache.org/jira/browse/SPARK-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Jones updated SPARK-25739: Affects Version/s: (was: 2.3.2) 2.3.1 > Double quote coming in as empty

[jira] [Updated] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

[ https://issues.apache.org/jira/browse/SPARK-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Jones updated SPARK-25739: Description:  Example code -  {code:scala} val df = List((1,""),(2,"hello"),(3,"hi"),(4,null)).toD

[jira] [Updated] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

[ https://issues.apache.org/jira/browse/SPARK-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Jones updated SPARK-25739: Description:  Example code -  {code} val df = List((1,""),(2,"hello"),(3,"hi"),(4,null)).toDF("key

[jira] [Updated] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

[ https://issues.apache.org/jira/browse/SPARK-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Jones updated SPARK-25739: Description:  Example code -  {code:java} val df = List((1,""),(2,"hello"),(3,"hi"),(4,null)).toDF

[jira] [Commented] (SPARK-25643) Performance issues querying wide rows

[ https://issues.apache.org/jira/browse/SPARK-25643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650866#comment-16650866 ] Bruce Robbins commented on SPARK-25643: --- [~viirya] Yes, in the case where I said "

[jira] [Comment Edited] (SPARK-25643) Performance issues querying wide rows

[ https://issues.apache.org/jira/browse/SPARK-25643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650866#comment-16650866 ] Bruce Robbins edited comment on SPARK-25643 at 10/15/18 10:08 PM:

[jira] [Resolved] (SPARK-25716) Project and Aggregate generate valid constraints with unnecessary operation

[ https://issues.apache.org/jira/browse/SPARK-25716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25716. - Resolution: Fixed Assignee: SongYadong Fix Version/s: 3.0.0 > Project and Aggregate gene

[jira] [Assigned] (SPARK-23257) Implement Kerberos Support in Kubernetes resource manager

[ https://issues.apache.org/jira/browse/SPARK-23257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23257: -- Assignee: Ilan Filonenko > Implement Kerberos Support in Kubernetes resource manager

[jira] [Resolved] (SPARK-23257) Implement Kerberos Support in Kubernetes resource manager

[ https://issues.apache.org/jira/browse/SPARK-23257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23257. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 21669 [https:

  1   2   >