[jira] [Commented] (SPARK-24980) add support for pandas/arrow etc for python2.7 and pypy builds

[ https://issues.apache.org/jira/browse/SPARK-24980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564813#comment-16564813 ] Hyukjin Kwon commented on SPARK-24980: -- Oh BTW, [~bryanc], do you remember if PyArr

[jira] [Comment Edited] (SPARK-24980) add support for pandas/arrow etc for python2.7 and pypy builds

[ https://issues.apache.org/jira/browse/SPARK-24980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564813#comment-16564813 ] Hyukjin Kwon edited comment on SPARK-24980 at 8/1/18 6:37 AM:

[jira] [Commented] (SPARK-24980) add support for pandas/arrow etc for python2.7 and pypy builds

[ https://issues.apache.org/jira/browse/SPARK-24980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564808#comment-16564808 ] Hyukjin Kwon commented on SPARK-24980: -- Thank you for cc'ing me Shane! > add suppo

[jira] [Updated] (SPARK-24984) Spark Streaming with xml data

[ https://issues.apache.org/jira/browse/SPARK-24984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24984: - Priority: Major (was: Critical) > Spark Streaming with xml data >

[jira] [Resolved] (SPARK-24984) Spark Streaming with xml data

[ https://issues.apache.org/jira/browse/SPARK-24984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24984. -- Resolution: Not A Problem It's something the package should implement and support . Please fil

[jira] [Updated] (SPARK-24984) Spark Streaming with xml data

[ https://issues.apache.org/jira/browse/SPARK-24984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24984: - Fix Version/s: (was: 0.8.2) > Spark Streaming with xml data > -

[jira] [Assigned] (SPARK-24966) Fix the precedence rule for set operations.

[ https://issues.apache.org/jira/browse/SPARK-24966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24966: Assignee: Apache Spark > Fix the precedence rule for set operations. > --

[jira] [Assigned] (SPARK-24966) Fix the precedence rule for set operations.

[ https://issues.apache.org/jira/browse/SPARK-24966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24966: Assignee: (was: Apache Spark) > Fix the precedence rule for set operations. > ---

[jira] [Commented] (SPARK-24966) Fix the precedence rule for set operations.

[ https://issues.apache.org/jira/browse/SPARK-24966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564772#comment-16564772 ] Apache Spark commented on SPARK-24966: -- User 'dilipbiswal' has created a pull reque

[jira] [Resolved] (SPARK-24951) Table valued functions should throw AnalysisException instead of IllegalArgumentException

[ https://issues.apache.org/jira/browse/SPARK-24951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24951. - Resolution: Fixed Fix Version/s: 2.4.0 > Table valued functions should throw AnalysisException in

[jira] [Resolved] (SPARK-24311) Refactor HDFSBackedStateStoreProvider to remove duplicated logic between operations on delta file and snapshot file

[ https://issues.apache.org/jira/browse/SPARK-24311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-24311. -- Resolution: Won't Fix As I got feedback on [https://github.com/apache/spark/pull/21357#issuec

[jira] [Resolved] (SPARK-24893) Remove the entire CaseWhen if all the outputs are semantic equivalence

[ https://issues.apache.org/jira/browse/SPARK-24893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24893. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21852 [https://gith

[jira] [Created] (SPARK-24986) OOM in BufferHolder during writes to a stream

Sanket Reddy created SPARK-24986: Summary: OOM in BufferHolder during writes to a stream Key: SPARK-24986 URL: https://issues.apache.org/jira/browse/SPARK-24986 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-24985) Executing SQL with "Full Outer Join" on top of large tables when there is data skew met OOM

sheperd huang created SPARK-24985: - Summary: Executing SQL with "Full Outer Join" on top of large tables when there is data skew met OOM Key: SPARK-24985 URL: https://issues.apache.org/jira/browse/SPARK-24985

[jira] [Commented] (SPARK-23874) Upgrade apache/arrow to 0.10.0

[ https://issues.apache.org/jira/browse/SPARK-23874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564570#comment-16564570 ] Apache Spark commented on SPARK-23874: -- User 'BryanCutler' has created a pull reque

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564562#comment-16564562 ] Saisai Shao commented on SPARK-24615: - Leveraging dynamic allocation to tear down an

[jira] [Comment Edited] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564562#comment-16564562 ] Saisai Shao edited comment on SPARK-24615 at 8/1/18 12:35 AM:

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564558#comment-16564558 ] Erik Erlandson commented on SPARK-24615: Am I understanding correctly that this

[jira] [Resolved] (SPARK-24976) Allow None for Decimal type conversion (specific to PyArrow 0.9.0)

[ https://issues.apache.org/jira/browse/SPARK-24976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-24976. -- Resolution: Fixed Fix Version/s: 2.3.2 2.4.0 Issue resolved by pull

[jira] [Assigned] (SPARK-24976) Allow None for Decimal type conversion (specific to PyArrow 0.9.0)

[ https://issues.apache.org/jira/browse/SPARK-24976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-24976: Assignee: Hyukjin Kwon > Allow None for Decimal type conversion (specific to PyArrow 0.9.

[jira] [Commented] (SPARK-19394) "assertion failed: Expected hostname" on macOS when self-assigned IP contains a percent sign

[ https://issues.apache.org/jira/browse/SPARK-19394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564532#comment-16564532 ] Yuming Wang commented on SPARK-19394: - Try to add {{::1             localhost}} to /

[jira] [Commented] (SPARK-24977) input_file_name() result can't save and use for partitionBy()

[ https://issues.apache.org/jira/browse/SPARK-24977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564490#comment-16564490 ] kevin yu commented on SPARK-24977: -- Hello Srinivasarao: Can you show the steps you enco

[jira] [Created] (SPARK-24984) Spark Streaming with xml data

Kavya created SPARK-24984: - Summary: Spark Streaming with xml data Key: SPARK-24984 URL: https://issues.apache.org/jira/browse/SPARK-24984 Project: Spark Issue Type: New Feature Components

[jira] [Updated] (SPARK-24983) Collapsing multiple project statements with dependent When-Otherwise statements on the same column can OOM the driver

[ https://issues.apache.org/jira/browse/SPARK-24983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Vogelbacher updated SPARK-24983: -- Description: I noticed that writing a spark job that includes many sequential {{when-

[jira] [Created] (SPARK-24983) Collapsing multiple project statements with dependent When-Otherwise statements on the same column can OOM the driver

David Vogelbacher created SPARK-24983: - Summary: Collapsing multiple project statements with dependent When-Otherwise statements on the same column can OOM the driver Key: SPARK-24983 URL: https://issues.apach

[jira] [Assigned] (SPARK-24982) UDAF resolution should not throw java.lang.AssertionError

[ https://issues.apache.org/jira/browse/SPARK-24982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24982: Assignee: Reynold Xin (was: Apache Spark) > UDAF resolution should not throw java.lang.A

[jira] [Commented] (SPARK-24982) UDAF resolution should not throw java.lang.AssertionError

[ https://issues.apache.org/jira/browse/SPARK-24982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564364#comment-16564364 ] Apache Spark commented on SPARK-24982: -- User 'rxin' has created a pull request for

[jira] [Assigned] (SPARK-24982) UDAF resolution should not throw java.lang.AssertionError

[ https://issues.apache.org/jira/browse/SPARK-24982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24982: Assignee: Apache Spark (was: Reynold Xin) > UDAF resolution should not throw java.lang.A

[jira] [Assigned] (SPARK-24982) UDAF resolution should not throw java.lang.AssertionError

[ https://issues.apache.org/jira/browse/SPARK-24982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-24982: --- Assignee: Reynold Xin > UDAF resolution should not throw java.lang.AssertionError > ---

[jira] [Comment Edited] (SPARK-24882) data source v2 API improvement

[ https://issues.apache.org/jira/browse/SPARK-24882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564362#comment-16564362 ] Ryan Blue edited comment on SPARK-24882 at 7/31/18 9:02 PM:

[jira] [Commented] (SPARK-24882) data source v2 API improvement

[ https://issues.apache.org/jira/browse/SPARK-24882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564362#comment-16564362 ] Ryan Blue commented on SPARK-24882: --- {quote}the problem is then we need to make `Catal

[jira] [Assigned] (SPARK-24973) Add numIter to Python ClusteringSummary

[ https://issues.apache.org/jira/browse/SPARK-24973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-24973: - Assignee: Huaxin Gao > Add numIter to Python ClusteringSummary > -

[jira] [Updated] (SPARK-18057) Update structured streaming kafka from 0.10.0.1 to 2.0.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18057: - Summary: Update structured streaming kafka from 0.10.0.1 to 2.0.0 (was: Update structured strea

[jira] [Resolved] (SPARK-18057) Update structured streaming kafka from 0.10.0.1 to 2.0.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-18057. -- Resolution: Fixed Assignee: Ted Yu Fix Version/s: 2.4.0 > Update structured st

[jira] [Commented] (SPARK-23914) High-order function: array_union(x, y) → array

[ https://issues.apache.org/jira/browse/SPARK-23914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564227#comment-16564227 ] Apache Spark commented on SPARK-23914: -- User 'kiszk' has created a pull request for

[jira] [Updated] (SPARK-24982) UDAF resolution should not throw java.lang.AssertionError

[ https://issues.apache.org/jira/browse/SPARK-24982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-24982: Description: See udaf.sql.out:   {code:java} – !query 3 SELECT default.myDoubleAvg(int_col1, 3)

[jira] [Created] (SPARK-24982) UDAF resolution should not throw java.lang.AssertionError

Reynold Xin created SPARK-24982: --- Summary: UDAF resolution should not throw java.lang.AssertionError Key: SPARK-24982 URL: https://issues.apache.org/jira/browse/SPARK-24982 Project: Spark Issue

[jira] [Commented] (SPARK-24981) ShutdownHook timeout causes job to fail when succeeded when SparkContext stop() not called by user program

[ https://issues.apache.org/jira/browse/SPARK-24981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564175#comment-16564175 ] Apache Spark commented on SPARK-24981: -- User 'hthuynh2' has created a pull request

[jira] [Assigned] (SPARK-24981) ShutdownHook timeout causes job to fail when succeeded when SparkContext stop() not called by user program

[ https://issues.apache.org/jira/browse/SPARK-24981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24981: Assignee: Apache Spark > ShutdownHook timeout causes job to fail when succeeded when Spar

[jira] [Assigned] (SPARK-24981) ShutdownHook timeout causes job to fail when succeeded when SparkContext stop() not called by user program

[ https://issues.apache.org/jira/browse/SPARK-24981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24981: Assignee: (was: Apache Spark) > ShutdownHook timeout causes job to fail when succeede

[jira] [Commented] (SPARK-14643) Remove overloaded methods which become ambiguous in Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564172#comment-16564172 ] Stavros Kontopoulos commented on SPARK-14643: - Cool thanks! > Remove overlo

[jira] [Comment Edited] (SPARK-14643) Remove overloaded methods which become ambiguous in Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564148#comment-16564148 ] Stavros Kontopoulos edited comment on SPARK-14643 at 7/31/18 6:40 PM:

[jira] [Updated] (SPARK-14643) Remove overloaded methods which become ambiguous in Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14643: -- Affects Version/s: 2.4.0 Issue Type: Task (was: Sub-task) Parent: (was:

[jira] [Updated] (SPARK-14643) Remove overloaded methods which become ambiguous in Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14643: -- Target Version/s: 3.0.0 Ah right, I had forgotten, and I see that thread again now. I'm going to mark

[jira] [Comment Edited] (SPARK-14643) Remove overloaded methods which become ambiguous in Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564148#comment-16564148 ] Stavros Kontopoulos edited comment on SPARK-14643 at 7/31/18 6:38 PM:

[jira] [Comment Edited] (SPARK-14643) Remove overloaded methods which become ambiguous in Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564148#comment-16564148 ] Stavros Kontopoulos edited comment on SPARK-14643 at 7/31/18 6:38 PM:

[jira] [Updated] (SPARK-24609) PySpark/SparkR doc doesn't explain RandomForestClassifier.featureSubsetStrategy well

[ https://issues.apache.org/jira/browse/SPARK-24609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-24609: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) > PySpark/SparkR doc doesn't e

[jira] [Assigned] (SPARK-24609) PySpark/SparkR doc doesn't explain RandomForestClassifier.featureSubsetStrategy well

[ https://issues.apache.org/jira/browse/SPARK-24609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-24609: - Assignee: zhengruifeng > PySpark/SparkR doc doesn't explain > RandomForestClassifier.featureSu

[jira] [Resolved] (SPARK-24609) PySpark/SparkR doc doesn't explain RandomForestClassifier.featureSubsetStrategy well

[ https://issues.apache.org/jira/browse/SPARK-24609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24609. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21788 [https://github.c

[jira] [Commented] (SPARK-24980) add support for pandas/arrow etc for python2.7 and pypy builds

[ https://issues.apache.org/jira/browse/SPARK-24980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564154#comment-16564154 ] shane knapp commented on SPARK-24980: - [~hyukjin.kwon] thought you might be interest

[jira] [Comment Edited] (SPARK-14643) Remove overloaded methods which become ambiguous in Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564148#comment-16564148 ] Stavros Kontopoulos edited comment on SPARK-14643 at 7/31/18 6:34 PM:

[jira] [Comment Edited] (SPARK-14643) Remove overloaded methods which become ambiguous in Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564148#comment-16564148 ] Stavros Kontopoulos edited comment on SPARK-14643 at 7/31/18 6:34 PM:

[jira] [Commented] (SPARK-14643) Remove overloaded methods which become ambiguous in Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564148#comment-16564148 ] Stavros Kontopoulos commented on SPARK-14643: - [~srowen] Right now there is

[jira] [Assigned] (SPARK-24773) support reading AVRO logical types - Timestamp with different precisions

[ https://issues.apache.org/jira/browse/SPARK-24773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24773: Assignee: (was: Apache Spark) > support reading AVRO logical types - Timestamp with d

[jira] [Commented] (SPARK-24773) support reading AVRO logical types - Timestamp with different precisions

[ https://issues.apache.org/jira/browse/SPARK-24773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564145#comment-16564145 ] Apache Spark commented on SPARK-24773: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-24773) support reading AVRO logical types - Timestamp with different precisions

[ https://issues.apache.org/jira/browse/SPARK-24773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24773: Assignee: Apache Spark > support reading AVRO logical types - Timestamp with different pr

[jira] [Resolved] (SPARK-24973) Add numIter to Python ClusteringSummary

[ https://issues.apache.org/jira/browse/SPARK-24973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24973. --- Resolution: Duplicate I agree, though I think this should be thought of as one big issue. > Add num

[jira] [Updated] (SPARK-24981) ShutdownHook timeout causes job to fail when succeeded when SparkContext stop() not called by user program

[ https://issues.apache.org/jira/browse/SPARK-24981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hieu Tri Huynh updated SPARK-24981: --- Priority: Minor (was: Major) > ShutdownHook timeout causes job to fail when succeeded when

[jira] [Updated] (SPARK-24981) ShutdownHook timeout causes job to fail when succeeded when SparkContext stop() not called by user program

[ https://issues.apache.org/jira/browse/SPARK-24981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hieu Tri Huynh updated SPARK-24981: --- Summary: ShutdownHook timeout causes job to fail when succeeded when SparkContext stop() not

[jira] [Created] (SPARK-24981) ShutdownHook timeout causes job to fail when succeeded when SparkContext stop() not called

Hieu Tri Huynh created SPARK-24981: -- Summary: ShutdownHook timeout causes job to fail when succeeded when SparkContext stop() not called Key: SPARK-24981 URL: https://issues.apache.org/jira/browse/SPARK-24981

[jira] [Updated] (SPARK-24287) Spark -packages option should support classifier, no-transitive, and custom conf

[ https://issues.apache.org/jira/browse/SPARK-24287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-24287: -- Issue Type: Improvement (was: Bug) This covers a lot of the same ground as  https://issues.apache.org

[jira] [Commented] (SPARK-24951) Table valued functions should throw AnalysisException instead of IllegalArgumentException

[ https://issues.apache.org/jira/browse/SPARK-24951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564075#comment-16564075 ] Apache Spark commented on SPARK-24951: -- User 'rxin' has created a pull request for

[jira] [Assigned] (SPARK-24951) Table valued functions should throw AnalysisException instead of IllegalArgumentException

[ https://issues.apache.org/jira/browse/SPARK-24951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24951: Assignee: Reynold Xin (was: Apache Spark) > Table valued functions should throw Analysis

[jira] [Assigned] (SPARK-24951) Table valued functions should throw AnalysisException instead of IllegalArgumentException

[ https://issues.apache.org/jira/browse/SPARK-24951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24951: Assignee: Apache Spark (was: Reynold Xin) > Table valued functions should throw Analysis

[jira] [Updated] (SPARK-24971) remove SupportsDeprecatedScanRow

[ https://issues.apache.org/jira/browse/SPARK-24971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-24971: Issue Type: Sub-task (was: Improvement) Parent: SPARK-22386 > remove SupportsDeprecatedSc

[jira] [Updated] (SPARK-24974) Spark put all file's paths into SharedInMemoryCache even for unused partitions.

[ https://issues.apache.org/jira/browse/SPARK-24974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] andrzej.stankev...@gmail.com updated SPARK-24974: - Description: SharedInMemoryCache has all  filestatus no matter w

[jira] [Created] (SPARK-24980) add support for pandas/arrow etc for python2.7 and pypy builds

shane knapp created SPARK-24980: --- Summary: add support for pandas/arrow etc for python2.7 and pypy builds Key: SPARK-24980 URL: https://issues.apache.org/jira/browse/SPARK-24980 Project: Spark

[jira] [Commented] (SPARK-14643) Remove overloaded methods which become ambiguous in Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563998#comment-16563998 ] Sean Owen commented on SPARK-14643: --- [~skonto] is the conclusion that there's a change

[jira] [Updated] (SPARK-14540) Support Scala 2.12 closures and Java 8 lambdas in ClosureCleaner

[ https://issues.apache.org/jira/browse/SPARK-14540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14540: -- Labels: release-notes (was: ) > Support Scala 2.12 closures and Java 8 lambdas in ClosureCleaner > --

[jira] [Assigned] (SPARK-24917) Sending a partition over netty results in 2x memory usage

[ https://issues.apache.org/jira/browse/SPARK-24917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24917: Assignee: (was: Apache Spark) > Sending a partition over netty results in 2x memory u

[jira] [Assigned] (SPARK-24917) Sending a partition over netty results in 2x memory usage

[ https://issues.apache.org/jira/browse/SPARK-24917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24917: Assignee: Apache Spark > Sending a partition over netty results in 2x memory usage >

[jira] [Commented] (SPARK-24917) Sending a partition over netty results in 2x memory usage

[ https://issues.apache.org/jira/browse/SPARK-24917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563918#comment-16563918 ] Apache Spark commented on SPARK-24917: -- User 'vincent-grosbois' has created a pull

[jira] [Assigned] (SPARK-24979) add AnalysisHelper#resolveOperatorsUp

[ https://issues.apache.org/jira/browse/SPARK-24979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24979: Assignee: Apache Spark (was: Wenchen Fan) > add AnalysisHelper#resolveOperatorsUp >

[jira] [Assigned] (SPARK-24979) add AnalysisHelper#resolveOperatorsUp

[ https://issues.apache.org/jira/browse/SPARK-24979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24979: Assignee: Wenchen Fan (was: Apache Spark) > add AnalysisHelper#resolveOperatorsUp >

[jira] [Commented] (SPARK-24979) add AnalysisHelper#resolveOperatorsUp

[ https://issues.apache.org/jira/browse/SPARK-24979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563896#comment-16563896 ] Apache Spark commented on SPARK-24979: -- User 'cloud-fan' has created a pull request

[jira] [Updated] (SPARK-24979) add AnalysisHelper#resolveOperatorsUp

[ https://issues.apache.org/jira/browse/SPARK-24979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-24979: Summary: add AnalysisHelper#resolveOperatorsUp (was: add resolveOperatorsUp) > add AnalysisHelpe

[jira] [Created] (SPARK-24979) add resolveOperatorsUp

Wenchen Fan created SPARK-24979: --- Summary: add resolveOperatorsUp Key: SPARK-24979 URL: https://issues.apache.org/jira/browse/SPARK-24979 Project: Spark Issue Type: Improvement Compon

[jira] [Resolved] (SPARK-24536) Query with nonsensical LIMIT hits AssertionError

[ https://issues.apache.org/jira/browse/SPARK-24536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24536. - Resolution: Fixed Fix Version/s: 2.4.0 2.3.2 > Query with nonsensical LIMIT hi

[jira] [Comment Edited] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563730#comment-16563730 ] Saisai Shao edited comment on SPARK-24615 at 7/31/18 2:16 PM:

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563730#comment-16563730 ] Saisai Shao commented on SPARK-24615: - Hi [~tgraves], I think eval() might unnecessa

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563711#comment-16563711 ] Thomas Graves commented on SPARK-24615: --- so I guess my question is this the right

[jira] [Commented] (SPARK-24579) SPIP: Standardize Optimized Data Exchange between Spark and DL/AI frameworks

[ https://issues.apache.org/jira/browse/SPARK-24579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563689#comment-16563689 ] Thomas Graves commented on SPARK-24579: --- going from Spark feeds data into DL/AI fr

[jira] [Resolved] (SPARK-24816) SQL interface support repartitionByRange

[ https://issues.apache.org/jira/browse/SPARK-24816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-24816. - Resolution: Won't Fix {{Order by}} is implement by {{rangepartitioning}}. > SQL interface suppo

[jira] [Assigned] (SPARK-24978) Add spark.sql.fast.hash.aggregate.row.max.capacity to configure the capacity of fast aggregation.

[ https://issues.apache.org/jira/browse/SPARK-24978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24978: Assignee: (was: Apache Spark) > Add spark.sql.fast.hash.aggregate.row.max.capacity to

[jira] [Commented] (SPARK-24978) Add spark.sql.fast.hash.aggregate.row.max.capacity to configure the capacity of fast aggregation.

[ https://issues.apache.org/jira/browse/SPARK-24978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563487#comment-16563487 ] Apache Spark commented on SPARK-24978: -- User 'heary-cao' has created a pull request

[jira] [Assigned] (SPARK-24978) Add spark.sql.fast.hash.aggregate.row.max.capacity to configure the capacity of fast aggregation.

[ https://issues.apache.org/jira/browse/SPARK-24978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24978: Assignee: Apache Spark > Add spark.sql.fast.hash.aggregate.row.max.capacity to configure

[jira] [Created] (SPARK-24978) Add spark.sql.fast.hash.aggregate.row.max.capacity to configure the capacity of fast aggregation.

caoxuewen created SPARK-24978: - Summary: Add spark.sql.fast.hash.aggregate.row.max.capacity to configure the capacity of fast aggregation. Key: SPARK-24978 URL: https://issues.apache.org/jira/browse/SPARK-24978

[jira] [Resolved] (SPARK-24968) Configurable Chunksize in ChunkedByteBufferOutputStream

[ https://issues.apache.org/jira/browse/SPARK-24968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincent resolved SPARK-24968. - Resolution: Fixed > Configurable Chunksize in ChunkedByteBufferOutputStream > --

[jira] [Commented] (SPARK-24968) Configurable Chunksize in ChunkedByteBufferOutputStream

[ https://issues.apache.org/jira/browse/SPARK-24968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563467#comment-16563467 ] Vincent commented on SPARK-24968: - indeed they are closely related I'll close this tick

[jira] [Commented] (SPARK-24946) PySpark - Allow np.Arrays and pd.Series in df.approxQuantile

[ https://issues.apache.org/jira/browse/SPARK-24946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563455#comment-16563455 ] Paul Westenthanner commented on SPARK-24946: Sure, go ahead :)  > PySpark -

[jira] [Commented] (SPARK-14540) Support Scala 2.12 closures and Java 8 lambdas in ClosureCleaner

[ https://issues.apache.org/jira/browse/SPARK-14540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563434#comment-16563434 ] Apache Spark commented on SPARK-14540: -- User 'skonto' has created a pull request fo

[jira] [Commented] (SPARK-24977) input_file_name() result can't save and use for partitionBy()

[ https://issues.apache.org/jira/browse/SPARK-24977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563424#comment-16563424 ] Srinivasarao Padala commented on SPARK-24977: - i could able to see the the f

[jira] [Created] (SPARK-24977) input_file_name() result can't save and use for paritionBy()

Srinivasarao Padala created SPARK-24977: --- Summary: input_file_name() result can't save and use for paritionBy() Key: SPARK-24977 URL: https://issues.apache.org/jira/browse/SPARK-24977 Project: S

[jira] [Updated] (SPARK-24977) input_file_name() result can't save and use for partitionBy()

[ https://issues.apache.org/jira/browse/SPARK-24977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srinivasarao Padala updated SPARK-24977: Summary: input_file_name() result can't save and use for partitionBy() (was: inpu

[jira] [Commented] (SPARK-24968) Configurable Chunksize in ChunkedByteBufferOutputStream

[ https://issues.apache.org/jira/browse/SPARK-24968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563274#comment-16563274 ] Hyukjin Kwon commented on SPARK-24968: -- So is it a duplicate of SPARK-24917 roughly

[jira] [Resolved] (SPARK-24949) pyspark.sql.Column breaks the iterable contract

[ https://issues.apache.org/jira/browse/SPARK-24949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24949. -- Resolution: Won't Fix Let me leave this resolved. Seems difficult to fix but I wonder if it's

[jira] [Commented] (SPARK-24970) Spark Kinesis streaming application fails to recover from streaming checkpoint due to ProvisionedThroughputExceededException

[ https://issues.apache.org/jira/browse/SPARK-24970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563267#comment-16563267 ] Apache Spark commented on SPARK-24970: -- User 'brucezhao11' has created a pull reque

[jira] [Assigned] (SPARK-24970) Spark Kinesis streaming application fails to recover from streaming checkpoint due to ProvisionedThroughputExceededException

[ https://issues.apache.org/jira/browse/SPARK-24970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24970: Assignee: (was: Apache Spark) > Spark Kinesis streaming application fails to recover

[jira] [Assigned] (SPARK-24970) Spark Kinesis streaming application fails to recover from streaming checkpoint due to ProvisionedThroughputExceededException

[ https://issues.apache.org/jira/browse/SPARK-24970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24970: Assignee: Apache Spark > Spark Kinesis streaming application fails to recover from stream

[jira] [Resolved] (SPARK-24975) Spark history server REST API /api/v1/version returns error 404

[ https://issues.apache.org/jira/browse/SPARK-24975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24975. - Resolution: Duplicate > Spark history server REST API /api/v1/version returns error 404 > --

  1   2   >