[jira] [Commented] (SPARK-25121) Support multi-part column name for hint resolution

[ https://issues.apache.org/jira/browse/SPARK-25121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580779#comment-16580779 ] Xiao Li commented on SPARK-25121: - cc [~ksunitha] [~dkbiswal] > Support multi-part colu

[jira] [Created] (SPARK-25121) Support multi-part column name for hint resolution

Xiao Li created SPARK-25121: --- Summary: Support multi-part column name for hint resolution Key: SPARK-25121 URL: https://issues.apache.org/jira/browse/SPARK-25121 Project: Spark Issue Type: Improvem

[jira] [Resolved] (SPARK-24505) Convert strings in codegen to blocks: Cast and BoundAttribute

[ https://issues.apache.org/jira/browse/SPARK-24505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24505. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21537 [https://gi

[jira] [Assigned] (SPARK-24505) Convert strings in codegen to blocks: Cast and BoundAttribute

[ https://issues.apache.org/jira/browse/SPARK-24505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24505: Assignee: Liang-Chi Hsieh > Convert strings in codegen to blocks: Cast and BoundAttribute

[jira] [Commented] (SPARK-25109) spark python should retry reading another datanode if the first one fails to connect

[ https://issues.apache.org/jira/browse/SPARK-25109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580692#comment-16580692 ] Hyukjin Kwon commented on SPARK-25109: -- It should be helpful if we can narrow down

[jira] [Commented] (SPARK-25109) spark python should retry reading another datanode if the first one fails to connect

[ https://issues.apache.org/jira/browse/SPARK-25109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580689#comment-16580689 ] Yuanbo Liu commented on SPARK-25109: [~hyukjin.kwon] Thanks for your comments. Not s

[jira] [Commented] (SPARK-25120) EventLogListener may miss driver SparkListenerBlockManagerAdded event

[ https://issues.apache.org/jira/browse/SPARK-25120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580670#comment-16580670 ] deshanxiao commented on SPARK-25120: Sure, I find the tab "Executors" in HistorySeve

[jira] [Commented] (SPARK-25120) EventLogListener may miss driver SparkListenerBlockManagerAdded event

[ https://issues.apache.org/jira/browse/SPARK-25120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580658#comment-16580658 ] Apache Spark commented on SPARK-25120: -- User 'deshanxiao' has created a pull reques

[jira] [Assigned] (SPARK-25120) EventLogListener may miss driver SparkListenerBlockManagerAdded event

[ https://issues.apache.org/jira/browse/SPARK-25120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25120: Assignee: (was: Apache Spark) > EventLogListener may miss driver SparkListenerBlockMa

[jira] [Assigned] (SPARK-25120) EventLogListener may miss driver SparkListenerBlockManagerAdded event

[ https://issues.apache.org/jira/browse/SPARK-25120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25120: Assignee: Apache Spark > EventLogListener may miss driver SparkListenerBlockManagerAdded

[jira] [Commented] (SPARK-25120) EventLogListener may miss driver SparkListenerBlockManagerAdded event

[ https://issues.apache.org/jira/browse/SPARK-25120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580648#comment-16580648 ] Hyukjin Kwon commented on SPARK-25120: -- Can you describe a bit more details? When d

[jira] [Commented] (SPARK-25109) spark python should retry reading another datanode if the first one fails to connect

[ https://issues.apache.org/jira/browse/SPARK-25109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580647#comment-16580647 ] Hyukjin Kwon commented on SPARK-25109: -- Does the same thing happen in Scala API too

[jira] [Commented] (SPARK-24771) Upgrade AVRO version from 1.7.7 to 1.8

[ https://issues.apache.org/jira/browse/SPARK-24771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580644#comment-16580644 ] Wenchen Fan commented on SPARK-24771: - I've sent the email, let's wait for the feedb

[jira] [Created] (SPARK-25120) EventLogListener may miss driver SparkListenerBlockManagerAdded event

deshanxiao created SPARK-25120: -- Summary: EventLogListener may miss driver SparkListenerBlockManagerAdded event Key: SPARK-25120 URL: https://issues.apache.org/jira/browse/SPARK-25120 Project: Spark

[jira] [Updated] (SPARK-25083) remove the type erasure hack in data source scan

[ https://issues.apache.org/jira/browse/SPARK-25083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25083: Target Version/s: 3.0.0 > remove the type erasure hack in data source scan > -

[jira] [Commented] (SPARK-25083) remove the type erasure hack in data source scan

[ https://issues.apache.org/jira/browse/SPARK-25083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580638#comment-16580638 ] Wenchen Fan commented on SPARK-25083: - I've set the target version as 3.0. It's a co

[jira] [Resolved] (SPARK-23874) Upgrade apache/arrow to 0.10.0

[ https://issues.apache.org/jira/browse/SPARK-23874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-23874. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21939 [https://gi

[jira] [Resolved] (SPARK-25115) Eliminate extra memory copy done when a ByteBuf is used that is backed by > 1 ByteBuffer.

[ https://issues.apache.org/jira/browse/SPARK-25115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-25115. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22105 [https://github.com/a

[jira] [Assigned] (SPARK-25115) Eliminate extra memory copy done when a ByteBuf is used that is backed by > 1 ByteBuffer.

[ https://issues.apache.org/jira/browse/SPARK-25115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-25115: --- Assignee: Norman Maurer > Eliminate extra memory copy done when a ByteBuf is used that is backe

[jira] [Resolved] (SPARK-25113) Add logging to CodeGenerator when any generated method's bytecode size goes above HugeMethodLimit

[ https://issues.apache.org/jira/browse/SPARK-25113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25113. - Resolution: Fixed Assignee: Kris Mok Fix Version/s: 2.4.0 > Add logging to CodeGenerator

[jira] [Updated] (SPARK-25119) stages in wrong order within job page DAG chart

[ https://issues.apache.org/jira/browse/SPARK-25119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yunjian Zhang updated SPARK-25119: -- Attachment: Screen Shot 2018-08-14 at 3.35.34 PM.png > stages in wrong order within job page D

[jira] [Updated] (SPARK-25119) stages in wrong order within job page DAG chart

[ https://issues.apache.org/jira/browse/SPARK-25119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yunjian Zhang updated SPARK-25119: -- Description: {color:#33}multiple stages for same job are shown with wrong order in DAG Vi

[jira] [Created] (SPARK-25119) stages in wrong order within job page DAG chart

Yunjian Zhang created SPARK-25119: - Summary: stages in wrong order within job page DAG chart Key: SPARK-25119 URL: https://issues.apache.org/jira/browse/SPARK-25119 Project: Spark Issue Type:

[jira] [Commented] (SPARK-20384) supporting value classes over primitives in DataSets

[ https://issues.apache.org/jira/browse/SPARK-20384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580498#comment-16580498 ] Minh Thai commented on SPARK-20384: --- _(from my comment in SPARK-17368)_ I think the m

[jira] [Commented] (SPARK-25092) Add RewriteExceptAll and RewriteIntersectAll in the list of nonExcludableRules

[ https://issues.apache.org/jira/browse/SPARK-25092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580464#comment-16580464 ] Apache Spark commented on SPARK-25092: -- User 'dilipbiswal' has created a pull reque

[jira] [Updated] (SPARK-25118) Need a solution to persist Spark application console outputs when running in shell/yarn client mode

[ https://issues.apache.org/jira/browse/SPARK-25118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Gupta updated SPARK-25118: Description: We execute Spark applications in YARN Client mode a lot of time. When we do so the S

[jira] [Created] (SPARK-25118) Need a solution to persist Spark application console outputs when running in shell/yarn client mode

Ankur Gupta created SPARK-25118: --- Summary: Need a solution to persist Spark application console outputs when running in shell/yarn client mode Key: SPARK-25118 URL: https://issues.apache.org/jira/browse/SPARK-25118

[jira] [Commented] (SPARK-16406) Reference resolution for large number of columns should be faster

[ https://issues.apache.org/jira/browse/SPARK-16406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580444#comment-16580444 ] antonkulaga commented on SPARK-16406: - Are you going to backport it to 2.3.2 as well

[jira] [Commented] (SPARK-23337) withWatermark raises an exception on struct objects

[ https://issues.apache.org/jira/browse/SPARK-23337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580440#comment-16580440 ] Aydin Kocas commented on SPARK-23337: - [~marmbrus] Can you give a hint how to do it

[jira] [Commented] (SPARK-24938) Understand usage of netty's onheap memory use, even with offheap pools

[ https://issues.apache.org/jira/browse/SPARK-24938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580299#comment-16580299 ] Imran Rashid commented on SPARK-24938: -- {quote} So perhaps the fix here is not to u

[jira] [Updated] (SPARK-24838) Support uncorrelated IN/EXISTS subqueries for more operators

[ https://issues.apache.org/jira/browse/SPARK-24838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24838: Target Version/s: 2.4.0 > Support uncorrelated IN/EXISTS subqueries for more operators >

[jira] [Commented] (SPARK-24838) Support uncorrelated IN/EXISTS subqueries for more operators

[ https://issues.apache.org/jira/browse/SPARK-24838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580298#comment-16580298 ] Xiao Li commented on SPARK-24838: - [~maurits] Any update? Also cc [~liwen] [~hvanhovel

[jira] [Updated] (SPARK-24838) Support uncorrelated IN/EXISTS subqueries for more operators

[ https://issues.apache.org/jira/browse/SPARK-24838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24838: Labels: (was: spree) > Support uncorrelated IN/EXISTS subqueries for more operators > -

[jira] [Commented] (SPARK-25105) Importing all of pyspark.sql.functions should bring PandasUDFType in as well

[ https://issues.apache.org/jira/browse/SPARK-25105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580285#comment-16580285 ] Apache Spark commented on SPARK-25105: -- User 'kevinyu98' has created a pull request

[jira] [Assigned] (SPARK-25105) Importing all of pyspark.sql.functions should bring PandasUDFType in as well

[ https://issues.apache.org/jira/browse/SPARK-25105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25105: Assignee: (was: Apache Spark) > Importing all of pyspark.sql.functions should bring P

[jira] [Assigned] (SPARK-25105) Importing all of pyspark.sql.functions should bring PandasUDFType in as well

[ https://issues.apache.org/jira/browse/SPARK-25105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25105: Assignee: Apache Spark > Importing all of pyspark.sql.functions should bring PandasUDFTyp

[jira] [Commented] (SPARK-21375) Add date and timestamp support to ArrowConverters for toPandas() collection

[ https://issues.apache.org/jira/browse/SPARK-21375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580275#comment-16580275 ] Eric Wohlstadter commented on SPARK-21375: -- [~bryanc] Hi Brian,  I'm using th

[jira] [Commented] (SPARK-24938) Understand usage of netty's onheap memory use, even with offheap pools

[ https://issues.apache.org/jira/browse/SPARK-24938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580260#comment-16580260 ] Marcelo Vanzin commented on SPARK-24938: I think that unless there's a measurabl

[jira] [Commented] (SPARK-22236) CSV I/O: does not respect RFC 4180

[ https://issues.apache.org/jira/browse/SPARK-22236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580229#comment-16580229 ] Joe Pallas commented on SPARK-22236: {quote}people with preexisting datasets exporte

[jira] [Commented] (SPARK-24938) Understand usage of netty's onheap memory use, even with offheap pools

[ https://issues.apache.org/jira/browse/SPARK-24938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580225#comment-16580225 ] Nihar Sheth commented on SPARK-24938: - His comment for that change is "The header is

[jira] [Assigned] (SPARK-25043) spark-sql should print the appId and master on startup

[ https://issues.apache.org/jira/browse/SPARK-25043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-25043: - Assignee: Alessandro Bellina > spark-sql should print the appId and master on startup >

[jira] [Commented] (SPARK-24938) Understand usage of netty's onheap memory use, even with offheap pools

[ https://issues.apache.org/jira/browse/SPARK-24938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580213#comment-16580213 ] Marcelo Vanzin commented on SPARK-24938: BTW the change from buffer() to heapBuf

[jira] [Resolved] (SPARK-25043) spark-sql should print the appId and master on startup

[ https://issues.apache.org/jira/browse/SPARK-25043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-25043. --- Resolution: Fixed Fix Version/s: 2.4.0 > spark-sql should print the appId and master

[jira] [Commented] (SPARK-25117) Add EXEPT ALL and INTERSECT ALL support in R.

[ https://issues.apache.org/jira/browse/SPARK-25117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580208#comment-16580208 ] Apache Spark commented on SPARK-25117: -- User 'dilipbiswal' has created a pull reque

[jira] [Assigned] (SPARK-25117) Add EXEPT ALL and INTERSECT ALL support in R.

[ https://issues.apache.org/jira/browse/SPARK-25117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25117: Assignee: (was: Apache Spark) > Add EXEPT ALL and INTERSECT ALL support in R. > -

[jira] [Assigned] (SPARK-25117) Add EXEPT ALL and INTERSECT ALL support in R.

[ https://issues.apache.org/jira/browse/SPARK-25117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25117: Assignee: Apache Spark > Add EXEPT ALL and INTERSECT ALL support in R. >

[jira] [Assigned] (SPARK-25116) Fix the "exit code 1" error when terminating Kafka tests

[ https://issues.apache.org/jira/browse/SPARK-25116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25116: Assignee: Shixiong Zhu (was: Apache Spark) > Fix the "exit code 1" error when terminatin

[jira] [Commented] (SPARK-25116) Fix the "exit code 1" error when terminating Kafka tests

[ https://issues.apache.org/jira/browse/SPARK-25116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580202#comment-16580202 ] Apache Spark commented on SPARK-25116: -- User 'zsxwing' has created a pull request f

[jira] [Assigned] (SPARK-25116) Fix the "exit code 1" error when terminating Kafka tests

[ https://issues.apache.org/jira/browse/SPARK-25116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25116: Assignee: Apache Spark (was: Shixiong Zhu) > Fix the "exit code 1" error when terminatin

[jira] [Commented] (SPARK-24938) Understand usage of netty's onheap memory use, even with offheap pools

[ https://issues.apache.org/jira/browse/SPARK-24938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580195#comment-16580195 ] Marcelo Vanzin commented on SPARK-24938: The line you mention is this, right? {

[jira] [Resolved] (SPARK-25088) Rest Server default & doc updates

[ https://issues.apache.org/jira/browse/SPARK-25088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25088. --- Resolution: Fixed Fix Version/s: 2.4.0 Resolved by https://github.com/apache/spark/pull/22071

[jira] [Created] (SPARK-25117) Add EXEPT ALL and INTERSECT ALL support in R.

Dilip Biswal created SPARK-25117: Summary: Add EXEPT ALL and INTERSECT ALL support in R. Key: SPARK-25117 URL: https://issues.apache.org/jira/browse/SPARK-25117 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-25116) Fix the "exit code 1" error when terminating Kafka tests

Shixiong Zhu created SPARK-25116: Summary: Fix the "exit code 1" error when terminating Kafka tests Key: SPARK-25116 URL: https://issues.apache.org/jira/browse/SPARK-25116 Project: Spark Issu

[jira] [Updated] (SPARK-25114) RecordBinaryComparator may return wrong result when subtraction between two words is divisible by Integer.MAX_VALUE

[ https://issues.apache.org/jira/browse/SPARK-25114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25114: Target Version/s: 2.3.2, 2.4.0 (was: 2.4.0) > RecordBinaryComparator may return wrong result when subtrac

[jira] [Commented] (SPARK-25114) RecordBinaryComparator may return wrong result when subtraction between two words is divisible by Integer.MAX_VALUE

[ https://issues.apache.org/jira/browse/SPARK-25114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580153#comment-16580153 ] Xiao Li commented on SPARK-25114: - [~jerryshao] Another blocker for 2.3 > RecordBinaryC

[jira] [Commented] (SPARK-24938) Understand usage of netty's onheap memory use, even with offheap pools

[ https://issues.apache.org/jira/browse/SPARK-24938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580140#comment-16580140 ] Imran Rashid commented on SPARK-24938: -- Cool, sounds like the info we need for maki

[jira] [Resolved] (SPARK-25051) where clause on dataset gives AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-25051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25051. - Resolution: Fixed Assignee: Marco Gaido Fix Version/s: 2.3.3 > where clause on dataset g

[jira] [Updated] (SPARK-25051) where clause on dataset gives AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-25051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25051: Fix Version/s: (was: 2.3.3) 2.3.2 > where clause on dataset gives AnalysisException

[jira] [Commented] (SPARK-24938) Understand usage of netty's onheap memory use, even with offheap pools

[ https://issues.apache.org/jira/browse/SPARK-24938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580113#comment-16580113 ] Nihar Sheth commented on SPARK-24938: - Your expectation is correct, the offheap pool

[jira] [Assigned] (SPARK-25115) Eliminate extra memory copy done when a ByteBuf is used that is backed by > 1 ByteBuffer.

[ https://issues.apache.org/jira/browse/SPARK-25115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25115: Assignee: Apache Spark > Eliminate extra memory copy done when a ByteBuf is used that

[jira] [Commented] (SPARK-25115) Eliminate extra memory copy done when a ByteBuf is used that is backed by > 1 ByteBuffer.

[ https://issues.apache.org/jira/browse/SPARK-25115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580106#comment-16580106 ] Apache Spark commented on SPARK-25115: -- User 'normanmaurer' has created a pull requ

[jira] [Assigned] (SPARK-25115) Eliminate extra memory copy done when a ByteBuf is used that is backed by > 1 ByteBuffer.

[ https://issues.apache.org/jira/browse/SPARK-25115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25115: Assignee: (was: Apache Spark) > Eliminate extra memory copy done when a ByteBuf i

[jira] [Commented] (SPARK-25115) Eliminate extra memory copy done when a ByteBuf is used that is backed by > 1 ByteBuffer.

[ https://issues.apache.org/jira/browse/SPARK-25115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580087#comment-16580087 ] Norman Maurer commented on SPARK-25115: --- I opened the following PR to fix it:  

[jira] [Created] (SPARK-25115) Eliminate extra memory copy done when a ByteBuf is used that is backed by > 1 ByteBuffer.

Norman Maurer created SPARK-25115: - Summary: Eliminate extra memory copy done when a ByteBuf is used that is backed by > 1 ByteBuffer. Key: SPARK-25115 URL: https://issues.apache.org/jira/browse/SPARK-25115

[jira] [Commented] (SPARK-24787) Events being dropped at an alarming rate due to hsync being slow for eventLogging

[ https://issues.apache.org/jira/browse/SPARK-24787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580064#comment-16580064 ] Marcelo Vanzin commented on SPARK-24787: In that case it might be good to only u

[jira] [Commented] (SPARK-24771) Upgrade AVRO version from 1.7.7 to 1.8

[ https://issues.apache.org/jira/browse/SPARK-24771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580062#comment-16580062 ] Marcelo Vanzin commented on SPARK-24771: Asking is a good start. But I have anec

[jira] [Commented] (SPARK-22236) CSV I/O: does not respect RFC 4180

[ https://issues.apache.org/jira/browse/SPARK-22236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579974#comment-16579974 ] Ondrej Kokes commented on SPARK-22236: -- Multiline=true by default would cause some

[jira] [Updated] (SPARK-25051) where clause on dataset gives AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-25051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25051: Component/s: (was: Spark Core) > where clause on dataset gives AnalysisException > ---

[jira] [Commented] (SPARK-25051) where clause on dataset gives AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-25051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579967#comment-16579967 ] Xiao Li commented on SPARK-25051: - [~mgaido] This breaks the backport rule. We are unabl

[jira] [Commented] (SPARK-24561) User-defined window functions with pandas udf (bounded window)

[ https://issues.apache.org/jira/browse/SPARK-24561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579965#comment-16579965 ] Li Jin commented on SPARK-24561: I am looking into this. Early investigation:  https://d

[jira] [Commented] (SPARK-25083) remove the type erasure hack in data source scan

[ https://issues.apache.org/jira/browse/SPARK-25083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579961#comment-16579961 ] Ryan Blue commented on SPARK-25083: --- [~cloud_fan], what release is this targeting? >

[jira] [Updated] (SPARK-24721) Failed to use PythonUDF with literal inputs in filter with data sources

[ https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jin updated SPARK-24721: --- Component/s: SQL > Failed to use PythonUDF with literal inputs in filter with data sources > ---

[jira] [Updated] (SPARK-24721) Failed to use PythonUDF with literal inputs in filter with data sources

[ https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jin updated SPARK-24721: --- Issue Type: Bug (was: Sub-task) Parent: (was: SPARK-22216) > Failed to use PythonUDF with liter

[jira] [Comment Edited] (SPARK-24721) Failed to use PythonUDF with literal inputs in filter with data sources

[ https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579956#comment-16579956 ] Li Jin edited comment on SPARK-24721 at 8/14/18 3:26 PM: - Update

[jira] [Commented] (SPARK-24721) Failed to use PythonUDF with literal inputs in filter with data sources

[ https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579956#comment-16579956 ] Li Jin commented on SPARK-24721: Updates Jira title to reflect the actual issue > Faile

[jira] [Updated] (SPARK-24721) Failed to use PythonUDF with literal inputs in filter with data sources

[ https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jin updated SPARK-24721: --- Summary: Failed to use PythonUDF with literal inputs in filter with data sources (was: Failed to call Pytho

[jira] [Commented] (SPARK-24941) Add RDDBarrier.coalesce() function

[ https://issues.apache.org/jira/browse/SPARK-24941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579930#comment-16579930 ] Jiang Xingbo commented on SPARK-24941: -- Shall we add something like `spark.default.

[jira] [Commented] (SPARK-24721) Failed to call PythonUDF whose input is the output of another PythonUDF

[ https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579883#comment-16579883 ] Apache Spark commented on SPARK-24721: -- User 'icexelloss' has created a pull reques

[jira] [Assigned] (SPARK-24721) Failed to call PythonUDF whose input is the output of another PythonUDF

[ https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24721: Assignee: (was: Apache Spark) > Failed to call PythonUDF whose input is the output of

[jira] [Assigned] (SPARK-24721) Failed to call PythonUDF whose input is the output of another PythonUDF

[ https://issues.apache.org/jira/browse/SPARK-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24721: Assignee: Apache Spark > Failed to call PythonUDF whose input is the output of another Py

[jira] [Assigned] (SPARK-25114) RecordBinaryComparator may return wrong result when subtraction between two words is divisible by Integer.MAX_VALUE

[ https://issues.apache.org/jira/browse/SPARK-25114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25114: Assignee: (was: Apache Spark) > RecordBinaryComparator may return wrong result when s

[jira] [Commented] (SPARK-25114) RecordBinaryComparator may return wrong result when subtraction between two words is divisible by Integer.MAX_VALUE

[ https://issues.apache.org/jira/browse/SPARK-25114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579880#comment-16579880 ] Apache Spark commented on SPARK-25114: -- User 'jiangxb1987' has created a pull reque

[jira] [Assigned] (SPARK-25114) RecordBinaryComparator may return wrong result when subtraction between two words is divisible by Integer.MAX_VALUE

[ https://issues.apache.org/jira/browse/SPARK-25114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25114: Assignee: Apache Spark > RecordBinaryComparator may return wrong result when subtraction

[jira] [Commented] (SPARK-25114) RecordBinaryComparator may return wrong result when subtraction between two words is divisible by Integer.MAX_VALUE

[ https://issues.apache.org/jira/browse/SPARK-25114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579879#comment-16579879 ] Jiang Xingbo commented on SPARK-25114: -- I created https://github.com/apache/spark/p

[jira] [Updated] (SPARK-25114) RecordBinaryComparator may return wrong result when subtraction between two words is divisible by Integer.MAX_VALUE

[ https://issues.apache.org/jira/browse/SPARK-25114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiang Xingbo updated SPARK-25114: - Labels: correctness (was: ) > RecordBinaryComparator may return wrong result when subtraction b

[jira] [Updated] (SPARK-25114) RecordBinaryComparator may return wrong result when subtraction between two words is divisible by Integer.MAX_VALUE

[ https://issues.apache.org/jira/browse/SPARK-25114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiang Xingbo updated SPARK-25114: - Priority: Blocker (was: Major) > RecordBinaryComparator may return wrong result when subtractio

[jira] [Created] (SPARK-25114) RecordBinaryComparator may return wrong result when subtraction between two words is divisible by Integer.MAX_VALUE

Jiang Xingbo created SPARK-25114: Summary: RecordBinaryComparator may return wrong result when subtraction between two words is divisible by Integer.MAX_VALUE Key: SPARK-25114 URL: https://issues.apache.org/jira/b

[jira] [Commented] (SPARK-24787) Events being dropped at an alarming rate due to hsync being slow for eventLogging

[ https://issues.apache.org/jira/browse/SPARK-24787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579854#comment-16579854 ] Thomas Graves commented on SPARK-24787: --- Yes it was caused by hsync, hsync has to

[jira] [Commented] (SPARK-24918) Executor Plugin API

[ https://issues.apache.org/jira/browse/SPARK-24918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579851#comment-16579851 ] Thomas Graves commented on SPARK-24918: --- Personally I like the explicit config on

[jira] [Updated] (SPARK-25051) where clause on dataset gives AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-25051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-25051: -- Priority: Blocker (was: Major) > where clause on dataset gives AnalysisException > --

[jira] [Resolved] (SPARK-23938) High-order function: map_zip_with(map, map, function) → map

[ https://issues.apache.org/jira/browse/SPARK-23938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-23938. --- Resolution: Fixed Assignee: Marek Novotny Fix Version/s: 2.4.0 Issue resolve

[jira] [Commented] (SPARK-24411) Adding native Java tests for `isInCollection`

[ https://issues.apache.org/jira/browse/SPARK-24411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579650#comment-16579650 ] Aleksei Izmalkin commented on SPARK-24411: -- I will work on this issue. > Addin

[jira] [Commented] (SPARK-25113) Add logging to CodeGenerator when any generated method's bytecode size goes above HugeMethodLimit

[ https://issues.apache.org/jira/browse/SPARK-25113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579645#comment-16579645 ] Apache Spark commented on SPARK-25113: -- User 'rednaxelafx' has created a pull reque

[jira] [Assigned] (SPARK-25113) Add logging to CodeGenerator when any generated method's bytecode size goes above HugeMethodLimit

[ https://issues.apache.org/jira/browse/SPARK-25113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25113: Assignee: (was: Apache Spark) > Add logging to CodeGenerator when any generated metho

[jira] [Assigned] (SPARK-25113) Add logging to CodeGenerator when any generated method's bytecode size goes above HugeMethodLimit

[ https://issues.apache.org/jira/browse/SPARK-25113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25113: Assignee: Apache Spark > Add logging to CodeGenerator when any generated method's bytecod

[jira] [Created] (SPARK-25113) Add logging to CodeGenerator when any generated method's bytecode size goes above HugeMethodLimit

Kris Mok created SPARK-25113: Summary: Add logging to CodeGenerator when any generated method's bytecode size goes above HugeMethodLimit Key: SPARK-25113 URL: https://issues.apache.org/jira/browse/SPARK-25113

[jira] [Commented] (SPARK-25102) Write Spark version information to Parquet file footers

[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579622#comment-16579622 ] Nikita Poberezkin commented on SPARK-25102: --- I will work on this issue > Writ

[jira] [Assigned] (SPARK-25051) where clause on dataset gives AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-25051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25051: Assignee: Apache Spark > where clause on dataset gives AnalysisException > --

[jira] [Commented] (SPARK-25051) where clause on dataset gives AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-25051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579526#comment-16579526 ] Apache Spark commented on SPARK-25051: -- User 'mgaido91' has created a pull request

[jira] [Assigned] (SPARK-25051) where clause on dataset gives AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-25051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25051: Assignee: (was: Apache Spark) > where clause on dataset gives AnalysisException > ---

  1   2   >