[jira] [Commented] (SPARK-26206) Spark structured streaming with kafka integration fails in update mode

[ https://issues.apache.org/jira/browse/SPARK-26206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16706103#comment-16706103 ] Jungtaek Lim commented on SPARK-26206: -- [~indraneelrr] Could you also provide some

[jira] [Updated] (SPARK-26244) Do not use case class as public API

[ https://issues.apache.org/jira/browse/SPARK-26244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-26244: Description: It's a bad idea to use case class as public API, as it has a very wide surface. For

[jira] [Updated] (SPARK-26244) Do not use case class as public API

[ https://issues.apache.org/jira/browse/SPARK-26244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-26244: Labels: release-notes (was: ) > Do not use case class as public API > ---

[jira] [Updated] (SPARK-26244) Do not use case class as public API

[ https://issues.apache.org/jira/browse/SPARK-26244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-26244: Description: It's a bad idea to use case class as public API, as it has a very wide surface. For

[jira] [Created] (SPARK-26244) Do not use case class as public API

Wenchen Fan created SPARK-26244: --- Summary: Do not use case class as public API Key: SPARK-26244 URL: https://issues.apache.org/jira/browse/SPARK-26244 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-26216) Do not use case class as public API (ScalaUDF)

[ https://issues.apache.org/jira/browse/SPARK-26216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-26216: Issue Type: Sub-task (was: Improvement) Parent: SPARK-26244 > Do not use case class as pu

[jira] [Assigned] (SPARK-26195) Correct exception messages in some classes

[ https://issues.apache.org/jira/browse/SPARK-26195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26195: --- Assignee: lichaoqun > Correct exception messages in some classes >

[jira] [Resolved] (SPARK-26195) Correct exception messages in some classes

[ https://issues.apache.org/jira/browse/SPARK-26195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26195. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23154 [https://gith

[jira] [Resolved] (SPARK-26216) Do not use case class as public API (ScalaUDF)

[ https://issues.apache.org/jira/browse/SPARK-26216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26216. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23178 [https://gith

[jira] [Resolved] (SPARK-26161) Ignore empty files in load

[ https://issues.apache.org/jira/browse/SPARK-26161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26161. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23130 [https://gith

[jira] [Assigned] (SPARK-26161) Ignore empty files in load

[ https://issues.apache.org/jira/browse/SPARK-26161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26161: --- Assignee: Maxim Gekk > Ignore empty files in load > -- > >

[jira] [Commented] (SPARK-26242) Leading slash breaks proxying

[ https://issues.apache.org/jira/browse/SPARK-26242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16706057#comment-16706057 ] Ryan Lovett commented on SPARK-26242: - Both of these work for all paths like addres

[jira] [Resolved] (SPARK-26241) Add queryId to IncrementalExecution

[ https://issues.apache.org/jira/browse/SPARK-26241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26241. - Resolution: Fixed Fix Version/s: 3.0.0 > Add queryId to IncrementalExecution > --

[jira] [Commented] (SPARK-26242) Leading slash breaks proxying

[ https://issues.apache.org/jira/browse/SPARK-26242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16706029#comment-16706029 ] Ryan Lovett commented on SPARK-26242: - Thanks! I'll try this and report back. > Lea

[jira] [Assigned] (SPARK-26165) Date and Timestamp column expression is getting converted to string in less than/greater than filter query even though valid date/timestamp string literal is used in th

[ https://issues.apache.org/jira/browse/SPARK-26165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26165: Assignee: Apache Spark > Date and Timestamp column expression is getting converted to str

[jira] [Commented] (SPARK-26165) Date and Timestamp column expression is getting converted to string in less than/greater than filter query even though valid date/timestamp string literal is used in t

[ https://issues.apache.org/jira/browse/SPARK-26165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705958#comment-16705958 ] Apache Spark commented on SPARK-26165: -- User 'sujith71955' has created a pull reque

[jira] [Assigned] (SPARK-26165) Date and Timestamp column expression is getting converted to string in less than/greater than filter query even though valid date/timestamp string literal is used in th

[ https://issues.apache.org/jira/browse/SPARK-26165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26165: Assignee: (was: Apache Spark) > Date and Timestamp column expression is getting conve

[jira] [Commented] (SPARK-26165) Date and Timestamp column expression is getting converted to string in less than/greater than filter query even though valid date/timestamp string literal is used in t

[ https://issues.apache.org/jira/browse/SPARK-26165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705957#comment-16705957 ] Apache Spark commented on SPARK-26165: -- User 'sujith71955' has created a pull reque

[jira] [Commented] (SPARK-26243) Use java.time API for parsing timestamps and dates from JSON

[ https://issues.apache.org/jira/browse/SPARK-26243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705953#comment-16705953 ] Apache Spark commented on SPARK-26243: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-26243) Use java.time API for parsing timestamps and dates from JSON

[ https://issues.apache.org/jira/browse/SPARK-26243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26243: Assignee: Apache Spark > Use java.time API for parsing timestamps and dates from JSON > -

[jira] [Commented] (SPARK-26243) Use java.time API for parsing timestamps and dates from JSON

[ https://issues.apache.org/jira/browse/SPARK-26243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705952#comment-16705952 ] Apache Spark commented on SPARK-26243: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-26243) Use java.time API for parsing timestamps and dates from JSON

[ https://issues.apache.org/jira/browse/SPARK-26243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26243: Assignee: (was: Apache Spark) > Use java.time API for parsing timestamps and dates fr

[jira] [Created] (SPARK-26243) Use java.time API for parsing timestamps and dates from JSON

Maxim Gekk created SPARK-26243: -- Summary: Use java.time API for parsing timestamps and dates from JSON Key: SPARK-26243 URL: https://issues.apache.org/jira/browse/SPARK-26243 Project: Spark Iss

[jira] [Assigned] (SPARK-26241) Add queryId to IncrementalExecution

[ https://issues.apache.org/jira/browse/SPARK-26241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26241: Assignee: Reynold Xin (was: Apache Spark) > Add queryId to IncrementalExecution > --

[jira] [Commented] (SPARK-26241) Add queryId to IncrementalExecution

[ https://issues.apache.org/jira/browse/SPARK-26241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705944#comment-16705944 ] Apache Spark commented on SPARK-26241: -- User 'rxin' has created a pull request for

[jira] [Commented] (SPARK-26241) Add queryId to IncrementalExecution

[ https://issues.apache.org/jira/browse/SPARK-26241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705943#comment-16705943 ] Apache Spark commented on SPARK-26241: -- User 'rxin' has created a pull request for

[jira] [Assigned] (SPARK-26241) Add queryId to IncrementalExecution

[ https://issues.apache.org/jira/browse/SPARK-26241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26241: Assignee: Apache Spark (was: Reynold Xin) > Add queryId to IncrementalExecution > --

[jira] [Commented] (SPARK-26165) Date and Timestamp column expression is getting converted to string in less than/greater than filter query even though valid date/timestamp string literal is used in t

[ https://issues.apache.org/jira/browse/SPARK-26165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705935#comment-16705935 ] Sujith commented on SPARK-26165: CC [~marmbrus]  [~yhuai] - PLEASE LET ME KNOW YOUR SUGG

[jira] [Commented] (SPARK-26242) Leading slash breaks proxying

[ https://issues.apache.org/jira/browse/SPARK-26242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705916#comment-16705916 ] Marco Gaido commented on SPARK-26242: - You can set {{spark.ui.proxyBase}} or the pro

[jira] [Commented] (SPARK-26213) Custom Receiver for Structured streaming

[ https://issues.apache.org/jira/browse/SPARK-26213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705886#comment-16705886 ] Gabor Somogyi commented on SPARK-26213: --- On the other hand looks like this is not

[jira] [Commented] (SPARK-26213) Custom Receiver for Structured streaming

[ https://issues.apache.org/jira/browse/SPARK-26213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705884#comment-16705884 ] Gabor Somogyi commented on SPARK-26213: --- Structured streaming doesn't have receive

[jira] [Updated] (SPARK-26206) Spark structured streaming with kafka integration fails in update mode

[ https://issues.apache.org/jira/browse/SPARK-26206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26206: - Priority: Major (was: Blocker) > Spark structured streaming with kafka integration fails in upd

[jira] [Commented] (SPARK-26199) Long expressions cause mutate to fail

[ https://issues.apache.org/jira/browse/SPARK-26199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705879#comment-16705879 ] Hyukjin Kwon commented on SPARK-26199: -- I haven't taken a close look but sounds rig

[jira] [Updated] (SPARK-26236) Kafka delegation token support documentation

[ https://issues.apache.org/jira/browse/SPARK-26236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-26236: -- Component/s: Documentation > Kafka delegation token support documentation > --

[jira] [Commented] (SPARK-26206) Spark structured streaming with kafka integration fails in update mode

[ https://issues.apache.org/jira/browse/SPARK-26206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705860#comment-16705860 ] Hyukjin Kwon commented on SPARK-26206: -- Please avoid to set Critical+ which is usua

[jira] [Commented] (SPARK-26225) Scan: track decoding time for row-based data sources

[ https://issues.apache.org/jira/browse/SPARK-26225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705846#comment-16705846 ] Sean Owen commented on SPARK-26225: --- [~Thincrs] let's not automatically add comments f

[jira] [Resolved] (SPARK-26210) Streaming of syslog

[ https://issues.apache.org/jira/browse/SPARK-26210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26210. -- Resolution: Invalid Hi [~manojrai]. If that's a question, the best place should be Spark maili

[jira] [Commented] (SPARK-26214) Add "broadcast" method to DataFrame

[ https://issues.apache.org/jira/browse/SPARK-26214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705848#comment-16705848 ] Hyukjin Kwon commented on SPARK-26214: -- I don't think we should add a new API when

[jira] [Commented] (SPARK-26175) PySpark cannot terminate worker process if user program reads from stdin

[ https://issues.apache.org/jira/browse/SPARK-26175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705842#comment-16705842 ] Hyukjin Kwon commented on SPARK-26175: -- Thanks for pinging me - just checked. Let m

[jira] [Updated] (SPARK-26229) Expose SizeEstimator as a developer API in pyspark

[ https://issues.apache.org/jira/browse/SPARK-26229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26229: - Description: SizeEstimator is not available in pyspark. This api will be helpful for understandi

[jira] [Commented] (SPARK-26233) Incorrect decimal value with java beans and first/last/max... functions

[ https://issues.apache.org/jira/browse/SPARK-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705835#comment-16705835 ] Hyukjin Kwon commented on SPARK-26233: -- Hi [~mcanes], looks it's not specific to Ja

[jira] [Commented] (SPARK-26240) [pyspark] Updating illegal column names with withColumnRenamed does not change schema changes, causing pyspark.sql.utils.AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-26240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705834#comment-16705834 ] Hyukjin Kwon commented on SPARK-26240: -- I think it was a mistake that Spark allowed

[jira] [Resolved] (SPARK-26212) Upgrade maven from 3.5.4 to 3.6.0

[ https://issues.apache.org/jira/browse/SPARK-26212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26212. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23177 [https://github.c

[jira] [Assigned] (SPARK-26212) Upgrade maven from 3.5.4 to 3.6.0

[ https://issues.apache.org/jira/browse/SPARK-26212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26212: - Assignee: Kazuaki Ishizaki > Upgrade maven from 3.5.4 to 3.6.0 > --

[jira] [Updated] (SPARK-26216) Do not use case class as public API (ScalaUDF)

[ https://issues.apache.org/jira/browse/SPARK-26216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-26216: -- Docs Text: The implementation of UserDefinedFunction changed to a trait in Spark 3.0.0. It should be s

[jira] [Commented] (SPARK-26236) Kafka delegation token support documentation

[ https://issues.apache.org/jira/browse/SPARK-26236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705811#comment-16705811 ] Apache Spark commented on SPARK-26236: -- User 'gaborgsomogyi' has created a pull req

[jira] [Assigned] (SPARK-26236) Kafka delegation token support documentation

[ https://issues.apache.org/jira/browse/SPARK-26236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26236: Assignee: (was: Apache Spark) > Kafka delegation token support documentation > --

[jira] [Assigned] (SPARK-26236) Kafka delegation token support documentation

[ https://issues.apache.org/jira/browse/SPARK-26236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26236: Assignee: Apache Spark > Kafka delegation token support documentation > -

[jira] [Commented] (SPARK-26236) Kafka delegation token support documentation

[ https://issues.apache.org/jira/browse/SPARK-26236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705810#comment-16705810 ] Apache Spark commented on SPARK-26236: -- User 'gaborgsomogyi' has created a pull req

[jira] [Resolved] (SPARK-26180) Add a withCreateTempDir function to the SparkCore test case

[ https://issues.apache.org/jira/browse/SPARK-26180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26180. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23151 [https://gi

[jira] [Assigned] (SPARK-26180) Add a withCreateTempDir function to the SparkCore test case

[ https://issues.apache.org/jira/browse/SPARK-26180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26180: Assignee: caoxuewen > Add a withCreateTempDir function to the SparkCore test case > -

[jira] [Commented] (SPARK-25401) Reorder the required ordering to match the table's output ordering for bucket join

[ https://issues.apache.org/jira/browse/SPARK-25401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705708#comment-16705708 ] David Vrba commented on SPARK-25401: If no one is working on it, i would like to wor