[jira] [Commented] (SPARK-32779) Spark/Hive3 interaction potentially causes deadlock

[ https://issues.apache.org/jira/browse/SPARK-32779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190014#comment-17190014 ] JinxinTang commented on SPARK-32779: Hi [~bersprockets], Seems spark cannot work with hive-3.1.x

[jira] [Commented] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176966#comment-17176966 ] JinxinTang commented on SPARK-32500: Kindly ping [~hyukjin.kwon] It's my pleasure :  ) > Query and

[jira] [Commented] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173633#comment-17173633 ] JinxinTang commented on SPARK-32500: I have found the root cause is

[jira] [Commented] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170095#comment-17170095 ] JinxinTang commented on SPARK-32500: Thank you for your issue, I have understand and will check this

[jira] [Comment Edited] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169653#comment-17169653 ] JinxinTang edited comment on SPARK-32515 at 8/2/20, 11:09 PM: -- Hi

[jira] [Comment Edited] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169653#comment-17169653 ] JinxinTang edited comment on SPARK-32515 at 8/2/20, 11:08 PM: -- Hi

[jira] [Comment Edited] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169653#comment-17169653 ] JinxinTang edited comment on SPARK-32515 at 8/2/20, 11:08 PM: -- Hi

[jira] [Commented] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169653#comment-17169653 ] JinxinTang commented on SPARK-32515: Hi [~tigaiii123],  Could you please try the data or could you

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-32515: --- Attachment: image-2020-08-03-07-03-55-716.png > Distinct Function Weird Bug >

[jira] [Commented] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169444#comment-17169444 ] JinxinTang commented on SPARK-32515: Hi [~tigaiii123] , Could you please provide reproduce code and

[jira] [Comment Edited] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169219#comment-17169219 ] JinxinTang edited comment on SPARK-32500 at 8/1/20, 2:24 AM: - Hi,

[jira] [Commented] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169219#comment-17169219 ] JinxinTang commented on SPARK-32500: Hi, [~abhishekd0907], I have test the code in master,

[jira] [Updated] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-32500: --- Attachment: image-2020-08-01-10-21-51-246.png > Query and Batch Id not set for Structured Streaming

[jira] [Commented] (SPARK-32483) spark-shell: error: value topByKey is not a member of org.apache.spark.rdd.RDD[(String, (String, Double))]

[ https://issues.apache.org/jira/browse/SPARK-32483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167158#comment-17167158 ] JinxinTang commented on SPARK-32483: [~ukiml] Please `import

[jira] [Commented] (SPARK-32425) Spark sequence() fails if start and end of range are identical timestamps

[ https://issues.apache.org/jira/browse/SPARK-32425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17165634#comment-17165634 ] JinxinTang commented on SPARK-32425: cc [~viirya]  Because I have fixed it by 

[jira] [Commented] (SPARK-32347) BROADCAST hint makes a weird message that "column can't be resolved" (it was OK in Spark 2.4)

[ https://issues.apache.org/jira/browse/SPARK-32347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160627#comment-17160627 ] JinxinTang commented on SPARK-32347: cc [~ibobak] Thanks for your issue, I have just raised a PR for

[jira] [Comment Edited] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160370#comment-17160370 ] JinxinTang edited comment on SPARK-32317 at 7/18/20, 8:55 AM: -- Please try

[jira] [Comment Edited] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160370#comment-17160370 ] JinxinTang edited comment on SPARK-32317 at 7/18/20, 8:54 AM: -- Please try

[jira] [Comment Edited] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160370#comment-17160370 ] JinxinTang edited comment on SPARK-32317 at 7/18/20, 8:53 AM: -- Please try

[jira] [Comment Edited] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160370#comment-17160370 ] JinxinTang edited comment on SPARK-32317 at 7/18/20, 8:51 AM: -- Please try

[jira] [Commented] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160370#comment-17160370 ] JinxinTang commented on SPARK-32317: Please try use `spark.read.*option("mergeSchema",

[jira] [Comment Edited] (SPARK-32347) BROADCAST hint makes a weird message that "column can't be resolved" (it was OK in Spark 2.4)

[ https://issues.apache.org/jira/browse/SPARK-32347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160284#comment-17160284 ] JinxinTang edited comment on SPARK-32347 at 7/18/20, 12:22 AM: --- The CTE

[jira] [Commented] (SPARK-32347) BROADCAST hint makes a weird message that "column can't be resolved" (it was OK in Spark 2.4)

[ https://issues.apache.org/jira/browse/SPARK-32347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160284#comment-17160284 ] JinxinTang commented on SPARK-32347: Same the CTE issue as 

[jira] [Comment Edited] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159864#comment-17159864 ] JinxinTang edited comment on SPARK-32317 at 7/17/20, 11:12 AM: --- Thanks for

[jira] [Comment Edited] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159864#comment-17159864 ] JinxinTang edited comment on SPARK-32317 at 7/17/20, 11:11 AM: --- Thanks for

[jira] [Comment Edited] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159864#comment-17159864 ] JinxinTang edited comment on SPARK-32317 at 7/17/20, 11:07 AM: --- Yes, maybe

[jira] [Commented] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159864#comment-17159864 ] JinxinTang commented on SPARK-32317: Yes, maybe we should check the decimal schema and throw related

[jira] [Issue Comment Deleted] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-32317: --- Comment: was deleted (was: Seems a little hard to solve, because current spark use

[jira] [Commented] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159796#comment-17159796 ] JinxinTang commented on SPARK-32317: Seems a little hard to solve, because current spark use

[jira] [Comment Edited] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153131#comment-17153131 ] JinxinTang edited comment on SPARK-32205 at 7/8/20, 5:10 AM: -

[jira] [Comment Edited] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153131#comment-17153131 ] JinxinTang edited comment on SPARK-32205 at 7/8/20, 12:41 AM: --

[jira] [Comment Edited] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153131#comment-17153131 ] JinxinTang edited comment on SPARK-32205 at 7/7/20, 11:31 PM: --

[jira] [Commented] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153131#comment-17153131 ] JinxinTang commented on SPARK-32205: [~nileshr.patil] Seems a issue, we can insert timestamp to

[jira] [Comment Edited] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152679#comment-17152679 ] JinxinTang edited comment on SPARK-32205 at 7/7/20, 11:44 AM: --

[jira] [Commented] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152679#comment-17152679 ] JinxinTang commented on SPARK-32205: Sure, this problem seems in mysql side, because we cannot

[jira] [Comment Edited] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151682#comment-17151682 ] JinxinTang edited comment on SPARK-32176 at 7/7/20, 9:21 AM: -  [~abhi92544]

[jira] [Comment Edited] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152574#comment-17152574 ] JinxinTang edited comment on SPARK-32205 at 7/7/20, 9:16 AM: -

[jira] [Commented] (SPARK-32206) Enable multi-line true could break the read csv in Azure Data Lake Storage gen2

[ https://issues.apache.org/jira/browse/SPARK-32206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152588#comment-17152588 ] JinxinTang commented on SPARK-32206: [~vicentezqh] Could you please reproduce it in local

[jira] [Commented] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152574#comment-17152574 ] JinxinTang commented on SPARK-32205: We can see from the doc in mysql Date type has 1000-01-01

[jira] [Issue Comment Deleted] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-32205: --- Comment: was deleted (was: Could you please try insert into mysql directly, it seems mysql not

[jira] [Commented] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152568#comment-17152568 ] JinxinTang commented on SPARK-32205: Could you please try insert into mysql directly, it seems mysql

[jira] [Commented] (SPARK-32206) Enable multi-line true could break the read csv in Azure Data Lake Storage gen2

[ https://issues.apache.org/jira/browse/SPARK-32206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152567#comment-17152567 ] JinxinTang commented on SPARK-32206: Please use

[jira] [Comment Edited] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151682#comment-17151682 ] JinxinTang edited comment on SPARK-32176 at 7/6/20, 12:47 AM: -- cc

[jira] [Comment Edited] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151682#comment-17151682 ] JinxinTang edited comment on SPARK-32176 at 7/6/20, 12:43 AM: -- cc

[jira] [Comment Edited] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151682#comment-17151682 ] JinxinTang edited comment on SPARK-32176 at 7/6/20, 12:43 AM: -- cc

[jira] [Comment Edited] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151682#comment-17151682 ] JinxinTang edited comment on SPARK-32176 at 7/6/20, 12:42 AM: -- cc

[jira] [Comment Edited] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151682#comment-17151682 ] JinxinTang edited comment on SPARK-32176 at 7/6/20, 12:41 AM: -- cc

[jira] [Comment Edited] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151682#comment-17151682 ] JinxinTang edited comment on SPARK-32176 at 7/6/20, 12:40 AM: -- cc

[jira] [Comment Edited] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151682#comment-17151682 ] JinxinTang edited comment on SPARK-32176 at 7/6/20, 12:40 AM: -- cc

[jira] [Comment Edited] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151682#comment-17151682 ] JinxinTang edited comment on SPARK-32176 at 7/6/20, 12:40 AM: -- cc

[jira] [Commented] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151682#comment-17151682 ] JinxinTang commented on SPARK-32176: cc [~abhi92544] Seems can not reproduce：

[jira] [Created] (SPARK-32156) SPARK-31061 has two very similar tests could merge and somewhere could be improved

JinxinTang created SPARK-32156: -- Summary: SPARK-31061 has two very similar tests could merge and somewhere could be improved Key: SPARK-32156 URL: https://issues.apache.org/jira/browse/SPARK-32156

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148501#comment-17148501 ] JinxinTang commented on SPARK-32130: [~gourav.sengupta] Nice notebook, is seems the row count() is 

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148393#comment-17148393 ] JinxinTang edited comment on SPARK-32130 at 6/30/20, 8:07 AM: -- [~lotus2you] 

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148393#comment-17148393 ] JinxinTang edited comment on SPARK-32130 at 6/30/20, 8:05 AM: -- [~lotus2you] 

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148393#comment-17148393 ] JinxinTang edited comment on SPARK-32130 at 6/30/20, 8:05 AM: -- [~lotus2you] 

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148393#comment-17148393 ] JinxinTang edited comment on SPARK-32130 at 6/30/20, 7:59 AM: -- [~lotus2you] 

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148393#comment-17148393 ] JinxinTang edited comment on SPARK-32130 at 6/30/20, 7:58 AM: -- [~lotus2you] 

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148393#comment-17148393 ] JinxinTang edited comment on SPARK-32130 at 6/30/20, 7:39 AM: -- [~lotus2you] 

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148393#comment-17148393 ] JinxinTang commented on SPARK-32130: [~lotus2you] Thank you for your feedback. Lots of time is used

[jira] [Updated] (SPARK-32133) Forbid time field steps for date start/end in Sequence

[ https://issues.apache.org/jira/browse/SPARK-32133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-32133: --- Description: *Sequence time field steps for date start/end looks strange in spark as follows:*

[jira] [Updated] (SPARK-32133) Forbid time field steps for date start/end in Sequence

[ https://issues.apache.org/jira/browse/SPARK-32133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-32133: --- Summary: Forbid time field steps for date start/end in Sequence (was: Forbid time field steps for

[jira] [Created] (SPARK-32133) Forbid time field steps for date start/end

JinxinTang created SPARK-32133: -- Summary: Forbid time field steps for date start/end Key: SPARK-32133 URL: https://issues.apache.org/jira/browse/SPARK-32133 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32068) Spark 3 UI task launch time show in error time zone

[ https://issues.apache.org/jira/browse/SPARK-32068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143619#comment-17143619 ] JinxinTang commented on SPARK-32068: Thank you for the issue, hope my PR can help. > Spark 3 UI

[jira] [Commented] (SPARK-31751) spark serde property path overwrites table property location

[ https://issues.apache.org/jira/browse/SPARK-31751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17141320#comment-17141320 ] JinxinTang commented on SPARK-31751: Thank you for your issue, In order to compatible with

[jira] [Comment Edited] (SPARK-31980) Spark sequence() fails if start and end of range are identical dates

[ https://issues.apache.org/jira/browse/SPARK-31980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136248#comment-17136248 ] JinxinTang edited comment on SPARK-31980 at 6/16/20, 3:01 AM: --

[jira] [Commented] (SPARK-31980) Spark sequence() fails if start and end of range are identical dates

[ https://issues.apache.org/jira/browse/SPARK-31980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136248#comment-17136248 ] JinxinTang commented on SPARK-31980: [~DaveDeCaprio] Maybe one of this PRs could rebase to solve

[jira] [Commented] (SPARK-31968) write.partitionBy() creates duplicate subdirectories when user provides duplicate columns

[ https://issues.apache.org/jira/browse/SPARK-31968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135342#comment-17135342 ] JinxinTang commented on SPARK-31968: [~qxzzxq] Thank you for this issue, it is my pleasure. >

[jira] [Commented] (SPARK-31968) write.partitionBy() creates duplicate subdirectories when user provides duplicate columns

[ https://issues.apache.org/jira/browse/SPARK-31968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135340#comment-17135340 ] JinxinTang commented on SPARK-31968: Hi, [~dongjoon], [~hyukjin.kwon], I am  TJX2014,  Thank you

[jira] [Updated] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

[ https://issues.apache.org/jira/browse/SPARK-31797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31797: --- Description: 1.Add and register three new functions: {{TIMESTAMP_SECONDS}},  {{TIMESTAMP_MILLIS}} 

[jira] [Updated] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

[ https://issues.apache.org/jira/browse/SPARK-31797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31797: --- Description: (was: ### What changes were proposed in this pull request? Add and register three

[jira] [Created] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

JinxinTang created SPARK-31797: -- Summary: Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions Key: SPARK-31797 URL: https://issues.apache.org/jira/browse/SPARK-31797 Project: Spark

[jira] [Comment Edited] (SPARK-31598) LegacySimpleTimestampFormatter incorrectly interprets pre-Gregorian timestamps

[ https://issues.apache.org/jira/browse/SPARK-31598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097202#comment-17097202 ] JinxinTang edited comment on SPARK-31598 at 5/1/20, 7:00 AM: - same as :

[jira] [Commented] (SPARK-31598) LegacySimpleTimestampFormatter incorrectly interprets pre-Gregorian timestamps

[ https://issues.apache.org/jira/browse/SPARK-31598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097202#comment-17097202 ] JinxinTang commented on SPARK-31598: already fix: 

[jira] [Issue Comment Deleted] (SPARK-31550) nondeterministic configurations with general meanings in sql configuration doc

[ https://issues.apache.org/jira/browse/SPARK-31550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31550: --- Comment: was deleted (was: try to specify conf in spark-defaults.conf spark.sql.warehouse.dir /tmp

[jira] [Issue Comment Deleted] (SPARK-31532) SparkSessionBuilder shoud not propagate static sql configurations to the existing active/default SparkSession

[ https://issues.apache.org/jira/browse/SPARK-31532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31532: --- Comment: was deleted (was: Thanks for your issue, these followings may not be allowed to modify

[jira] [Commented] (SPARK-31550) nondeterministic configurations with general meanings in sql configuration doc

[ https://issues.apache.org/jira/browse/SPARK-31550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091218#comment-17091218 ] JinxinTang commented on SPARK-31550: try to specify conf in spark-defaults.conf

[jira] [Comment Edited] (SPARK-31532) SparkSessionBuilder shoud not propagate static sql configurations to the existing active/default SparkSession

[ https://issues.apache.org/jira/browse/SPARK-31532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091067#comment-17091067 ] JinxinTang edited comment on SPARK-31532 at 4/24/20, 1:08 AM: -- Thanks for

[jira] [Commented] (SPARK-31532) SparkSessionBuilder shoud not propagate static sql configurations to the existing active/default SparkSession

[ https://issues.apache.org/jira/browse/SPARK-31532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091067#comment-17091067 ] JinxinTang commented on SPARK-31532: Thanks for your issue, these followings not be modified after

[jira] [Comment Edited] (SPARK-31508) string type compare with numberic cause data inaccurate

[ https://issues.apache.org/jira/browse/SPARK-31508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089596#comment-17089596 ] JinxinTang edited comment on SPARK-31508 at 4/22/20, 12:00 PM: --- for

[jira] [Issue Comment Deleted] (SPARK-31508) string type compare with numberic cause data inaccurate

[ https://issues.apache.org/jira/browse/SPARK-31508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31508: --- Comment: was deleted (was: such '2...' is far larger than long, I think we should not covert to

[jira] [Comment Edited] (SPARK-31508) string type compare with numberic cause data inaccurate

[ https://issues.apache.org/jira/browse/SPARK-31508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089598#comment-17089598 ] JinxinTang edited comment on SPARK-31508 at 4/22/20, 11:49 AM: --- such

[jira] [Commented] (SPARK-31508) string type compare with numberic cause data inaccurate

[ https://issues.apache.org/jira/browse/SPARK-31508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089598#comment-17089598 ] JinxinTang commented on SPARK-31508: such '2...' is far larger than big, I think we should not

[jira] [Commented] (SPARK-31508) string type compare with numberic cause data inaccurate

[ https://issues.apache.org/jira/browse/SPARK-31508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089596#comment-17089596 ] JinxinTang commented on SPARK-31508: for example： select * from default.test1 where id > 3D add

[jira] [Commented] (SPARK-31508) string type compare with numberic cause data inaccurate

[ https://issues.apache.org/jira/browse/SPARK-31508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089583#comment-17089583 ] JinxinTang commented on SPARK-31508: You are right,We can see from code gen in branch 2.4 which will

[jira] [Issue Comment Deleted] (SPARK-31508) string type compare with numberic cause data inaccurate

[ https://issues.apache.org/jira/browse/SPARK-31508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31508: --- Comment: was deleted (was: You are right,We can see from code gen in branch 2.4 which will convet

[jira] [Comment Edited] (SPARK-31508) string type compare with numberic cause data inaccurate

[ https://issues.apache.org/jira/browse/SPARK-31508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089449#comment-17089449 ] JinxinTang edited comment on SPARK-31508 at 4/22/20, 8:33 AM: -- select *

[jira] [Commented] (SPARK-31508) string type compare with numberic cause data inaccurate

[ https://issues.apache.org/jira/browse/SPARK-31508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089449#comment-17089449 ] JinxinTang commented on SPARK-31508: select id from test1 where id>'0';  may be seems more normal.

[jira] [Commented] (SPARK-31515) Canonicalize Cast should consider the value of needTimeZone

[ https://issues.apache.org/jira/browse/SPARK-31515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089372#comment-17089372 ] JinxinTang commented on SPARK-31515: Could you please provide code piece or picture to depict this.

[jira] [Commented] (SPARK-31400) The catalogString doesn't distinguish Vectors in ml and mllib

[ https://issues.apache.org/jira/browse/SPARK-31400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089317#comment-17089317 ] JinxinTang commented on SPARK-31400: Thanks for report this issue, this is in progress:

[jira] [Issue Comment Deleted] (SPARK-31256) Dropna doesn't work for struct columns

[ https://issues.apache.org/jira/browse/SPARK-31256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31256: --- Comment: was deleted (was: This feature may not be supported since spark 2.4.5，due to former bug

[jira] [Commented] (SPARK-31256) Dropna doesn't work for struct columns

[ https://issues.apache.org/jira/browse/SPARK-31256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17087306#comment-17087306 ] JinxinTang commented on SPARK-31256: This feature may not be supported since spark 2.4.5，due to

[jira] [Commented] (SPARK-31386) Reading broadcast in UDF raises MemoryError when spark.executor.pyspark.memory is set

[ https://issues.apache.org/jira/browse/SPARK-31386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085455#comment-17085455 ] JinxinTang commented on SPARK-31386: hi [~hyukjin.kwon] , I test pyspark from my local spark

[jira] [Updated] (SPARK-31386) Reading broadcast in UDF raises MemoryError when spark.executor.pyspark.memory is set

[ https://issues.apache.org/jira/browse/SPARK-31386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31386: --- Attachment: 选区_001.png > Reading broadcast in UDF raises MemoryError when >

[jira] [Issue Comment Deleted] (SPARK-31386) Reading broadcast in UDF raises MemoryError when spark.executor.pyspark.memory is set

[ https://issues.apache.org/jira/browse/SPARK-31386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31386: --- Comment: was deleted (was: hi [~hyukjin.kwon] , I test pyspark from my local spark cluster with

[jira] [Comment Edited] (SPARK-31386) Reading broadcast in UDF raises MemoryError when spark.executor.pyspark.memory is set

[ https://issues.apache.org/jira/browse/SPARK-31386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085451#comment-17085451 ] JinxinTang edited comment on SPARK-31386 at 4/17/20, 5:45 AM: -- hi

[jira] [Commented] (SPARK-31386) Reading broadcast in UDF raises MemoryError when spark.executor.pyspark.memory is set

[ https://issues.apache.org/jira/browse/SPARK-31386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085451#comment-17085451 ] JinxinTang commented on SPARK-31386: hi [~hyukjin.kwon] , I test pyspark from my local spark

[jira] [Updated] (SPARK-31386) Reading broadcast in UDF raises MemoryError when spark.executor.pyspark.memory is set

[ https://issues.apache.org/jira/browse/SPARK-31386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31386: --- Attachment: 选区_001.png > Reading broadcast in UDF raises MemoryError when >

[jira] [Issue Comment Deleted] (SPARK-31256) Dropna doesn't work for struct columns

[ https://issues.apache.org/jira/browse/SPARK-31256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31256: --- Comment: was deleted (was: I will try to solve it) > Dropna doesn't work for struct columns >

[jira] [Commented] (SPARK-31256) Dropna doesn't work for struct columns

[ https://issues.apache.org/jira/browse/SPARK-31256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081084#comment-17081084 ] JinxinTang commented on SPARK-31256: I will try to solve it > Dropna doesn't work for struct

  1   2   >