[jira] [Created] (SPARK-29934) Dataset support GraphX

darion yaphet created SPARK-29934: - Summary: Dataset support GraphX Key: SPARK-29934 URL: https://issues.apache.org/jira/browse/SPARK-29934 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-29758) json_tuple truncates fields

[ https://issues.apache.org/jira/browse/SPARK-29758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976099#comment-16976099 ] Maxim Gekk commented on SPARK-29758: I have reproduced the issue on 2.4. The problem is in Jackson

[jira] [Updated] (SPARK-29644) Corrected ShortType and ByteType mapping to SmallInt and TinyInt in JDBCUtils

[ https://issues.apache.org/jira/browse/SPARK-29644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-29644: - Fix Version/s: 2.4.5 > Corrected ShortType and ByteType mapping to SmallInt and TinyInt in

[jira] [Resolved] (SPARK-29456) Add tooltip information for Session Statistics Table column in JDBC/ODBC Server Tab

[ https://issues.apache.org/jira/browse/SPARK-29456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-29456. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 26138

[jira] [Comment Edited] (SPARK-29758) json_tuple truncates fields

[ https://issues.apache.org/jira/browse/SPARK-29758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976106#comment-16976106 ] Maxim Gekk edited comment on SPARK-29758 at 11/17/19 6:17 PM: -- Another

[jira] [Assigned] (SPARK-29930) Remove SQL configs declared to be removed in Spark 3.0

[ https://issues.apache.org/jira/browse/SPARK-29930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29930: - Assignee: Maxim Gekk > Remove SQL configs declared to be removed in Spark 3.0 >

[jira] [Resolved] (SPARK-29930) Remove SQL configs declared to be removed in Spark 3.0

[ https://issues.apache.org/jira/browse/SPARK-29930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29930. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26559

[jira] [Resolved] (SPARK-29932) lint-r should do non-zero exit in case of errors

[ https://issues.apache.org/jira/browse/SPARK-29932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29932. --- Fix Version/s: 3.0.0 2.4.5 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-29932) lint-r should do non-zero exit in case of errors

[ https://issues.apache.org/jira/browse/SPARK-29932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29932: - Assignee: Dongjoon Hyun > lint-r should do non-zero exit in case of errors >

[jira] [Created] (SPARK-29933) ThriftServerQueryTestSuite runs tests with wrong settings

Maxim Gekk created SPARK-29933: -- Summary: ThriftServerQueryTestSuite runs tests with wrong settings Key: SPARK-29933 URL: https://issues.apache.org/jira/browse/SPARK-29933 Project: Spark Issue

[jira] [Updated] (SPARK-29456) Add tooltip information for Session Statistics Table column in JDBC/ODBC Server Tab

[ https://issues.apache.org/jira/browse/SPARK-29456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-29456: - Fix Version/s: (was: 3.1.0) 3.0.0 > Add tooltip information for Session

[jira] [Assigned] (SPARK-29456) Add tooltip information for Session Statistics Table column in JDBC/ODBC Server Tab

[ https://issues.apache.org/jira/browse/SPARK-29456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-29456: Assignee: pavithra ramachandran > Add tooltip information for Session Statistics Table

[jira] [Updated] (SPARK-29456) Add tooltip information for Session Statistics Table column in JDBC/ODBC Server Tab

[ https://issues.apache.org/jira/browse/SPARK-29456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-29456: - Priority: Minor (was: Major) > Add tooltip information for Session Statistics Table column in

[jira] [Commented] (SPARK-29758) json_tuple truncates fields

[ https://issues.apache.org/jira/browse/SPARK-29758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976106#comment-16976106 ] Maxim Gekk commented on SPARK-29758: Another solution is to remove this optimization:

[jira] [Updated] (SPARK-29933) ThriftServerQueryTestSuite runs tests with wrong settings

[ https://issues.apache.org/jira/browse/SPARK-29933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-29933: --- Attachment: filter_tests.patch > ThriftServerQueryTestSuite runs tests with wrong settings >

[jira] [Commented] (SPARK-29575) from_json can produce nulls for fields which are marked as non-nullable

[ https://issues.apache.org/jira/browse/SPARK-29575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976102#comment-16976102 ] Maxim Gekk commented on SPARK-29575: This is intentional behavior. User's schema is forcibly set as

[jira] [Commented] (SPARK-29931) Declare all SQL legacy configs as will be removed in Spark 4.0

[ https://issues.apache.org/jira/browse/SPARK-29931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975944#comment-16975944 ] Maxim Gekk commented on SPARK-29931: > It's conceivable there could a reason to do it later, or

[jira] [Resolved] (SPARK-29807) Rename "spark.sql.ansi.enabled" to "spark.sql.dialect.spark.ansi.enabled"

[ https://issues.apache.org/jira/browse/SPARK-29807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29807. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26444

[jira] [Commented] (SPARK-29929) Allow V2 Datasources to require a data distribution

[ https://issues.apache.org/jira/browse/SPARK-29929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976294#comment-16976294 ] Jungtaek Lim commented on SPARK-29929: -- Possibly duplicated with SPARK-23889 , though no one is

[jira] [Assigned] (SPARK-25694) URL.setURLStreamHandlerFactory causing incompatible HttpURLConnection issue

[ https://issues.apache.org/jira/browse/SPARK-25694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-25694: --- Assignee: Dongjoon Hyun > URL.setURLStreamHandlerFactory causing incompatible HttpURLConnection

[jira] [Created] (SPARK-29938) Add batching in alter table add partition flow

Prakhar Jain created SPARK-29938: Summary: Add batching in alter table add partition flow Key: SPARK-29938 URL: https://issues.apache.org/jira/browse/SPARK-29938 Project: Spark Issue Type:

[jira] [Updated] (SPARK-29938) Add batching in alter table add partition flow

[ https://issues.apache.org/jira/browse/SPARK-29938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prakhar Jain updated SPARK-29938: - Description: When lot of new partitions are added by an Insert query on a partitioned

[jira] [Updated] (SPARK-29938) Add batching in alter table add partition flow

[ https://issues.apache.org/jira/browse/SPARK-29938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prakhar Jain updated SPARK-29938: - Description: When lot of new partitions are added by an Insert query on a partitioned

[jira] [Created] (SPARK-29939) Add a conf for CompressionCodec for Ser/Deser of MapOutputStatus

Xiao Li created SPARK-29939: --- Summary: Add a conf for CompressionCodec for Ser/Deser of MapOutputStatus Key: SPARK-29939 URL: https://issues.apache.org/jira/browse/SPARK-29939 Project: Spark

[jira] [Created] (SPARK-29940) Whether contains schema for this parameter "spark.yarn.historyServer.address"

hehuiyuan created SPARK-29940: - Summary: Whether contains schema for this parameter "spark.yarn.historyServer.address" Key: SPARK-29940 URL: https://issues.apache.org/jira/browse/SPARK-29940 Project:

[jira] [Updated] (SPARK-29940) Whether contains schema for this parameter "spark.yarn.historyServer.address"

[ https://issues.apache.org/jira/browse/SPARK-29940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hehuiyuan updated SPARK-29940: -- Description:     !image-2019-11-18-15-44-10-358.png|width=815,height=156!  

[jira] [Updated] (SPARK-29940) Whether contains schema for this parameter "spark.yarn.historyServer.address"

[ https://issues.apache.org/jira/browse/SPARK-29940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hehuiyuan updated SPARK-29940: -- Attachment: image-2019-11-18-15-45-33-295.png > Whether contains schema for this parameter

[jira] [Resolved] (SPARK-25694) URL.setURLStreamHandlerFactory causing incompatible HttpURLConnection issue

[ https://issues.apache.org/jira/browse/SPARK-25694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-25694. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26530

[jira] [Updated] (SPARK-29940) Whether contains schema for this parameter "spark.yarn.historyServer.address"

[ https://issues.apache.org/jira/browse/SPARK-29940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hehuiyuan updated SPARK-29940: -- Description:     !image-2019-11-18-15-44-10-358.png|width=815,height=156!   was:

[jira] [Updated] (SPARK-29940) Whether contains schema for this parameter "spark.yarn.historyServer.address"

[ https://issues.apache.org/jira/browse/SPARK-29940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hehuiyuan updated SPARK-29940: -- Attachment: image-2019-11-18-15-44-10-358.png > Whether contains schema for this parameter

[jira] [Resolved] (SPARK-29783) Support SQL Standard output style for interval type

[ https://issues.apache.org/jira/browse/SPARK-29783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29783. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26418

[jira] [Assigned] (SPARK-29783) Support SQL Standard output style for interval type

[ https://issues.apache.org/jira/browse/SPARK-29783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29783: --- Assignee: Kent Yao > Support SQL Standard output style for interval type >

[jira] [Assigned] (SPARK-29907) Move DELETE/UPDATE/MERGE relative rules to dmlStatementNoWith to support cte.

[ https://issues.apache.org/jira/browse/SPARK-29907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29907: --- Assignee: Xianyin Xin > Move DELETE/UPDATE/MERGE relative rules to dmlStatementNoWith to

[jira] [Resolved] (SPARK-29907) Move DELETE/UPDATE/MERGE relative rules to dmlStatementNoWith to support cte.

[ https://issues.apache.org/jira/browse/SPARK-29907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29907. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26536

[jira] [Comment Edited] (SPARK-29600) array_contains built in function is not backward compatible in 3.0

[ https://issues.apache.org/jira/browse/SPARK-29600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976319#comment-16976319 ] Udbhav Agrawal edited comment on SPARK-29600 at 11/18/19 6:56 AM: -- Hi

[jira] [Resolved] (SPARK-29020) Unifying behaviour between array_sort and sort_array

[ https://issues.apache.org/jira/browse/SPARK-29020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29020. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25728

[jira] [Assigned] (SPARK-29020) Unifying behaviour between array_sort and sort_array

[ https://issues.apache.org/jira/browse/SPARK-29020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-29020: Assignee: German Schiavon Matteo > Unifying behaviour between array_sort and sort_array

[jira] [Assigned] (SPARK-25694) URL.setURLStreamHandlerFactory causing incompatible HttpURLConnection issue

[ https://issues.apache.org/jira/browse/SPARK-25694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25694: - Assignee: Zhou Jiang (was: Dongjoon Hyun) > URL.setURLStreamHandlerFactory causing

[jira] [Assigned] (SPARK-29870) Unify the logic of multi-units interval string to CalendarInterval

[ https://issues.apache.org/jira/browse/SPARK-29870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29870: --- Assignee: Kent Yao > Unify the logic of multi-units interval string to CalendarInterval >

[jira] [Resolved] (SPARK-29870) Unify the logic of multi-units interval string to CalendarInterval

[ https://issues.apache.org/jira/browse/SPARK-29870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29870. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26491

[jira] [Commented] (SPARK-29903) Add documentation for recursiveFileLookup

[ https://issues.apache.org/jira/browse/SPARK-29903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976252#comment-16976252 ] Nicholas Chammas commented on SPARK-29903: -- Happy to do that. Going to wait for [this

[jira] [Resolved] (SPARK-29936) Fix SparkR lint errors and add lint-r GitHub Action

[ https://issues.apache.org/jira/browse/SPARK-29936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29936. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26564

[jira] [Assigned] (SPARK-29936) Fix SparkR lint errors and add lint-r GitHub Action

[ https://issues.apache.org/jira/browse/SPARK-29936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29936: - Assignee: Dongjoon Hyun > Fix SparkR lint errors and add lint-r GitHub Action >

[jira] [Issue Comment Deleted] (SPARK-29587) Real data type is not supported in Spark SQL which is supporting in postgresql

[ https://issues.apache.org/jira/browse/SPARK-29587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Raj Boudh updated SPARK-29587: Comment: was deleted (was: I will analyse this issue) > Real data type is not supported

[jira] [Commented] (SPARK-29600) array_contains built in function is not backward compatible in 3.0

[ https://issues.apache.org/jira/browse/SPARK-29600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976319#comment-16976319 ] Udbhav Agrawal commented on SPARK-29600: Hi [~hyukjin.kwon], this failure is because we cannot

[jira] [Created] (SPARK-29936) Add `lint-r` GitHub Action

Dongjoon Hyun created SPARK-29936: - Summary: Add `lint-r` GitHub Action Key: SPARK-29936 URL: https://issues.apache.org/jira/browse/SPARK-29936 Project: Spark Issue Type: Task

[jira] [Commented] (SPARK-29935) Remove `Spark QA Compile` Jenkins Dashboard (and jobs)

[ https://issues.apache.org/jira/browse/SPARK-29935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976197#comment-16976197 ] Sean R. Owen commented on SPARK-29935: -- Do the github actions work reliably now? I haven't watched

[jira] [Resolved] (SPARK-29581) Enable cleanup old event log files

[ https://issues.apache.org/jira/browse/SPARK-29581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-29581. -- Resolution: Invalid We took the different approach: see SPARK-29779 > Enable cleanup old

[jira] [Resolved] (SPARK-16872) Impl Gaussian Naive Bayes Classifier

[ https://issues.apache.org/jira/browse/SPARK-16872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-16872. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 26413

[jira] [Updated] (SPARK-29935) Remove `Spark QA Compile` Jenkins Dashboard (and jobs)

[ https://issues.apache.org/jira/browse/SPARK-29935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29935: -- Priority: Minor (was: Major) > Remove `Spark QA Compile` Jenkins Dashboard (and jobs) >

[jira] [Commented] (SPARK-29935) Remove `Spark QA Compile` Jenkins Dashboard (and jobs)

[ https://issues.apache.org/jira/browse/SPARK-29935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976208#comment-16976208 ] Dongjoon Hyun commented on SPARK-29935: --- Yes, it's much better now. Also, we can re-trigger the

[jira] [Comment Edited] (SPARK-29935) Remove `Spark QA Compile` Jenkins Dashboard (and jobs)

[ https://issues.apache.org/jira/browse/SPARK-29935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976208#comment-16976208 ] Dongjoon Hyun edited comment on SPARK-29935 at 11/18/19 12:19 AM: -- Yes,

[jira] [Commented] (SPARK-29934) Dataset support GraphX

[ https://issues.apache.org/jira/browse/SPARK-29934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976151#comment-16976151 ] Dongjoon Hyun commented on SPARK-29934: --- Hi, [~darion]. According to your context, the issue type

[jira] [Commented] (SPARK-29931) Declare all SQL legacy configs as will be removed in Spark 4.0

[ https://issues.apache.org/jira/browse/SPARK-29931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976134#comment-16976134 ] Reynold Xin commented on SPARK-29931: - You can say "This config will be removed in Spark 4.0 or a

[jira] [Commented] (SPARK-29935) Remove `Spark QA Compile` Jenkins Dashboard (and jobs)

[ https://issues.apache.org/jira/browse/SPARK-29935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976209#comment-16976209 ] Dongjoon Hyun commented on SPARK-29935: --- SPARK-29936 will recover Lint-R in GitHub Action. >

[jira] [Updated] (SPARK-29936) Add `lint-r` GitHub Action

[ https://issues.apache.org/jira/browse/SPARK-29936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29936: -- Component/s: (was: Tests) SparkR > Add `lint-r` GitHub Action >

[jira] [Updated] (SPARK-29936) Fix SparkR lint errors and add lint-r GitHub Action

[ https://issues.apache.org/jira/browse/SPARK-29936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29936: -- Issue Type: Bug (was: Task) > Fix SparkR lint errors and add lint-r GitHub Action >

[jira] [Updated] (SPARK-29936) Fix SparkR lint errors and add lint-r GitHub Action

[ https://issues.apache.org/jira/browse/SPARK-29936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29936: -- Summary: Fix SparkR lint errors and add lint-r GitHub Action (was: Add `lint-r` GitHub

[jira] [Created] (SPARK-29935) Remove `Spark QA Compile` Jenkins Dashboard (and jobs)

Dongjoon Hyun created SPARK-29935: - Summary: Remove `Spark QA Compile` Jenkins Dashboard (and jobs) Key: SPARK-29935 URL: https://issues.apache.org/jira/browse/SPARK-29935 Project: Spark

[jira] [Commented] (SPARK-29935) Remove `Spark QA Compile` Jenkins Dashboard (and jobs)

[ https://issues.apache.org/jira/browse/SPARK-29935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976153#comment-16976153 ] Dongjoon Hyun commented on SPARK-29935: --- cc [~shaneknapp], [~srowen] > Remove `Spark QA Compile`

[jira] [Created] (SPARK-29937) Make FileSourceScanExec class fields lazy

ulysses you created SPARK-29937: --- Summary: Make FileSourceScanExec class fields lazy Key: SPARK-29937 URL: https://issues.apache.org/jira/browse/SPARK-29937 Project: Spark Issue Type: