[jira] [Resolved] (SPARK-29612) ALTER TABLE (RECOVER PARTITIONS) should look up catalog/table like v2 commands

2019-10-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29612. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26269

[jira] [Assigned] (SPARK-29612) ALTER TABLE (RECOVER PARTITIONS) should look up catalog/table like v2 commands

2019-10-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29612: --- Assignee: Huaxin Gao > ALTER TABLE (RECOVER PARTITIONS) should look up catalog/table like

[jira] [Updated] (SPARK-29573) Spark should work as PostgreSQL when using + Operator

2019-10-28 Thread ABHISHEK KUMAR GUPTA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ABHISHEK KUMAR GUPTA updated SPARK-29573: - Description: Spark and PostgreSQL result is different when concatenating as

[jira] [Commented] (SPARK-24918) Executor Plugin API

2019-10-28 Thread Brandon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961672#comment-16961672 ] Brandon commented on SPARK-24918: - [~nsheth] placing the plugin class inside a jar and passing as

[jira] [Updated] (SPARK-29601) JDBC ODBC Tab Statement column should be provided ellipsis for bigger SQL statement

2019-10-28 Thread ABHISHEK KUMAR GUPTA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ABHISHEK KUMAR GUPTA updated SPARK-29601: - Description: Statement column in JDBC/ODBC gives whole SQL statement and page

[jira] [Updated] (SPARK-29600) array_contains built in function is not backward compatible in 3.0

2019-10-28 Thread ABHISHEK KUMAR GUPTA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ABHISHEK KUMAR GUPTA updated SPARK-29600: - Description: SELECT array_contains(array(0,0.1,0.2,0.3,0.5,0.02,0.033), .2);

[jira] [Commented] (SPARK-29477) Improve tooltip information for Streaming Tab

2019-10-28 Thread ABHISHEK KUMAR GUPTA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961662#comment-16961662 ] ABHISHEK KUMAR GUPTA commented on SPARK-29477: -- [~hyukjin.kwon]: When user submit streaming

[jira] [Commented] (SPARK-29222) Flaky test: pyspark.mllib.tests.test_streaming_algorithms.StreamingLinearRegressionWithTests.test_parameter_convergence

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961636#comment-16961636 ] Hyukjin Kwon commented on SPARK-29222: -- Can you make a PR to increase the timeout? > Flaky test:

[jira] [Created] (SPARK-29627) array_contains should allow column instances in PySpark

2019-10-28 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-29627: Summary: array_contains should allow column instances in PySpark Key: SPARK-29627 URL: https://issues.apache.org/jira/browse/SPARK-29627 Project: Spark

[jira] [Created] (SPARK-29626) notEqual() should return true when the one is null, the other is not null

2019-10-28 Thread zhouhuazheng (Jira)
zhouhuazheng created SPARK-29626: Summary: notEqual() should return true when the one is null, the other is not null Key: SPARK-29626 URL: https://issues.apache.org/jira/browse/SPARK-29626 Project:

[jira] [Assigned] (SPARK-29566) Imputer should support single-column input/ouput

2019-10-28 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng reassigned SPARK-29566: Assignee: Huaxin Gao > Imputer should support single-column input/ouput >

[jira] [Resolved] (SPARK-29566) Imputer should support single-column input/ouput

2019-10-28 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-29566. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26247

[jira] [Resolved] (SPARK-29549) Union of DataSourceV2 datasources leads to duplicated results

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29549. -- Resolution: Cannot Reproduce > Union of DataSourceV2 datasources leads to duplicated results

[jira] [Commented] (SPARK-29549) Union of DataSourceV2 datasources leads to duplicated results

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961596#comment-16961596 ] Hyukjin Kwon commented on SPARK-29549: -- There will be no more releases in 2.3.x line. > Union of

[jira] [Updated] (SPARK-27842) Inconsistent results of Statistics.corr() and PearsonCorrelation.computeCorrelationMatrix()

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27842: - Affects Version/s: 2.4.3 > Inconsistent results of Statistics.corr() and >

[jira] [Commented] (SPARK-29624) Jenkins fails with "Python versions prior to 2.7 are not supported."

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961595#comment-16961595 ] Hyukjin Kwon commented on SPARK-29624: -- Awesome [~shaneknapp]! > Jenkins fails with "Python

[jira] [Updated] (SPARK-29625) Spark Structure Streaming Kafka Wrong Reset Offset twice

2019-10-28 Thread Sandish Kumar HN (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandish Kumar HN updated SPARK-29625: - Description: Spark Structure Streaming Kafka Reset Offset twice, once with right

[jira] [Updated] (SPARK-29625) Spark Structure Streaming Kafka Wrong Reset Offset twice

2019-10-28 Thread Sandish Kumar HN (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandish Kumar HN updated SPARK-29625: - Summary: Spark Structure Streaming Kafka Wrong Reset Offset twice (was: Spark Stucture

[jira] [Created] (SPARK-29625) Spark Stucture Streaming Kafka Reset Offset twice

2019-10-28 Thread Sandish Kumar HN (Jira)
Sandish Kumar HN created SPARK-29625: Summary: Spark Stucture Streaming Kafka Reset Offset twice Key: SPARK-29625 URL: https://issues.apache.org/jira/browse/SPARK-29625 Project: Spark

[jira] [Updated] (SPARK-29625) Spark Structure Streaming Kafka Wrong Reset Offset twice

2019-10-28 Thread Sandish Kumar HN (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandish Kumar HN updated SPARK-29625: - Description: Spark Structure Streaming Kafka Reset Offset twice, once with right

[jira] [Assigned] (SPARK-29609) DataSourceV2: Support DROP NAMESPACE

2019-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29609: - Assignee: Terry Kim > DataSourceV2: Support DROP NAMESPACE >

[jira] [Resolved] (SPARK-29609) DataSourceV2: Support DROP NAMESPACE

2019-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29609. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26262

[jira] [Commented] (SPARK-28841) Spark cannot read a relative path containing ":"

2019-10-28 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961412#comment-16961412 ] Shixiong Zhu commented on SPARK-28841: -- [~srowen] Yep. ":" is not a valid char in a HDFS path. But

[jira] [Commented] (SPARK-28841) Spark cannot read a relative path containing ":"

2019-10-28 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961408#comment-16961408 ] Sean R. Owen commented on SPARK-28841: -- Oh I see, OK. So is this mostly a Hadoop-side issue? >

[jira] [Comment Edited] (SPARK-27592) Set the bucketed data source table SerDe correctly

2019-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961404#comment-16961404 ] Dongjoon Hyun edited comment on SPARK-27592 at 10/28/19 7:44 PM: - Ping

[jira] [Commented] (SPARK-27592) Set the bucketed data source table SerDe correctly

2019-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961404#comment-16961404 ] Dongjoon Hyun commented on SPARK-27592: --- Ping [~Patnaik] since you asked about this on

[jira] [Commented] (SPARK-28841) Spark cannot read a relative path containing ":"

2019-10-28 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961402#comment-16961402 ] Shixiong Zhu commented on SPARK-28841: -- [~srowen] "?" is to trigger glob pattern codes. It matches

[jira] [Commented] (SPARK-29234) bucketed table created by Spark SQL DataFrame is in SequenceFile format

2019-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961395#comment-16961395 ] Dongjoon Hyun commented on SPARK-29234: --- [~Patnaik]. As you see, this is closed as `Duplicated`.

[jira] [Assigned] (SPARK-29624) Jenkins fails with "Python versions prior to 2.7 are not supported."

2019-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29624: - Assignee: Shane Knapp > Jenkins fails with "Python versions prior to 2.7 are not

[jira] [Resolved] (SPARK-29624) Jenkins fails with "Python versions prior to 2.7 are not supported."

2019-10-28 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Knapp resolved SPARK-29624. - Resolution: Fixed > Jenkins fails with "Python versions prior to 2.7 are not supported." >

[jira] [Commented] (SPARK-29624) Jenkins fails with "Python versions prior to 2.7 are not supported."

2019-10-28 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961377#comment-16961377 ] Shane Knapp commented on SPARK-29624: - alright, a pull request build has successfully made it past

[jira] [Commented] (SPARK-29624) Jenkins fails with "Python versions prior to 2.7 are not supported."

2019-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961369#comment-16961369 ] Dongjoon Hyun commented on SPARK-29624: --- Thank you so much! > Jenkins fails with "Python versions

[jira] [Resolved] (SPARK-29509) Deduplicate code blocks in Kafka data source

2019-10-28 Thread Marcelo Masiero Vanzin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29509. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-29509) Deduplicate code blocks in Kafka data source

2019-10-28 Thread Marcelo Masiero Vanzin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-29509: -- Assignee: Jungtaek Lim > Deduplicate code blocks in Kafka data

[jira] [Commented] (SPARK-29624) Jenkins fails with "Python versions prior to 2.7 are not supported."

2019-10-28 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961320#comment-16961320 ] Shane Knapp commented on SPARK-29624: - alright, i triggered a job and checked the console, and the

[jira] [Commented] (SPARK-29624) Jenkins fails with "Python versions prior to 2.7 are not supported."

2019-10-28 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961290#comment-16961290 ] Shane Knapp commented on SPARK-29624: - nothing changed... the PATH env vars for each worker got

[jira] [Commented] (SPARK-29624) Jenkins fails with "Python versions prior to 2.7 are not supported."

2019-10-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961287#comment-16961287 ] Dongjoon Hyun commented on SPARK-29624: --- Hi, [~shaneknapp]. Jenkins server seems to be changed.

[jira] [Created] (SPARK-29624) Jenkins fails with "Python versions prior to 2.7 are not supported."

2019-10-28 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-29624: - Summary: Jenkins fails with "Python versions prior to 2.7 are not supported." Key: SPARK-29624 URL: https://issues.apache.org/jira/browse/SPARK-29624 Project:

[jira] [Resolved] (SPARK-29521) LOAD DATA INTO TABLE should look up catalog/table like v2 commands

2019-10-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29521. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26178

[jira] [Commented] (SPARK-22000) org.codehaus.commons.compiler.CompileException: toString method is not declared

2019-10-28 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961171#comment-16961171 ] Sean R. Owen commented on SPARK-22000: -- As indicated, it's fixed in 3.0, not 2.4.x. The change is

[jira] [Commented] (SPARK-22000) org.codehaus.commons.compiler.CompileException: toString method is not declared

2019-10-28 Thread Shyam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961159#comment-16961159 ] Shyam commented on SPARK-22000: --- [~srowen] I am using spark.2.4.1 version getting same error ... for more

[jira] [Created] (SPARK-29623) do not allow multiple unit TO unit statements in interval literal syntax

2019-10-28 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-29623: --- Summary: do not allow multiple unit TO unit statements in interval literal syntax Key: SPARK-29623 URL: https://issues.apache.org/jira/browse/SPARK-29623 Project:

[jira] [Created] (SPARK-29622) do not allow leading 'interval' in the interval string format

2019-10-28 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-29622: --- Summary: do not allow leading 'interval' in the interval string format Key: SPARK-29622 URL: https://issues.apache.org/jira/browse/SPARK-29622 Project: Spark

[jira] [Resolved] (SPARK-29599) Support pagination for session table in JDBC/ODBC Tab

2019-10-28 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-29599. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26253

[jira] [Assigned] (SPARK-29599) Support pagination for session table in JDBC/ODBC Tab

2019-10-28 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-29599: Assignee: angerszhu > Support pagination for session table in JDBC/ODBC Tab >

[jira] [Updated] (SPARK-29621) Querying internal corrupt record column should not be allowed in filter operation

2019-10-28 Thread Suchintak Patnaik (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suchintak Patnaik updated SPARK-29621: -- Labels: PySpark SparkSQL (was: ) > Querying internal corrupt record column should

[jira] [Created] (SPARK-29621) Querying internal corrupt record column should not be allowed in filter operation

2019-10-28 Thread Suchintak Patnaik (Jira)
Suchintak Patnaik created SPARK-29621: - Summary: Querying internal corrupt record column should not be allowed in filter operation Key: SPARK-29621 URL: https://issues.apache.org/jira/browse/SPARK-29621

[jira] [Commented] (SPARK-29549) Union of DataSourceV2 datasources leads to duplicated results

2019-10-28 Thread Miguel Molina (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961043#comment-16961043 ] Miguel Molina commented on SPARK-29549: --- Hi [~hyukjin.kwon]. In Spark 2.4 it works correctly, this

[jira] [Updated] (SPARK-29620) UnsafeKVExternalSorterSuite failure on bigendian system

2019-10-28 Thread salamani (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] salamani updated SPARK-29620: - Description: spark/sql/core# ../../build/mvn -Dtest=none

[jira] [Created] (SPARK-29620) UnsafeKVExternalSorterSuite failure on bigendian system

2019-10-28 Thread salamani (Jira)
salamani created SPARK-29620: Summary: UnsafeKVExternalSorterSuite failure on bigendian system Key: SPARK-29620 URL: https://issues.apache.org/jira/browse/SPARK-29620 Project: Spark Issue Type:

[jira] [Commented] (SPARK-29234) bucketed table created by Spark SQL DataFrame is in SequenceFile format

2019-10-28 Thread Suchintak Patnaik (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961028#comment-16961028 ] Suchintak Patnaik commented on SPARK-29234: --- [~dongjoon] [~yumwang] Are the PRs backported to

[jira] [Resolved] (SPARK-29459) Spark takes lot of time to initialize when the network is not connected to internet.

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29459. -- Resolution: Incomplete 1.5.2 is too old and EOL release. Can you try higher versions and see

[jira] [Commented] (SPARK-29476) Add tooltip information for Thread Dump links and Thread details table columns in Executors Tab

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961007#comment-16961007 ] Hyukjin Kwon commented on SPARK-29476: -- What does this Jira mean? > Add tooltip information for

[jira] [Commented] (SPARK-29477) Improve tooltip information for Streaming Tab

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961005#comment-16961005 ] Hyukjin Kwon commented on SPARK-29477: -- Please describe what each Jira means clearly,

[jira] [Commented] (SPARK-29590) Support hiding table in JDBC/ODBC server page in WebUI

2019-10-28 Thread shahid (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961006#comment-16961006 ] shahid commented on SPARK-29590: Hi [~hyukjin.kwon], This JIRA is similar to the JIRA

[jira] [Updated] (SPARK-29505) desc extended is case sensitive

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29505: - Description: {code} create table customer(id int, name String, *CName String*, address String,

[jira] [Updated] (SPARK-29505) desc extended is case sensitive

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29505: - Description: {code} create table customer(id int, name String, *CName String*, address String,

[jira] [Updated] (SPARK-29510) JobGroup ID is not set for the job submitted from Spark-SQL and Spark -Shell

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29510: - Description: When user submit jobs from Spark-shell or SQL Job group id is not set. UI Screen

[jira] [Commented] (SPARK-29534) Hanging tasks in DiskBlockObjectWriter.commitAndGet while calling native FileDispatcherImpl.size0

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961001#comment-16961001 ] Hyukjin Kwon commented on SPARK-29534: -- Can you post the codes you ran as well? > Hanging tasks

[jira] [Commented] (SPARK-29549) Union of DataSourceV2 datasources leads to duplicated results

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960999#comment-16960999 ] Hyukjin Kwon commented on SPARK-29549: -- Hi [~erizocosmico]. Spark 2.3.x is EOL. Can you try it out

[jira] [Commented] (SPARK-29561) Large Case Statement Code Generation OOM

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960996#comment-16960996 ] Hyukjin Kwon commented on SPARK-29561: -- Seems like it was just the leak of memory. Does it work

[jira] [Commented] (SPARK-29573) Spark should work as PostgreSQL when using + Operator

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960995#comment-16960995 ] Hyukjin Kwon commented on SPARK-29573: -- [~abhishek.akg] can you please use \{code\} ... \{code\}

[jira] [Updated] (SPARK-29573) Spark should work as PostgreSQL when using + Operator

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29573: - Description: Spark and PostgreSQL result is different when concatenating as below : Spark :

[jira] [Updated] (SPARK-29575) from_json can produce nulls for fields which are marked as non-nullable

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29575: - Component/s: SQL > from_json can produce nulls for fields which are marked as non-nullable >

[jira] [Commented] (SPARK-29590) Support hiding table in JDBC/ODBC server page in WebUI

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960992#comment-16960992 ] Hyukjin Kwon commented on SPARK-29590: -- [~shahid] what does this JIRA mean? > Support hiding table

[jira] [Commented] (SPARK-29598) Support search option for statistics in JDBC/ODBC server page

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960991#comment-16960991 ] Hyukjin Kwon commented on SPARK-29598: -- [~jobitmathew] can you please elaborate what this JIRA

[jira] [Commented] (SPARK-29600) array_contains built in function is not backward compatible in 3.0

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960988#comment-16960988 ] Hyukjin Kwon commented on SPARK-29600: -- Please go ahead but please just don't copy and paste.

[jira] [Commented] (SPARK-29601) JDBC ODBC Tab Statement column should be provided ellipsis for bigger SQL statement

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960987#comment-16960987 ] Hyukjin Kwon commented on SPARK-29601: -- Please go ahead but [~abhishek.akg] can you please don't

[jira] [Resolved] (SPARK-29602) How does the spark from_json json and dataframe transform ignore the case of the json key

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29602. -- Resolution: Invalid Please ask questions to stackoverflow or mailing list. > How does the

[jira] [Resolved] (SPARK-29610) Keys with Null values are discarded when using to_json function

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29610. -- Resolution: Duplicate > Keys with Null values are discarded when using to_json function >

[jira] [Resolved] (SPARK-28158) Hive UDFs supports UDT type

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28158. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 24961

[jira] [Assigned] (SPARK-28158) Hive UDFs supports UDT type

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28158: Assignee: Genmao Yu > Hive UDFs supports UDT type > --- > >

[jira] [Updated] (SPARK-29528) Upgrade scala-maven-plugin to 4.2.4 for Scala 2.13.1

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29528: - Fix Version/s: (was: 3.0.0) > Upgrade scala-maven-plugin to 4.2.4 for Scala 2.13.1 >

[jira] [Commented] (SPARK-29528) Upgrade scala-maven-plugin to 4.2.4 for Scala 2.13.1

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960958#comment-16960958 ] Hyukjin Kwon commented on SPARK-29528: -- Reverted at

[jira] [Reopened] (SPARK-29528) Upgrade scala-maven-plugin to 4.2.4 for Scala 2.13.1

2019-10-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-29528: -- > Upgrade scala-maven-plugin to 4.2.4 for Scala 2.13.1 >

[jira] [Updated] (SPARK-29619) Improve the exception message when reading the daemon port and add retry times.

2019-10-28 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-29619: --- Description: In production environment, my pyspark application occurs an exception and it's

[jira] [Created] (SPARK-29619) Improve the exception message when reading the daemon port and add retry times.

2019-10-28 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-29619: -- Summary: Improve the exception message when reading the daemon port and add retry times. Key: SPARK-29619 URL: https://issues.apache.org/jira/browse/SPARK-29619 Project:

[jira] [Created] (SPARK-29618) remove V1_BATCH_WRITE table capability

2019-10-28 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-29618: --- Summary: remove V1_BATCH_WRITE table capability Key: SPARK-29618 URL: https://issues.apache.org/jira/browse/SPARK-29618 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-29580) Add kerberos debug messages for Kafka secure tests

2019-10-28 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960869#comment-16960869 ] Gabor Somogyi commented on SPARK-29580: --- [~dongjoon] thanks for taking care, ping me if reproduced