[jira] [Issue Comment Deleted] (SPARK-27800) Example for xor function has a wrong answer

[ https://issues.apache.org/jira/browse/SPARK-27800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated SPARK-27800: - Comment: was deleted (was:  A PR created to fix this, Would anyone take a look. :)

[jira] [Commented] (SPARK-27800) Example for xor function has a wrong answer

[ https://issues.apache.org/jira/browse/SPARK-27800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845429#comment-16845429 ] Alex Liu commented on SPARK-27800: --  A PR created to fix this, Would anyone take a look. :)

[jira] [Updated] (SPARK-27800) Example for xor function has a wrong answer

[ https://issues.apache.org/jira/browse/SPARK-27800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated SPARK-27800: - Description: See [https://spark.apache.org/docs/latest/api/sql/index.html#_14] 3 ^ 5 should be 6 rather

[jira] [Created] (SPARK-27800) Example for xor function has a wrong answer

Alex Liu created SPARK-27800: Summary: Example for xor function has a wrong answer Key: SPARK-27800 URL: https://issues.apache.org/jira/browse/SPARK-27800 Project: Spark Issue Type:

[jira] [Created] (SPARK-18786) pySpark SQLContext.getOrCreate(sc) take stopped sparkContext

Alex Liu created SPARK-18786: Summary: pySpark SQLContext.getOrCreate(sc) take stopped sparkContext Key: SPARK-18786 URL: https://issues.apache.org/jira/browse/SPARK-18786 Project: Spark Issue

[jira] [Commented] (SPARK-18050) spark 2.0.1 enable hive throw AlreadyExistsException(message:Database default already exists)

[ https://issues.apache.org/jira/browse/SPARK-18050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15686930#comment-15686930 ] Alex Liu commented on SPARK-18050: -- This error logging msg is very annoying. Is there any workaround to

[jira] [Commented] (SPARK-17767) Spark SQL ExternalCatalog API custom implementation support

[ https://issues.apache.org/jira/browse/SPARK-17767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543539#comment-15543539 ] Alex Liu commented on SPARK-17767: -- It looks good to me. How about hive thrift server, will you patch

[jira] [Commented] (SPARK-12985) Spark Hive thrift server big decimal data issue

[ https://issues.apache.org/jira/browse/SPARK-12985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542909#comment-15542909 ] Alex Liu commented on SPARK-12985: -- It's ok to close it, It could be fixed from Simba side. > Spark

[jira] [Created] (SPARK-17767) Spark SQL ExternalCatalog API custom implementation support

Alex Liu created SPARK-17767: Summary: Spark SQL ExternalCatalog API custom implementation support Key: SPARK-17767 URL: https://issues.apache.org/jira/browse/SPARK-17767 Project: Spark Issue

[jira] [Created] (SPARK-12985) Spark Hive thrift server big decimal data issue

Alex Liu created SPARK-12985: Summary: Spark Hive thrift server big decimal data issue Key: SPARK-12985 URL: https://issues.apache.org/jira/browse/SPARK-12985 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11609) Resolve permanent Hive UDFs

[ https://issues.apache.org/jira/browse/SPARK-11609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15081761#comment-15081761 ] Alex Liu commented on SPARK-11609: -- duplicate as SPARK-12051 > Resolve permanent Hive UDFs >

[jira] [Created] (SPARK-12051) Can't register UDF from Hive thrift server

Alex Liu created SPARK-12051: Summary: Can't register UDF from Hive thrift server Key: SPARK-12051 URL: https://issues.apache.org/jira/browse/SPARK-12051 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11320) DROP TABLE IF EXISTS throws exception if the table does not exist.

[ https://issues.apache.org/jira/browse/SPARK-11320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14974594#comment-14974594 ] Alex Liu commented on SPARK-11320: -- It's a hive problem. > DROP TABLE IF EXISTS throws exception if

[jira] [Created] (SPARK-11320) DROP TABLE IF EXISTS throws exception if the table does not exist.

Alex Liu created SPARK-11320: Summary: DROP TABLE IF EXISTS throws exception if the table does not exist. Key: SPARK-11320 URL: https://issues.apache.org/jira/browse/SPARK-11320 Project: Spark

[jira] [Commented] (SPARK-11067) Spark SQL thrift server fails to handle decimal value

[ https://issues.apache.org/jira/browse/SPARK-11067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14957806#comment-14957806 ] Alex Liu commented on SPARK-11067: -- Let's keep this ticket simply fix the exception. We could implement

[jira] [Commented] (SPARK-11067) Spark SQL thrift server fails to handle decimal value

[ https://issues.apache.org/jira/browse/SPARK-11067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14955341#comment-14955341 ] Alex Liu commented on SPARK-11067: -- [~navis] Can you elaborate what could be done to improve Hive JDBC

[jira] [Created] (SPARK-11067) Spark SQL thrift server fails to handle decimal value

Alex Liu created SPARK-11067: Summary: Spark SQL thrift server fails to handle decimal value Key: SPARK-11067 URL: https://issues.apache.org/jira/browse/SPARK-11067 Project: Spark Issue Type:

[jira] [Commented] (SPARK-10501) support UUID as an atomic type

[ https://issues.apache.org/jira/browse/SPARK-10501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739228#comment-14739228 ] Alex Liu commented on SPARK-10501: -- Cassandra has InetAddress and BigInteger types. Can we add those as

[jira] [Created] (SPARK-10159) Hive 1.3.x GenericUDFDate NPE issue

Alex Liu created SPARK-10159: Summary: Hive 1.3.x GenericUDFDate NPE issue Key: SPARK-10159 URL: https://issues.apache.org/jira/browse/SPARK-10159 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-10159) Hive 1.3.x GenericUDFDate NPE issue

[ https://issues.apache.org/jira/browse/SPARK-10159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated SPARK-10159: - Description: When run sql query with HiveContext, Hive 1.3.x GenericUDFDate NPE issue. The following is

[jira] [Commented] (SPARK-9844) File appender race condition during SparkWorker shutdown

[ https://issues.apache.org/jira/browse/SPARK-9844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14693666#comment-14693666 ] Alex Liu commented on SPARK-9844: - Please provide a pull for it if you could File

[jira] [Created] (SPARK-9845) Add built-in UDF

Alex Liu created SPARK-9845: --- Summary: Add built-in UDF Key: SPARK-9845 URL: https://issues.apache.org/jira/browse/SPARK-9845 Project: Spark Issue Type: Improvement Components: SQL

[jira] [Created] (SPARK-9844) File appender race condition during SparkWorker shutdown

Alex Liu created SPARK-9844: --- Summary: File appender race condition during SparkWorker shutdown Key: SPARK-9844 URL: https://issues.apache.org/jira/browse/SPARK-9844 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-9842) Push down Spark SQL UDF to datasource UDF

Alex Liu created SPARK-9842: --- Summary: Push down Spark SQL UDF to datasource UDF Key: SPARK-9842 URL: https://issues.apache.org/jira/browse/SPARK-9842 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-4300) Race condition during SparkWorker shutdown

[ https://issues.apache.org/jira/browse/SPARK-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682472#comment-14682472 ] Alex Liu commented on SPARK-4300: - I created SPARK-9844 for the unsolved issue. Race

[jira] [Updated] (SPARK-9844) File appender race condition during SparkWorker shutdown

[ https://issues.apache.org/jira/browse/SPARK-9844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated SPARK-9844: Description: We find this issue still exists in 1.3.1 {code} ERROR [Thread-6] 2015-07-28 22:49:57,653

[jira] [Commented] (SPARK-4300) Race condition during SparkWorker shutdown

[ https://issues.apache.org/jira/browse/SPARK-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14645331#comment-14645331 ] Alex Liu commented on SPARK-4300: - We find this issue still exists in 1.3.1 {code} ERROR

[jira] [Created] (SPARK-9269) Add Set to the matching type in ArrayConverter

Alex Liu created SPARK-9269: --- Summary: Add Set to the matching type in ArrayConverter Key: SPARK-9269 URL: https://issues.apache.org/jira/browse/SPARK-9269 Project: Spark Issue Type: Improvement

[jira] [Closed] (SPARK-7095) Pass DataType to source.Filter classes

[ https://issues.apache.org/jira/browse/SPARK-7095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu closed SPARK-7095. --- Resolution: Not A Problem Spark SQL only pushes filter to datasource if the value is the same data type as

[jira] [Created] (SPARK-7095) Pass DataType to source.Filter classes

Alex Liu created SPARK-7095: --- Summary: Pass DataType to source.Filter classes Key: SPARK-7095 URL: https://issues.apache.org/jira/browse/SPARK-7095 Project: Spark Issue Type: Improvement

[jira] [Closed] (SPARK-6745) Develop a general filter function to be used in PrunedFilteredScan and CatalystScan

[ https://issues.apache.org/jira/browse/SPARK-6745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu closed SPARK-6745. --- Resolution: Won't Fix I close it if some one is interested in it, please reopen it. Develop a general

[jira] [Commented] (SPARK-6745) Develop a general filter function to be used in PrunedFilteredScan and CatalystScan

[ https://issues.apache.org/jira/browse/SPARK-6745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14494765#comment-14494765 ] Alex Liu commented on SPARK-6745: - Use Expression.transform method to transform

[jira] [Created] (SPARK-6745) Develop a general filter function to be used in PrunedFilteredScan and CatalystScan

Alex Liu created SPARK-6745: --- Summary: Develop a general filter function to be used in PrunedFilteredScan and CatalystScan Key: SPARK-6745 URL: https://issues.apache.org/jira/browse/SPARK-6745 Project:

[jira] [Updated] (SPARK-6730) Can't have table as identifier in OPTIONS

[ https://issues.apache.org/jira/browse/SPARK-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated SPARK-6730: Description: The following query fails because there is an identifier table in OPTIONS {code} CREATE

[jira] [Created] (SPARK-6730) Can't have table as identifier in OPTIONS

Alex Liu created SPARK-6730: --- Summary: Can't have table as identifier in OPTIONS Key: SPARK-6730 URL: https://issues.apache.org/jira/browse/SPARK-6730 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-6540) Spark SQL thrift server fails to pass settings to following query in the same session

[ https://issues.apache.org/jira/browse/SPARK-6540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated SPARK-6540: Description: e.g. By using beeline set cassandra.username=user; set cassandra.password = pass; show

[jira] [Updated] (SPARK-6540) Spark SQL thrift server fails to pass settings to following query in the same session

[ https://issues.apache.org/jira/browse/SPARK-6540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated SPARK-6540: Description: e.g. By using beeline set cassandra.username=user; set cassandra.password = pass; show

[jira] [Updated] (SPARK-6540) Spark SQL thrift server fails to pass settings to following query in the same session

[ https://issues.apache.org/jira/browse/SPARK-6540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated SPARK-6540: Description: e.g. By using beeline set cassandra.username=user; set cassandra.password = pass; show

[jira] [Commented] (SPARK-6540) Spark SQL thrift server fails to pass settings to following query in the same session

[ https://issues.apache.org/jira/browse/SPARK-6540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381140#comment-14381140 ] Alex Liu commented on SPARK-6540: - Pass session HiveConf to NativeCommand(sql, hiveconf),

[jira] [Commented] (SPARK-5624) Can't find new column

[ https://issues.apache.org/jira/browse/SPARK-5624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14308261#comment-14308261 ] Alex Liu commented on SPARK-5624: - Test it on the latest master branch it doesn't have

[jira] [Created] (SPARK-5622) Add connector/handler hive configuration settings to hive-thrift-server

Alex Liu created SPARK-5622: --- Summary: Add connector/handler hive configuration settings to hive-thrift-server Key: SPARK-5622 URL: https://issues.apache.org/jira/browse/SPARK-5622 Project: Spark

[jira] [Commented] (SPARK-4943) Parsing error for query with table name having dot

[ https://issues.apache.org/jira/browse/SPARK-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14265212#comment-14265212 ] Alex Liu commented on SPARK-4943: - Catalog part of table identifier should be handled by

[jira] [Commented] (SPARK-4943) Parsing error for query with table name having dot

[ https://issues.apache.org/jira/browse/SPARK-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14265224#comment-14265224 ] Alex Liu commented on SPARK-4943: - For each catalog, the configuration settings should

[jira] [Commented] (SPARK-4943) Parsing error for query with table name having dot

[ https://issues.apache.org/jira/browse/SPARK-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14265351#comment-14265351 ] Alex Liu commented on SPARK-4943: - No changes to your approach. Regarding

[jira] [Commented] (SPARK-4943) Parsing error for query with table name having dot

[ https://issues.apache.org/jira/browse/SPARK-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14265089#comment-14265089 ] Alex Liu commented on SPARK-4943: - The approach of {code} case class

[jira] [Commented] (SPARK-4943) Parsing error for query with table name having dot

[ https://issues.apache.org/jira/browse/SPARK-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14263230#comment-14263230 ] Alex Liu commented on SPARK-4943: - Should we also change the signatures of Catalog methods

[jira] [Commented] (SPARK-4943) Parsing error for query with table name having dot

[ https://issues.apache.org/jira/browse/SPARK-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14262442#comment-14262442 ] Alex Liu commented on SPARK-4943: - The seq can be like {code} tableName, databaseName,

[jira] [Created] (SPARK-4943) Parsing error for query with table name having dot

Alex Liu created SPARK-4943: --- Summary: Parsing error for query with table name having dot Key: SPARK-4943 URL: https://issues.apache.org/jira/browse/SPARK-4943 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-4925) Publish Spark SQL hive-thriftserver maven artifact

Alex Liu created SPARK-4925: --- Summary: Publish Spark SQL hive-thriftserver maven artifact Key: SPARK-4925 URL: https://issues.apache.org/jira/browse/SPARK-4925 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4925) Publish Spark SQL hive-thriftserver maven artifact

[ https://issues.apache.org/jira/browse/SPARK-4925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14256405#comment-14256405 ] Alex Liu commented on SPARK-4925: - Our build.xml downloads hive-thriftserver maven

[jira] [Issue Comment Deleted] (SPARK-4345) Spark SQL Hive throws exception when drop a none-exist table

[ https://issues.apache.org/jira/browse/SPARK-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated SPARK-4345: Comment: was deleted (was: Swallow NoSuchObjectException exception when drop a none-exist hive table. pull

[jira] [Commented] (SPARK-4345) Spark SQL Hive throws exception when drop a none-exist table

[ https://issues.apache.org/jira/browse/SPARK-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14213121#comment-14213121 ] Alex Liu commented on SPARK-4345: - It looks like a bug in Hive

[jira] [Resolved] (SPARK-4345) Spark SQL Hive throws exception when drop a none-exist table

[ https://issues.apache.org/jira/browse/SPARK-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu resolved SPARK-4345. - Resolution: Won't Fix Spark SQL Hive throws exception when drop a none-exist table

[jira] [Created] (SPARK-4345) Spark SQL Hive throws exception when drop a none-exist table

Alex Liu created SPARK-4345: --- Summary: Spark SQL Hive throws exception when drop a none-exist table Key: SPARK-4345 URL: https://issues.apache.org/jira/browse/SPARK-4345 Project: Spark Issue

[jira] [Commented] (SPARK-4345) Spark SQL Hive throws exception when drop a none-exist table

[ https://issues.apache.org/jira/browse/SPARK-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14207171#comment-14207171 ] Alex Liu commented on SPARK-4345: - Swallow NoSuchObjectException exception when drop a

[jira] [Created] (SPARK-4300) Race condition during SparkWorker shutdown

Alex Liu created SPARK-4300: --- Summary: Race condition during SparkWorker shutdown Key: SPARK-4300 URL: https://issues.apache.org/jira/browse/SPARK-4300 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-3816) Add configureInputJobPropertiesForStorageHandler to JobConf in SparkHadoopWriter

Alex Liu created SPARK-3816: --- Summary: Add configureInputJobPropertiesForStorageHandler to JobConf in SparkHadoopWriter Key: SPARK-3816 URL: https://issues.apache.org/jira/browse/SPARK-3816 Project: Spark

[jira] [Updated] (SPARK-3816) Add configureInputJobPropertiesForStorageHandler to JobConf in SparkHadoopWriter

[ https://issues.apache.org/jira/browse/SPARK-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated SPARK-3816: Description: It's similar to SPARK-2846. We should add

[jira] [Updated] (SPARK-3816) Add configureOutputJobPropertiesForStorageHandler to JobConf in SparkHadoopWriter

[ https://issues.apache.org/jira/browse/SPARK-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated SPARK-3816: Summary: Add configureOutputJobPropertiesForStorageHandler to JobConf in SparkHadoopWriter (was: Add

[jira] [Created] (SPARK-3823) Spark Hive SQL readColumn is not reset each time for a new query

Alex Liu created SPARK-3823: --- Summary: Spark Hive SQL readColumn is not reset each time for a new query Key: SPARK-3823 URL: https://issues.apache.org/jira/browse/SPARK-3823 Project: Spark Issue

[jira] [Updated] (SPARK-3823) Spark Hive SQL readColumn is not reset each time for a new query

[ https://issues.apache.org/jira/browse/SPARK-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated SPARK-3823: Description: After a few queries running in the same hiveContext, hive.io.file.readcolumn.ids and

[jira] [Created] (SPARK-3109) Sql query with OR condition should be handled above PhysicalOperation layer

Alex Liu created SPARK-3109: --- Summary: Sql query with OR condition should be handled above PhysicalOperation layer Key: SPARK-3109 URL: https://issues.apache.org/jira/browse/SPARK-3109 Project: Spark

[jira] [Commented] (SPARK-2846) Spark SQL hive implementation bypass StorageHandler which breaks any customized StorageHandler

[ https://issues.apache.org/jira/browse/SPARK-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095879#comment-14095879 ] Alex Liu commented on SPARK-2846: - pull @ https://github.com/apache/spark/pull/1927

[jira] [Updated] (SPARK-2846) Add configureInputJobPropertiesForStorageHandler to initialization of job conf

[ https://issues.apache.org/jira/browse/SPARK-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated SPARK-2846: Summary: Add configureInputJobPropertiesForStorageHandler to initialization of job conf (was: Spark SQL

[jira] [Updated] (SPARK-2846) Spark SQL hive implementation bypass StorageHandler which breaks any customized StorageHandler

[ https://issues.apache.org/jira/browse/SPARK-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu updated SPARK-2846: Component/s: SQL Spark SQL hive implementation bypass StorageHandler which breaks any customized

[jira] [Created] (SPARK-2847) SPARK SQL Hive misses SHOW CREATE TABLE command

Alex Liu created SPARK-2847: --- Summary: SPARK SQL Hive misses SHOW CREATE TABLE command Key: SPARK-2847 URL: https://issues.apache.org/jira/browse/SPARK-2847 Project: Spark Issue Type: Improvement