[jira] [Commented] (SPARK-19416) Dataset.schema is inconsistent with Dataset in handling columns with periods

[ https://issues.apache.org/jira/browse/SPARK-19416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859401#comment-15859401 ] Thomas Sebastian commented on SPARK-19416: -- [~josephkb] If I understand correctly the consistent

[jira] [Commented] (SPARK-19473) Several DataFrame Methods still fail with dot in column names

[ https://issues.apache.org/jira/browse/SPARK-19473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859135#comment-15859135 ] Thomas Sebastian commented on SPARK-19473: -- As mentioned in the PR:

[jira] [Commented] (SPARK-18120) QueryExecutionListener method doesnt' get executed for DataFrameWriter methods

[ https://issues.apache.org/jira/browse/SPARK-18120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15830307#comment-15830307 ] Thomas Sebastian commented on SPARK-18120: -- Hi [~salilsurendran] Are you working on this, please

[jira] [Commented] (SPARK-19059) Unable to retrieve data from a parquet table whose name starts with underscore

[ https://issues.apache.org/jira/browse/SPARK-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828559#comment-15828559 ] Thomas Sebastian commented on SPARK-19059: -- Me and [~jayadevan.m] working on it. > Unable to

[jira] [Commented] (SPARK-18536) Failed to save to hive table when case class with empty field

[ https://issues.apache.org/jira/browse/SPARK-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15708442#comment-15708442 ] Thomas Sebastian commented on SPARK-18536: -- [~pin_zhang] and [~rxin] In this case, what we need

[jira] [Comment Edited] (SPARK-18097) Can't drop a table from Hive if the schema is corrupt

[ https://issues.apache.org/jira/browse/SPARK-18097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654685#comment-15654685 ] Thomas Sebastian edited comment on SPARK-18097 at 11/30/16 11:39 AM: -

[jira] [Created] (SPARK-18626) Concurrent write to table fails from spark

Thomas Sebastian created SPARK-18626: Summary: Concurrent write to table fails from spark Key: SPARK-18626 URL: https://issues.apache.org/jira/browse/SPARK-18626 Project: Spark Issue

[jira] [Commented] (SPARK-18120) QueryExecutionListener method doesnt' get executed for DataFrameWriter methods

[ https://issues.apache.org/jira/browse/SPARK-18120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15699523#comment-15699523 ] Thomas Sebastian commented on SPARK-18120: -- [~salilsurendran] [~rxin] [~jayadevan.m] Is it

[jira] [Comment Edited] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15698420#comment-15698420 ] Thomas Sebastian edited comment on SPARK-15396 at 11/26/16 7:54 PM:

[jira] [Comment Edited] (SPARK-15034) Use the value of spark.sql.warehouse.dir as the warehouse location instead of using hive.metastore.warehouse.dir

[ https://issues.apache.org/jira/browse/SPARK-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15698424#comment-15698424 ] Thomas Sebastian edited comment on SPARK-15034 at 11/26/16 7:54 PM:

[jira] [Comment Edited] (SPARK-15034) Use the value of spark.sql.warehouse.dir as the warehouse location instead of using hive.metastore.warehouse.dir

[ https://issues.apache.org/jira/browse/SPARK-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15698424#comment-15698424 ] Thomas Sebastian edited comment on SPARK-15034 at 11/26/16 7:53 PM:

[jira] [Commented] (SPARK-15034) Use the value of spark.sql.warehouse.dir as the warehouse location instead of using hive.metastore.warehouse.dir

[ https://issues.apache.org/jira/browse/SPARK-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15698424#comment-15698424 ] Thomas Sebastian commented on SPARK-15034: -- Hi [~yhuai] Could you suggest to resolve this: I

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15698420#comment-15698420 ] Thomas Sebastian commented on SPARK-15396: -- Hi, [~yizishou] and [~redlighter] I am facing the

[jira] [Commented] (SPARK-18527) UDAFPercentile (bigint, array) needs explicity cast to double

[ https://issues.apache.org/jira/browse/SPARK-18527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15696598#comment-15696598 ] Thomas Sebastian commented on SPARK-18527: -- I am interested to work on this. > UDAFPercentile

[jira] [Commented] (SPARK-18543) SaveAsTable(CTAS) using overwrite could change table definition

[ https://issues.apache.org/jira/browse/SPARK-18543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15696586#comment-15696586 ] Thomas Sebastian commented on SPARK-18543: -- I would like to take a look at this fix, if you have

[jira] [Commented] (SPARK-18097) Can't drop a table from Hive if the schema is corrupt

[ https://issues.apache.org/jira/browse/SPARK-18097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654685#comment-15654685 ] Thomas Sebastian commented on SPARK-18097: -- But, if we blindly delete the table, just on

[jira] [Comment Edited] (SPARK-18097) Can't drop a table from Hive if the schema is corrupt

[ https://issues.apache.org/jira/browse/SPARK-18097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654685#comment-15654685 ] Thomas Sebastian edited comment on SPARK-18097 at 11/10/16 5:56 PM:

[jira] [Comment Edited] (SPARK-18352) Parse normal, multi-line JSON files (not just JSON Lines)

[ https://issues.apache.org/jira/browse/SPARK-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15649982#comment-15649982 ] Thomas Sebastian edited comment on SPARK-18352 at 11/9/16 6:49 AM: --- Hi

[jira] [Commented] (SPARK-18352) Parse normal, multi-line JSON files (not just JSON Lines)

[ https://issues.apache.org/jira/browse/SPARK-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15649982#comment-15649982 ] Thomas Sebastian commented on SPARK-18352: -- Hi Reynold, So, do you mean that stream API need not

[jira] [Commented] (SPARK-17334) Provide management tools for broadcasted variables

[ https://issues.apache.org/jira/browse/SPARK-17334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15647392#comment-15647392 ] Thomas Sebastian commented on SPARK-17334: -- This looks to be a good feature. I would take a look

[jira] [Commented] (SPARK-16753) Spark SQL doesn't handle skewed dataset joins properly

[ https://issues.apache.org/jira/browse/SPARK-16753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406348#comment-15406348 ] Thomas Sebastian commented on SPARK-16753: -- [~jurriaanpruis]try a repartition before your

[jira] [Commented] (SPARK-16827) Query with Join produces excessive amount of shuffle data

[ https://issues.apache.org/jira/browse/SPARK-16827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406328#comment-15406328 ] Thomas Sebastian commented on SPARK-16827: -- I understand what you have mentioned is w.r.t to the

[jira] [Commented] (SPARK-12941) Spark-SQL JDBC Oracle dialect fails to map string datatypes to Oracle VARCHAR datatype

[ https://issues.apache.org/jira/browse/SPARK-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157410#comment-15157410 ] Thomas Sebastian commented on SPARK-12941: -- Submitting the PR with [~jayadevan.m] > Spark-SQL

[jira] [Commented] (SPARK-13260) count(*) does not work with CSV data source

[ https://issues.apache.org/jira/browse/SPARK-13260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15142295#comment-15142295 ] Thomas Sebastian commented on SPARK-13260: -- HI [~falaki] Could you give more details about this

[jira] [Commented] (SPARK-13197) When trying to select from the data frame which contains the columns with . in it, it is throwing exception.

[ https://issues.apache.org/jira/browse/SPARK-13197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132515#comment-15132515 ] Thomas Sebastian commented on SPARK-13197: -- Working on the code fix [~jayadevan.m] > When

[jira] [Commented] (SPARK-12982) SQLContext: temporary table registration does not accept valid identifier

[ https://issues.apache.org/jira/browse/SPARK-12982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15130471#comment-15130471 ] Thomas Sebastian commented on SPARK-12982: -- Adding changes in the SQLContext.scala and testing

[jira] [Commented] (SPARK-12982) SQLContext: temporary table registration does not accept valid identifier

[ https://issues.apache.org/jira/browse/SPARK-12982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15130478#comment-15130478 ] Thomas Sebastian commented on SPARK-12982: -- One main difference for this with the description of

[jira] [Commented] (SPARK-12982) SQLContext: temporary table registration does not accept valid identifier

[ https://issues.apache.org/jira/browse/SPARK-12982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123403#comment-15123403 ] Thomas Sebastian commented on SPARK-12982: -- Need to make change in the SQLConext.scala, to

[jira] [Comment Edited] (SPARK-12982) SQLContext: temporary table registration does not accept valid identifier

[ https://issues.apache.org/jira/browse/SPARK-12982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123403#comment-15123403 ] Thomas Sebastian edited comment on SPARK-12982 at 1/29/16 12:08 PM:

[jira] [Comment Edited] (SPARK-12982) SQLContext: temporary table registration does not accept valid identifier

[ https://issues.apache.org/jira/browse/SPARK-12982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123248#comment-15123248 ] Thomas Sebastian edited comment on SPARK-12982 at 1/29/16 9:30 AM: ---

[jira] [Commented] (SPARK-12982) SQLContext: temporary table registration does not accept valid identifier

[ https://issues.apache.org/jira/browse/SPARK-12982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123248#comment-15123248 ] Thomas Sebastian commented on SPARK-12982: -- @Reynold Xin @Michael Armbrust Shall we add the

[jira] [Commented] (SPARK-12941) Spark-SQL JDBC Oracle dialect fails to map string datatypes to Oracle VARCHAR datatype

[ https://issues.apache.org/jira/browse/SPARK-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115716#comment-15115716 ] Thomas Sebastian commented on SPARK-12941: -- Added a pull request

[jira] [Issue Comment Deleted] (SPARK-12941) Spark-SQL JDBC Oracle dialect fails to map string datatypes to Oracle VARCHAR datatype

[ https://issues.apache.org/jira/browse/SPARK-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Sebastian updated SPARK-12941: - Comment: was deleted (was: Added a pull request

[jira] [Commented] (SPARK-12987) Drop fails when columns contain dots

[ https://issues.apache.org/jira/browse/SPARK-12987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116589#comment-15116589 ] Thomas Sebastian commented on SPARK-12987: -- I am able to replicate the issue. Looking at it. >

[jira] [Commented] (SPARK-12941) Spark-SQL JDBC Oracle dialect fails to map string datatypes to Oracle VARCHAR datatype

[ https://issues.apache.org/jira/browse/SPARK-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15114860#comment-15114860 ] Thomas Sebastian commented on SPARK-12941: -- Sure. Working on it. > Spark-SQL JDBC Oracle

[jira] [Commented] (SPARK-12941) Spark-SQL JDBC Oracle dialect fails to map string datatypes to Oracle VARCHAR datatype

[ https://issues.apache.org/jira/browse/SPARK-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110124#comment-15110124 ] Thomas Sebastian commented on SPARK-12941: -- [~jayadevan.m] and [~jpoblete] As per the details,

[jira] [Commented] (SPARK-11139) Make SparkContext.stop() exception-safe

[ https://issues.apache.org/jira/browse/SPARK-11139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15105519#comment-15105519 ] Thomas Sebastian commented on SPARK-11139: -- I have re-opened the issue SPARK-11137 , for this

[jira] [Reopened] (SPARK-11137) Make StreamingContext.stop() exception-safe

[ https://issues.apache.org/jira/browse/SPARK-11137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Sebastian reopened SPARK-11137: -- Re-opening this issue based on the discussion at

[jira] [Commented] (SPARK-12577) better support of parentheses in partition by and order by clause of window function's over clause

[ https://issues.apache.org/jira/browse/SPARK-12577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15083590#comment-15083590 ] Thomas Sebastian commented on SPARK-12577: -- Hi Reynold, Would you share some thoughts on how did

[jira] [Comment Edited] (SPARK-12577) better support of parentheses in partition by and order by clause of window function's over clause

[ https://issues.apache.org/jira/browse/SPARK-12577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15083590#comment-15083590 ] Thomas Sebastian edited comment on SPARK-12577 at 1/5/16 7:41 PM: -- Hi

[jira] [Comment Edited] (SPARK-12533) hiveContext.table() throws the wrong exception

[ https://issues.apache.org/jira/browse/SPARK-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15076330#comment-15076330 ] Thomas Sebastian edited comment on SPARK-12533 at 1/1/16 4:43 PM: --

[jira] [Commented] (SPARK-12533) hiveContext.table() throws the wrong exception

[ https://issues.apache.org/jira/browse/SPARK-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15076330#comment-15076330 ] Thomas Sebastian commented on SPARK-12533: -- [~jayadevan.m], it is added. [~marmbrus] , The

[jira] [Comment Edited] (SPARK-12533) hiveContext.table() throws the wrong exception

[ https://issues.apache.org/jira/browse/SPARK-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15075451#comment-15075451 ] Thomas Sebastian edited comment on SPARK-12533 at 12/31/15 5:52 AM:

[jira] [Commented] (SPARK-12533) hiveContext.table() throws the wrong exception

[ https://issues.apache.org/jira/browse/SPARK-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15075278#comment-15075278 ] Thomas Sebastian commented on SPARK-12533: -- Hi Michael, I was able to replicate the issue. I am

[jira] [Commented] (SPARK-12533) hiveContext.table() throws the wrong exception

[ https://issues.apache.org/jira/browse/SPARK-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15075451#comment-15075451 ] Thomas Sebastian commented on SPARK-12533: -- Submitted the pull request > hiveContext.table()

[jira] [Commented] (SPARK-12533) hiveContext.table() throws the wrong exception

[ https://issues.apache.org/jira/browse/SPARK-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072447#comment-15072447 ] Thomas Sebastian commented on SPARK-12533: -- Hi Michael,Can you tell the scenario to replicate

[jira] [Commented] (SPARK-12437) Reserved words (like table) throws error when writing a data frame to JDBC

[ https://issues.apache.org/jira/browse/SPARK-12437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065295#comment-15065295 ] Thomas Sebastian commented on SPARK-12437: -- I looked at the redshift similar problem; and I