[jira] [Comment Edited] (SPARK-13832) TPC-DS Query 36 fails with Parser error

[ https://issues.apache.org/jira/browse/SPARK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15196377#comment-15196377 ] Xin Wu edited comment on SPARK-13832 at 3/15/16 10:41 PM: -- Trying this query in

[jira] [Commented] (SPARK-13832) TPC-DS Query 36 fails with Parser error

[ https://issues.apache.org/jira/browse/SPARK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15196377#comment-15196377 ] Xin Wu commented on SPARK-13832: Trying this query in Spark 2.0 and I needed to change grouping__id to

[jira] [Issue Comment Deleted] (SPARK-13832) TPC-DS Query 36 fails with Parser error

[ https://issues.apache.org/jira/browse/SPARK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Wu updated SPARK-13832: --- Comment: was deleted (was: Hi Roy, I will validate and look into this one. ) > TPC-DS Query 36 fails with

[jira] [Commented] (SPARK-13832) TPC-DS Query 36 fails with Parser error

[ https://issues.apache.org/jira/browse/SPARK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15196020#comment-15196020 ] Xin Wu commented on SPARK-13832: Hi Roy, I will validate and look into this one. > TPC-DS Query 36

[jira] [Comment Edited] (SPARK-9761) Inconsistent metadata handling with ALTER TABLE

[ https://issues.apache.org/jira/browse/SPARK-9761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15012655#comment-15012655 ] Xin Wu edited comment on SPARK-9761 at 11/19/15 2:31 AM: - One thing I notice is

[jira] [Commented] (SPARK-9761) Inconsistent metadata handling with ALTER TABLE

[ https://issues.apache.org/jira/browse/SPARK-9761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15012655#comment-15012655 ] Xin Wu commented on SPARK-9761: --- One thing I notice is that if I create the table explicitly before letting

[jira] [Commented] (SPARK-9761) Inconsistent metadata handling with ALTER TABLE

[ https://issues.apache.org/jira/browse/SPARK-9761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15009947#comment-15009947 ] Xin Wu commented on SPARK-9761: --- I can recreate this with the following code {code} val conf = new

[jira] [Issue Comment Deleted] (SPARK-10673) spark.sql.hive.verifyPartitionPath Attempts to Verify Unregistered Partitions

[ https://issues.apache.org/jira/browse/SPARK-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Wu updated SPARK-10673: --- Comment: was deleted (was: The fix is being tested.. will submit PR shortly.) >

[jira] [Issue Comment Deleted] (SPARK-10673) spark.sql.hive.verifyPartitionPath Attempts to Verify Unregistered Partitions

[ https://issues.apache.org/jira/browse/SPARK-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Wu updated SPARK-10673: --- Comment: was deleted (was: if the default is false, {code} if (!sc.conf.verifyPartitionPath) {

[jira] [Issue Comment Deleted] (SPARK-10673) spark.sql.hive.verifyPartitionPath Attempts to Verify Unregistered Partitions

[ https://issues.apache.org/jira/browse/SPARK-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Wu updated SPARK-10673: --- Comment: was deleted (was: I will look into this one. ) > spark.sql.hive.verifyPartitionPath Attempts to

[jira] [Commented] (SPARK-11748) Result is null after alter column name of table stored as Parquet

[ https://issues.apache.org/jira/browse/SPARK-11748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15007828#comment-15007828 ] Xin Wu commented on SPARK-11748: I can recreate it for parquet format.. but not for orc or default text

[jira] [Commented] (SPARK-10673) spark.sql.hive.verifyPartitionPath Attempts to Verify Unregistered Partitions

[ https://issues.apache.org/jira/browse/SPARK-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005249#comment-15005249 ] Xin Wu commented on SPARK-10673: if the default is false, {code} if (!sc.conf.verifyPartitionPath) {

[jira] [Comment Edited] (SPARK-10673) spark.sql.hive.verifyPartitionPath Attempts to Verify Unregistered Partitions

[ https://issues.apache.org/jira/browse/SPARK-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005249#comment-15005249 ] Xin Wu edited comment on SPARK-10673 at 11/14/15 8:19 AM: -- if the default is

[jira] [Commented] (SPARK-10673) spark.sql.hive.verifyPartitionPath Attempts to Verify Unregistered Partitions

[ https://issues.apache.org/jira/browse/SPARK-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005577#comment-15005577 ] Xin Wu commented on SPARK-10673: The fix is being tested.. will submit PR shortly. >

[jira] [Commented] (SPARK-10673) spark.sql.hive.verifyPartitionPath Attempts to Verify Unregistered Partitions

[ https://issues.apache.org/jira/browse/SPARK-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15004240#comment-15004240 ] Xin Wu commented on SPARK-10673: I will look into this one. > spark.sql.hive.verifyPartitionPath

[jira] [Commented] (SPARK-10848) Applied JSON Schema Works for json RDD but not when loading json file

[ https://issues.apache.org/jira/browse/SPARK-10848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003178#comment-15003178 ] Xin Wu commented on SPARK-10848: With rdd created in the application, as following: {code} val jsonRdd =

[jira] [Commented] (SPARK-11683) DataFrameWriter.insertInto() fails on non-default schema

[ https://issues.apache.org/jira/browse/SPARK-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003251#comment-15003251 ] Xin Wu commented on SPARK-11683: It seems like in 1.6.0, this issue is resolved: {code} scala> val df =

[jira] [Issue Comment Deleted] (SPARK-11657) Bad Dataframe data read from parquet

[ https://issues.apache.org/jira/browse/SPARK-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Wu updated SPARK-11657: --- Comment: was deleted (was: I tried this sample data as local file mode. and it seems working to me. Have

[jira] [Commented] (SPARK-11657) Bad Dataframe data read from parquet

[ https://issues.apache.org/jira/browse/SPARK-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15000607#comment-15000607 ] Xin Wu commented on SPARK-11657: I tried this sample data as local file mode. and it seems working to me.

[jira] [Commented] (SPARK-10848) Applied JSON Schema Works for json RDD but not when loading json file

[ https://issues.apache.org/jira/browse/SPARK-10848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001404#comment-15001404 ] Xin Wu commented on SPARK-10848: I can recreate this as describe. Looking into the code. > Applied JSON

[jira] [Commented] (SPARK-10848) Applied JSON Schema Works for json RDD but not when loading json file

[ https://issues.apache.org/jira/browse/SPARK-10848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001667#comment-15001667 ] Xin Wu commented on SPARK-10848: Debugging through the code and found that in ResolvedDataSource.apply

[jira] [Commented] (SPARK-10848) Applied JSON Schema Works for json RDD but not when loading json file

[ https://issues.apache.org/jira/browse/SPARK-10848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001685#comment-15001685 ] Xin Wu commented on SPARK-10848: One case I found where there is inconsistency between using asNullable

[jira] [Commented] (SPARK-11356) Option to refresh information about parquet partitions

[ https://issues.apache.org/jira/browse/SPARK-11356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997651#comment-14997651 ] Xin Wu commented on SPARK-11356: [~maver1ck] If I only execute {code}sqlContext.sql("select * from t

[jira] [Commented] (SPARK-9465) Could not read parquet table after recreating it with the same table name

[ https://issues.apache.org/jira/browse/SPARK-9465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14995771#comment-14995771 ] Xin Wu commented on SPARK-9465: --- [~stanzhai] Would you be able to recreate on the later releases? > Could

[jira] [Updated] (SPARK-11522) input_file_name() returns "" for external tables

[ https://issues.apache.org/jira/browse/SPARK-11522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Wu updated SPARK-11522: --- Description: Given an external table definition where the data consists of many CSV files,

[jira] [Commented] (SPARK-11522) input_file_name() returns "" for external tables

[ https://issues.apache.org/jira/browse/SPARK-11522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994763#comment-14994763 ] Xin Wu commented on SPARK-11522: running full test against the fix now. will submit the PR soon. >

[jira] [Commented] (SPARK-11522) input_file_name() returns "" for external tables

[ https://issues.apache.org/jira/browse/SPARK-11522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992183#comment-14992183 ] Xin Wu commented on SPARK-11522: I can recreate this issue with simpler case on 1.6.0 code base too.

[jira] [Commented] (SPARK-11522) input_file_name() returns "" for external tables

[ https://issues.apache.org/jira/browse/SPARK-11522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14993212#comment-14993212 ] Xin Wu commented on SPARK-11522: Actually this seems not related to whether the table is created as

[jira] [Commented] (SPARK-9465) Could not read parquet table after recreating it with the same table name

[ https://issues.apache.org/jira/browse/SPARK-9465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990890#comment-14990890 ] Xin Wu commented on SPARK-9465: --- I tried on both 1.5.1 and 1.6.0, I can not recreate the issue {code}

[jira] [Commented] (SPARK-9465) Could not read parquet table after recreating it with the same table name

[ https://issues.apache.org/jira/browse/SPARK-9465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990909#comment-14990909 ] Xin Wu commented on SPARK-9465: --- I can not recreate the issue on 1.5.1 or 1.6.0.. {code} scala>

[jira] [Issue Comment Deleted] (SPARK-9465) Could not read parquet table after recreating it with the same table name

[ https://issues.apache.org/jira/browse/SPARK-9465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Wu updated SPARK-9465: -- Comment: was deleted (was: I can not recreate the issue on 1.5.1 or 1.6.0.. {code} scala>

[jira] [Commented] (SPARK-11246) [1.5] Table cache for Parquet broken in 1.5

[ https://issues.apache.org/jira/browse/SPARK-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980652#comment-14980652 ] Xin Wu commented on SPARK-11246: [~yhuai] Thank you! > [1.5] Table cache for Parquet broken in 1.5 >

[jira] [Commented] (SPARK-10046) Hive warehouse dir not set in current directory when not providing hive-site.xml

[ https://issues.apache.org/jira/browse/SPARK-10046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981331#comment-14981331 ] Xin Wu commented on SPARK-10046: Ok. Let me update the doc then. > Hive warehouse dir not set in current

[jira] [Commented] (SPARK-10046) Hive warehouse dir not set in current directory when not providing hive-site.xml

[ https://issues.apache.org/jira/browse/SPARK-10046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981282#comment-14981282 ] Xin Wu commented on SPARK-10046: The default HiveConf without hite-site.xml has

[jira] [Issue Comment Deleted] (SPARK-10046) Hive warehouse dir not set in current directory when not providing hive-site.xml

[ https://issues.apache.org/jira/browse/SPARK-10046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Wu updated SPARK-10046: --- Comment: was deleted (was: pull request is created: https://github.com/apache/spark/pull/9365) > Hive

[jira] [Commented] (SPARK-10046) Hive warehouse dir not set in current directory when not providing hive-site.xml

[ https://issues.apache.org/jira/browse/SPARK-10046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981439#comment-14981439 ] Xin Wu commented on SPARK-10046: pull request is created: https://github.com/apache/spark/pull/9365 >

[jira] [Commented] (SPARK-11246) [1.5] Table cache for Parquet broken in 1.5

[ https://issues.apache.org/jira/browse/SPARK-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970071#comment-14970071 ] Xin Wu commented on SPARK-11246: Root cause found, fix is being tested. will submit PR shortly. > [1.5]

[jira] [Commented] (SPARK-11229) NPE in JoinedRow.isNullAt when spark.shuffle.memoryFraction=0

[ https://issues.apache.org/jira/browse/SPARK-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967694#comment-14967694 ] Xin Wu commented on SPARK-11229: Here is what I tried: 1. start spark-shell with command

[jira] [Commented] (SPARK-11229) NPE in JoinedRow.isNullAt when spark.shuffle.memoryFraction=0

[ https://issues.apache.org/jira/browse/SPARK-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967811#comment-14967811 ] Xin Wu commented on SPARK-11229: I am able to recreate it on 1.5.1 {code} scala> data02.show 15/10/21

[jira] [Commented] (SPARK-11229) NPE in JoinedRow.isNullAt when spark.shuffle.memoryFraction=0

[ https://issues.apache.org/jira/browse/SPARK-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968089#comment-14968089 ] Xin Wu commented on SPARK-11229: [~marmbrus] Please take a look if this is safe to close as fixed?

[jira] [Commented] (SPARK-11246) [1.5] Table cache for Parquet broken in 1.5

[ https://issues.apache.org/jira/browse/SPARK-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968293#comment-14968293 ] Xin Wu commented on SPARK-11246: I am able to recreate this issue on 1.6.0 with hive 1.2.1.. I am looking

[jira] [Commented] (SPARK-10747) add support for window specification to include how NULLS are ordered

[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14956094#comment-14956094 ] Xin Wu commented on SPARK-10747: I ran this query on the released Hive 1.2.1 version, and this is not

[jira] [Commented] (SPARK-10505) windowed form of count ( star ) fails with No handler for udf class

[ https://issues.apache.org/jira/browse/SPARK-10505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14953580#comment-14953580 ] Xin Wu commented on SPARK-10505: [~the6campbells] I also tried this on Spark 1.5.1 and it is not

[jira] [Commented] (SPARK-10505) windowed form of count ( star ) fails with No handler for udf class

[ https://issues.apache.org/jira/browse/SPARK-10505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945791#comment-14945791 ] Xin Wu commented on SPARK-10505: [~the6campbells], I am trying to recreate the issue with master branch

[jira] [Commented] (SPARK-10505) windowed form of count ( star ) fails with No handler for udf class

[ https://issues.apache.org/jira/browse/SPARK-10505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940675#comment-14940675 ] Xin Wu commented on SPARK-10505: This error is triggered in function

<    1   2