[jira] [Updated] (SPARK-16914) NodeManager crash when spark are registering executor infomartion into leveldb

2016-08-05 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-16914: Description: {noformat} Stack: [0x7fb5b53de000,0x7fb5b54df000], sp=0x7fb5b54dcba8, free

[jira] [Updated] (SPARK-16914) NodeManager crash when spark are registering executor infomartion into leveldb

2016-08-05 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-16914: Description: {noformat} Stack: [0x7fb5b53de000,0x7fb5b54df000], sp=0x7fb5b54dcba8, free

[jira] [Commented] (SPARK-11990) DataFrame recompute UDF in some situation.

2015-11-25 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15028222#comment-15028222 ] Yi Tian commented on SPARK-11990: - [~rekhajoshm], yes, that's it > DataFrame recompute UDF in some

[jira] [Commented] (SPARK-11990) DataFrame recompute UDF in some situation.

2015-11-25 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15026876#comment-15026876 ] Yi Tian commented on SPARK-11990: - I found TRACE logs like this: {code} === Applying Rule

[jira] [Commented] (SPARK-11990) DataFrame recompute UDF in some situation.

2015-11-25 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15026882#comment-15026882 ] Yi Tian commented on SPARK-11990: - [~liancheng] [~marmbrus] This problem will waste multiple resources

[jira] [Created] (SPARK-11990) DataFrame recompute UDF in some situation.

2015-11-25 Thread Yi Tian (JIRA)
Yi Tian created SPARK-11990: --- Summary: DataFrame recompute UDF in some situation. Key: SPARK-11990 URL: https://issues.apache.org/jira/browse/SPARK-11990 Project: Spark Issue Type: Bug

[jira] [Issue Comment Deleted] (SPARK-9141) DataFrame recomputed instead of using cached parent.

2015-11-24 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-9141: --- Comment: was deleted (was: When u run these codes, you can find the UDF was called 20 times, not the

[jira] [Comment Edited] (SPARK-9141) DataFrame recomputed instead of using cached parent.

2015-11-24 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15026105#comment-15026105 ] Yi Tian edited comment on SPARK-9141 at 11/25/15 7:00 AM: -- [~marmbrus] Here is my

[jira] [Comment Edited] (SPARK-9141) DataFrame recomputed instead of using cached parent.

2015-11-24 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15026105#comment-15026105 ] Yi Tian edited comment on SPARK-9141 at 11/25/15 7:24 AM: -- [~marmbrus] Sorry,

[jira] [Commented] (SPARK-9141) DataFrame recomputed instead of using cached parent.

2015-11-24 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15026105#comment-15026105 ] Yi Tian commented on SPARK-9141: [~marmbrus] Here is my codes: {quote} val rdd = sc.parallelize(1 to

[jira] [Comment Edited] (SPARK-9141) DataFrame recomputed instead of using cached parent.

2015-11-24 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15026105#comment-15026105 ] Yi Tian edited comment on SPARK-9141 at 11/25/15 4:00 AM: -- [~marmbrus] Here is my

[jira] [Commented] (SPARK-9141) DataFrame recomputed instead of using cached parent.

2015-11-24 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15024314#comment-15024314 ] Yi Tian commented on SPARK-9141: [~marmbrus] It seems that your patch did not fix this problem. I found

[jira] [Updated] (SPARK-10610) Using AppName instead of AppId in the name of all metrics

2015-10-23 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-10610: Summary: Using AppName instead of AppId in the name of all metrics (was: Using AppName instead AppId in

[jira] [Created] (SPARK-10610) Using AppName instead AppId in the name of all metrics

2015-09-15 Thread Yi Tian (JIRA)
Yi Tian created SPARK-10610: --- Summary: Using AppName instead AppId in the name of all metrics Key: SPARK-10610 URL: https://issues.apache.org/jira/browse/SPARK-10610 Project: Spark Issue Type: New

[jira] [Created] (SPARK-8552) Using incorrect database in multiple sessions

2015-06-22 Thread Yi Tian (JIRA)
Yi Tian created SPARK-8552: -- Summary: Using incorrect database in multiple sessions Key: SPARK-8552 URL: https://issues.apache.org/jira/browse/SPARK-8552 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-8552) Using incorrect database in multiple sessions

2015-06-22 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-8552: --- Priority: Critical (was: Blocker) Using incorrect database in multiple sessions

[jira] [Created] (SPARK-7519) There are some bugs in the new WebUI in thrift server

2015-05-10 Thread Yi Tian (JIRA)
Yi Tian created SPARK-7519: -- Summary: There are some bugs in the new WebUI in thrift server Key: SPARK-7519 URL: https://issues.apache.org/jira/browse/SPARK-7519 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-6926) Word2Vec's transform method throw IllegalStateException if a word not in vocabulary, but findSynonyms(word: String, num: Int) method neither try catch exception nor thro

2015-04-15 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-6926: --- Description: {code} val doc = sc.textFile(/xxx/segrestxt, args(3).toInt).map(line = line.split(

[jira] [Updated] (SPARK-6926) Word2Vec's transform method throw IllegalStateException if a word not in vocabulary, but findSynonyms(word: String, num: Int) method neither try catch exception nor thro

2015-04-15 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-6926: --- Description: {code} val doc = sc.textFile(/xxx/segrestxt, args(3).toInt).map(line = line.split( ).toSeq)

[jira] [Updated] (SPARK-6926) Word2Vec's transform method throw IllegalStateException if a word not in vocabulary, but findSynonyms(word: String, num: Int) method neither try catch exception nor thro

2015-04-15 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-6926: --- Description: {code} val doc = sc.textFile(/xxx/segrestxt, args(3).toInt).map(line = line.split(

[jira] [Commented] (SPARK-6221) SparkSQL should support auto merging output files

2015-03-09 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354166#comment-14354166 ] Yi Tian commented on SPARK-6221: [~srowen], thanks for your comment. I think we should add

[jira] [Comment Edited] (SPARK-6221) SparkSQL should support auto merging output files

2015-03-09 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354166#comment-14354166 ] Yi Tian edited comment on SPARK-6221 at 3/10/15 3:25 AM: -

[jira] [Created] (SPARK-6221) SparkSQL should support auto mergiing

2015-03-09 Thread Yi Tian (JIRA)
Yi Tian created SPARK-6221: -- Summary: SparkSQL should support auto mergiing Key: SPARK-6221 URL: https://issues.apache.org/jira/browse/SPARK-6221 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-6221) SparkSQL should support auto merging output files

2015-03-09 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-6221: --- Summary: SparkSQL should support auto merging output files (was: SparkSQL should support auto mergiing)

[jira] [Commented] (SPARK-6061) File source dstream can not include the old file which timestamp is before the system time

2015-03-04 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348160#comment-14348160 ] Yi Tian commented on SPARK-6061: In spark 1.2.1, when you set the {{newFilesOnly}} to

[jira] [Commented] (SPARK-6061) File source dstream can not include the old file which timestamp is before the system time

2015-03-04 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348253#comment-14348253 ] Yi Tian commented on SPARK-6061: [~tdas] Could you explain about

[jira] [Commented] (SPARK-3365) Failure to save Lists to Parquet

2015-02-12 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14317768#comment-14317768 ] Yi Tian commented on SPARK-3365: The reason is Spark generated wrong schema for type

[jira] [Commented] (SPARK-5792) hive udfs like get_json_object and json_tuple doesnot work in spark 1.2.0

2015-02-12 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319618#comment-14319618 ] Yi Tian commented on SPARK-5792: Can you provide more information about this issue? Like:

[jira] [Updated] (SPARK-3688) LogicalPlan can't resolve column correctlly

2015-02-10 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-3688: --- Description: How to reproduce this problem: {code} CREATE TABLE t1(x INT); CREATE TABLE t2(a STRUCTx: INT, k

[jira] [Commented] (SPARK-5446) Parquet column pruning should work for Map and Struct

2015-01-28 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14294925#comment-14294925 ] Yi Tian commented on SPARK-5446: Is it related to

[jira] [Updated] (SPARK-5100) Spark Thrift server monitor page

2015-01-08 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-5100: --- Attachment: (was: Spark Thrift-server monitor page.pdf) Spark Thrift server monitor page

[jira] [Updated] (SPARK-5100) Spark Thrift server monitor page

2015-01-08 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-5100: --- Attachment: Spark Thrift-server monitor page.pdf design doc Spark Thrift server monitor page

[jira] [Updated] (SPARK-5100) Spark Thrift server monitor page

2015-01-06 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-5100: --- Attachment: prototype-screenshot.png Spark Thrift server monitor page

[jira] [Updated] (SPARK-5100) Spark Thrift server monitor page

2015-01-06 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-5100: --- Attachment: (was: Spark Thrift-server monitor page.pdf) Spark Thrift server monitor page

[jira] [Updated] (SPARK-5100) Spark Thrift server monitor page

2015-01-06 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-5100: --- Attachment: Spark Thrift-server monitor page.pdf design doc Spark Thrift server monitor page

[jira] [Commented] (SPARK-4817) [streaming]Print the specified number of data and handle all of the elements in RDD

2014-12-10 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242193#comment-14242193 ] Yi Tian commented on SPARK-4817: Hi, [~sowen] I think the main idea of [~surq] is * For

[jira] [Created] (SPARK-4483) Optimization about reduce memory costs during the HashOuterJoin

2014-11-18 Thread Yi Tian (JIRA)
Yi Tian created SPARK-4483: -- Summary: Optimization about reduce memory costs during the HashOuterJoin Key: SPARK-4483 URL: https://issues.apache.org/jira/browse/SPARK-4483 Project: Spark Issue

[jira] [Updated] (SPARK-4483) Optimization about reduce memory costs during the HashOuterJoin

2014-11-18 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-4483: --- Description: In {{HashOuterJoin.scala}}, spark read data from both side of join operation before zip them

[jira] [Commented] (SPARK-3687) Spark hang while processing more than 100 sequence files

2014-10-01 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14154893#comment-14154893 ] Yi Tian commented on SPARK-3687: yes Spark hang while processing more than 100 sequence

[jira] [Commented] (SPARK-3687) Spark hang while processing more than 100 sequence files

2014-09-29 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151683#comment-14151683 ] Yi Tian commented on SPARK-3687: The stack you print is the worker process. I think you

[jira] [Commented] (SPARK-3324) YARN module has nonstandard structure which cause compile error In IntelliJ

2014-09-27 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150873#comment-14150873 ] Yi Tian commented on SPARK-3324: I think the deeper problem is the codes in common and

[jira] [Created] (SPARK-3688) LogicalPlan can't resolve column correctlly

2014-09-25 Thread Yi Tian (JIRA)
Yi Tian created SPARK-3688: -- Summary: LogicalPlan can't resolve column correctlly Key: SPARK-3688 URL: https://issues.apache.org/jira/browse/SPARK-3688 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3688) LogicalPlan can't resolve column correctlly

2014-09-25 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14147520#comment-14147520 ] Yi Tian commented on SPARK-3688: As we know, the hive support complex colunm datatype like

[jira] [Updated] (SPARK-3688) LogicalPlan can't resolve column correctlly

2014-09-25 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-3688: --- Description: How to reproduce this problem: create a table: {code} create table test (a string, b string);

[jira] [Commented] (SPARK-3324) YARN module has nonstandard structure which cause compile error In IntelliJ

2014-08-31 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116782#comment-14116782 ] Yi Tian commented on SPARK-3324: Thanks [~srowen] explain that. In my beginning idea, I'd

[jira] [Commented] (SPARK-3324) YARN module has nonstandard structure which cause compile error In IntelliJ

2014-08-31 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116785#comment-14116785 ] Yi Tian commented on SPARK-3324: BTW, [~pwendell] there is another problem for IntelliJ.

[jira] [Created] (SPARK-3324) YARN module has nonstandard structure which cause compile error In IntelliJ

2014-08-30 Thread Yi Tian (JIRA)
Yi Tian created SPARK-3324: -- Summary: YARN module has nonstandard structure which cause compile error In IntelliJ Key: SPARK-3324 URL: https://issues.apache.org/jira/browse/SPARK-3324 Project: Spark

[jira] [Updated] (SPARK-3324) YARN module has nonstandard structure which cause compile error In IntelliJ

2014-08-30 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Tian updated SPARK-3324: --- Description: The YARN module has nonstandard path structure like: {code} ${SPARK_HOME} |--yarn

[jira] [Commented] (SPARK-2087) Clean Multi-user semantics for thrift JDBC/ODBC server.

2014-08-25 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110026#comment-14110026 ] Yi Tian commented on SPARK-2087: You mean the CACHE TABLE ... AS SELECT... syntax will

[jira] [Commented] (SPARK-2087) Clean Multi-user semantics for thrift JDBC/ODBC server.

2014-08-18 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14101541#comment-14101541 ] Yi Tian commented on SPARK-2087: What is the temporary table mean for the thrift server?

[jira] [Created] (SPARK-2817) add show create table support

2014-08-03 Thread Yi Tian (JIRA)
Yi Tian created SPARK-2817: -- Summary: add show create table support Key: SPARK-2817 URL: https://issues.apache.org/jira/browse/SPARK-2817 Project: Spark Issue Type: Bug Components: SQL