[jira] [Created] (HUPA-111) Using gwt-polymer-elements for theme
dongxu created HUPA-111: --- Summary: Using gwt-polymer-elements for theme Key: HUPA-111 URL: https://issues.apache.org/jira/browse/HUPA-111 Project: James Hupa Issue Type: Improvement Reporter: dongxu Assignee: dongxu Priority: Minor Replace the current theme by polymer-elements -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Resolved] (HUPA-110) All tool buttons are disabled when refreshing some message page.
[ https://issues.apache.org/jira/browse/HUPA-110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dongxu resolved HUPA-110. - Resolution: Fixed Fix Version/s: 0.1 All tool buttons are disabled when refreshing some message page. Key: HUPA-110 URL: https://issues.apache.org/jira/browse/HUPA-110 Project: James Hupa Issue Type: Bug Components: client Affects Versions: 0.1 Reporter: dongxu Assignee: dongxu Fix For: 0.1 When refreshing on some message page, like : http://127.0.0.1:/hupa/Hupa.html#message:INBOX:15205 Even though the tool buttons are active style, but actually they are not clickable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Created] (HUPA-110) All tool buttons are disabled when refreshing some message page.
dongxu created HUPA-110: --- Summary: All tool buttons are disabled when refreshing some message page. Key: HUPA-110 URL: https://issues.apache.org/jira/browse/HUPA-110 Project: James Hupa Issue Type: Bug Components: client Affects Versions: 0.1 Reporter: dongxu Assignee: dongxu When refreshing on some message page, like : http://127.0.0.1:/hupa/Hupa.html#message:INBOX:15205 Even though the tool buttons are active style, but actually they are not clickable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Updated] (SPARK-6644) [SPARK-SQL]when the partition schema does not match table schema(ADD COLUMN), new column value is NULL
[
https://issues.apache.org/jira/browse/SPARK-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dongxu updated SPARK-6644:
--
Description:
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
Some problems have been solved at PR4289
(https://github.com/apache/spark/pull/4289),
but if you add a new column,put new data into the old partition,new column
value is NULL
[According to the following steps]:
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize((1 to 10).map(i = TestData(i,
i.toString))).toDF()
testData.registerTempTable(testData)
//inititi
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value string)
PARTITIONED by (ds string) location '${tmpDir.toURI.toString}' )
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value FROM testData)
// add column to table
sql(ALTER TABLE table_with_partition ADD COLUMNS(key1 string))
sql(ALTER TABLE table_with_partition ADD COLUMNS(destlng double))
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value,'test',1.11 FROM testData)
sql(select * from table_with_partition where ds='1'
).collect().foreach(println)
result :
[1,1,null,null,1]
[2,2,null,null,1]
result we expect:
[1,1,test,1.11,1]
[2,2,test,1.11,1]
This bug will cause the wrong queyr number ,when we query like that:
select count(1) from table_with_partition where key1 is not NULL
was:
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
some problems is solved(https://github.com/apache/spark/pull/4289),
but if you add a new column,put new data into the old partition,new column
value is NULL
[According to the following steps]:
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize((1 to 10).map(i = TestData(i,
i.toString))).toDF()
testData.registerTempTable(testData)
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value string)
PARTITIONED by (ds string) location '${tmpDir.toURI.toString}' )
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value FROM testData)
// add column to table
sql(ALTER TABLE table_with_partition ADD COLUMNS(key1 string))
sql(ALTER TABLE table_with_partition ADD COLUMNS(destlng double))
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value,'test',1.11 FROM testData)
sql(select * from table_with_partition where ds='1'
).collect().foreach(println)
result :
[1,1,null,null,1]
[2,2,null,null,1]
result we expect:
[1,1,test,1.11,1]
[2,2,test,1.11,1]
This bug will cause the wrong queyr number ,when we query like that:
select count(1) from table_with_partition where key1 is not NULL
[SPARK-SQL]when the partition schema does not match table schema(ADD COLUMN),
new column value is NULL
--
Key: SPARK-6644
URL: https://issues.apache.org/jira/browse/SPARK-6644
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.3.0
Reporter: dongxu
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
Some problems have been solved at PR4289
(https://github.com/apache/spark/pull/4289),
but if you add a new column,put new data into the old partition,new column
value is NULL
[According to the following steps]:
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize((1 to 10).map(i =
TestData(i, i.toString))).toDF()
testData.registerTempTable(testData)
//inititi
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value
string) PARTITIONED by (ds string) location '${tmpDir.toURI.toString}' )
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value FROM testData)
// add column to table
sql(ALTER TABLE table_with_partition ADD COLUMNS(key1 string))
sql(ALTER TABLE table_with_partition ADD COLUMNS(destlng double))
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value,'test',1.11 FROM testData)
sql(select * from table_with_partition where ds='1'
).collect().foreach(println)
[jira] [Updated] (SPARK-6644) [SPARK-SQL]when the partition schema does not match table schema(ADD COLUMN), new column value is NULL
[
https://issues.apache.org/jira/browse/SPARK-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dongxu updated SPARK-6644:
--
Description:
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
Some problems have been solved at PR4289
(https://github.com/apache/spark/pull/4289),
but if you add a new column,put new data into the old partition,new column
value is NULL
[According to the following steps]:
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize((1 to 2).map(i = TestData(i,
i.toString))).toDF()
testData.registerTempTable(testData)
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value string)
PARTITIONED by (ds string) location '${tmpDir.toURI.toString}' )
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value FROM testData)
// add column to table
sql(ALTER TABLE table_with_partition ADD COLUMNS(key1 string))
sql(ALTER TABLE table_with_partition ADD COLUMNS(destlng double))
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value,'test',1.11 FROM testData)
sql(select * from table_with_partition where ds='1'
).collect().foreach(println)
result :
[1,1,null,null,1]
[2,2,null,null,1]
result we expect:
[1,1,test,1.11,1]
[2,2,test,1.11,1]
This bug will cause the wrong queyr number ,when we query like that:
select count(1) from table_with_partition where key1 is not NULL
was:
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
Some problems have been solved at PR4289
(https://github.com/apache/spark/pull/4289),
but if you add a new column,put new data into the old partition,new column
value is NULL
[According to the following steps]:
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize((1 to 10).map(i = TestData(i,
i.toString))).toDF()
testData.registerTempTable(testData)
//inititi
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value string)
PARTITIONED by (ds string) location '${tmpDir.toURI.toString}' )
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value FROM testData)
// add column to table
sql(ALTER TABLE table_with_partition ADD COLUMNS(key1 string))
sql(ALTER TABLE table_with_partition ADD COLUMNS(destlng double))
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value,'test',1.11 FROM testData)
sql(select * from table_with_partition where ds='1'
).collect().foreach(println)
result :
[1,1,null,null,1]
[2,2,null,null,1]
result we expect:
[1,1,test,1.11,1]
[2,2,test,1.11,1]
This bug will cause the wrong queyr number ,when we query like that:
select count(1) from table_with_partition where key1 is not NULL
[SPARK-SQL]when the partition schema does not match table schema(ADD COLUMN),
new column value is NULL
--
Key: SPARK-6644
URL: https://issues.apache.org/jira/browse/SPARK-6644
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.3.0
Reporter: dongxu
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
Some problems have been solved at PR4289
(https://github.com/apache/spark/pull/4289),
but if you add a new column,put new data into the old partition,new column
value is NULL
[According to the following steps]:
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize((1 to 2).map(i =
TestData(i, i.toString))).toDF()
testData.registerTempTable(testData)
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value
string) PARTITIONED by (ds string) location '${tmpDir.toURI.toString}' )
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value FROM testData)
// add column to table
sql(ALTER TABLE table_with_partition ADD COLUMNS(key1 string))
sql(ALTER TABLE table_with_partition ADD COLUMNS(destlng double))
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value,'test',1.11 FROM testData)
sql(select * from table_with_partition where ds='1'
[jira] [Updated] (SPARK-6644) [SPARK-SQL]when the partition schema does not match table schema(ADD COLUMN), new column value is NULL
[
https://issues.apache.org/jira/browse/SPARK-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dongxu updated SPARK-6644:
--
Description:
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
Some problems have been solved at PR4289
(https://github.com/apache/spark/pull/4289),
but if you add a new column,put new data into the old partition,new column
value is NULL
[According to the following steps]:
--
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize((1 to 2).map(i = TestData(i,
i.toString))).toDF()
testData.registerTempTable(testData)
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value string)
PARTITIONED by (ds string) location '${tmpDir.toURI.toString}' )
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value FROM testData)
// add column to table
sql(ALTER TABLE table_with_partition ADD COLUMNS(key1 string))
sql(ALTER TABLE table_with_partition ADD COLUMNS(destlng double))
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value,'test',1.11 FROM testData)
sql(select * from table_with_partition where ds='1'
).collect().foreach(println)
-
result:
[1,1,null,null,1]
[2,2,null,null,1]
result we expect:
[1,1,test,1.11,1]
[2,2,test,1.11,1]
This bug will cause the wrong query number ,when we query :
select count(1) from table_with_partition where key1 is not NULL
was:
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
Some problems have been solved at PR4289
(https://github.com/apache/spark/pull/4289),
but if you add a new column,put new data into the old partition,new column
value is NULL
[According to the following steps]:
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize((1 to 2).map(i = TestData(i,
i.toString))).toDF()
testData.registerTempTable(testData)
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value string)
PARTITIONED by (ds string) location '${tmpDir.toURI.toString}' )
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value FROM testData)
// add column to table
sql(ALTER TABLE table_with_partition ADD COLUMNS(key1 string))
sql(ALTER TABLE table_with_partition ADD COLUMNS(destlng double))
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value,'test',1.11 FROM testData)
sql(select * from table_with_partition where ds='1'
).collect().foreach(println)
result :
[1,1,null,null,1]
[2,2,null,null,1]
result we expect:
[1,1,test,1.11,1]
[2,2,test,1.11,1]
This bug will cause the wrong queyr number ,when we query like that:
select count(1) from table_with_partition where key1 is not NULL
[SPARK-SQL]when the partition schema does not match table schema(ADD COLUMN),
new column value is NULL
--
Key: SPARK-6644
URL: https://issues.apache.org/jira/browse/SPARK-6644
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.3.0
Reporter: dongxu
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
Some problems have been solved at PR4289
(https://github.com/apache/spark/pull/4289),
but if you add a new column,put new data into the old partition,new column
value is NULL
[According to the following steps]:
--
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize((1 to 2).map(i =
TestData(i, i.toString))).toDF()
testData.registerTempTable(testData)
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value
string) PARTITIONED by (ds string) location '${tmpDir.toURI.toString}' )
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value FROM testData)
// add column to table
sql(ALTER TABLE
[jira] [Updated] (SPARK-6644) [SPARK-SQL]when the partition schema does not match table schema(ADD COLUMN), new column value is NULL
[
https://issues.apache.org/jira/browse/SPARK-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dongxu updated SPARK-6644:
--
Description:
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
some problems is solved(https://github.com/apache/spark/pull/4289),
but if you add a new column,put new data into the old partition,new column
value is NULL
[According to the following steps]:
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize((1 to 10).map(i = TestData(i,
i.toString))).toDF()
testData.registerTempTable(testData)
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value string)
PARTITIONED by (ds string) location '${tmpDir.toURI.toString}' )
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value FROM testData)
// add column to table
sql(ALTER TABLE table_with_partition ADD COLUMNS(key1 string))
sql(ALTER TABLE table_with_partition ADD COLUMNS(destlng double))
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value,'test',1.11 FROM testData)
sql(select * from table_with_partition where ds='1'
).collect().foreach(println)
result :
[1,1,null,null,1]
[2,2,null,null,1]
result we expect:
[1,1,test,1.11,1]
[2,2,test,1.11,1]
This bug will cause the wrong queyr number ,when we query like that:
select count(1) from table_with_partition where key1 is not NULL
was:
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
some problems is solved(https://github.com/apache/spark/pull/4289),
but if you add a new column,put new data into the old partition,new column
value is NULL
[According to the following steps]:
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize(
(1 to 10).map(i = TestData(i, i.toString))).toDF()
testData.registerTempTable(testData)
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value string)
PARTITIONED by (ds string) location '${tmpDir.toURI.toString}' )
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value FROM testData)
// add column to table
sql(ALTER TABLE table_with_partition ADD COLUMNS(key1 string))
sql(ALTER TABLE table_with_partition ADD COLUMNS(destlng double))
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value,'test',1.11 FROM testData)
sql(select * from table_with_partition where ds='1'
).collect().foreach(println)
result :
[1,1,null,null,1]
[2,2,null,null,1]
result we expect:
[1,1,test,1.11,1]
[2,2,test,1.11,1]
[SPARK-SQL]when the partition schema does not match table schema(ADD COLUMN),
new column value is NULL
--
Key: SPARK-6644
URL: https://issues.apache.org/jira/browse/SPARK-6644
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.3.0
Reporter: dongxu
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
some problems is solved(https://github.com/apache/spark/pull/4289),
but if you add a new column,put new data into the old partition,new column
value is NULL
[According to the following steps]:
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize((1 to 10).map(i =
TestData(i, i.toString))).toDF()
testData.registerTempTable(testData)
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value
string) PARTITIONED by (ds string) location '${tmpDir.toURI.toString}' )
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value FROM testData)
// add column to table
sql(ALTER TABLE table_with_partition ADD COLUMNS(key1 string))
sql(ALTER TABLE table_with_partition ADD COLUMNS(destlng double))
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value,'test',1.11 FROM testData)
sql(select * from table_with_partition where ds='1'
).collect().foreach(println)
result :
[1,1,null,null,1]
[2,2,null,null,1]
result we expect:
[1,1,test,1.11,1]
[2,2,test,1.11,1]
This bug will cause the wrong queyr number ,when we query like that:
select
[jira] [Updated] (SPARK-6644) [SPARK-SQL]when the partition schema does not match table schema(ADD COLUMN), new column value is NULL
[
https://issues.apache.org/jira/browse/SPARK-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dongxu updated SPARK-6644:
--
Description:
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
Some problems have been solved at PR4289
(https://github.com/apache/spark/pull/4289),
but if we add new column, and put new data into the old partition schema,new
column value is NULL
[According to the following steps]:
--
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize((1 to 2).map(i = TestData(i,
i.toString))).toDF()
testData.registerTempTable(testData)
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value string)
PARTITIONED by (ds string) location '${tmpDir.toURI.toString}' )
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value FROM testData)
// add column to table
sql(ALTER TABLE table_with_partition ADD COLUMNS(key1 string))
sql(ALTER TABLE table_with_partition ADD COLUMNS(destlng double))
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value,'test',1.11 FROM testData)
sql(select * from table_with_partition where ds='1'
).collect().foreach(println)
-
result:
[1,1,null,null,1]
[2,2,null,null,1]
result we expect:
[1,1,test,1.11,1]
[2,2,test,1.11,1]
This bug will cause the wrong query number ,when we query :
select count(1) from table_with_partition where key1 is not NULL
was:
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
Some problems have been solved at PR4289
(https://github.com/apache/spark/pull/4289),
but if you add a new column,put new data into the old partition schema,new
column value is NULL
[According to the following steps]:
--
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize((1 to 2).map(i = TestData(i,
i.toString))).toDF()
testData.registerTempTable(testData)
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value string)
PARTITIONED by (ds string) location '${tmpDir.toURI.toString}' )
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value FROM testData)
// add column to table
sql(ALTER TABLE table_with_partition ADD COLUMNS(key1 string))
sql(ALTER TABLE table_with_partition ADD COLUMNS(destlng double))
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value,'test',1.11 FROM testData)
sql(select * from table_with_partition where ds='1'
).collect().foreach(println)
-
result:
[1,1,null,null,1]
[2,2,null,null,1]
result we expect:
[1,1,test,1.11,1]
[2,2,test,1.11,1]
This bug will cause the wrong query number ,when we query :
select count(1) from table_with_partition where key1 is not NULL
[SPARK-SQL]when the partition schema does not match table schema(ADD COLUMN),
new column value is NULL
--
Key: SPARK-6644
URL: https://issues.apache.org/jira/browse/SPARK-6644
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.3.0
Reporter: dongxu
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
Some problems have been solved at PR4289
(https://github.com/apache/spark/pull/4289),
but if we add new column, and put new data into the old partition schema,new
column value is NULL
[According to the following steps]:
--
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize((1 to 2).map(i =
TestData(i, i.toString))).toDF()
testData.registerTempTable(testData)
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value
string) PARTITIONED by
[jira] [Updated] (SPARK-6644) [SPARK-SQL]when the partition schema does not match table schema(ADD COLUMN), new column value is NULL
[
https://issues.apache.org/jira/browse/SPARK-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dongxu updated SPARK-6644:
--
Summary: [SPARK-SQL]when the partition schema does not match table
schema(ADD COLUMN), new column value is NULL (was: [SPARK-SQL]when the
partition schema does not match table schema(ADD COLUMN), new column is NULL)
[SPARK-SQL]when the partition schema does not match table schema(ADD COLUMN),
new column value is NULL
--
Key: SPARK-6644
URL: https://issues.apache.org/jira/browse/SPARK-6644
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.3.0
Reporter: dongxu
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
some problems is solved(https://github.com/apache/spark/pull/4289),
but if you add a new column,put new data into the old partition,new column
value is NULL
[According to the following steps]:
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize(
(1 to 10).map(i = TestData(i, i.toString))).toDF()
testData.registerTempTable(testData)
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value
string) PARTITIONED by (ds string) location '${tmpDir.toURI.toString}' )
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value FROM testData)
// add column to table
sql(ALTER TABLE table_with_partition ADD COLUMNS(key1 string))
sql(ALTER TABLE table_with_partition ADD COLUMNS(destlng double))
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value,'test',1.11 FROM testData)
sql(select * from table_with_partition where ds='1'
).collect().foreach(println)
result :
[1,1,null,null,1]
[2,2,null,null,1]
result we expect:
[1,1,test,1.11,1]
[2,2,test,1.11,1]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-6644) [SPARK-SQL]when the partition schema does not match table schema(ADD COLUMN), new column is NULL
dongxu created SPARK-6644:
-
Summary: [SPARK-SQL]when the partition schema does not match table
schema(ADD COLUMN), new column is NULL
Key: SPARK-6644
URL: https://issues.apache.org/jira/browse/SPARK-6644
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.3.0
Reporter: dongxu
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
some problems is solved(https://github.com/apache/spark/pull/4289),
but if you add a new column,put new data into the old partition,new column
value is NULL
[According to the following steps]:
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize(
(1 to 10).map(i = TestData(i, i.toString))).toDF()
testData.registerTempTable(testData)
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value string)
PARTITIONED by (ds string) location '${tmpDir.toURI.toString}' )
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value FROM testData)
// add column to table
sql(ALTER TABLE table_with_partition ADD COLUMNS(key1 string))
sql(ALTER TABLE table_with_partition ADD COLUMNS(destlng double))
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value,'test',1.11 FROM testData)
sql(select * from table_with_partition where ds='1'
).collect().foreach(println)
result :
[1,1,null,null,1]
[2,2,null,null,1]
result we expect:
[1,1,test,1.11,1]
[2,2,test,1.11,1]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-6644) [SPARK-SQL]when the partition schema does not match table schema(ADD COLUMN), new column value is NULL
[
https://issues.apache.org/jira/browse/SPARK-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dongxu updated SPARK-6644:
--
Description:
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
Some problems have been solved at PR4289
(https://github.com/apache/spark/pull/4289),
but if you add a new column,put new data into the old partition schema,new
column value is NULL
[According to the following steps]:
--
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize((1 to 2).map(i = TestData(i,
i.toString))).toDF()
testData.registerTempTable(testData)
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value string)
PARTITIONED by (ds string) location '${tmpDir.toURI.toString}' )
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value FROM testData)
// add column to table
sql(ALTER TABLE table_with_partition ADD COLUMNS(key1 string))
sql(ALTER TABLE table_with_partition ADD COLUMNS(destlng double))
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value,'test',1.11 FROM testData)
sql(select * from table_with_partition where ds='1'
).collect().foreach(println)
-
result:
[1,1,null,null,1]
[2,2,null,null,1]
result we expect:
[1,1,test,1.11,1]
[2,2,test,1.11,1]
This bug will cause the wrong query number ,when we query :
select count(1) from table_with_partition where key1 is not NULL
was:
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
Some problems have been solved at PR4289
(https://github.com/apache/spark/pull/4289),
but if you add a new column,put new data into the old partition,new column
value is NULL
[According to the following steps]:
--
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize((1 to 2).map(i = TestData(i,
i.toString))).toDF()
testData.registerTempTable(testData)
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value string)
PARTITIONED by (ds string) location '${tmpDir.toURI.toString}' )
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value FROM testData)
// add column to table
sql(ALTER TABLE table_with_partition ADD COLUMNS(key1 string))
sql(ALTER TABLE table_with_partition ADD COLUMNS(destlng double))
sql(INSERT OVERWRITE TABLE table_with_partition partition (ds='1') SELECT
key,value,'test',1.11 FROM testData)
sql(select * from table_with_partition where ds='1'
).collect().foreach(println)
-
result:
[1,1,null,null,1]
[2,2,null,null,1]
result we expect:
[1,1,test,1.11,1]
[2,2,test,1.11,1]
This bug will cause the wrong query number ,when we query :
select count(1) from table_with_partition where key1 is not NULL
[SPARK-SQL]when the partition schema does not match table schema(ADD COLUMN),
new column value is NULL
--
Key: SPARK-6644
URL: https://issues.apache.org/jira/browse/SPARK-6644
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.3.0
Reporter: dongxu
In hive,the schema of partition may be difference from the table schema. For
example, we add new column. When we use spark-sql to query the data of
partition which schema is difference from the table schema.
Some problems have been solved at PR4289
(https://github.com/apache/spark/pull/4289),
but if you add a new column,put new data into the old partition schema,new
column value is NULL
[According to the following steps]:
--
case class TestData(key: Int, value: String)
val testData = TestHive.sparkContext.parallelize((1 to 2).map(i =
TestData(i, i.toString))).toDF()
testData.registerTempTable(testData)
sql(DROP TABLE IF EXISTS table_with_partition )
sql(sCREATE TABLE IF NOT EXISTS table_with_partition(key int,value
string) PARTITIONED by (ds
[jira] [Updated] (SPARK-5616) Add examples for PySpark API
[ https://issues.apache.org/jira/browse/SPARK-5616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dongxu updated SPARK-5616: -- Description: PySpark API examples are less than Spark scala API. For example: 1.Broadcast: how to use broadcast operation API 2.Module: how to import a other python file in zip file. Add more examples for freshman who wanna use PySpark. was: PySpark API examples are less than Spark scala API. For example: 1.Boardcast: how to use boardcast operation APi 2.Module: how to import a other python file in zip file. Add more examples for freshman who wanna use PySpark. Add examples for PySpark API Key: SPARK-5616 URL: https://issues.apache.org/jira/browse/SPARK-5616 Project: Spark Issue Type: Improvement Components: PySpark Reporter: dongxu Priority: Minor Labels: examples, pyspark, python PySpark API examples are less than Spark scala API. For example: 1.Broadcast: how to use broadcast operation API 2.Module: how to import a other python file in zip file. Add more examples for freshman who wanna use PySpark. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-5616) Add examples for PySpark API
dongxu created SPARK-5616: - Summary: Add examples for PySpark API Key: SPARK-5616 URL: https://issues.apache.org/jira/browse/SPARK-5616 Project: Spark Issue Type: Improvement Components: PySpark Reporter: dongxu Fix For: 1.3.0 PySpark API examples are less than Spark scala API. For example: 1.Boardcast: how to use boardcast operation APi 2.Module: how to import a other python file in zip file. Add more examples for freshman who wanna use PySpark. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-5527) Add standalone document configiration to explain how to make cluster conf file consistency
[ https://issues.apache.org/jira/browse/SPARK-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dongxu updated SPARK-5527: -- Description: We must make all node conf file consistent when we start our standalone cluster. For example, we set SPARK_WORKER_INSTANCES=2 to start 2 worker on each machine. I see this code at $SPARK_HOME/sbin/spark-daemon.sh if [ $SPARK_MASTER != ]; then echo rsync from $SPARK_MASTER rsync -a -e ssh --delete --exclude=.svn --exclude='logs/*' --exclude='contrib/hod/logs/*' $SPARK_MASTER/ $SPARK_HOME fi I think we better mention it at document . was: We need make all node conf file consistency when we start our standalone cluster. For example, we set SPARK_WORKER_INSTANCES=2 to start 2 worker on each machine. I see this code at $SPARK_HOME/sbin/spark-daemon.sh if [ $SPARK_MASTER != ]; then echo rsync from $SPARK_MASTER rsync -a -e ssh --delete --exclude=.svn --exclude='logs/*' --exclude='contrib/hod/logs/*' $SPARK_MASTER/ $SPARK_HOME fi I think we better mention it at document . Add standalone document configiration to explain how to make cluster conf file consistency --- Key: SPARK-5527 URL: https://issues.apache.org/jira/browse/SPARK-5527 Project: Spark Issue Type: Documentation Components: Documentation Affects Versions: 1.2.0, 1.3.0 Reporter: dongxu Priority: Minor Labels: docuentation, starter Original Estimate: 10m Remaining Estimate: 10m We must make all node conf file consistent when we start our standalone cluster. For example, we set SPARK_WORKER_INSTANCES=2 to start 2 worker on each machine. I see this code at $SPARK_HOME/sbin/spark-daemon.sh if [ $SPARK_MASTER != ]; then echo rsync from $SPARK_MASTER rsync -a -e ssh --delete --exclude=.svn --exclude='logs/*' --exclude='contrib/hod/logs/*' $SPARK_MASTER/ $SPARK_HOME fi I think we better mention it at document . -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-5527) Improvements to standalone doc. - how to make cluster conf file consistency
[ https://issues.apache.org/jira/browse/SPARK-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dongxu updated SPARK-5527: -- Summary: Improvements to standalone doc. - how to make cluster conf file consistency (was: Add standalone document configiration to explain how to make cluster conf file consistency) Improvements to standalone doc. - how to make cluster conf file consistency --- Key: SPARK-5527 URL: https://issues.apache.org/jira/browse/SPARK-5527 Project: Spark Issue Type: Documentation Components: Documentation Affects Versions: 1.2.0, 1.3.0 Reporter: dongxu Priority: Minor Labels: docuentation, starter Original Estimate: 10m Remaining Estimate: 10m We must make all node conf file consistent when we start our standalone cluster. For example, we set SPARK_WORKER_INSTANCES=2 to start 2 worker on each machine. I see this code at $SPARK_HOME/sbin/spark-daemon.sh if [ $SPARK_MASTER != ]; then echo rsync from $SPARK_MASTER rsync -a -e ssh --delete --exclude=.svn --exclude='logs/*' --exclude='contrib/hod/logs/*' $SPARK_MASTER/ $SPARK_HOME fi I think we better mention it at document . -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-5527) Improvements to standalone doc. - how to sync cluster conf file
[ https://issues.apache.org/jira/browse/SPARK-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dongxu updated SPARK-5527: -- Summary: Improvements to standalone doc. - how to sync cluster conf file (was: Improvements to standalone doc. - how to make cluster conf file consistency) Improvements to standalone doc. - how to sync cluster conf file --- Key: SPARK-5527 URL: https://issues.apache.org/jira/browse/SPARK-5527 Project: Spark Issue Type: Documentation Components: Documentation Affects Versions: 1.2.0, 1.3.0 Reporter: dongxu Priority: Minor Labels: docuentation, starter Original Estimate: 10m Remaining Estimate: 10m We must make all node conf file consistent when we start our standalone cluster. For example, we set SPARK_WORKER_INSTANCES=2 to start 2 worker on each machine. I see this code at $SPARK_HOME/sbin/spark-daemon.sh if [ $SPARK_MASTER != ]; then echo rsync from $SPARK_MASTER rsync -a -e ssh --delete --exclude=.svn --exclude='logs/*' --exclude='contrib/hod/logs/*' $SPARK_MASTER/ $SPARK_HOME fi I think we better mention it at document . -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-5527) Add standalone document configiration to explain how to make cluster conf file consistency
dongxu created SPARK-5527: - Summary: Add standalone document configiration to explain how to make cluster conf file consistency Key: SPARK-5527 URL: https://issues.apache.org/jira/browse/SPARK-5527 Project: Spark Issue Type: Documentation Components: Documentation Affects Versions: 1.2.0, 1.3.0 Reporter: dongxu Priority: Minor We need make all node conf file consistency when we start our standalone cluster. For example, we set SPARK_WORKER_INSTANCES=2 to start 2 worker on each machine. I see this code at $SPARK_HOME/sbin/spark-daemon.sh if [ $SPARK_MASTER != ]; then echo rsync from $SPARK_MASTER rsync -a -e ssh --delete --exclude=.svn --exclude='logs/*' --exclude='contrib/hod/logs/*' $SPARK_MASTER/ $SPARK_HOME fi I think we better mention it at document . -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-4201) Can't use concat() on partition column in where condition (Hive compatibility problem)
dongxu created SPARK-4201: - Summary: Can't use concat() on partition column in where condition (Hive compatibility problem) Key: SPARK-4201 URL: https://issues.apache.org/jira/browse/SPARK-4201 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 1.1.0, 1.0.0 Environment: Hive 0.12+hadoop 2.4/hadoop 2.2 +spark 1.1 Reporter: dongxu Priority: Minor The team used hive to query,we try to move it to spark-sql. when I search sentences like that. select count(1) from gulfstream_day_driver_base_2 where concat(year,month,day) = '20140929'; It can't work ,but it work well in hive. I have to rewrite the sql to select count(1) from gulfstream_day_driver_base_2 where year = 2014 and month = 09 day= 29. There are some error logs. 14/11/03 15:05:03 ERROR SparkSQLDriver: Failed in [select count(1) from gulfstream_day_driver_base_2 where concat(year,month,day) = '20140929'] org.apache.spark.sql.catalyst.errors.package$TreeNodeException: execute, tree: Aggregate false, [], [SUM(PartialCount#1390L) AS c_0#1337L] Exchange SinglePartition Aggregate true, [], [COUNT(1) AS PartialCount#1390L] HiveTableScan [], (MetastoreRelation default, gulfstream_day_driver_base_2, None), Some((HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFConcat(year#1339,month#1340,day#1341) = 20140929)) at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:47) at org.apache.spark.sql.execution.Aggregate.execute(Aggregate.scala:126) at org.apache.spark.sql.hive.HiveContext$QueryExecution.toRdd$lzycompute(HiveContext.scala:360) at org.apache.spark.sql.hive.HiveContext$QueryExecution.toRdd(HiveContext.scala:360) at org.apache.spark.sql.hive.HiveContext$QueryExecution.stringResult(HiveContext.scala:415) at org.apache.spark.sql.hive.thriftserver.SparkSQLDriver.run(SparkSQLDriver.scala:59) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processCmd(SparkSQLCLIDriver.scala:291) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:226) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:328) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: execute, tree: Exchange SinglePartition Aggregate true, [], [COUNT(1) AS PartialCount#1390L] HiveTableScan [], (MetastoreRelation default, gulfstream_day_driver_base_2, None), Some((HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFConcat(year#1339,month#1340,day#1341) = 20140929)) at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:47) at org.apache.spark.sql.execution.Exchange.execute(Exchange.scala:44) at org.apache.spark.sql.execution.Aggregate$$anonfun$execute$1.apply(Aggregate.scala:128) at org.apache.spark.sql.execution.Aggregate$$anonfun$execute$1.apply(Aggregate.scala:127) at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:46) ... 16 more Caused by: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: execute, tree: Aggregate true, [], [COUNT(1) AS PartialCount#1390L] HiveTableScan [], (MetastoreRelation default, gulfstream_day_driver_base_2, None), Some((HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFConcat(year#1339,month#1340,day#1341) = 20140929)) at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:47) at org.apache.spark.sql.execution.Aggregate.execute(Aggregate.scala:126) at org.apache.spark.sql.execution.Exchange$$anonfun$execute$1.apply(Exchange.scala:86) at org.apache.spark.sql.execution.Exchange$$anonfun$execute$1.apply(Exchange.scala:45) at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:46) ... 20 more Caused by: org.apache.spark.SparkException: Task not serializable at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:166) at org.apache.spark.util.ClosureCleaner$.clean(ClosureCleaner.scala:158) at org.apache.spark.SparkContext.clean(SparkContext.scala:1242) at
[jira] [Updated] (SPARK-4201) Can't use concat() on partition column in where condition (Hive compatibility problem)
[ https://issues.apache.org/jira/browse/SPARK-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dongxu updated SPARK-4201: -- Description: The team used hive to query,we try to move it to spark-sql. when I search sentences like that. select count(1) from gulfstream_day_driver_base_2 where concat(year,month,day) = '20140929'; It can't work ,but it work well in hive. I have to rewrite the sql to select count(1) from gulfstream_day_driver_base_2 where year = 2014 and month = 09 day= 29. There are some error log. 14/11/03 15:05:03 ERROR SparkSQLDriver: Failed in [select count(1) from gulfstream_day_driver_base_2 where concat(year,month,day) = '20140929'] org.apache.spark.sql.catalyst.errors.package$TreeNodeException: execute, tree: Aggregate false, [], [SUM(PartialCount#1390L) AS c_0#1337L] Exchange SinglePartition Aggregate true, [], [COUNT(1) AS PartialCount#1390L] HiveTableScan [], (MetastoreRelation default, gulfstream_day_driver_base_2, None), Some((HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFConcat(year#1339,month#1340,day#1341) = 20140929)) at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:47) at org.apache.spark.sql.execution.Aggregate.execute(Aggregate.scala:126) at org.apache.spark.sql.hive.HiveContext$QueryExecution.toRdd$lzycompute(HiveContext.scala:360) at org.apache.spark.sql.hive.HiveContext$QueryExecution.toRdd(HiveContext.scala:360) at org.apache.spark.sql.hive.HiveContext$QueryExecution.stringResult(HiveContext.scala:415) at org.apache.spark.sql.hive.thriftserver.SparkSQLDriver.run(SparkSQLDriver.scala:59) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processCmd(SparkSQLCLIDriver.scala:291) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:226) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:328) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: execute, tree: Exchange SinglePartition Aggregate true, [], [COUNT(1) AS PartialCount#1390L] HiveTableScan [], (MetastoreRelation default, gulfstream_day_driver_base_2, None), Some((HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFConcat(year#1339,month#1340,day#1341) = 20140929)) at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:47) at org.apache.spark.sql.execution.Exchange.execute(Exchange.scala:44) at org.apache.spark.sql.execution.Aggregate$$anonfun$execute$1.apply(Aggregate.scala:128) at org.apache.spark.sql.execution.Aggregate$$anonfun$execute$1.apply(Aggregate.scala:127) at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:46) ... 16 more Caused by: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: execute, tree: Aggregate true, [], [COUNT(1) AS PartialCount#1390L] HiveTableScan [], (MetastoreRelation default, gulfstream_day_driver_base_2, None), Some((HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFConcat(year#1339,month#1340,day#1341) = 20140929)) at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:47) at org.apache.spark.sql.execution.Aggregate.execute(Aggregate.scala:126) at org.apache.spark.sql.execution.Exchange$$anonfun$execute$1.apply(Exchange.scala:86) at org.apache.spark.sql.execution.Exchange$$anonfun$execute$1.apply(Exchange.scala:45) at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:46) ... 20 more Caused by: org.apache.spark.SparkException: Task not serializable at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:166) at org.apache.spark.util.ClosureCleaner$.clean(ClosureCleaner.scala:158) at org.apache.spark.SparkContext.clean(SparkContext.scala:1242) at org.apache.spark.rdd.RDD.mapPartitions(RDD.scala:597) at org.apache.spark.sql.execution.Aggregate$$anonfun$execute$1.apply(Aggregate.scala:128) at org.apache.spark.sql.execution.Aggregate$$anonfun$execute$1.apply(Aggregate.scala:127) at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:46)
[jira] [Updated] (ROL-2046) Sorry! We couldn't find your document
[ https://issues.apache.org/jira/browse/ROL-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dongxu updated ROL-2046: Description: After we navigate to the design tab of setting when logging in, we can choose the 'basic mobile' for theme.Then we preview the mobile theme and click the 'View Mobile Weblog', this complain would take place, with complaining that Sorry! We couldn't find your document. [USER SOLUTION]: If you want to go back to normal, you have to remove the cookie key roller_user_request_type under roller domain in your browser. However, we have to say it is a bug. was: After we navigate to the design tab of setting when logging in, we can choose the 'basic mobile' for theme.Then we preview the mobile theme and click the 'View Mobile Weblog', this complain would take place, with complaining that Sorry! We couldn't find your document. [USER SOLUTION], you have to remove the cookie key roller_user_request_type under roller domain in your browser. However, we have to say it is a bug. Sorry! We couldn't find your document - Key: ROL-2046 URL: https://issues.apache.org/jira/browse/ROL-2046 Project: Apache Roller Issue Type: Bug Components: Themes and Macros, User Interface - General Environment: Tomcat 7.0.55 installed on Ubuntu 12.04 with MySQL 5.1 Reporter: dongxu Assignee: Roller Unassigned After we navigate to the design tab of setting when logging in, we can choose the 'basic mobile' for theme.Then we preview the mobile theme and click the 'View Mobile Weblog', this complain would take place, with complaining that Sorry! We couldn't find your document. [USER SOLUTION]: If you want to go back to normal, you have to remove the cookie key roller_user_request_type under roller domain in your browser. However, we have to say it is a bug. -- This message was sent by Atlassian JIRA (v6.2#6252)
