[jira] [Created] (HAWQ-852) Quey hang when retrieve data from view defined using CTE when filter using like is used
Ruilong Huo created HAWQ-852:
Summary: Quey hang when retrieve data from view defined using CTE
when filter using like is used
Key: HAWQ-852
URL: https://issues.apache.org/jira/browse/HAWQ-852
Project: Apache HAWQ
Issue Type: Bug
Components: Core, Optimizer, Query Execution
Reporter: Ruilong Huo
Assignee: Amr El-Helw
It hangs to retrieve data from view defined using CTE when filter using like is
used. Here are the steps to reproduce:
Step 1: prepare schema and data by running attached world.sql
{noformat}
psql -a -d postgres -f world.sql > world.out 2>&1
{noformat}
Step 2: create view defined using CTE by running attached
{noformat}
create view view_with_shared_scans as
(
with longlivingregions as
(
select FOO.*,count(distinct language) as "lang_count"
from(
select
sum(population) as "REGION_POP",
sum(gnp) as "REGION_GNP",
avg(lifeexpectancy) as "REGION_LIFETIME",region
from
country
group by region
) FOO,countrylanguage,country
where
country.code = countrylanguage.countrycode
and FOO.region = country.region
group by
FOO.region,foo."REGION_POP",foo."REGION_GNP",foo."REGION_LIFETIME"),
denseregions as
(
select FOO.*,count(distinct language) as "lang_count",
sum(surfacearea) as "REGION_SURFACE_AREA"
from(
select
sum(population) as "REGION_POP",
sum(gnp) as "REGION_GNP",
region
from
country
group by region
) FOO,countrylanguage,country
where
country.code = countrylanguage.countrycode
and FOO.region = country.region
and FOO."REGION_POP" != 0
group by
FOO.region,foo."REGION_POP",foo."REGION_GNP"
order by sum(surfacearea)/foo."REGION_POP" desc),
allcountrystats as
( select country.code,country.name,count(distinct city.id) CITY_CNT,
count(distinct countrylanguage.language) LANG_CNT
from country,city,countrylanguage
where country.code = city.countrycode
and country.code = countrylanguage.countrycode
group by country.code,country.name
)
select allcountrystats.CITY_CNT,allcountrystats.LANG_CNT,allcountrystats.name,
"REGION_SURFACE_AREA","REGION_LIFETIME",longlivingregions."REGION_POP",longlivingregions.lang_count,longlivingregions."REGION_GNP",longlivingregions.region
from longlivingregions,denseregions,allcountrystats,country
where longlivingregions.region = denseregions.region and allcountrystats.code =
country.code and country.region = longlivingregions.region
and country.indepyear between 1800 and 1850
UNION ALL
select allcountrystats.CITY_CNT,allcountrystats.LANG_CNT,allcountrystats.name,
"REGION_SURFACE_AREA","REGION_LIFETIME",longlivingregions."REGION_POP",longlivingregions.lang_count,longlivingregions."REGION_GNP",longlivingregions.region
from longlivingregions,denseregions,allcountrystats,country
where longlivingregions.region = denseregions.region and allcountrystats.code =
country.code and country.region = longlivingregions.region
and country.indepyear between 1850 and 1900
UNION ALL
select allcountrystats.CITY_CNT,allcountrystats.LANG_CNT,allcountrystats.name,
"REGION_SURFACE_AREA","REGION_LIFETIME",longlivingregions."REGION_POP",longlivingregions.lang_count,longlivingregions."REGION_GNP",longlivingregions.region
from longlivingregions,denseregions,allcountrystats,country
where longlivingregions.region = denseregions.region and allcountrystats.code =
country.code and country.region = longlivingregions.region
and country.indepyear > 1900
);
{noformat}
Step 3: retrieve all data from the view succeed
{noformat}
select * from view_with_shared_scans;
city_cnt | lang_cnt | name |
REGION_SURFACE_AREA | REGION_LIFETIME | REGION_POP | lang_count | REGION_GNP |
region
--+--+---+-+--++++---
49 | 12 | Canada|
2.36342e+08 | 75.816948242 | 309632000 | 18 | 9111890.00 | North
America
58 |8 | Italy |
5.8452e+06 | 76.5285720825195 | 144674200 | 22 | 2012289.00 | Southern
Europe
29 |6 | Romania |
2.14732e+08 | 69.925422363 | 307026000 | 28 | 659980.00 | Eastern
Europe
5 |2 | United Arab Emirates |
1.01537e+07 | 70.571329074 | 188380700 | 21 | 677260.00 | Middle
East
1 |2 | Bahrain |
1.01537e+07 | 70.571329074 | 188380700 | 21 | 677260.00 | Middle
East
14 |3 | Israel|
1.01537e+07 | 70.571329074 | 188380700 | 21 | 677260.00 | Middle
East
2
[jira] [Assigned] (HAWQ-852) Quey hang when retrieve data from view defined using CTE when filter using like is used
[
https://issues.apache.org/jira/browse/HAWQ-852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruilong Huo reassigned HAWQ-852:
Assignee: Ruilong Huo (was: Amr El-Helw)
> Quey hang when retrieve data from view defined using CTE when filter using
> like is used
> ---
>
> Key: HAWQ-852
> URL: https://issues.apache.org/jira/browse/HAWQ-852
> Project: Apache HAWQ
> Issue Type: Bug
> Components: Core, Optimizer, Query Execution
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
>
> It hangs to retrieve data from view defined using CTE when filter using like
> is used. Here are the steps to reproduce:
> Step 1: prepare schema and data by running attached world.sql
> {noformat}
> psql -a -d postgres -f world.sql > world.out 2>&1
> {noformat}
> Step 2: create view defined using CTE by running attached
> {noformat}
> create view view_with_shared_scans as
> (
> with longlivingregions as
> (
> select FOO.*,count(distinct language) as "lang_count"
> from(
> select
>sum(population) as "REGION_POP",
>sum(gnp) as "REGION_GNP",
>avg(lifeexpectancy) as "REGION_LIFETIME",region
> from
> country
> group by region
> ) FOO,countrylanguage,country
> where
>country.code = countrylanguage.countrycode
>and FOO.region = country.region
> group by
> FOO.region,foo."REGION_POP",foo."REGION_GNP",foo."REGION_LIFETIME"),
> denseregions as
> (
> select FOO.*,count(distinct language) as "lang_count",
>sum(surfacearea) as "REGION_SURFACE_AREA"
> from(
> select
>sum(population) as "REGION_POP",
>sum(gnp) as "REGION_GNP",
>region
> from
> country
> group by region
> ) FOO,countrylanguage,country
> where
>country.code = countrylanguage.countrycode
>and FOO.region = country.region
>and FOO."REGION_POP" != 0
> group by
> FOO.region,foo."REGION_POP",foo."REGION_GNP"
> order by sum(surfacearea)/foo."REGION_POP" desc),
> allcountrystats as
> ( select country.code,country.name,count(distinct city.id) CITY_CNT,
> count(distinct countrylanguage.language) LANG_CNT
> from country,city,countrylanguage
> where country.code = city.countrycode
> and country.code = countrylanguage.countrycode
> group by country.code,country.name
> )
> select allcountrystats.CITY_CNT,allcountrystats.LANG_CNT,allcountrystats.name,
>
> "REGION_SURFACE_AREA","REGION_LIFETIME",longlivingregions."REGION_POP",longlivingregions.lang_count,longlivingregions."REGION_GNP",longlivingregions.region
> from longlivingregions,denseregions,allcountrystats,country
> where longlivingregions.region = denseregions.region and allcountrystats.code
> = country.code and country.region = longlivingregions.region
> and country.indepyear between 1800 and 1850
> UNION ALL
> select allcountrystats.CITY_CNT,allcountrystats.LANG_CNT,allcountrystats.name,
>
> "REGION_SURFACE_AREA","REGION_LIFETIME",longlivingregions."REGION_POP",longlivingregions.lang_count,longlivingregions."REGION_GNP",longlivingregions.region
> from longlivingregions,denseregions,allcountrystats,country
> where longlivingregions.region = denseregions.region and allcountrystats.code
> = country.code and country.region = longlivingregions.region
> and country.indepyear between 1850 and 1900
> UNION ALL
> select allcountrystats.CITY_CNT,allcountrystats.LANG_CNT,allcountrystats.name,
>
> "REGION_SURFACE_AREA","REGION_LIFETIME",longlivingregions."REGION_POP",longlivingregions.lang_count,longlivingregions."REGION_GNP",longlivingregions.region
> from longlivingregions,denseregions,allcountrystats,country
> where longlivingregions.region = denseregions.region and allcountrystats.code
> = country.code and country.region = longlivingregions.region
> and country.indepyear > 1900
> );
> {noformat}
> Step 3: retrieve all data from the view succeed
> {noformat}
> select * from view_with_shared_scans;
> city_cnt | lang_cnt | name |
> REGION_SURFACE_AREA | REGION_LIFETIME | REGION_POP | lang_count | REGION_GNP
> | region
> --+--+---+-+--++++---
>49 | 12 | Canada|
> 2.36342e+08 | 75.816948242 | 309632000 | 18 | 9111890.00 | North
> America
>58 |8 | Italy |
> 5.8452e+06 | 76.5285720825195 | 144674200 | 22 | 2012289.00 |
> Southern Europe
>29 |6 | Romania |
> 2.14732e+08 | 69.925422363 | 307026000 | 28 | 659980.00 |
> Eastern Europe
> 5 |2 | United Ara
[jira] [Updated] (HAWQ-852) Quey hang when retrieve data from view defined using CTE when filter using like is used
[
https://issues.apache.org/jira/browse/HAWQ-852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruilong Huo updated HAWQ-852:
-
Attachment: cte_query_like.cte_share_off.sql
cte_query_like.cte_share_off.out
cte_query_like.cte_share_on.sql
cte_query_like.cte_share_on.out
world.sql
> Quey hang when retrieve data from view defined using CTE when filter using
> like is used
> ---
>
> Key: HAWQ-852
> URL: https://issues.apache.org/jira/browse/HAWQ-852
> Project: Apache HAWQ
> Issue Type: Bug
> Components: Core, Optimizer, Query Execution
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Attachments: cte_query_like.cte_share_off.out,
> cte_query_like.cte_share_off.sql, cte_query_like.cte_share_on.out,
> cte_query_like.cte_share_on.sql, world.sql
>
>
> It hangs to retrieve data from view defined using CTE when filter using like
> is used. Here are the steps to reproduce:
> Step 1: prepare schema and data by running attached world.sql
> {noformat}
> psql -a -d postgres -f world.sql > world.out 2>&1
> {noformat}
> Step 2: create view defined using CTE by running attached
> {noformat}
> create view view_with_shared_scans as
> (
> with longlivingregions as
> (
> select FOO.*,count(distinct language) as "lang_count"
> from(
> select
>sum(population) as "REGION_POP",
>sum(gnp) as "REGION_GNP",
>avg(lifeexpectancy) as "REGION_LIFETIME",region
> from
> country
> group by region
> ) FOO,countrylanguage,country
> where
>country.code = countrylanguage.countrycode
>and FOO.region = country.region
> group by
> FOO.region,foo."REGION_POP",foo."REGION_GNP",foo."REGION_LIFETIME"),
> denseregions as
> (
> select FOO.*,count(distinct language) as "lang_count",
>sum(surfacearea) as "REGION_SURFACE_AREA"
> from(
> select
>sum(population) as "REGION_POP",
>sum(gnp) as "REGION_GNP",
>region
> from
> country
> group by region
> ) FOO,countrylanguage,country
> where
>country.code = countrylanguage.countrycode
>and FOO.region = country.region
>and FOO."REGION_POP" != 0
> group by
> FOO.region,foo."REGION_POP",foo."REGION_GNP"
> order by sum(surfacearea)/foo."REGION_POP" desc),
> allcountrystats as
> ( select country.code,country.name,count(distinct city.id) CITY_CNT,
> count(distinct countrylanguage.language) LANG_CNT
> from country,city,countrylanguage
> where country.code = city.countrycode
> and country.code = countrylanguage.countrycode
> group by country.code,country.name
> )
> select allcountrystats.CITY_CNT,allcountrystats.LANG_CNT,allcountrystats.name,
>
> "REGION_SURFACE_AREA","REGION_LIFETIME",longlivingregions."REGION_POP",longlivingregions.lang_count,longlivingregions."REGION_GNP",longlivingregions.region
> from longlivingregions,denseregions,allcountrystats,country
> where longlivingregions.region = denseregions.region and allcountrystats.code
> = country.code and country.region = longlivingregions.region
> and country.indepyear between 1800 and 1850
> UNION ALL
> select allcountrystats.CITY_CNT,allcountrystats.LANG_CNT,allcountrystats.name,
>
> "REGION_SURFACE_AREA","REGION_LIFETIME",longlivingregions."REGION_POP",longlivingregions.lang_count,longlivingregions."REGION_GNP",longlivingregions.region
> from longlivingregions,denseregions,allcountrystats,country
> where longlivingregions.region = denseregions.region and allcountrystats.code
> = country.code and country.region = longlivingregions.region
> and country.indepyear between 1850 and 1900
> UNION ALL
> select allcountrystats.CITY_CNT,allcountrystats.LANG_CNT,allcountrystats.name,
>
> "REGION_SURFACE_AREA","REGION_LIFETIME",longlivingregions."REGION_POP",longlivingregions.lang_count,longlivingregions."REGION_GNP",longlivingregions.region
> from longlivingregions,denseregions,allcountrystats,country
> where longlivingregions.region = denseregions.region and allcountrystats.code
> = country.code and country.region = longlivingregions.region
> and country.indepyear > 1900
> );
> {noformat}
> Step 3: retrieve all data from the view succeed
> {noformat}
> select * from view_with_shared_scans;
> city_cnt | lang_cnt | name |
> REGION_SURFACE_AREA | REGION_LIFETIME | REGION_POP | lang_count | REGION_GNP
> | region
> --+--+---+-+--++++---
>49 | 12 | Canada|
> 2.36342e+08 | 75.816948242 | 309632000 | 18 | 9111890.00 | North
> America
>58 |8 | I
[jira] [Updated] (HAWQ-852) Quey hang when retrieve data from view defined using CTE when filter using like is used
[
https://issues.apache.org/jira/browse/HAWQ-852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruilong Huo updated HAWQ-852:
-
Issue Type: Sub-task (was: Bug)
Parent: HAWQ-830
> Quey hang when retrieve data from view defined using CTE when filter using
> like is used
> ---
>
> Key: HAWQ-852
> URL: https://issues.apache.org/jira/browse/HAWQ-852
> Project: Apache HAWQ
> Issue Type: Sub-task
> Components: Core, Optimizer, Query Execution
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Attachments: cte_query_like.cte_share_off.out,
> cte_query_like.cte_share_off.sql, cte_query_like.cte_share_on.out,
> cte_query_like.cte_share_on.sql, world.sql
>
>
> It hangs to retrieve data from view defined using CTE when filter using like
> is used. Here are the steps to reproduce:
> Step 1: prepare schema and data by running attached world.sql
> {noformat}
> psql -a -d postgres -f world.sql > world.out 2>&1
> {noformat}
> Step 2: create view defined using CTE by running attached
> {noformat}
> create view view_with_shared_scans as
> (
> with longlivingregions as
> (
> select FOO.*,count(distinct language) as "lang_count"
> from(
> select
>sum(population) as "REGION_POP",
>sum(gnp) as "REGION_GNP",
>avg(lifeexpectancy) as "REGION_LIFETIME",region
> from
> country
> group by region
> ) FOO,countrylanguage,country
> where
>country.code = countrylanguage.countrycode
>and FOO.region = country.region
> group by
> FOO.region,foo."REGION_POP",foo."REGION_GNP",foo."REGION_LIFETIME"),
> denseregions as
> (
> select FOO.*,count(distinct language) as "lang_count",
>sum(surfacearea) as "REGION_SURFACE_AREA"
> from(
> select
>sum(population) as "REGION_POP",
>sum(gnp) as "REGION_GNP",
>region
> from
> country
> group by region
> ) FOO,countrylanguage,country
> where
>country.code = countrylanguage.countrycode
>and FOO.region = country.region
>and FOO."REGION_POP" != 0
> group by
> FOO.region,foo."REGION_POP",foo."REGION_GNP"
> order by sum(surfacearea)/foo."REGION_POP" desc),
> allcountrystats as
> ( select country.code,country.name,count(distinct city.id) CITY_CNT,
> count(distinct countrylanguage.language) LANG_CNT
> from country,city,countrylanguage
> where country.code = city.countrycode
> and country.code = countrylanguage.countrycode
> group by country.code,country.name
> )
> select allcountrystats.CITY_CNT,allcountrystats.LANG_CNT,allcountrystats.name,
>
> "REGION_SURFACE_AREA","REGION_LIFETIME",longlivingregions."REGION_POP",longlivingregions.lang_count,longlivingregions."REGION_GNP",longlivingregions.region
> from longlivingregions,denseregions,allcountrystats,country
> where longlivingregions.region = denseregions.region and allcountrystats.code
> = country.code and country.region = longlivingregions.region
> and country.indepyear between 1800 and 1850
> UNION ALL
> select allcountrystats.CITY_CNT,allcountrystats.LANG_CNT,allcountrystats.name,
>
> "REGION_SURFACE_AREA","REGION_LIFETIME",longlivingregions."REGION_POP",longlivingregions.lang_count,longlivingregions."REGION_GNP",longlivingregions.region
> from longlivingregions,denseregions,allcountrystats,country
> where longlivingregions.region = denseregions.region and allcountrystats.code
> = country.code and country.region = longlivingregions.region
> and country.indepyear between 1850 and 1900
> UNION ALL
> select allcountrystats.CITY_CNT,allcountrystats.LANG_CNT,allcountrystats.name,
>
> "REGION_SURFACE_AREA","REGION_LIFETIME",longlivingregions."REGION_POP",longlivingregions.lang_count,longlivingregions."REGION_GNP",longlivingregions.region
> from longlivingregions,denseregions,allcountrystats,country
> where longlivingregions.region = denseregions.region and allcountrystats.code
> = country.code and country.region = longlivingregions.region
> and country.indepyear > 1900
> );
> {noformat}
> Step 3: retrieve all data from the view succeed
> {noformat}
> select * from view_with_shared_scans;
> city_cnt | lang_cnt | name |
> REGION_SURFACE_AREA | REGION_LIFETIME | REGION_POP | lang_count | REGION_GNP
> | region
> --+--+---+-+--++++---
>49 | 12 | Canada|
> 2.36342e+08 | 75.816948242 | 309632000 | 18 | 9111890.00 | North
> America
>58 |8 | Italy |
> 5.8452e+06 | 76.5285720825195 | 144674200 | 22 | 2012289.00 |
> Southern Europe
>29
[jira] [Commented] (HAWQ-849) Remove apache orc codebase from hawq
[ https://issues.apache.org/jira/browse/HAWQ-849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15343836#comment-15343836 ] ASF GitHub Bot commented on HAWQ-849: - Github user changleicn commented on the issue: https://github.com/apache/incubator-hawq/pull/737 LGTM > Remove apache orc codebase from hawq > > > Key: HAWQ-849 > URL: https://issues.apache.org/jira/browse/HAWQ-849 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Storage >Reporter: hongwu >Assignee: hongwu > > We discuss a way to don't modify original orc repo to support read in HDFS, > orc codebase might be imported as a git submodule in future. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #737: HAWQ-849. Remove apache orc codebase from hawq
Github user changleicn commented on the issue: https://github.com/apache/incubator-hawq/pull/737 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (HAWQ-850) Planner supports refineCachedPlan interface.
[ https://issues.apache.org/jira/browse/HAWQ-850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei Chang updated HAWQ-850: --- Issue Type: Sub-task (was: Improvement) Parent: HAWQ-800 > Planner supports refineCachedPlan interface. > > > Key: HAWQ-850 > URL: https://issues.apache.org/jira/browse/HAWQ-850 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Core >Reporter: Hubert Zhang >Assignee: Hubert Zhang > > in PBE(prepare bind execution) cases, plan will be cached, and only be > generated once in postgres. > But in HAWQ since resource is dynamic or elastic, the plan maybe changed > during the different execution of PBE. Also, the file split allocation result > which is stored in plan is dynamically too, so the split-vseg mapping may > also be changed. > For example, We call a plpgsql function which do "insert into t select * from > t". > in Prepare we cache the plan, with fixed split-vseg mapping(e.g. 10 blocks), > but after insert, there would be 20 blocks so the old split-vseg mapping is > invalid. We need to update it. Also when we call this function again and > again, the data size of t may request more resource to handle it than the > first call, so we should recalculate the entire plan with the new vseg number. > This function is implemented in a interface called refineCachedPlan. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-852) Quey hang when retrieve data from view defined using CTE when filter using like is used
[
https://issues.apache.org/jira/browse/HAWQ-852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruilong Huo updated HAWQ-852:
-
Issue Type: Bug (was: Sub-task)
Parent: (was: HAWQ-830)
> Quey hang when retrieve data from view defined using CTE when filter using
> like is used
> ---
>
> Key: HAWQ-852
> URL: https://issues.apache.org/jira/browse/HAWQ-852
> Project: Apache HAWQ
> Issue Type: Bug
> Components: Core, Optimizer, Query Execution
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Attachments: cte_query_like.cte_share_off.out,
> cte_query_like.cte_share_off.sql, cte_query_like.cte_share_on.out,
> cte_query_like.cte_share_on.sql, world.sql
>
>
> It hangs to retrieve data from view defined using CTE when filter using like
> is used. Here are the steps to reproduce:
> Step 1: prepare schema and data by running attached world.sql
> {noformat}
> psql -a -d postgres -f world.sql > world.out 2>&1
> {noformat}
> Step 2: create view defined using CTE by running attached
> {noformat}
> create view view_with_shared_scans as
> (
> with longlivingregions as
> (
> select FOO.*,count(distinct language) as "lang_count"
> from(
> select
>sum(population) as "REGION_POP",
>sum(gnp) as "REGION_GNP",
>avg(lifeexpectancy) as "REGION_LIFETIME",region
> from
> country
> group by region
> ) FOO,countrylanguage,country
> where
>country.code = countrylanguage.countrycode
>and FOO.region = country.region
> group by
> FOO.region,foo."REGION_POP",foo."REGION_GNP",foo."REGION_LIFETIME"),
> denseregions as
> (
> select FOO.*,count(distinct language) as "lang_count",
>sum(surfacearea) as "REGION_SURFACE_AREA"
> from(
> select
>sum(population) as "REGION_POP",
>sum(gnp) as "REGION_GNP",
>region
> from
> country
> group by region
> ) FOO,countrylanguage,country
> where
>country.code = countrylanguage.countrycode
>and FOO.region = country.region
>and FOO."REGION_POP" != 0
> group by
> FOO.region,foo."REGION_POP",foo."REGION_GNP"
> order by sum(surfacearea)/foo."REGION_POP" desc),
> allcountrystats as
> ( select country.code,country.name,count(distinct city.id) CITY_CNT,
> count(distinct countrylanguage.language) LANG_CNT
> from country,city,countrylanguage
> where country.code = city.countrycode
> and country.code = countrylanguage.countrycode
> group by country.code,country.name
> )
> select allcountrystats.CITY_CNT,allcountrystats.LANG_CNT,allcountrystats.name,
>
> "REGION_SURFACE_AREA","REGION_LIFETIME",longlivingregions."REGION_POP",longlivingregions.lang_count,longlivingregions."REGION_GNP",longlivingregions.region
> from longlivingregions,denseregions,allcountrystats,country
> where longlivingregions.region = denseregions.region and allcountrystats.code
> = country.code and country.region = longlivingregions.region
> and country.indepyear between 1800 and 1850
> UNION ALL
> select allcountrystats.CITY_CNT,allcountrystats.LANG_CNT,allcountrystats.name,
>
> "REGION_SURFACE_AREA","REGION_LIFETIME",longlivingregions."REGION_POP",longlivingregions.lang_count,longlivingregions."REGION_GNP",longlivingregions.region
> from longlivingregions,denseregions,allcountrystats,country
> where longlivingregions.region = denseregions.region and allcountrystats.code
> = country.code and country.region = longlivingregions.region
> and country.indepyear between 1850 and 1900
> UNION ALL
> select allcountrystats.CITY_CNT,allcountrystats.LANG_CNT,allcountrystats.name,
>
> "REGION_SURFACE_AREA","REGION_LIFETIME",longlivingregions."REGION_POP",longlivingregions.lang_count,longlivingregions."REGION_GNP",longlivingregions.region
> from longlivingregions,denseregions,allcountrystats,country
> where longlivingregions.region = denseregions.region and allcountrystats.code
> = country.code and country.region = longlivingregions.region
> and country.indepyear > 1900
> );
> {noformat}
> Step 3: retrieve all data from the view succeed
> {noformat}
> select * from view_with_shared_scans;
> city_cnt | lang_cnt | name |
> REGION_SURFACE_AREA | REGION_LIFETIME | REGION_POP | lang_count | REGION_GNP
> | region
> --+--+---+-+--++++---
>49 | 12 | Canada|
> 2.36342e+08 | 75.816948242 | 309632000 | 18 | 9111890.00 | North
> America
>58 |8 | Italy |
> 5.8452e+06 | 76.5285720825195 | 144674200 | 22 | 2012289.00 |
> Southern Europe
>
[jira] [Commented] (HAWQ-847) Build PLR error on Mac
[ https://issues.apache.org/jira/browse/HAWQ-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15343868#comment-15343868 ] Paul Guo commented on HAWQ-847: --- Hi Ming, I tried building the whole hawq with plr support on mac, and it runs well. Commands see below. brew tap homebrew/science brew install r ./configure --with-r make -j4 DLSUFFIX was defined in src/Makefile.port which is linked to a platform dependent makefile. lrwxr-xr-x 1 pguo staff32B Jun 22 14:00 Makefile.port -> ../src/makefiles/Makefile.darwin The link was created during configure. Please check whether this file exist, else you could rerun configure. I probably saw the Makefile.port link breakage long time ago, but did not look into it at that time. > Build PLR error on Mac > -- > > Key: HAWQ-847 > URL: https://issues.apache.org/jira/browse/HAWQ-847 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Ming LI >Assignee: Lei Chang > > Reproduce steps: > 1) download and install R-3.1.3.pkg from https://www.r-project.org/ > 2) set env variable: > export R_HOME=/Library/Frameworks/R.framework/Resources > 3) ./configure --with-r > 4) make > Error occurs: > gcc -O0 -std=gnu99 -Wall -Wmissing-prototypes -Wpointer-arith > -Wendif-labels -Wformat-security -fno-strict-aliasing -fwrapv -g -ggdb > -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/libxml2 > -I"/Library/Frameworks/R.framework/Resources"/include -I. > -I../../../src/include > -I/Users/mili/workspace/hawq2/apache-hawq/depends/libhdfs3/build/install/Users/mili/workspace/hawq2/hawq-db-devel/include > > -I/Users/mili/workspace/hawq2/apache-hawq/depends/libyarn/build/install/Users/mili/workspace/hawq2/hawq-db-devel/include > -c -o pg_backend_support.o pg_backend_support.c > pg_backend_support.c:296:2: error: "DLSUFFIX must be defined to compile this > file." > #error "DLSUFFIX must be defined to compile this file." > ^ > pg_backend_support.c:333:37: error: use of undeclared identifier 'DLSUFFIX' > new = palloc(strlen(name) + strlen(DLSUFFIX) + 1); >^ > pg_backend_support.c:335:14: error: use of undeclared identifier 'DLSUFFIX' > strcat(new, DLSUFFIX); > ^ > 3 errors generated. > make[3]: *** [pg_backend_support.o] Error 1 > make[2]: *** [all] Error 1 > make[1]: *** [all] Error 2 > make: *** [all] Error 2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-853) Master standby should avoid incomplete split operation
Lin Wen created HAWQ-853: Summary: Master standby should avoid incomplete split operation Key: HAWQ-853 URL: https://issues.apache.org/jira/browse/HAWQ-853 Project: Apache HAWQ Issue Type: Bug Components: Standby master Reporter: Lin Wen Assignee: Lei Chang Master standby performs replay of xlog records within a range of checkpoints similar to crash recovery. During this is it may replay incomplete multi-step operation at the end of the range. This generates xlog. But Master standby should not generate xlog as it does not have WAL subsystem active. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-853) Master standby should avoid incomplete split operation
[ https://issues.apache.org/jira/browse/HAWQ-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15343889#comment-15343889 ] ASF GitHub Bot commented on HAWQ-853: - GitHub user linwen opened a pull request: https://github.com/apache/incubator-hawq/pull/739 HAWQ-853. Master standby should avoid incomplete split operation Please review, thanks! You can merge this pull request into a Git repository by running: $ git pull https://github.com/linwen/incubator-hawq hawq-853 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/739.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #739 commit 17d76245a73406dff9a399cc395e13ed5fbf87c6 Author: Wen Lin Date: 2016-06-22T08:14:58Z HAWQ-853. Master standby should avoid incomplete split operation Master standby performs replay of xlog records within a range of checkpoints similar to crash recovery. During this is it may replay incomplete multi-step operation at the end of the range. This generates xlog. But Master standby should not generate xlog as it does not have WAL subsystem active. This fix avoids completing such operation e.g. Btree split if the system is in standby mode. > Master standby should avoid incomplete split operation > -- > > Key: HAWQ-853 > URL: https://issues.apache.org/jira/browse/HAWQ-853 > Project: Apache HAWQ > Issue Type: Bug > Components: Standby master >Reporter: Lin Wen >Assignee: Lei Chang > > Master standby performs replay of xlog records within a range of > checkpoints similar to crash recovery. During this is it may replay > incomplete multi-step operation at the end of the range. This generates > xlog. But Master standby should not generate xlog as it does not have > WAL subsystem active. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #739: HAWQ-853. Master standby should avoid inco...
GitHub user linwen opened a pull request: https://github.com/apache/incubator-hawq/pull/739 HAWQ-853. Master standby should avoid incomplete split operation Please review, thanks! You can merge this pull request into a Git repository by running: $ git pull https://github.com/linwen/incubator-hawq hawq-853 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/739.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #739 commit 17d76245a73406dff9a399cc395e13ed5fbf87c6 Author: Wen Lin Date: 2016-06-22T08:14:58Z HAWQ-853. Master standby should avoid incomplete split operation Master standby performs replay of xlog records within a range of checkpoints similar to crash recovery. During this is it may replay incomplete multi-step operation at the end of the range. This generates xlog. But Master standby should not generate xlog as it does not have WAL subsystem active. This fix avoids completing such operation e.g. Btree split if the system is in standby mode. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (HAWQ-854) Remove directories from %files in plr.spec
Paul Guo created HAWQ-854: - Summary: Remove directories from %files in plr.spec Key: HAWQ-854 URL: https://issues.apache.org/jira/browse/HAWQ-854 Project: Apache HAWQ Issue Type: Bug Components: Build Reporter: Paul Guo Assignee: Lei Chang This prevents annoying conflict warning when directories are in another rpm package. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #740: HAWQ-854. Remove directories from %files i...
GitHub user paul-guo- opened a pull request: https://github.com/apache/incubator-hawq/pull/740 HAWQ-854. Remove directories from %files in plr.spec You can merge this pull request into a Git repository by running: $ git pull https://github.com/paul-guo-/incubator-hawq plr Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/740.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #740 commit 595b5b2c96631f3b93bc91f1f3c1a6995a4c2d7a Author: Paul Guo Date: 2016-06-22T10:00:58Z HAWQ-854. Remove directories from %files in plr.spec --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-854) Remove directories from %files in plr.spec
[ https://issues.apache.org/jira/browse/HAWQ-854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15344049#comment-15344049 ] ASF GitHub Bot commented on HAWQ-854: - GitHub user paul-guo- opened a pull request: https://github.com/apache/incubator-hawq/pull/740 HAWQ-854. Remove directories from %files in plr.spec You can merge this pull request into a Git repository by running: $ git pull https://github.com/paul-guo-/incubator-hawq plr Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/740.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #740 commit 595b5b2c96631f3b93bc91f1f3c1a6995a4c2d7a Author: Paul Guo Date: 2016-06-22T10:00:58Z HAWQ-854. Remove directories from %files in plr.spec > Remove directories from %files in plr.spec > -- > > Key: HAWQ-854 > URL: https://issues.apache.org/jira/browse/HAWQ-854 > Project: Apache HAWQ > Issue Type: Bug > Components: Build >Reporter: Paul Guo >Assignee: Lei Chang > > This prevents annoying conflict warning when directories are in another rpm > package. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #740: HAWQ-854. Remove directories from %files in plr.s...
Github user huor commented on the issue: https://github.com/apache/incubator-hawq/pull/740 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #740: HAWQ-854. Remove directories from %files in plr.s...
Github user xunzhang commented on the issue: https://github.com/apache/incubator-hawq/pull/740 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-854) Remove directories from %files in plr.spec
[ https://issues.apache.org/jira/browse/HAWQ-854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15344060#comment-15344060 ] ASF GitHub Bot commented on HAWQ-854: - Github user huor commented on the issue: https://github.com/apache/incubator-hawq/pull/740 +1 > Remove directories from %files in plr.spec > -- > > Key: HAWQ-854 > URL: https://issues.apache.org/jira/browse/HAWQ-854 > Project: Apache HAWQ > Issue Type: Bug > Components: Build >Reporter: Paul Guo >Assignee: Lei Chang > > This prevents annoying conflict warning when directories are in another rpm > package. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-854) Remove directories from %files in plr.spec
[ https://issues.apache.org/jira/browse/HAWQ-854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15344063#comment-15344063 ] ASF GitHub Bot commented on HAWQ-854: - Github user xunzhang commented on the issue: https://github.com/apache/incubator-hawq/pull/740 +1 > Remove directories from %files in plr.spec > -- > > Key: HAWQ-854 > URL: https://issues.apache.org/jira/browse/HAWQ-854 > Project: Apache HAWQ > Issue Type: Bug > Components: Build >Reporter: Paul Guo >Assignee: Lei Chang > > This prevents annoying conflict warning when directories are in another rpm > package. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #740: HAWQ-854. Remove directories from %files i...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-hawq/pull/740 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-854) Remove directories from %files in plr.spec
[ https://issues.apache.org/jira/browse/HAWQ-854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15344068#comment-15344068 ] ASF GitHub Bot commented on HAWQ-854: - Github user asfgit closed the pull request at: https://github.com/apache/incubator-hawq/pull/740 > Remove directories from %files in plr.spec > -- > > Key: HAWQ-854 > URL: https://issues.apache.org/jira/browse/HAWQ-854 > Project: Apache HAWQ > Issue Type: Bug > Components: Build >Reporter: Paul Guo >Assignee: Lei Chang > > This prevents annoying conflict warning when directories are in another rpm > package. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (HAWQ-854) Remove directories from %files in plr.spec
[ https://issues.apache.org/jira/browse/HAWQ-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Guo closed HAWQ-854. - Resolution: Fixed Assignee: Paul Guo (was: Lei Chang) > Remove directories from %files in plr.spec > -- > > Key: HAWQ-854 > URL: https://issues.apache.org/jira/browse/HAWQ-854 > Project: Apache HAWQ > Issue Type: Bug > Components: Build >Reporter: Paul Guo >Assignee: Paul Guo > > This prevents annoying conflict warning when directories are in another rpm > package. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-855) Make resource manager ask for cluster report quickly when YARN is detected down
[ https://issues.apache.org/jira/browse/HAWQ-855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Jin reassigned HAWQ-855: --- Assignee: Yi Jin (was: Lei Chang) > Make resource manager ask for cluster report quickly when YARN is detected > down > --- > > Key: HAWQ-855 > URL: https://issues.apache.org/jira/browse/HAWQ-855 > Project: Apache HAWQ > Issue Type: Improvement > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > > This improvement is for facilitating test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-855) Make resource manager ask for cluster report quickly when YARN is detected down
Yi Jin created HAWQ-855: --- Summary: Make resource manager ask for cluster report quickly when YARN is detected down Key: HAWQ-855 URL: https://issues.apache.org/jira/browse/HAWQ-855 Project: Apache HAWQ Issue Type: Improvement Components: Resource Manager Reporter: Yi Jin Assignee: Lei Chang This improvement is for facilitating test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #652: HAWQ-713 Make lc_numeric guc to have GUC_GPDB_ADD...
Github user karthijrk commented on the issue: https://github.com/apache/incubator-hawq/pull/652 Thanks for the review and merge --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #652: HAWQ-713 Make lc_numeric guc to have GUC_G...
Github user karthijrk closed the pull request at: https://github.com/apache/incubator-hawq/pull/652 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-713) lc_numeric guc doesn't behave as expected after some time
[
https://issues.apache.org/jira/browse/HAWQ-713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15344561#comment-15344561
]
ASF GitHub Bot commented on HAWQ-713:
-
Github user karthijrk commented on the issue:
https://github.com/apache/incubator-hawq/pull/652
Thanks for the review and merge
> lc_numeric guc doesn't behave as expected after some time
> -
>
> Key: HAWQ-713
> URL: https://issues.apache.org/jira/browse/HAWQ-713
> Project: Apache HAWQ
> Issue Type: Bug
> Components: Core
>Reporter: Karthikeyan Jambu Rajaraman
>Assignee: Lei Chang
> Attachments: lc_numeric_check.out, lc_numeric_check.sql
>
>
> Create a simple table with a 1,1 value in it that we will change based on
> lc_numeric. Default we are expecting will be 11 but 1.1 under de_DE
> {code}
> gpadmin=# \d tbl_lc_numeric_test
> Table "public.tbl_lc_numeric_test"
> Column | Type | Modifiers
> +---+---
> a | text |
> s | character varying(50) |
> Distributed by: (a)
> gpadmin=# select * from tbl_lc_numeric_test;
> a | s
> -+---
> 3 | blablabla
> 1,1 | bla
> 2 | blabla
> (3 rows)
> {code}
> When lc_numeric is changed 'de_DE.utf8' as shown below, '1,1' is printed as
> '1.1' as expected.
> {code}
> gpadmin=# set lc_numeric='de_DE.utf8';
> SET
> gpadmin=# show lc_numeric
> gpadmin-# ;
> lc_numeric
>
> de_DE.utf8
> (1 row)
> gpadmin=# \echo `date`
> Thu Apr 21 10:05:00 PDT 2016
> gpadmin=# select to_number(a,'99D9')::numeric(10,5), s from
> tbl_lc_numeric_test;
> to_number | s
> ---+---
>1.1 | bla
>3.0 | blablabla
>2.0 | blabla
> (3 rows)
> {code}
> But if we run above select again after some time (after idle gang timeout),
> then we saw the value as 11 instead of 1.1.
> {code}
> gpadmin=# \echo `date`
> Thu Apr 21 10:05:30 PDT 2016
> gpadmin=# select to_number(a,'99D9')::numeric(10,5), s from
> tbl_lc_numeric_test;
> to_number | s
> ---+---
>3.0 | blablabla
>2.0 | blabla
> 11.0 | bla
> (3 rows)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HAWQ-713) lc_numeric guc doesn't behave as expected after some time
[
https://issues.apache.org/jira/browse/HAWQ-713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15344562#comment-15344562
]
ASF GitHub Bot commented on HAWQ-713:
-
Github user karthijrk closed the pull request at:
https://github.com/apache/incubator-hawq/pull/652
> lc_numeric guc doesn't behave as expected after some time
> -
>
> Key: HAWQ-713
> URL: https://issues.apache.org/jira/browse/HAWQ-713
> Project: Apache HAWQ
> Issue Type: Bug
> Components: Core
>Reporter: Karthikeyan Jambu Rajaraman
>Assignee: Lei Chang
> Attachments: lc_numeric_check.out, lc_numeric_check.sql
>
>
> Create a simple table with a 1,1 value in it that we will change based on
> lc_numeric. Default we are expecting will be 11 but 1.1 under de_DE
> {code}
> gpadmin=# \d tbl_lc_numeric_test
> Table "public.tbl_lc_numeric_test"
> Column | Type | Modifiers
> +---+---
> a | text |
> s | character varying(50) |
> Distributed by: (a)
> gpadmin=# select * from tbl_lc_numeric_test;
> a | s
> -+---
> 3 | blablabla
> 1,1 | bla
> 2 | blabla
> (3 rows)
> {code}
> When lc_numeric is changed 'de_DE.utf8' as shown below, '1,1' is printed as
> '1.1' as expected.
> {code}
> gpadmin=# set lc_numeric='de_DE.utf8';
> SET
> gpadmin=# show lc_numeric
> gpadmin-# ;
> lc_numeric
>
> de_DE.utf8
> (1 row)
> gpadmin=# \echo `date`
> Thu Apr 21 10:05:00 PDT 2016
> gpadmin=# select to_number(a,'99D9')::numeric(10,5), s from
> tbl_lc_numeric_test;
> to_number | s
> ---+---
>1.1 | bla
>3.0 | blablabla
>2.0 | blabla
> (3 rows)
> {code}
> But if we run above select again after some time (after idle gang timeout),
> then we saw the value as 11 instead of 1.1.
> {code}
> gpadmin=# \echo `date`
> Thu Apr 21 10:05:30 PDT 2016
> gpadmin=# select to_number(a,'99D9')::numeric(10,5), s from
> tbl_lc_numeric_test;
> to_number | s
> ---+---
>3.0 | blablabla
>2.0 | blabla
> 11.0 | bla
> (3 rows)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HAWQ-713) lc_numeric guc doesn't behave as expected after some time
[
https://issues.apache.org/jira/browse/HAWQ-713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthikeyan Jambu Rajaraman resolved HAWQ-713.
--
Resolution: Fixed
> lc_numeric guc doesn't behave as expected after some time
> -
>
> Key: HAWQ-713
> URL: https://issues.apache.org/jira/browse/HAWQ-713
> Project: Apache HAWQ
> Issue Type: Bug
> Components: Core
>Reporter: Karthikeyan Jambu Rajaraman
>Assignee: Lei Chang
> Attachments: lc_numeric_check.out, lc_numeric_check.sql
>
>
> Create a simple table with a 1,1 value in it that we will change based on
> lc_numeric. Default we are expecting will be 11 but 1.1 under de_DE
> {code}
> gpadmin=# \d tbl_lc_numeric_test
> Table "public.tbl_lc_numeric_test"
> Column | Type | Modifiers
> +---+---
> a | text |
> s | character varying(50) |
> Distributed by: (a)
> gpadmin=# select * from tbl_lc_numeric_test;
> a | s
> -+---
> 3 | blablabla
> 1,1 | bla
> 2 | blabla
> (3 rows)
> {code}
> When lc_numeric is changed 'de_DE.utf8' as shown below, '1,1' is printed as
> '1.1' as expected.
> {code}
> gpadmin=# set lc_numeric='de_DE.utf8';
> SET
> gpadmin=# show lc_numeric
> gpadmin-# ;
> lc_numeric
>
> de_DE.utf8
> (1 row)
> gpadmin=# \echo `date`
> Thu Apr 21 10:05:00 PDT 2016
> gpadmin=# select to_number(a,'99D9')::numeric(10,5), s from
> tbl_lc_numeric_test;
> to_number | s
> ---+---
>1.1 | bla
>3.0 | blablabla
>2.0 | blabla
> (3 rows)
> {code}
> But if we run above select again after some time (after idle gang timeout),
> then we saw the value as 11 instead of 1.1.
> {code}
> gpadmin=# \echo `date`
> Thu Apr 21 10:05:30 PDT 2016
> gpadmin=# select to_number(a,'99D9')::numeric(10,5), s from
> tbl_lc_numeric_test;
> to_number | s
> ---+---
>3.0 | blablabla
>2.0 | blabla
> 11.0 | bla
> (3 rows)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] incubator-hawq issue #737: HAWQ-849. Remove apache orc codebase from hawq
Github user rvs commented on the issue: https://github.com/apache/incubator-hawq/pull/737 @xunzhang happy to help and thanks for the patience! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-849) Remove apache orc codebase from hawq
[ https://issues.apache.org/jira/browse/HAWQ-849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15344776#comment-15344776 ] ASF GitHub Bot commented on HAWQ-849: - Github user rvs commented on the issue: https://github.com/apache/incubator-hawq/pull/737 @xunzhang happy to help and thanks for the patience! > Remove apache orc codebase from hawq > > > Key: HAWQ-849 > URL: https://issues.apache.org/jira/browse/HAWQ-849 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Storage >Reporter: hongwu >Assignee: hongwu > > We discuss a way to don't modify original orc repo to support read in HDFS, > orc codebase might be imported as a git submodule in future. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-763) Remove traceflag that allows ORCA to run in multiple threads
[ https://issues.apache.org/jira/browse/HAWQ-763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15344783#comment-15344783 ] ASF GitHub Bot commented on HAWQ-763: - Github user asfgit closed the pull request at: https://github.com/apache/incubator-hawq/pull/644 > Remove traceflag that allows ORCA to run in multiple threads > > > Key: HAWQ-763 > URL: https://issues.apache.org/jira/browse/HAWQ-763 > Project: Apache HAWQ > Issue Type: Improvement > Components: Optimizer >Reporter: Haisheng Yuan >Assignee: Lei Chang > > We are integrating VEM with GPOS memory allocator. VMEM is not thread safe. > So, ORCA/GPOS also need to be single threaded. Currently, HAWQ users are > using only the single threaded Orca. > So need to remove the traceflag that allows ORCA to run in multiple threads > so that HAWQ users do not accidentally run ORCA in multiple threads. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #644: Remove traceflag that allows ORCA to run i...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-hawq/pull/644 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #646: Remove CTranslatorPlStmtToDXL Deadcode [#1...
Github user vraghavan78 closed the pull request at: https://github.com/apache/incubator-hawq/pull/646 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-764) Remove CTranslatorPlStmtToDXL Deadcode
[ https://issues.apache.org/jira/browse/HAWQ-764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15344885#comment-15344885 ] ASF GitHub Bot commented on HAWQ-764: - Github user vraghavan78 closed the pull request at: https://github.com/apache/incubator-hawq/pull/646 > Remove CTranslatorPlStmtToDXL Deadcode > -- > > Key: HAWQ-764 > URL: https://issues.apache.org/jira/browse/HAWQ-764 > Project: Apache HAWQ > Issue Type: Improvement > Components: Optimizer >Reporter: Haisheng Yuan >Assignee: Lei Chang > > When we did not have optimization modules in Orca and just DXL, we used > CTranslatorPlStmtToDXL to test correctness of translation of DXL to PlStmt. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #661: Deadcode #119780063
Github user vraghavan78 closed the pull request at: https://github.com/apache/incubator-hawq/pull/661 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-763) Remove traceflag that allows ORCA to run in multiple threads
[ https://issues.apache.org/jira/browse/HAWQ-763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15344942#comment-15344942 ] Venkatesh commented on HAWQ-763: This fix has been merged. Who closes this? > Remove traceflag that allows ORCA to run in multiple threads > > > Key: HAWQ-763 > URL: https://issues.apache.org/jira/browse/HAWQ-763 > Project: Apache HAWQ > Issue Type: Improvement > Components: Optimizer >Reporter: Haisheng Yuan >Assignee: Lei Chang > > We are integrating VEM with GPOS memory allocator. VMEM is not thread safe. > So, ORCA/GPOS also need to be single threaded. Currently, HAWQ users are > using only the single threaded Orca. > So need to remove the traceflag that allows ORCA to run in multiple threads > so that HAWQ users do not accidentally run ORCA in multiple threads. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HAWQ-763) Remove traceflag that allows ORCA to run in multiple threads
[ https://issues.apache.org/jira/browse/HAWQ-763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan resolved HAWQ-763. Resolution: Fixed > Remove traceflag that allows ORCA to run in multiple threads > > > Key: HAWQ-763 > URL: https://issues.apache.org/jira/browse/HAWQ-763 > Project: Apache HAWQ > Issue Type: Improvement > Components: Optimizer >Reporter: Haisheng Yuan >Assignee: Lei Chang > > We are integrating VEM with GPOS memory allocator. VMEM is not thread safe. > So, ORCA/GPOS also need to be single threaded. Currently, HAWQ users are > using only the single threaded Orca. > So need to remove the traceflag that allows ORCA to run in multiple threads > so that HAWQ users do not accidentally run ORCA in multiple threads. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-763) Remove traceflag that allows ORCA to run in multiple threads
[ https://issues.apache.org/jira/browse/HAWQ-763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345144#comment-15345144 ] Haisheng Yuan commented on HAWQ-763: closed. Thanks~ Haisheng Yuan > Remove traceflag that allows ORCA to run in multiple threads > > > Key: HAWQ-763 > URL: https://issues.apache.org/jira/browse/HAWQ-763 > Project: Apache HAWQ > Issue Type: Improvement > Components: Optimizer >Reporter: Haisheng Yuan >Assignee: Lei Chang > > We are integrating VEM with GPOS memory allocator. VMEM is not thread safe. > So, ORCA/GPOS also need to be single threaded. Currently, HAWQ users are > using only the single threaded Orca. > So need to remove the traceflag that allows ORCA to run in multiple threads > so that HAWQ users do not accidentally run ORCA in multiple threads. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-856) Explicitly initialize GPOPT and its dependencies.
Venkatesh created HAWQ-856: -- Summary: Explicitly initialize GPOPT and its dependencies. Key: HAWQ-856 URL: https://issues.apache.org/jira/browse/HAWQ-856 Project: Apache HAWQ Issue Type: New Feature Components: Optimizer Reporter: Venkatesh Assignee: Amr El-Helw These were initialized by constructors earlier. To pass any parameters for initializing GPOPT or any of its dependencies, we need to do that explicitly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #654: Explicitly initialize GPOPT and its dependencies.
Github user vraghavan78 commented on the issue: https://github.com/apache/incubator-hawq/pull/654 Created a HAWQ JIRA: https://issues.apache.org/jira/browse/HAWQ-856 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-856) Explicitly initialize GPOPT and its dependencies.
[ https://issues.apache.org/jira/browse/HAWQ-856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345270#comment-15345270 ] ASF GitHub Bot commented on HAWQ-856: - Github user vraghavan78 commented on the issue: https://github.com/apache/incubator-hawq/pull/654 Created a HAWQ JIRA: https://issues.apache.org/jira/browse/HAWQ-856 > Explicitly initialize GPOPT and its dependencies. > -- > > Key: HAWQ-856 > URL: https://issues.apache.org/jira/browse/HAWQ-856 > Project: Apache HAWQ > Issue Type: New Feature > Components: Optimizer >Reporter: Venkatesh >Assignee: Amr El-Helw > > These were initialized by constructors earlier. To pass any parameters > for initializing GPOPT or any of its dependencies, we need to do that > explicitly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #654: Explicitly initialize GPOPT and its depend...
Github user hardikar closed the pull request at: https://github.com/apache/incubator-hawq/pull/654 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #654: Explicitly initialize GPOPT and its dependencies.
Github user vraghavan78 commented on the issue: https://github.com/apache/incubator-hawq/pull/654 @hardikar please close the PR. I pushed it to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #654: Explicitly initialize GPOPT and its dependencies.
Github user hardikar commented on the issue: https://github.com/apache/incubator-hawq/pull/654 @vraghavan78 Thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (HAWQ-856) Explicitly initialize GPOPT and its dependencies.
[ https://issues.apache.org/jira/browse/HAWQ-856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkatesh resolved HAWQ-856. Resolution: Fixed > Explicitly initialize GPOPT and its dependencies. > -- > > Key: HAWQ-856 > URL: https://issues.apache.org/jira/browse/HAWQ-856 > Project: Apache HAWQ > Issue Type: New Feature > Components: Optimizer >Reporter: Venkatesh >Assignee: Amr El-Helw > > These were initialized by constructors earlier. To pass any parameters > for initializing GPOPT or any of its dependencies, we need to do that > explicitly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-857) Remove dead code in the Algebrizer
Venkatesh created HAWQ-857: -- Summary: Remove dead code in the Algebrizer Key: HAWQ-857 URL: https://issues.apache.org/jira/browse/HAWQ-857 Project: Apache HAWQ Issue Type: Improvement Reporter: Venkatesh Assignee: Lei Chang Remove unreachable code paths in the Algebrizer -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-857) Remove dead code in the Algebrizer
[ https://issues.apache.org/jira/browse/HAWQ-857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345308#comment-15345308 ] ASF GitHub Bot commented on HAWQ-857: - GitHub user vraghavan78 opened a pull request: https://github.com/apache/incubator-hawq/pull/741 HAWQ-857. Remove dead code in the Algebrizer Remove unreachable (stale) code paths in the Algebrizer. You can merge this pull request into a Git repository by running: $ git pull https://github.com/vraghavan78/incubator-hawq algebrizer-deadcode Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/741.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #741 commit 9a40f99f5fd3f592ecda02223dca5b1c5f458fac Author: Venkatesh (Venky) Raghavan Date: 2016-06-22T22:41:30Z HAWQ-857. Remove dead code in the Algebrizer > Remove dead code in the Algebrizer > --- > > Key: HAWQ-857 > URL: https://issues.apache.org/jira/browse/HAWQ-857 > Project: Apache HAWQ > Issue Type: Improvement >Reporter: Venkatesh >Assignee: Lei Chang > > Remove unreachable code paths in the Algebrizer -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #741: HAWQ-857. Remove dead code in the Algebriz...
GitHub user vraghavan78 opened a pull request: https://github.com/apache/incubator-hawq/pull/741 HAWQ-857. Remove dead code in the Algebrizer Remove unreachable (stale) code paths in the Algebrizer. You can merge this pull request into a Git repository by running: $ git pull https://github.com/vraghavan78/incubator-hawq algebrizer-deadcode Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/741.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #741 commit 9a40f99f5fd3f592ecda02223dca5b1c5f458fac Author: Venkatesh (Venky) Raghavan Date: 2016-06-22T22:41:30Z HAWQ-857. Remove dead code in the Algebrizer --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (HAWQ-858) Fix parser to understand case / when expression in group by
Venkatesh created HAWQ-858: -- Summary: Fix parser to understand case / when expression in group by Key: HAWQ-858 URL: https://issues.apache.org/jira/browse/HAWQ-858 Project: Apache HAWQ Issue Type: Bug Components: Parser Reporter: Venkatesh Assignee: Lei Chang [~lei_chang] please port this parser changes into HAWQ https://github.com/greenplum-db/gpdb4/commit/30b33a10f4b0a4468a9ed80cf3779fd12f176abf -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-859) Fix the naming of output columns of append node
Venkatesh created HAWQ-859:
--
Summary: Fix the naming of output columns of append node
Key: HAWQ-859
URL: https://issues.apache.org/jira/browse/HAWQ-859
Project: Apache HAWQ
Issue Type: Bug
Components: Optimizer
Reporter: Venkatesh
Assignee: Amr El-Helw
repro:
{code}
pivotal=# select * from (values (1),(2)) as v(k);
column1
-
1
2
(2 rows)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HAWQ-860) Optimizer generates wrong plan when correlated subquery contains set-returning functions
Venkatesh created HAWQ-860: -- Summary: Optimizer generates wrong plan when correlated subquery contains set-returning functions Key: HAWQ-860 URL: https://issues.apache.org/jira/browse/HAWQ-860 Project: Apache HAWQ Issue Type: Bug Components: Optimizer Reporter: Venkatesh Assignee: Amr El-Helw Bump ORCA to 1.634 [#119042413] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HAWQ-849) Remove apache orc codebase from hawq
[ https://issues.apache.org/jira/browse/HAWQ-849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu resolved HAWQ-849. - Resolution: Fixed > Remove apache orc codebase from hawq > > > Key: HAWQ-849 > URL: https://issues.apache.org/jira/browse/HAWQ-849 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Storage >Reporter: hongwu >Assignee: hongwu > > We discuss a way to don't modify original orc repo to support read in HDFS, > orc codebase might be imported as a git submodule in future. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-861) Wrong logerror position in insert into hashtable select * from gpfdist external table.
[ https://issues.apache.org/jira/browse/HAWQ-861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hubert Zhang reassigned HAWQ-861: - Assignee: Hubert Zhang (was: Lei Chang) > Wrong logerror position in insert into hashtable select * from gpfdist > external table. > -- > > Key: HAWQ-861 > URL: https://issues.apache.org/jira/browse/HAWQ-861 > Project: Apache HAWQ > Issue Type: Bug > Components: Core >Reporter: Hubert Zhang >Assignee: Hubert Zhang > > For statement insert into hashtable select * from gpfdist external table, > when hashtable bucket number is less than location number of gpfdist external > table, error should be reported in cdbdatalocality module, with error message > like this: > ERROR: Could not allocate enough memory! bucket number of result hash table > and external table should match each other (cdbdatalocality.c:4222) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-861) Wrong logerror position in insert into hashtable select * from gpfdist external table.
Hubert Zhang created HAWQ-861: - Summary: Wrong logerror position in insert into hashtable select * from gpfdist external table. Key: HAWQ-861 URL: https://issues.apache.org/jira/browse/HAWQ-861 Project: Apache HAWQ Issue Type: Bug Components: Core Reporter: Hubert Zhang Assignee: Lei Chang For statement insert into hashtable select * from gpfdist external table, when hashtable bucket number is less than location number of gpfdist external table, error should be reported in cdbdatalocality module, with error message like this: ERROR: Could not allocate enough memory! bucket number of result hash table and external table should match each other (cdbdatalocality.c:4222) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-862) Make user defined function get_ao_distribution work.
[
https://issues.apache.org/jira/browse/HAWQ-862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hubert Zhang updated HAWQ-862:
--
Description:
In Hawq we had a functon that could do a fast row-count on AO tables based on
querying the catalog versus issuing 'select count(*) ' SQL. Was a lot faster
for larger tables. I found the script and tried using it with HAWQ but one of
the underlying Postgres functions that gets called (get_ao_distribution) bombs
since it doesn't seem to be able to find the table files when they are stored
in HDFS.
gpadmin=# select sum(tupcount) from get_ao_distribution('public.foo');
ERROR: could not open relation 1663/24731/24740: No such file or directory
(seg0 localhost.localdomain:4 pid=82964)
DETAIL: Database directory "base/24731" does not exist
CONTEXT: SQL statement "select gp_segment_id, sum(tupcount) from
gp_dist_random('pg_aoseg.pg_aoseg_24738') group by (gp_segment_id)"
gpadmin=#
was:
In Hawq we had a functon that could do a fast row-count on AO tables based on
querying the catalog versus issuing 'select count(*)' SQL. Was a lot faster
for larger tables. I found the script and tried using it with HAWQ but one of
the underlying Postgres functions that gets called (get_ao_distribution) bombs
since it doesn't seem to be able to find the table files when they are stored
in HDFS.
gpadmin=# select sum(tupcount) from get_ao_distribution('public.foo');
ERROR: could not open relation 1663/24731/24740: No such file or directory
(seg0 localhost.localdomain:4 pid=82964)
DETAIL: Database directory "base/24731" does not exist
CONTEXT: SQL statement "select gp_segment_id, sum(tupcount) from
gp_dist_random('pg_aoseg.pg_aoseg_24738') group by (gp_segment_id)"
gpadmin=#
> Make user defined function get_ao_distribution work.
>
>
> Key: HAWQ-862
> URL: https://issues.apache.org/jira/browse/HAWQ-862
> Project: Apache HAWQ
> Issue Type: Bug
> Components: Core
>Reporter: Hubert Zhang
>Assignee: Hubert Zhang
>
> In Hawq we had a functon that could do a fast row-count on AO tables based on
> querying the catalog versus issuing 'select count(*) ' SQL. Was a lot faster
> for larger tables. I found the script and tried using it with HAWQ but one
> of the underlying Postgres functions that gets called (get_ao_distribution)
> bombs since it doesn't seem to be able to find the table files when they are
> stored in HDFS.
> gpadmin=# select sum(tupcount) from get_ao_distribution('public.foo');
> ERROR: could not open relation 1663/24731/24740: No such file or directory
> (seg0 localhost.localdomain:4 pid=82964)
> DETAIL: Database directory "base/24731" does not exist
> CONTEXT: SQL statement "select gp_segment_id, sum(tupcount) from
> gp_dist_random('pg_aoseg.pg_aoseg_24738') group by (gp_segment_id)"
> gpadmin=#
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HAWQ-862) Make user defined function get_ao_distribution work.
Hubert Zhang created HAWQ-862:
-
Summary: Make user defined function get_ao_distribution work.
Key: HAWQ-862
URL: https://issues.apache.org/jira/browse/HAWQ-862
Project: Apache HAWQ
Issue Type: Bug
Components: Core
Reporter: Hubert Zhang
Assignee: Lei Chang
In Hawq we had a functon that could do a fast row-count on AO tables based on
querying the catalog versus issuing 'select count(*)' SQL. Was a lot faster
for larger tables. I found the script and tried using it with HAWQ but one of
the underlying Postgres functions that gets called (get_ao_distribution) bombs
since it doesn't seem to be able to find the table files when they are stored
in HDFS.
gpadmin=# select sum(tupcount) from get_ao_distribution('public.foo');
ERROR: could not open relation 1663/24731/24740: No such file or directory
(seg0 localhost.localdomain:4 pid=82964)
DETAIL: Database directory "base/24731" does not exist
CONTEXT: SQL statement "select gp_segment_id, sum(tupcount) from
gp_dist_random('pg_aoseg.pg_aoseg_24738') group by (gp_segment_id)"
gpadmin=#
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HAWQ-862) Make user defined function get_ao_distribution work.
[
https://issues.apache.org/jira/browse/HAWQ-862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hubert Zhang reassigned HAWQ-862:
-
Assignee: Hubert Zhang (was: Lei Chang)
> Make user defined function get_ao_distribution work.
>
>
> Key: HAWQ-862
> URL: https://issues.apache.org/jira/browse/HAWQ-862
> Project: Apache HAWQ
> Issue Type: Bug
> Components: Core
>Reporter: Hubert Zhang
>Assignee: Hubert Zhang
>
> In Hawq we had a functon that could do a fast row-count on AO tables based on
> querying the catalog versus issuing 'select count(*)' SQL. Was a lot faster
> for larger tables. I found the script and tried using it with HAWQ but one
> of the underlying Postgres functions that gets called (get_ao_distribution)
> bombs since it doesn't seem to be able to find the table files when they are
> stored in HDFS.
> gpadmin=# select sum(tupcount) from get_ao_distribution('public.foo');
> ERROR: could not open relation 1663/24731/24740: No such file or directory
> (seg0 localhost.localdomain:4 pid=82964)
> DETAIL: Database directory "base/24731" does not exist
> CONTEXT: SQL statement "select gp_segment_id, sum(tupcount) from
> gp_dist_random('pg_aoseg.pg_aoseg_24738') group by (gp_segment_id)"
> gpadmin=#
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] incubator-hawq issue #739: HAWQ-853. Master standby should avoid incomplete ...
Github user yaoj2 commented on the issue: https://github.com/apache/incubator-hawq/pull/739 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-853) Master standby should avoid incomplete split operation
[ https://issues.apache.org/jira/browse/HAWQ-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345536#comment-15345536 ] ASF GitHub Bot commented on HAWQ-853: - Github user yaoj2 commented on the issue: https://github.com/apache/incubator-hawq/pull/739 LGTM > Master standby should avoid incomplete split operation > -- > > Key: HAWQ-853 > URL: https://issues.apache.org/jira/browse/HAWQ-853 > Project: Apache HAWQ > Issue Type: Bug > Components: Standby master >Reporter: Lin Wen >Assignee: Lei Chang > > Master standby performs replay of xlog records within a range of > checkpoints similar to crash recovery. During this is it may replay > incomplete multi-step operation at the end of the range. This generates > xlog. But Master standby should not generate xlog as it does not have > WAL subsystem active. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-823) Amazon S3 External Table Support
[ https://issues.apache.org/jira/browse/HAWQ-823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345540#comment-15345540 ] Kyle R Dunn commented on HAWQ-823: -- I'm happy to investigate how to interface this with the FDW-based approaches if thats the direction things are headed for HAWQ. If nothing else I thought this might be a good, lightweight MVP for S3 until the FDW framework details are worked out. Maybe we can point to the isolated module testing the GPDB team has already done for this code base, since there were almost no source-level changes for this port and it uses the "same" PROTOCOL interface. I certainly won't be offended if this doesn't meet the bigger picture requirements for S3 functionality but it does seem to be an immediately useful "bolt on" through the existing `CREATE PROTOCOL` functionality in HAWQ, if we're open to relaxing the existing restrictions on that interface from the "userland" entry point (i.e. allowing the command outside of utility mode). > Amazon S3 External Table Support > > > Key: HAWQ-823 > URL: https://issues.apache.org/jira/browse/HAWQ-823 > Project: Apache HAWQ > Issue Type: Wish > Components: External Tables >Reporter: Kyle R Dunn >Assignee: Lei Chang > > As a cloud user, I'd like to be able to create readable external tables with > data in Amazon S3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #739: HAWQ-853. Master standby should avoid incomplete ...
Github user huor commented on the issue: https://github.com/apache/incubator-hawq/pull/739 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-853) Master standby should avoid incomplete split operation
[ https://issues.apache.org/jira/browse/HAWQ-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345561#comment-15345561 ] ASF GitHub Bot commented on HAWQ-853: - Github user huor commented on the issue: https://github.com/apache/incubator-hawq/pull/739 +1 > Master standby should avoid incomplete split operation > -- > > Key: HAWQ-853 > URL: https://issues.apache.org/jira/browse/HAWQ-853 > Project: Apache HAWQ > Issue Type: Bug > Components: Standby master >Reporter: Lin Wen >Assignee: Lei Chang > > Master standby performs replay of xlog records within a range of > checkpoints similar to crash recovery. During this is it may replay > incomplete multi-step operation at the end of the range. This generates > xlog. But Master standby should not generate xlog as it does not have > WAL subsystem active. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-853) Master standby should avoid incomplete split operation
[ https://issues.apache.org/jira/browse/HAWQ-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345570#comment-15345570 ] ASF GitHub Bot commented on HAWQ-853: - Github user linwen closed the pull request at: https://github.com/apache/incubator-hawq/pull/739 > Master standby should avoid incomplete split operation > -- > > Key: HAWQ-853 > URL: https://issues.apache.org/jira/browse/HAWQ-853 > Project: Apache HAWQ > Issue Type: Bug > Components: Standby master >Reporter: Lin Wen >Assignee: Lei Chang > > Master standby performs replay of xlog records within a range of > checkpoints similar to crash recovery. During this is it may replay > incomplete multi-step operation at the end of the range. This generates > xlog. But Master standby should not generate xlog as it does not have > WAL subsystem active. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #739: HAWQ-853. Master standby should avoid inco...
Github user linwen closed the pull request at: https://github.com/apache/incubator-hawq/pull/739 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (HAWQ-853) Master standby should avoid incomplete split operation
[ https://issues.apache.org/jira/browse/HAWQ-853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Wen resolved HAWQ-853. -- Resolution: Fixed > Master standby should avoid incomplete split operation > -- > > Key: HAWQ-853 > URL: https://issues.apache.org/jira/browse/HAWQ-853 > Project: Apache HAWQ > Issue Type: Bug > Components: Standby master >Reporter: Lin Wen >Assignee: Lin Wen > > Master standby performs replay of xlog records within a range of > checkpoints similar to crash recovery. During this is it may replay > incomplete multi-step operation at the end of the range. This generates > xlog. But Master standby should not generate xlog as it does not have > WAL subsystem active. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-853) Master standby should avoid incomplete split operation
[ https://issues.apache.org/jira/browse/HAWQ-853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Wen reassigned HAWQ-853: Assignee: Lin Wen (was: Lei Chang) > Master standby should avoid incomplete split operation > -- > > Key: HAWQ-853 > URL: https://issues.apache.org/jira/browse/HAWQ-853 > Project: Apache HAWQ > Issue Type: Bug > Components: Standby master >Reporter: Lin Wen >Assignee: Lin Wen > > Master standby performs replay of xlog records within a range of > checkpoints similar to crash recovery. During this is it may replay > incomplete multi-step operation at the end of the range. This generates > xlog. But Master standby should not generate xlog as it does not have > WAL subsystem active. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-863) Add python module 'pycrypto'
Radar Lei created HAWQ-863: -- Summary: Add python module 'pycrypto' Key: HAWQ-863 URL: https://issues.apache.org/jira/browse/HAWQ-863 Project: Apache HAWQ Issue Type: Task Components: Build, Command Line Tools Reporter: Radar Lei Assignee: Lei Chang Previously we added python modules which HAWQ needed, but 'pycrypto' not added as we think it's not needed by HAWQ now. Actually 'hawq ssh-exkeys' still need it, and it's license is 'Public domain' which is compatible with HAWQ. So we should add it in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-863) Add python module 'pycrypto'
[ https://issues.apache.org/jira/browse/HAWQ-863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radar Lei reassigned HAWQ-863: -- Assignee: Radar Lei (was: Lei Chang) > Add python module 'pycrypto' > > > Key: HAWQ-863 > URL: https://issues.apache.org/jira/browse/HAWQ-863 > Project: Apache HAWQ > Issue Type: Task > Components: Build, Command Line Tools >Reporter: Radar Lei >Assignee: Radar Lei > > Previously we added python modules which HAWQ needed, but 'pycrypto' not > added as we think it's not needed by HAWQ now. > Actually 'hawq ssh-exkeys' still need it, and it's license is 'Public domain' > which is compatible with HAWQ. So we should add it in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #742: HAWQ-863. Add python module pycrypto
GitHub user radarwave opened a pull request: https://github.com/apache/incubator-hawq/pull/742 HAWQ-863. Add python module pycrypto Add this module to reduce user install works. So user don't need to do 'pip install pycrypto'. You can merge this pull request into a Git repository by running: $ git pull https://github.com/radarwave/incubator-hawq pycrypto Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/742.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #742 commit 2c87cdcbdde5cf349083e3254a65b6c619388491 Author: rlei Date: 2016-06-23T02:06:34Z HAWQ-863. Add python module pycrypto --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-863) Add python module 'pycrypto'
[ https://issues.apache.org/jira/browse/HAWQ-863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345590#comment-15345590 ] ASF GitHub Bot commented on HAWQ-863: - GitHub user radarwave opened a pull request: https://github.com/apache/incubator-hawq/pull/742 HAWQ-863. Add python module pycrypto Add this module to reduce user install works. So user don't need to do 'pip install pycrypto'. You can merge this pull request into a Git repository by running: $ git pull https://github.com/radarwave/incubator-hawq pycrypto Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/742.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #742 commit 2c87cdcbdde5cf349083e3254a65b6c619388491 Author: rlei Date: 2016-06-23T02:06:34Z HAWQ-863. Add python module pycrypto > Add python module 'pycrypto' > > > Key: HAWQ-863 > URL: https://issues.apache.org/jira/browse/HAWQ-863 > Project: Apache HAWQ > Issue Type: Task > Components: Build, Command Line Tools >Reporter: Radar Lei >Assignee: Radar Lei > > Previously we added python modules which HAWQ needed, but 'pycrypto' not > added as we think it's not needed by HAWQ now. > Actually 'hawq ssh-exkeys' still need it, and it's license is 'Public domain' > which is compatible with HAWQ. So we should add it in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #742: HAWQ-863. Add python module pycrypto
Github user yaoj2 commented on the issue: https://github.com/apache/incubator-hawq/pull/742 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-863) Add python module 'pycrypto'
[ https://issues.apache.org/jira/browse/HAWQ-863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345592#comment-15345592 ] ASF GitHub Bot commented on HAWQ-863: - Github user yaoj2 commented on the issue: https://github.com/apache/incubator-hawq/pull/742 +1 > Add python module 'pycrypto' > > > Key: HAWQ-863 > URL: https://issues.apache.org/jira/browse/HAWQ-863 > Project: Apache HAWQ > Issue Type: Task > Components: Build, Command Line Tools >Reporter: Radar Lei >Assignee: Radar Lei > > Previously we added python modules which HAWQ needed, but 'pycrypto' not > added as we think it's not needed by HAWQ now. > Actually 'hawq ssh-exkeys' still need it, and it's license is 'Public domain' > which is compatible with HAWQ. So we should add it in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #742: HAWQ-863. Add python module pycrypto
Github user xunzhang commented on the issue: https://github.com/apache/incubator-hawq/pull/742 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-863) Add python module 'pycrypto'
[ https://issues.apache.org/jira/browse/HAWQ-863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345593#comment-15345593 ] ASF GitHub Bot commented on HAWQ-863: - Github user xunzhang commented on the issue: https://github.com/apache/incubator-hawq/pull/742 +1 > Add python module 'pycrypto' > > > Key: HAWQ-863 > URL: https://issues.apache.org/jira/browse/HAWQ-863 > Project: Apache HAWQ > Issue Type: Task > Components: Build, Command Line Tools >Reporter: Radar Lei >Assignee: Radar Lei > > Previously we added python modules which HAWQ needed, but 'pycrypto' not > added as we think it's not needed by HAWQ now. > Actually 'hawq ssh-exkeys' still need it, and it's license is 'Public domain' > which is compatible with HAWQ. So we should add it in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #742: HAWQ-863. Add python module pycrypto
Github user asfgit closed the pull request at: https://github.com/apache/incubator-hawq/pull/742 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-863) Add python module 'pycrypto'
[ https://issues.apache.org/jira/browse/HAWQ-863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345594#comment-15345594 ] ASF GitHub Bot commented on HAWQ-863: - Github user asfgit closed the pull request at: https://github.com/apache/incubator-hawq/pull/742 > Add python module 'pycrypto' > > > Key: HAWQ-863 > URL: https://issues.apache.org/jira/browse/HAWQ-863 > Project: Apache HAWQ > Issue Type: Task > Components: Build, Command Line Tools >Reporter: Radar Lei >Assignee: Radar Lei > > Previously we added python modules which HAWQ needed, but 'pycrypto' not > added as we think it's not needed by HAWQ now. > Actually 'hawq ssh-exkeys' still need it, and it's license is 'Public domain' > which is compatible with HAWQ. So we should add it in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-858) Fix parser to understand case / when expression in group by
[ https://issues.apache.org/jira/browse/HAWQ-858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345617#comment-15345617 ] Lili Ma commented on HAWQ-858: -- [~abhi90] What's the target version in HAWQ for this bug fix? > Fix parser to understand case / when expression in group by > --- > > Key: HAWQ-858 > URL: https://issues.apache.org/jira/browse/HAWQ-858 > Project: Apache HAWQ > Issue Type: Bug > Components: Parser >Reporter: Venkatesh >Assignee: Lei Chang > > [~lei_chang] please port this parser changes into HAWQ > https://github.com/greenplum-db/gpdb4/commit/30b33a10f4b0a4468a9ed80cf3779fd12f176abf -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-857) Remove dead code in the Algebrizer
[ https://issues.apache.org/jira/browse/HAWQ-857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lili Ma updated HAWQ-857: - Component/s: PXF > Remove dead code in the Algebrizer > --- > > Key: HAWQ-857 > URL: https://issues.apache.org/jira/browse/HAWQ-857 > Project: Apache HAWQ > Issue Type: Improvement > Components: PXF >Reporter: Venkatesh >Assignee: Lei Chang > > Remove unreachable code paths in the Algebrizer -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #739: HAWQ-853. Master standby should avoid incomplete ...
Github user jiny2 commented on the issue: https://github.com/apache/incubator-hawq/pull/739 +1 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-853) Master standby should avoid incomplete split operation
[ https://issues.apache.org/jira/browse/HAWQ-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345622#comment-15345622 ] ASF GitHub Bot commented on HAWQ-853: - Github user jiny2 commented on the issue: https://github.com/apache/incubator-hawq/pull/739 +1 LGTM > Master standby should avoid incomplete split operation > -- > > Key: HAWQ-853 > URL: https://issues.apache.org/jira/browse/HAWQ-853 > Project: Apache HAWQ > Issue Type: Bug > Components: Standby master >Reporter: Lin Wen >Assignee: Lin Wen > > Master standby performs replay of xlog records within a range of > checkpoints similar to crash recovery. During this is it may replay > incomplete multi-step operation at the end of the range. This generates > xlog. But Master standby should not generate xlog as it does not have > WAL subsystem active. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #743: HAWQ-855. Make resource manager ask for cl...
GitHub user jiny2 opened a pull request: https://github.com/apache/incubator-hawq/pull/743 HAWQ-855. Make resource manager ask for cluster report quickly when YARN is detected down You can merge this pull request into a Git repository by running: $ git pull https://github.com/jiny2/incubator-hawq HAWQ-855 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/743.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #743 commit 4c0ffc5861a89c77b18128300a2ce846b5ebde12 Author: YI JIN Date: 2016-06-23T03:15:52Z HAWQ-855. Make resource manager ask for cluster report quickly when YARN is detected down --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #743: HAWQ-855. Make resource manager ask for cl...
Github user jiny2 closed the pull request at: https://github.com/apache/incubator-hawq/pull/743 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #743: HAWQ-855. Make resource manager ask for cluster r...
Github user jiny2 commented on the issue: https://github.com/apache/incubator-hawq/pull/743 Close and open another one, this is not a clean commit --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-855) Make resource manager ask for cluster report quickly when YARN is detected down
[ https://issues.apache.org/jira/browse/HAWQ-855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345635#comment-15345635 ] ASF GitHub Bot commented on HAWQ-855: - GitHub user jiny2 opened a pull request: https://github.com/apache/incubator-hawq/pull/743 HAWQ-855. Make resource manager ask for cluster report quickly when YARN is detected down You can merge this pull request into a Git repository by running: $ git pull https://github.com/jiny2/incubator-hawq HAWQ-855 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/743.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #743 commit 4c0ffc5861a89c77b18128300a2ce846b5ebde12 Author: YI JIN Date: 2016-06-23T03:15:52Z HAWQ-855. Make resource manager ask for cluster report quickly when YARN is detected down > Make resource manager ask for cluster report quickly when YARN is detected > down > --- > > Key: HAWQ-855 > URL: https://issues.apache.org/jira/browse/HAWQ-855 > Project: Apache HAWQ > Issue Type: Improvement > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > > This improvement is for facilitating test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-855) Make resource manager ask for cluster report quickly when YARN is detected down
[ https://issues.apache.org/jira/browse/HAWQ-855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345638#comment-15345638 ] ASF GitHub Bot commented on HAWQ-855: - Github user jiny2 closed the pull request at: https://github.com/apache/incubator-hawq/pull/743 > Make resource manager ask for cluster report quickly when YARN is detected > down > --- > > Key: HAWQ-855 > URL: https://issues.apache.org/jira/browse/HAWQ-855 > Project: Apache HAWQ > Issue Type: Improvement > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > > This improvement is for facilitating test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-855) Make resource manager ask for cluster report quickly when YARN is detected down
[ https://issues.apache.org/jira/browse/HAWQ-855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345637#comment-15345637 ] ASF GitHub Bot commented on HAWQ-855: - Github user jiny2 commented on the issue: https://github.com/apache/incubator-hawq/pull/743 Close and open another one, this is not a clean commit > Make resource manager ask for cluster report quickly when YARN is detected > down > --- > > Key: HAWQ-855 > URL: https://issues.apache.org/jira/browse/HAWQ-855 > Project: Apache HAWQ > Issue Type: Improvement > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > > This improvement is for facilitating test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #744: HAWQ-855. Make resource manager ask for cl...
GitHub user jiny2 opened a pull request: https://github.com/apache/incubator-hawq/pull/744 HAWQ-855. Make resource manager ask for cluster report quickly when YARN is detected down You can merge this pull request into a Git repository by running: $ git pull https://github.com/jiny2/incubator-hawq HAWQ-855-2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/744.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #744 commit dd64de1f660430c1a712c469eb10f13ca846b040 Author: YI JIN Date: 2016-06-23T03:20:25Z HAWQ-855. Make resource manager ask for cluster report quickly when YARN is detected down --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-855) Make resource manager ask for cluster report quickly when YARN is detected down
[ https://issues.apache.org/jira/browse/HAWQ-855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345642#comment-15345642 ] ASF GitHub Bot commented on HAWQ-855: - GitHub user jiny2 opened a pull request: https://github.com/apache/incubator-hawq/pull/744 HAWQ-855. Make resource manager ask for cluster report quickly when YARN is detected down You can merge this pull request into a Git repository by running: $ git pull https://github.com/jiny2/incubator-hawq HAWQ-855-2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/744.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #744 commit dd64de1f660430c1a712c469eb10f13ca846b040 Author: YI JIN Date: 2016-06-23T03:20:25Z HAWQ-855. Make resource manager ask for cluster report quickly when YARN is detected down > Make resource manager ask for cluster report quickly when YARN is detected > down > --- > > Key: HAWQ-855 > URL: https://issues.apache.org/jira/browse/HAWQ-855 > Project: Apache HAWQ > Issue Type: Improvement > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > > This improvement is for facilitating test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #744: HAWQ-855. Make resource manager ask for cluster r...
Github user linwen commented on the issue: https://github.com/apache/incubator-hawq/pull/744 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-855) Make resource manager ask for cluster report quickly when YARN is detected down
[ https://issues.apache.org/jira/browse/HAWQ-855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345645#comment-15345645 ] ASF GitHub Bot commented on HAWQ-855: - Github user linwen commented on the issue: https://github.com/apache/incubator-hawq/pull/744 +1 > Make resource manager ask for cluster report quickly when YARN is detected > down > --- > > Key: HAWQ-855 > URL: https://issues.apache.org/jira/browse/HAWQ-855 > Project: Apache HAWQ > Issue Type: Improvement > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > > This improvement is for facilitating test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #744: HAWQ-855. Make resource manager ask for cluster r...
Github user ictmalili commented on the issue: https://github.com/apache/incubator-hawq/pull/744 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-855) Make resource manager ask for cluster report quickly when YARN is detected down
[ https://issues.apache.org/jira/browse/HAWQ-855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345649#comment-15345649 ] ASF GitHub Bot commented on HAWQ-855: - Github user ictmalili commented on the issue: https://github.com/apache/incubator-hawq/pull/744 +1 > Make resource manager ask for cluster report quickly when YARN is detected > down > --- > > Key: HAWQ-855 > URL: https://issues.apache.org/jira/browse/HAWQ-855 > Project: Apache HAWQ > Issue Type: Improvement > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > > This improvement is for facilitating test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #745: HAWQ-850. Planner supports refineCachedPla...
GitHub user zhangh43 opened a pull request: https://github.com/apache/incubator-hawq/pull/745 HAWQ-850. Planner supports refineCachedPlan interface. in PBE(prepare bind execution) cases, plan will be cached, and only be generated once in postgres. But in HAWQ since resource is dynamic or elastic, the plan maybe changed during the different execution of PBE. Also, the file split allocation result which is stored in plan is dynamically too, so the split-vseg mapping may also be changed. For example, We call a plpgsql function which do "insert into t select * from t". in Prepare we cache the plan, with fixed split-vseg mapping(e.g. 10 blocks), but after insert, there would be 20 blocks so the old split-vseg mapping is invalid. We need to update it. Also when we call this function again and again, the data size of t may request more resource to handle it than the first call, so we should recalculate the entire plan with the new vseg number. This function is implemented in a interface called refineCachedPlan. [Link to Jira](https://issues.apache.org/jira/browse/HAWQ-850) You can merge this pull request into a Git repository by running: $ git pull https://github.com/zhangh43/incubator-hawq hawq850 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/745.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #745 commit ee4cdf7a694ccf62f0548dd0a52ddb874f9976dd Author: hzhang2 Date: 2016-06-22T06:36:51Z HAWQ-850. Planner supports refineCachedPlan interface. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-850) Planner supports refineCachedPlan interface.
[ https://issues.apache.org/jira/browse/HAWQ-850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345730#comment-15345730 ] ASF GitHub Bot commented on HAWQ-850: - GitHub user zhangh43 opened a pull request: https://github.com/apache/incubator-hawq/pull/745 HAWQ-850. Planner supports refineCachedPlan interface. in PBE(prepare bind execution) cases, plan will be cached, and only be generated once in postgres. But in HAWQ since resource is dynamic or elastic, the plan maybe changed during the different execution of PBE. Also, the file split allocation result which is stored in plan is dynamically too, so the split-vseg mapping may also be changed. For example, We call a plpgsql function which do "insert into t select * from t". in Prepare we cache the plan, with fixed split-vseg mapping(e.g. 10 blocks), but after insert, there would be 20 blocks so the old split-vseg mapping is invalid. We need to update it. Also when we call this function again and again, the data size of t may request more resource to handle it than the first call, so we should recalculate the entire plan with the new vseg number. This function is implemented in a interface called refineCachedPlan. [Link to Jira](https://issues.apache.org/jira/browse/HAWQ-850) You can merge this pull request into a Git repository by running: $ git pull https://github.com/zhangh43/incubator-hawq hawq850 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/745.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #745 commit ee4cdf7a694ccf62f0548dd0a52ddb874f9976dd Author: hzhang2 Date: 2016-06-22T06:36:51Z HAWQ-850. Planner supports refineCachedPlan interface. > Planner supports refineCachedPlan interface. > > > Key: HAWQ-850 > URL: https://issues.apache.org/jira/browse/HAWQ-850 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Core >Reporter: Hubert Zhang >Assignee: Hubert Zhang > > in PBE(prepare bind execution) cases, plan will be cached, and only be > generated once in postgres. > But in HAWQ since resource is dynamic or elastic, the plan maybe changed > during the different execution of PBE. Also, the file split allocation result > which is stored in plan is dynamically too, so the split-vseg mapping may > also be changed. > For example, We call a plpgsql function which do "insert into t select * from > t". > in Prepare we cache the plan, with fixed split-vseg mapping(e.g. 10 blocks), > but after insert, there would be 20 blocks so the old split-vseg mapping is > invalid. We need to update it. Also when we call this function again and > again, the data size of t may request more resource to handle it than the > first call, so we should recalculate the entire plan with the new vseg number. > This function is implemented in a interface called refineCachedPlan. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-786) Framework to support pluggable formats and file systems
[ https://issues.apache.org/jira/browse/HAWQ-786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei Chang updated HAWQ-786: --- Summary: Framework to support pluggable formats and file systems (was: HAWQ supports ORC as a native storage format) > Framework to support pluggable formats and file systems > --- > > Key: HAWQ-786 > URL: https://issues.apache.org/jira/browse/HAWQ-786 > Project: Apache HAWQ > Issue Type: Wish > Components: Storage >Reporter: Lei Chang >Assignee: hongwu > Attachments: ORCdesign-v0.1-2016-06-17.pdf > > > In current HAWQ, two native formats are supported: AO and parquet. Now we > want to support ORC. A framework to support naive c/c++ pluggable format is > needed to support ORC more easily. And it can also be potentially used for > fast external data access. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-786) Framework to support pluggable formats and file systems
[ https://issues.apache.org/jira/browse/HAWQ-786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei Chang updated HAWQ-786: --- Description: In current HAWQ, two native formats are supported: AO and parquet. Now we want to support ORC. A framework to support naive c/c++ pluggable format is needed to support ORC more easily. And it can also be potentially used for fast external data access. And there are a lot of requests for supporting S3, Ceph and other file systems, and this is closely related to pluggable formats, so this JIRA is proposing a framework to support both. was:In current HAWQ, two native formats are supported: AO and parquet. Now we want to support ORC. A framework to support naive c/c++ pluggable format is needed to support ORC more easily. And it can also be potentially used for fast external data access. > Framework to support pluggable formats and file systems > --- > > Key: HAWQ-786 > URL: https://issues.apache.org/jira/browse/HAWQ-786 > Project: Apache HAWQ > Issue Type: Wish > Components: Storage >Reporter: Lei Chang >Assignee: hongwu > Attachments: ORCdesign-v0.1-2016-06-17.pdf > > > In current HAWQ, two native formats are supported: AO and parquet. Now we > want to support ORC. A framework to support naive c/c++ pluggable format is > needed to support ORC more easily. And it can also be potentially used for > fast external data access. > And there are a lot of requests for supporting S3, Ceph and other file > systems, and this is closely related to pluggable formats, so this JIRA is > proposing a framework to support both. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-864) Support ORC as a native file format
Lei Chang created HAWQ-864: -- Summary: Support ORC as a native file format Key: HAWQ-864 URL: https://issues.apache.org/jira/browse/HAWQ-864 Project: Apache HAWQ Issue Type: Wish Components: Storage Reporter: Lei Chang Assignee: Lei Chang -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-864) Support ORC as a native file format
[ https://issues.apache.org/jira/browse/HAWQ-864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei Chang updated HAWQ-864: --- Description: ORC (Optimized Row Columnar) is a very popular open source format adopted in some major components in Hadoop ecosystem. It is also used by a lot of users. The advantages of supporting ORC storage in HAWQ are in two folds: firstly, it makes HAWQ more Hadoop native which interacts with other components more easily; secondly, ORC stores some meta info for query optimization, thus, it might potentially outperform two native formats (i.e., AO, Parquet) if it is available. > Support ORC as a native file format > --- > > Key: HAWQ-864 > URL: https://issues.apache.org/jira/browse/HAWQ-864 > Project: Apache HAWQ > Issue Type: Wish > Components: Storage >Reporter: Lei Chang >Assignee: Lei Chang > > ORC (Optimized Row Columnar) is a very popular open source format adopted in > some major > components in Hadoop ecosystem. It is also used by a lot of users. The > advantages of > supporting ORC storage in HAWQ are in two folds: firstly, it makes HAWQ more > Hadoop native > which interacts with other components more easily; secondly, ORC stores some > meta info for > query optimization, thus, it might potentially outperform two native formats > (i.e., AO, Parquet) if it > is available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-864) Support ORC as a native file format
[ https://issues.apache.org/jira/browse/HAWQ-864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei Chang updated HAWQ-864: --- Description: ORC (Optimized Row Columnar) is a very popular open source format adopted in some major components in Hadoop ecosystem. It is also used by a lot of users. The advantages of supporting ORC storage in HAWQ are in two folds: firstly, it makes HAWQ more Hadoop native which interacts with other components more easily; secondly, ORC stores some meta info for query optimization, thus, it might potentially outperform two native formats (i.e., AO, Parquet) if it is available. The implementation can be based on the framework proposed in HAWQ-786. was: ORC (Optimized Row Columnar) is a very popular open source format adopted in some major components in Hadoop ecosystem. It is also used by a lot of users. The advantages of supporting ORC storage in HAWQ are in two folds: firstly, it makes HAWQ more Hadoop native which interacts with other components more easily; secondly, ORC stores some meta info for query optimization, thus, it might potentially outperform two native formats (i.e., AO, Parquet) if it is available. > Support ORC as a native file format > --- > > Key: HAWQ-864 > URL: https://issues.apache.org/jira/browse/HAWQ-864 > Project: Apache HAWQ > Issue Type: Wish > Components: Storage >Reporter: Lei Chang >Assignee: Lei Chang > > ORC (Optimized Row Columnar) is a very popular open source format adopted in > some major > components in Hadoop ecosystem. It is also used by a lot of users. The > advantages of > supporting ORC storage in HAWQ are in two folds: firstly, it makes HAWQ more > Hadoop native > which interacts with other components more easily; secondly, ORC stores some > meta info for > query optimization, thus, it might potentially outperform two native formats > (i.e., AO, Parquet) if it > is available. > The implementation can be based on the framework proposed in HAWQ-786. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
