[jira] [Closed] (HAWQ-862) Make user defined function get_ao_distribution work.

[Hubert Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hubert Zhang closed HAWQ-862.
-
Resolution: Fixed

> Make user defined function get_ao_distribution work.
> 
>
> Key: HAWQ-862
> URL: https://issues.apache.org/jira/browse/HAWQ-862
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core
>Reporter: Hubert Zhang
>Assignee: Hubert Zhang
>
> In Hawq we had a functon that could do a fast row-count on AO tables based on 
> querying the catalog versus issuing 'select count(*) ' SQL.  Was a lot faster 
> for larger tables.  I found the script and tried using it with HAWQ but one 
> of the underlying Postgres functions that gets called (get_ao_distribution) 
> bombs since it doesn't seem to be able to find the table files when they are 
> stored in HDFS.
> gpadmin=# select sum(tupcount) from get_ao_distribution('public.foo');
> ERROR:  could not open relation 1663/24731/24740: No such file or directory 
> (seg0 localhost.localdomain:4 pid=82964)
> DETAIL:   Database directory "base/24731" does not exist
> CONTEXT:  SQL statement "select gp_segment_id, sum(tupcount) from 
> gp_dist_random('pg_aoseg.pg_aoseg_24738') group by (gp_segment_id)"
> gpadmin=#



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (HAWQ-861) Wrong logerror position in insert into hashtable select * from gpfdist external table.

[Hubert Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hubert Zhang closed HAWQ-861.
-
Resolution: Fixed

> Wrong logerror position in insert into hashtable select * from gpfdist 
> external table.
> --
>
> Key: HAWQ-861
> URL: https://issues.apache.org/jira/browse/HAWQ-861
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core
>Reporter: Hubert Zhang
>Assignee: Hubert Zhang
>
> For statement insert into hashtable select * from gpfdist external table,
> when hashtable bucket number is less than location number of gpfdist external 
> table, error should be reported in cdbdatalocality module, with error message 
> like this:
> ERROR:  Could not allocate enough memory! bucket number of result hash table 
> and external table should match each other (cdbdatalocality.c:4222)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-712) Add GUC gp_vmem_idle_resource_timeout back to do timeout for gang release in idle session

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo updated HAWQ-712:
-
Fix Version/s: (was: 2.1.0)
   2.0.0

> Add GUC gp_vmem_idle_resource_timeout back to do timeout for gang release in 
> idle session
> -
>
> Key: HAWQ-712
> URL: https://issues.apache.org/jira/browse/HAWQ-712
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> The guc gp_vmem_idle_resource_timeout has been removed during GUC cleanup for 
> hawq 2.0 by mistake. That means hawq now uses default timeout value and we 
> don't have a way to set/show it. Please find details 
> [HAWQ-242|https://issues.apache.org/jira/browse/HAWQ-242] and its 
> [commit|https://github.com/apache/incubator-hawq/commit/8ac320522baf507432d0c5b16b8df29244312785#diff-4428096e6ef1d602d60cb6b14ee1e38f].
> We need to add it back so that the timeout value for gang release in idle 
> session can be customized.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (HAWQ-712) Add GUC gp_vmem_idle_resource_timeout back to do timeout for gang release in idle session

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo closed HAWQ-712.


> Add GUC gp_vmem_idle_resource_timeout back to do timeout for gang release in 
> idle session
> -
>
> Key: HAWQ-712
> URL: https://issues.apache.org/jira/browse/HAWQ-712
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> The guc gp_vmem_idle_resource_timeout has been removed during GUC cleanup for 
> hawq 2.0 by mistake. That means hawq now uses default timeout value and we 
> don't have a way to set/show it. Please find details 
> [HAWQ-242|https://issues.apache.org/jira/browse/HAWQ-242] and its 
> [commit|https://github.com/apache/incubator-hawq/commit/8ac320522baf507432d0c5b16b8df29244312785#diff-4428096e6ef1d602d60cb6b14ee1e38f].
> We need to add it back so that the timeout value for gang release in idle 
> session can be customized.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (HAWQ-716) Support code coverage build and report in hawq Makefile

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo closed HAWQ-716.


> Support code coverage build and report in hawq Makefile
> ---
>
> Key: HAWQ-716
> URL: https://issues.apache.org/jira/browse/HAWQ-716
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Build
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> We need to support code coverage build and report in hawq Makefile. The scope 
> include:
> 1. Build hawq/libhdfs3/libyarn binaries with code coverage enabled
> 2. Generate unified code coverage report for hawq/libhdfs3/libyarn



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HAWQ-716) Support code coverage build and report in hawq Makefile

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo reopened HAWQ-716:
--

Reopening to add the fix version.

> Support code coverage build and report in hawq Makefile
> ---
>
> Key: HAWQ-716
> URL: https://issues.apache.org/jira/browse/HAWQ-716
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Build
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> We need to support code coverage build and report in hawq Makefile. The scope 
> include:
> 1. Build hawq/libhdfs3/libyarn binaries with code coverage enabled
> 2. Generate unified code coverage report for hawq/libhdfs3/libyarn



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HAWQ-716) Support code coverage build and report in hawq Makefile

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo resolved HAWQ-716.
--
Resolution: Fixed

> Support code coverage build and report in hawq Makefile
> ---
>
> Key: HAWQ-716
> URL: https://issues.apache.org/jira/browse/HAWQ-716
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: Build
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> We need to support code coverage build and report in hawq Makefile. The scope 
> include:
> 1. Build hawq/libhdfs3/libyarn binaries with code coverage enabled
> 2. Generate unified code coverage report for hawq/libhdfs3/libyarn



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-733) Command utility takes about 75 sec to process result

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo updated HAWQ-733:
-
Fix Version/s: (was: 2.1.0)
   2.0.0

> Command utility takes about 75 sec to process result
> 
>
> Key: HAWQ-733
> URL: https://issues.apache.org/jira/browse/HAWQ-733
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
>Priority: Minor
> Fix For: 2.0.0
>
>
> Command utility is designed to run a shell command in hawq test. It follows 
> below steps to run command:
> {noformat}
> Step 1: prepare the shell command string
> Step 2: run the command using popen
> Step 3: save the result of command in memory (i.e., stdio returned from popen)
> Step 4: check the status of the command (i.e., status returned from popen)
> Step 5: save the result to file if output file is specified
> {noformat}
> Now it takes about 75 sec to save the result of command in memory in step 3 
> if the command is to run SQL file using psql. This needs to be fixed so as to 
> improved the performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HAWQ-733) Command utility takes about 75 sec to process result

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo reopened HAWQ-733:
--

Reopening to add the fix version.

> Command utility takes about 75 sec to process result
> 
>
> Key: HAWQ-733
> URL: https://issues.apache.org/jira/browse/HAWQ-733
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
>Priority: Minor
> Fix For: 2.0.0
>
>
> Command utility is designed to run a shell command in hawq test. It follows 
> below steps to run command:
> {noformat}
> Step 1: prepare the shell command string
> Step 2: run the command using popen
> Step 3: save the result of command in memory (i.e., stdio returned from popen)
> Step 4: check the status of the command (i.e., status returned from popen)
> Step 5: save the result to file if output file is specified
> {noformat}
> Now it takes about 75 sec to save the result of command in memory in step 3 
> if the command is to run SQL file using psql. This needs to be fixed so as to 
> improved the performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (HAWQ-733) Command utility takes about 75 sec to process result

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo closed HAWQ-733.


> Command utility takes about 75 sec to process result
> 
>
> Key: HAWQ-733
> URL: https://issues.apache.org/jira/browse/HAWQ-733
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
>Priority: Minor
> Fix For: 2.0.0
>
>
> Command utility is designed to run a shell command in hawq test. It follows 
> below steps to run command:
> {noformat}
> Step 1: prepare the shell command string
> Step 2: run the command using popen
> Step 3: save the result of command in memory (i.e., stdio returned from popen)
> Step 4: check the status of the command (i.e., status returned from popen)
> Step 5: save the result to file if output file is specified
> {noformat}
> Now it takes about 75 sec to save the result of command in memory in step 3 
> if the command is to run SQL file using psql. This needs to be fixed so as to 
> improved the performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HAWQ-733) Command utility takes about 75 sec to process result

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo resolved HAWQ-733.
--
Resolution: Fixed

> Command utility takes about 75 sec to process result
> 
>
> Key: HAWQ-733
> URL: https://issues.apache.org/jira/browse/HAWQ-733
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
>Priority: Minor
> Fix For: 2.0.0
>
>
> Command utility is designed to run a shell command in hawq test. It follows 
> below steps to run command:
> {noformat}
> Step 1: prepare the shell command string
> Step 2: run the command using popen
> Step 3: save the result of command in memory (i.e., stdio returned from popen)
> Step 4: check the status of the command (i.e., status returned from popen)
> Step 5: save the result to file if output file is specified
> {noformat}
> Now it takes about 75 sec to save the result of command in memory in step 3 
> if the command is to run SQL file using psql. This needs to be fixed so as to 
> improved the performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HAWQ-738) Allocate query resource twice in function call through jdbc

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo resolved HAWQ-738.
--
Resolution: Fixed

> Allocate query resource twice in function call through jdbc
> ---
>
> Key: HAWQ-738
> URL: https://issues.apache.org/jira/browse/HAWQ-738
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> It allocates query resource twice in function call through jdbc, one in 
> parse, and the other in bind. Though the same thing works with psql.
> Use runme.sh in attached bug.zip to reproduce the issue. It may raise below 
> error on host with limited resource (i.e., low memory, etc).
> {noformat}
> [gpadmin@localhost debug]$ ./runme.sh 
> java -classpath /home/gpadmin/debug/Bug.jar:/home/gpadmin/debug/gpdb.jar Bug 
> localhost 5432 gpadmin gpadmin changeme
> gpServer: hdp23
> gpPort: 5432
> gpDatabase: gpadmin
> gpUserName: gpadmin
> gpPassword: changeme
> DriverManager.getConnection("jdbc:postgresql://hdp23:5432/gpadmin")
> trying sun.jdbc.odbc.JdbcOdbcDriver
> *Driver.connect (jdbc:postgresql://hdp23:5432/gpadmin)
> trying org.postgresql.Driver
> getConnection returning org.postgresql.Driver
> strSQL: DROP TABLE IF EXISTS public.debug;
> CREATE TABLE public.debug
> (id int, foo_bar text)
> DISTRIBUTED RANDOMLY;
> strSQL: INSERT INTO public.debug
> SELECT i, 'foo_' || i from generate_series(1,100) AS i;
> strSQL: CREATE OR REPLACE FUNCTION public.fn_debug() RETURNS text AS
> $$
> DECLARE
>   v_return text;
> BEGIN
>   SELECT foo_bar
>   INTO v_return
>   FROM public.debug
>   WHERE id = 1;
>   RETURN v_return;
> END;
> $$
> LANGUAGE plpgsql;
> strSQL: SELECT public.fn_debug()
> org.postgresql.util.PSQLException: ERROR: failed to acquire resource from 
> resource manager, session 32 deadlock is detected (pquery.c:804)
>   at 
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2102)
>   at 
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1835)
>   at 
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
>   at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:500)
>   at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:374)
>   at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:254)
>   at Bug.getFunctionResults(Bug.java:144)
>   at Bug.main(Bug.java:41)
> SQLException: SQLState(XX000)
> ERROR: failed to acquire resource from resource manager, session 32 deadlock 
> is detected (pquery.c:804)
> Exception in thread "main" java.sql.SQLException: ERROR: failed to acquire 
> resource from resource manager, session 32 deadlock is detected (pquery.c:804)
>   at Bug.main(Bug.java:49)
> {noformat}
> while the expected result is as below:
> {noformat}
> [gpadmin@localhost hawq_bug]$ ./runme.sh
> java -classpath 
> /home/gpadmin/huor/hawq_bug/Bug.jar:/home/gpadmin/huor/hawq_bug/gpdb.jar Bug 
> localhost 5432 gptest gpadmin changeme
> gpServer: localhost
> gpPort: 5432
> gpDatabase: gptest
> gpUserName: gpadmin
> gpPassword: changeme
> DriverManager.getConnection("jdbc:postgresql://localhost:5432/gptest")
> trying sun.jdbc.odbc.JdbcOdbcDriver
> *Driver.connect (jdbc:postgresql://localhost:5432/gptest)
> trying org.postgresql.Driver
> getConnection returning org.postgresql.Driver
> strSQL: DROP TABLE IF EXISTS public.debug;
> CREATE TABLE public.debug
> (id int, foo_bar text)
> DISTRIBUTED RANDOMLY;
> SQLWarning:
> strSQL: INSERT INTO public.debug
> SELECT i, 'foo_' || i from generate_series(1,100) AS i;
> strSQL: CREATE OR REPLACE FUNCTION public.fn_debug() RETURNS text AS
> $$
> DECLARE
> v_return text;
> BEGIN
> SELECT foo_bar
> INTO v_return
> FROM public.debug
> WHERE id = 1;
> RETURN v_return;
> END;
> $$
> LANGUAGE plpgsql;
> strSQL: SELECT public.fn_debug()
> output: foo_1
> {noformat}
> If you look into the pg_log on master, you can see it allocate query resource 
> twice for the function call:
> {noformat}
> rhuo-mbp:jdbc rhuo$ cat hawq-2016-05-14_00.csv
> 2016-05-16 13:50:50.255504 
> EDT,,,p4522,th4240651520,con4,,seg-1,"LOG","0","Resource 
> manager adjusts segment hdp23.localdomain original resource capacity from 
> (2048 MB, 48 CORE) to (2048 MB, 2 CORE)",,,0,,"resourcepool.c",4688,
> 2016-05-16 13:51:20.275204 
> EDT,,,p4522,th4240651520,con4,,seg-1,"LOG","0","Resource 
> manager adjusts segment hdp23.localdomain original resource capacity from 
> (2048 MB, 48 CORE) to (2048 MB, 2 

[jira] [Updated] (HAWQ-738) Allocate query resource twice in function call through jdbc

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo updated HAWQ-738:
-
Fix Version/s: (was: 2.1.0)
   2.0.0

> Allocate query resource twice in function call through jdbc
> ---
>
> Key: HAWQ-738
> URL: https://issues.apache.org/jira/browse/HAWQ-738
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> It allocates query resource twice in function call through jdbc, one in 
> parse, and the other in bind. Though the same thing works with psql.
> Use runme.sh in attached bug.zip to reproduce the issue. It may raise below 
> error on host with limited resource (i.e., low memory, etc).
> {noformat}
> [gpadmin@localhost debug]$ ./runme.sh 
> java -classpath /home/gpadmin/debug/Bug.jar:/home/gpadmin/debug/gpdb.jar Bug 
> localhost 5432 gpadmin gpadmin changeme
> gpServer: hdp23
> gpPort: 5432
> gpDatabase: gpadmin
> gpUserName: gpadmin
> gpPassword: changeme
> DriverManager.getConnection("jdbc:postgresql://hdp23:5432/gpadmin")
> trying sun.jdbc.odbc.JdbcOdbcDriver
> *Driver.connect (jdbc:postgresql://hdp23:5432/gpadmin)
> trying org.postgresql.Driver
> getConnection returning org.postgresql.Driver
> strSQL: DROP TABLE IF EXISTS public.debug;
> CREATE TABLE public.debug
> (id int, foo_bar text)
> DISTRIBUTED RANDOMLY;
> strSQL: INSERT INTO public.debug
> SELECT i, 'foo_' || i from generate_series(1,100) AS i;
> strSQL: CREATE OR REPLACE FUNCTION public.fn_debug() RETURNS text AS
> $$
> DECLARE
>   v_return text;
> BEGIN
>   SELECT foo_bar
>   INTO v_return
>   FROM public.debug
>   WHERE id = 1;
>   RETURN v_return;
> END;
> $$
> LANGUAGE plpgsql;
> strSQL: SELECT public.fn_debug()
> org.postgresql.util.PSQLException: ERROR: failed to acquire resource from 
> resource manager, session 32 deadlock is detected (pquery.c:804)
>   at 
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2102)
>   at 
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1835)
>   at 
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
>   at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:500)
>   at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:374)
>   at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:254)
>   at Bug.getFunctionResults(Bug.java:144)
>   at Bug.main(Bug.java:41)
> SQLException: SQLState(XX000)
> ERROR: failed to acquire resource from resource manager, session 32 deadlock 
> is detected (pquery.c:804)
> Exception in thread "main" java.sql.SQLException: ERROR: failed to acquire 
> resource from resource manager, session 32 deadlock is detected (pquery.c:804)
>   at Bug.main(Bug.java:49)
> {noformat}
> while the expected result is as below:
> {noformat}
> [gpadmin@localhost hawq_bug]$ ./runme.sh
> java -classpath 
> /home/gpadmin/huor/hawq_bug/Bug.jar:/home/gpadmin/huor/hawq_bug/gpdb.jar Bug 
> localhost 5432 gptest gpadmin changeme
> gpServer: localhost
> gpPort: 5432
> gpDatabase: gptest
> gpUserName: gpadmin
> gpPassword: changeme
> DriverManager.getConnection("jdbc:postgresql://localhost:5432/gptest")
> trying sun.jdbc.odbc.JdbcOdbcDriver
> *Driver.connect (jdbc:postgresql://localhost:5432/gptest)
> trying org.postgresql.Driver
> getConnection returning org.postgresql.Driver
> strSQL: DROP TABLE IF EXISTS public.debug;
> CREATE TABLE public.debug
> (id int, foo_bar text)
> DISTRIBUTED RANDOMLY;
> SQLWarning:
> strSQL: INSERT INTO public.debug
> SELECT i, 'foo_' || i from generate_series(1,100) AS i;
> strSQL: CREATE OR REPLACE FUNCTION public.fn_debug() RETURNS text AS
> $$
> DECLARE
> v_return text;
> BEGIN
> SELECT foo_bar
> INTO v_return
> FROM public.debug
> WHERE id = 1;
> RETURN v_return;
> END;
> $$
> LANGUAGE plpgsql;
> strSQL: SELECT public.fn_debug()
> output: foo_1
> {noformat}
> If you look into the pg_log on master, you can see it allocate query resource 
> twice for the function call:
> {noformat}
> rhuo-mbp:jdbc rhuo$ cat hawq-2016-05-14_00.csv
> 2016-05-16 13:50:50.255504 
> EDT,,,p4522,th4240651520,con4,,seg-1,"LOG","0","Resource 
> manager adjusts segment hdp23.localdomain original resource capacity from 
> (2048 MB, 48 CORE) to (2048 MB, 2 CORE)",,,0,,"resourcepool.c",4688,
> 2016-05-16 13:51:20.275204 
> EDT,,,p4522,th4240651520,con4,,seg-1,"LOG","0","Resource 
> manager adjusts segment hdp23.localdomain original resource capacity from 
> (2048 MB, 48 CORE) to 

[jira] [Closed] (HAWQ-738) Allocate query resource twice in function call through jdbc

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo closed HAWQ-738.


> Allocate query resource twice in function call through jdbc
> ---
>
> Key: HAWQ-738
> URL: https://issues.apache.org/jira/browse/HAWQ-738
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> It allocates query resource twice in function call through jdbc, one in 
> parse, and the other in bind. Though the same thing works with psql.
> Use runme.sh in attached bug.zip to reproduce the issue. It may raise below 
> error on host with limited resource (i.e., low memory, etc).
> {noformat}
> [gpadmin@localhost debug]$ ./runme.sh 
> java -classpath /home/gpadmin/debug/Bug.jar:/home/gpadmin/debug/gpdb.jar Bug 
> localhost 5432 gpadmin gpadmin changeme
> gpServer: hdp23
> gpPort: 5432
> gpDatabase: gpadmin
> gpUserName: gpadmin
> gpPassword: changeme
> DriverManager.getConnection("jdbc:postgresql://hdp23:5432/gpadmin")
> trying sun.jdbc.odbc.JdbcOdbcDriver
> *Driver.connect (jdbc:postgresql://hdp23:5432/gpadmin)
> trying org.postgresql.Driver
> getConnection returning org.postgresql.Driver
> strSQL: DROP TABLE IF EXISTS public.debug;
> CREATE TABLE public.debug
> (id int, foo_bar text)
> DISTRIBUTED RANDOMLY;
> strSQL: INSERT INTO public.debug
> SELECT i, 'foo_' || i from generate_series(1,100) AS i;
> strSQL: CREATE OR REPLACE FUNCTION public.fn_debug() RETURNS text AS
> $$
> DECLARE
>   v_return text;
> BEGIN
>   SELECT foo_bar
>   INTO v_return
>   FROM public.debug
>   WHERE id = 1;
>   RETURN v_return;
> END;
> $$
> LANGUAGE plpgsql;
> strSQL: SELECT public.fn_debug()
> org.postgresql.util.PSQLException: ERROR: failed to acquire resource from 
> resource manager, session 32 deadlock is detected (pquery.c:804)
>   at 
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2102)
>   at 
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1835)
>   at 
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
>   at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:500)
>   at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:374)
>   at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:254)
>   at Bug.getFunctionResults(Bug.java:144)
>   at Bug.main(Bug.java:41)
> SQLException: SQLState(XX000)
> ERROR: failed to acquire resource from resource manager, session 32 deadlock 
> is detected (pquery.c:804)
> Exception in thread "main" java.sql.SQLException: ERROR: failed to acquire 
> resource from resource manager, session 32 deadlock is detected (pquery.c:804)
>   at Bug.main(Bug.java:49)
> {noformat}
> while the expected result is as below:
> {noformat}
> [gpadmin@localhost hawq_bug]$ ./runme.sh
> java -classpath 
> /home/gpadmin/huor/hawq_bug/Bug.jar:/home/gpadmin/huor/hawq_bug/gpdb.jar Bug 
> localhost 5432 gptest gpadmin changeme
> gpServer: localhost
> gpPort: 5432
> gpDatabase: gptest
> gpUserName: gpadmin
> gpPassword: changeme
> DriverManager.getConnection("jdbc:postgresql://localhost:5432/gptest")
> trying sun.jdbc.odbc.JdbcOdbcDriver
> *Driver.connect (jdbc:postgresql://localhost:5432/gptest)
> trying org.postgresql.Driver
> getConnection returning org.postgresql.Driver
> strSQL: DROP TABLE IF EXISTS public.debug;
> CREATE TABLE public.debug
> (id int, foo_bar text)
> DISTRIBUTED RANDOMLY;
> SQLWarning:
> strSQL: INSERT INTO public.debug
> SELECT i, 'foo_' || i from generate_series(1,100) AS i;
> strSQL: CREATE OR REPLACE FUNCTION public.fn_debug() RETURNS text AS
> $$
> DECLARE
> v_return text;
> BEGIN
> SELECT foo_bar
> INTO v_return
> FROM public.debug
> WHERE id = 1;
> RETURN v_return;
> END;
> $$
> LANGUAGE plpgsql;
> strSQL: SELECT public.fn_debug()
> output: foo_1
> {noformat}
> If you look into the pg_log on master, you can see it allocate query resource 
> twice for the function call:
> {noformat}
> rhuo-mbp:jdbc rhuo$ cat hawq-2016-05-14_00.csv
> 2016-05-16 13:50:50.255504 
> EDT,,,p4522,th4240651520,con4,,seg-1,"LOG","0","Resource 
> manager adjusts segment hdp23.localdomain original resource capacity from 
> (2048 MB, 48 CORE) to (2048 MB, 2 CORE)",,,0,,"resourcepool.c",4688,
> 2016-05-16 13:51:20.275204 
> EDT,,,p4522,th4240651520,con4,,seg-1,"LOG","0","Resource 
> manager adjusts segment hdp23.localdomain original resource capacity from 
> (2048 MB, 48 CORE) to (2048 MB, 2 CORE)",,,0,,"resourcepool.c",4688,
> 2016-05-16 

[jira] [Reopened] (HAWQ-738) Allocate query resource twice in function call through jdbc

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo reopened HAWQ-738:
--

Reopening to add the fix version.

> Allocate query resource twice in function call through jdbc
> ---
>
> Key: HAWQ-738
> URL: https://issues.apache.org/jira/browse/HAWQ-738
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> It allocates query resource twice in function call through jdbc, one in 
> parse, and the other in bind. Though the same thing works with psql.
> Use runme.sh in attached bug.zip to reproduce the issue. It may raise below 
> error on host with limited resource (i.e., low memory, etc).
> {noformat}
> [gpadmin@localhost debug]$ ./runme.sh 
> java -classpath /home/gpadmin/debug/Bug.jar:/home/gpadmin/debug/gpdb.jar Bug 
> localhost 5432 gpadmin gpadmin changeme
> gpServer: hdp23
> gpPort: 5432
> gpDatabase: gpadmin
> gpUserName: gpadmin
> gpPassword: changeme
> DriverManager.getConnection("jdbc:postgresql://hdp23:5432/gpadmin")
> trying sun.jdbc.odbc.JdbcOdbcDriver
> *Driver.connect (jdbc:postgresql://hdp23:5432/gpadmin)
> trying org.postgresql.Driver
> getConnection returning org.postgresql.Driver
> strSQL: DROP TABLE IF EXISTS public.debug;
> CREATE TABLE public.debug
> (id int, foo_bar text)
> DISTRIBUTED RANDOMLY;
> strSQL: INSERT INTO public.debug
> SELECT i, 'foo_' || i from generate_series(1,100) AS i;
> strSQL: CREATE OR REPLACE FUNCTION public.fn_debug() RETURNS text AS
> $$
> DECLARE
>   v_return text;
> BEGIN
>   SELECT foo_bar
>   INTO v_return
>   FROM public.debug
>   WHERE id = 1;
>   RETURN v_return;
> END;
> $$
> LANGUAGE plpgsql;
> strSQL: SELECT public.fn_debug()
> org.postgresql.util.PSQLException: ERROR: failed to acquire resource from 
> resource manager, session 32 deadlock is detected (pquery.c:804)
>   at 
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2102)
>   at 
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1835)
>   at 
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
>   at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:500)
>   at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:374)
>   at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:254)
>   at Bug.getFunctionResults(Bug.java:144)
>   at Bug.main(Bug.java:41)
> SQLException: SQLState(XX000)
> ERROR: failed to acquire resource from resource manager, session 32 deadlock 
> is detected (pquery.c:804)
> Exception in thread "main" java.sql.SQLException: ERROR: failed to acquire 
> resource from resource manager, session 32 deadlock is detected (pquery.c:804)
>   at Bug.main(Bug.java:49)
> {noformat}
> while the expected result is as below:
> {noformat}
> [gpadmin@localhost hawq_bug]$ ./runme.sh
> java -classpath 
> /home/gpadmin/huor/hawq_bug/Bug.jar:/home/gpadmin/huor/hawq_bug/gpdb.jar Bug 
> localhost 5432 gptest gpadmin changeme
> gpServer: localhost
> gpPort: 5432
> gpDatabase: gptest
> gpUserName: gpadmin
> gpPassword: changeme
> DriverManager.getConnection("jdbc:postgresql://localhost:5432/gptest")
> trying sun.jdbc.odbc.JdbcOdbcDriver
> *Driver.connect (jdbc:postgresql://localhost:5432/gptest)
> trying org.postgresql.Driver
> getConnection returning org.postgresql.Driver
> strSQL: DROP TABLE IF EXISTS public.debug;
> CREATE TABLE public.debug
> (id int, foo_bar text)
> DISTRIBUTED RANDOMLY;
> SQLWarning:
> strSQL: INSERT INTO public.debug
> SELECT i, 'foo_' || i from generate_series(1,100) AS i;
> strSQL: CREATE OR REPLACE FUNCTION public.fn_debug() RETURNS text AS
> $$
> DECLARE
> v_return text;
> BEGIN
> SELECT foo_bar
> INTO v_return
> FROM public.debug
> WHERE id = 1;
> RETURN v_return;
> END;
> $$
> LANGUAGE plpgsql;
> strSQL: SELECT public.fn_debug()
> output: foo_1
> {noformat}
> If you look into the pg_log on master, you can see it allocate query resource 
> twice for the function call:
> {noformat}
> rhuo-mbp:jdbc rhuo$ cat hawq-2016-05-14_00.csv
> 2016-05-16 13:50:50.255504 
> EDT,,,p4522,th4240651520,con4,,seg-1,"LOG","0","Resource 
> manager adjusts segment hdp23.localdomain original resource capacity from 
> (2048 MB, 48 CORE) to (2048 MB, 2 CORE)",,,0,,"resourcepool.c",4688,
> 2016-05-16 13:51:20.275204 
> EDT,,,p4522,th4240651520,con4,,seg-1,"LOG","0","Resource 
> manager adjusts segment hdp23.localdomain original resource capacity from 
> (2048 MB, 48 CORE) to (2048 MB, 2 

[jira] [Resolved] (HAWQ-766) Add feature test for partition using new feature test framework

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo resolved HAWQ-766.
--
Resolution: Fixed

> Add feature test for partition using new feature test framework
> ---
>
> Key: HAWQ-766
> URL: https://issues.apache.org/jira/browse/HAWQ-766
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> Need to migrate goh_partition test in installcheck-good to new feature test 
> framework.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-766) Add feature test for partition using new feature test framework

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo updated HAWQ-766:
-
Fix Version/s: (was: 2.1.0)
   2.0.0

> Add feature test for partition using new feature test framework
> ---
>
> Key: HAWQ-766
> URL: https://issues.apache.org/jira/browse/HAWQ-766
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> Need to migrate goh_partition test in installcheck-good to new feature test 
> framework.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (HAWQ-766) Add feature test for partition using new feature test framework

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo closed HAWQ-766.


> Add feature test for partition using new feature test framework
> ---
>
> Key: HAWQ-766
> URL: https://issues.apache.org/jira/browse/HAWQ-766
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> Need to migrate goh_partition test in installcheck-good to new feature test 
> framework.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HAWQ-766) Add feature test for partition using new feature test framework

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo reopened HAWQ-766:
--

Reopening to add the fix version.

> Add feature test for partition using new feature test framework
> ---
>
> Key: HAWQ-766
> URL: https://issues.apache.org/jira/browse/HAWQ-766
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.1.0
>
>
> Need to migrate goh_partition test in installcheck-good to new feature test 
> framework.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-769) Add feature test for function, function_extensions and set_functions of user-defined function using new feature test framework

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo updated HAWQ-769:
-
Fix Version/s: 2.0.0

> Add feature test for function, function_extensions and set_functions of 
> user-defined function using new feature test framework
> --
>
> Key: HAWQ-769
> URL: https://issues.apache.org/jira/browse/HAWQ-769
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> Need to migrate function, function_extensions, set_functions test in 
> installcheck-good to new feature test framework.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (HAWQ-769) Add feature test for function, function_extensions and set_functions of user-defined function using new feature test framework

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo closed HAWQ-769.


> Add feature test for function, function_extensions and set_functions of 
> user-defined function using new feature test framework
> --
>
> Key: HAWQ-769
> URL: https://issues.apache.org/jira/browse/HAWQ-769
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> Need to migrate function, function_extensions, set_functions test in 
> installcheck-good to new feature test framework.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HAWQ-769) Add feature test for function, function_extensions and set_functions of user-defined function using new feature test framework

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo resolved HAWQ-769.
--
Resolution: Fixed

> Add feature test for function, function_extensions and set_functions of 
> user-defined function using new feature test framework
> --
>
> Key: HAWQ-769
> URL: https://issues.apache.org/jira/browse/HAWQ-769
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> Need to migrate function, function_extensions, set_functions test in 
> installcheck-good to new feature test framework.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HAWQ-815) KeyError while compiling PLPython function due to deletion of non-existent record from Python global dict

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo resolved HAWQ-815.
--
Resolution: Fixed

> KeyError while compiling PLPython function due to deletion of non-existent 
> record from Python global dict
> -
>
> Key: HAWQ-815
> URL: https://issues.apache.org/jira/browse/HAWQ-815
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Query Execution
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> It raise KeyError while compiling PLPython function due to deletion of 
> non-existent record from Python global dict. Below are two example to 
> reproduce the issue:
> Example 1:
> {noformat}
> -- 1) prepare schema and data
> CREATE LANGUAGE plpythonu;
> CREATE TABLE gp_single_row(a int) DISTRIBUTED BY (a);
> INSERT INTO gp_single_row VALUES (1);
> CREATE OR REPLACE FUNCTION test_return_table(s TEXT)
> RETURNS TABLE(first TEXT, second INT4)
> AS $$
>exec('y = ' + s)
>return y
> $$ LANGUAGE plpythonu;
> SELECT (test_return_table('[]')).*;
> SELECT (test_return_table('[]')).*
> FROM gp_single_row;
> -- 2) Actual output
> SELECT (test_return_table('[]')).*;
>  first | second
> ---+
> (0 rows)
> SELECT (test_return_table('[]')).*
> FROM gp_single_row;
> ERROR:  could not compile PL/Python function "test_return_table" 
> (plpython.c:4651)  (seg5 test1:31100 pid=36826) (dispatcher.c:1801)
> DETAIL:  KeyError: 's'
> -- 3) Expected output
> SELECT (test_return_table('[]')).*;
>  first | second
> ---+
> (0 rows)
> SELECT (test_return_table('[]')).*
> FROM gp_single_row;
>  first | second
> ---+
> (0 rows)
> {noformat}
> Example 2:
> {noformat}
> -- 1) prepare schema and data
> CREATE LANGUAGE plpythonu;
> CREATE OR REPLACE FUNCTION func1(var_cause_bug int4[])
> RETURNS SETOF INT4 AS $$
> for el in var_cause_bug:
> yield el
> $$ LANGUAGE plpythonu;
> CREATE OR REPLACE FUNCTION func2()
> RETURNS INT4 AS $$
> return 1
> $$ LANGUAGE plpythonu;
> SELECT func1(ARRAY[1,2,3]), func1(ARRAY[1,2,3]);
> SELECT func2();
> -- 2) Actual output
> SELECT func1(ARRAY[1,2,3]), func1(ARRAY[1,2,3]);
>  func1 | func1 
> ---+---
>  1 | 1
>  2 | 2
>  3 | 3
> (3 rows)
> SELECT func2();
> ERROR:  could not compile PL/Python function "func2" (plpython.c:4648)
> DETAIL:  KeyError: 'var_cause_bug'
> -- 3) Expected output
> SELECT func1(ARRAY[1,2,3]), func1(ARRAY[1,2,3]);
>  func1 | func1
> ---+---
>  1 | 1
>  2 | 2
>  3 | 3
> (3 rows)
> SELECT func2();
>  func2
> ---
>  1
> (1 row)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-815) KeyError while compiling PLPython function due to deletion of non-existent record from Python global dict

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo updated HAWQ-815:
-
Fix Version/s: 2.0.0

> KeyError while compiling PLPython function due to deletion of non-existent 
> record from Python global dict
> -
>
> Key: HAWQ-815
> URL: https://issues.apache.org/jira/browse/HAWQ-815
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Query Execution
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> It raise KeyError while compiling PLPython function due to deletion of 
> non-existent record from Python global dict. Below are two example to 
> reproduce the issue:
> Example 1:
> {noformat}
> -- 1) prepare schema and data
> CREATE LANGUAGE plpythonu;
> CREATE TABLE gp_single_row(a int) DISTRIBUTED BY (a);
> INSERT INTO gp_single_row VALUES (1);
> CREATE OR REPLACE FUNCTION test_return_table(s TEXT)
> RETURNS TABLE(first TEXT, second INT4)
> AS $$
>exec('y = ' + s)
>return y
> $$ LANGUAGE plpythonu;
> SELECT (test_return_table('[]')).*;
> SELECT (test_return_table('[]')).*
> FROM gp_single_row;
> -- 2) Actual output
> SELECT (test_return_table('[]')).*;
>  first | second
> ---+
> (0 rows)
> SELECT (test_return_table('[]')).*
> FROM gp_single_row;
> ERROR:  could not compile PL/Python function "test_return_table" 
> (plpython.c:4651)  (seg5 test1:31100 pid=36826) (dispatcher.c:1801)
> DETAIL:  KeyError: 's'
> -- 3) Expected output
> SELECT (test_return_table('[]')).*;
>  first | second
> ---+
> (0 rows)
> SELECT (test_return_table('[]')).*
> FROM gp_single_row;
>  first | second
> ---+
> (0 rows)
> {noformat}
> Example 2:
> {noformat}
> -- 1) prepare schema and data
> CREATE LANGUAGE plpythonu;
> CREATE OR REPLACE FUNCTION func1(var_cause_bug int4[])
> RETURNS SETOF INT4 AS $$
> for el in var_cause_bug:
> yield el
> $$ LANGUAGE plpythonu;
> CREATE OR REPLACE FUNCTION func2()
> RETURNS INT4 AS $$
> return 1
> $$ LANGUAGE plpythonu;
> SELECT func1(ARRAY[1,2,3]), func1(ARRAY[1,2,3]);
> SELECT func2();
> -- 2) Actual output
> SELECT func1(ARRAY[1,2,3]), func1(ARRAY[1,2,3]);
>  func1 | func1 
> ---+---
>  1 | 1
>  2 | 2
>  3 | 3
> (3 rows)
> SELECT func2();
> ERROR:  could not compile PL/Python function "func2" (plpython.c:4648)
> DETAIL:  KeyError: 'var_cause_bug'
> -- 3) Expected output
> SELECT func1(ARRAY[1,2,3]), func1(ARRAY[1,2,3]);
>  func1 | func1
> ---+---
>  1 | 1
>  2 | 2
>  3 | 3
> (3 rows)
> SELECT func2();
>  func2
> ---
>  1
> (1 row)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (HAWQ-815) KeyError while compiling PLPython function due to deletion of non-existent record from Python global dict

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo closed HAWQ-815.


> KeyError while compiling PLPython function due to deletion of non-existent 
> record from Python global dict
> -
>
> Key: HAWQ-815
> URL: https://issues.apache.org/jira/browse/HAWQ-815
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Query Execution
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> It raise KeyError while compiling PLPython function due to deletion of 
> non-existent record from Python global dict. Below are two example to 
> reproduce the issue:
> Example 1:
> {noformat}
> -- 1) prepare schema and data
> CREATE LANGUAGE plpythonu;
> CREATE TABLE gp_single_row(a int) DISTRIBUTED BY (a);
> INSERT INTO gp_single_row VALUES (1);
> CREATE OR REPLACE FUNCTION test_return_table(s TEXT)
> RETURNS TABLE(first TEXT, second INT4)
> AS $$
>exec('y = ' + s)
>return y
> $$ LANGUAGE plpythonu;
> SELECT (test_return_table('[]')).*;
> SELECT (test_return_table('[]')).*
> FROM gp_single_row;
> -- 2) Actual output
> SELECT (test_return_table('[]')).*;
>  first | second
> ---+
> (0 rows)
> SELECT (test_return_table('[]')).*
> FROM gp_single_row;
> ERROR:  could not compile PL/Python function "test_return_table" 
> (plpython.c:4651)  (seg5 test1:31100 pid=36826) (dispatcher.c:1801)
> DETAIL:  KeyError: 's'
> -- 3) Expected output
> SELECT (test_return_table('[]')).*;
>  first | second
> ---+
> (0 rows)
> SELECT (test_return_table('[]')).*
> FROM gp_single_row;
>  first | second
> ---+
> (0 rows)
> {noformat}
> Example 2:
> {noformat}
> -- 1) prepare schema and data
> CREATE LANGUAGE plpythonu;
> CREATE OR REPLACE FUNCTION func1(var_cause_bug int4[])
> RETURNS SETOF INT4 AS $$
> for el in var_cause_bug:
> yield el
> $$ LANGUAGE plpythonu;
> CREATE OR REPLACE FUNCTION func2()
> RETURNS INT4 AS $$
> return 1
> $$ LANGUAGE plpythonu;
> SELECT func1(ARRAY[1,2,3]), func1(ARRAY[1,2,3]);
> SELECT func2();
> -- 2) Actual output
> SELECT func1(ARRAY[1,2,3]), func1(ARRAY[1,2,3]);
>  func1 | func1 
> ---+---
>  1 | 1
>  2 | 2
>  3 | 3
> (3 rows)
> SELECT func2();
> ERROR:  could not compile PL/Python function "func2" (plpython.c:4648)
> DETAIL:  KeyError: 'var_cause_bug'
> -- 3) Expected output
> SELECT func1(ARRAY[1,2,3]), func1(ARRAY[1,2,3]);
>  func1 | func1
> ---+---
>  1 | 1
>  2 | 2
>  3 | 3
> (3 rows)
> SELECT func2();
>  func2
> ---
>  1
> (1 row)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HAWQ-815) KeyError while compiling PLPython function due to deletion of non-existent record from Python global dict

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo reopened HAWQ-815:
--

Reopening to add the fix version.

> KeyError while compiling PLPython function due to deletion of non-existent 
> record from Python global dict
> -
>
> Key: HAWQ-815
> URL: https://issues.apache.org/jira/browse/HAWQ-815
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Query Execution
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
>
> It raise KeyError while compiling PLPython function due to deletion of 
> non-existent record from Python global dict. Below are two example to 
> reproduce the issue:
> Example 1:
> {noformat}
> -- 1) prepare schema and data
> CREATE LANGUAGE plpythonu;
> CREATE TABLE gp_single_row(a int) DISTRIBUTED BY (a);
> INSERT INTO gp_single_row VALUES (1);
> CREATE OR REPLACE FUNCTION test_return_table(s TEXT)
> RETURNS TABLE(first TEXT, second INT4)
> AS $$
>exec('y = ' + s)
>return y
> $$ LANGUAGE plpythonu;
> SELECT (test_return_table('[]')).*;
> SELECT (test_return_table('[]')).*
> FROM gp_single_row;
> -- 2) Actual output
> SELECT (test_return_table('[]')).*;
>  first | second
> ---+
> (0 rows)
> SELECT (test_return_table('[]')).*
> FROM gp_single_row;
> ERROR:  could not compile PL/Python function "test_return_table" 
> (plpython.c:4651)  (seg5 test1:31100 pid=36826) (dispatcher.c:1801)
> DETAIL:  KeyError: 's'
> -- 3) Expected output
> SELECT (test_return_table('[]')).*;
>  first | second
> ---+
> (0 rows)
> SELECT (test_return_table('[]')).*
> FROM gp_single_row;
>  first | second
> ---+
> (0 rows)
> {noformat}
> Example 2:
> {noformat}
> -- 1) prepare schema and data
> CREATE LANGUAGE plpythonu;
> CREATE OR REPLACE FUNCTION func1(var_cause_bug int4[])
> RETURNS SETOF INT4 AS $$
> for el in var_cause_bug:
> yield el
> $$ LANGUAGE plpythonu;
> CREATE OR REPLACE FUNCTION func2()
> RETURNS INT4 AS $$
> return 1
> $$ LANGUAGE plpythonu;
> SELECT func1(ARRAY[1,2,3]), func1(ARRAY[1,2,3]);
> SELECT func2();
> -- 2) Actual output
> SELECT func1(ARRAY[1,2,3]), func1(ARRAY[1,2,3]);
>  func1 | func1 
> ---+---
>  1 | 1
>  2 | 2
>  3 | 3
> (3 rows)
> SELECT func2();
> ERROR:  could not compile PL/Python function "func2" (plpython.c:4648)
> DETAIL:  KeyError: 'var_cause_bug'
> -- 3) Expected output
> SELECT func1(ARRAY[1,2,3]), func1(ARRAY[1,2,3]);
>  func1 | func1
> ---+---
>  1 | 1
>  2 | 2
>  3 | 3
> (3 rows)
> SELECT func2();
>  func2
> ---
>  1
> (1 row)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (HAWQ-820) Correct expected output for query in basic udf suite in new feature test framework

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo closed HAWQ-820.


> Correct expected output for query in basic udf suite in new feature test 
> framework
> --
>
> Key: HAWQ-820
> URL: https://issues.apache.org/jira/browse/HAWQ-820
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> Need to correct expected output for query in basic udf suite in new feature 
> test framework.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-820) Correct expected output for query in basic udf suite in new feature test framework

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo updated HAWQ-820:
-
Fix Version/s: 2.0.0

> Correct expected output for query in basic udf suite in new feature test 
> framework
> --
>
> Key: HAWQ-820
> URL: https://issues.apache.org/jira/browse/HAWQ-820
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> Need to correct expected output for query in basic udf suite in new feature 
> test framework.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HAWQ-820) Correct expected output for query in basic udf suite in new feature test framework

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo resolved HAWQ-820.
--
Resolution: Fixed

> Correct expected output for query in basic udf suite in new feature test 
> framework
> --
>
> Key: HAWQ-820
> URL: https://issues.apache.org/jira/browse/HAWQ-820
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> Need to correct expected output for query in basic udf suite in new feature 
> test framework.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HAWQ-820) Correct expected output for query in basic udf suite in new feature test framework

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo reopened HAWQ-820:
--

Reopening to add the fix version.

> Correct expected output for query in basic udf suite in new feature test 
> framework
> --
>
> Key: HAWQ-820
> URL: https://issues.apache.org/jira/browse/HAWQ-820
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> Need to correct expected output for query in basic udf suite in new feature 
> test framework.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HAWQ-822) Add string replacement utility in feature test framework to support convert from source to sql and ans files

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo resolved HAWQ-822.
--
Resolution: Fixed

> Add string replacement utility in feature test framework to support convert 
> from source to sql and ans files
> 
>
> Key: HAWQ-822
> URL: https://issues.apache.org/jira/browse/HAWQ-822
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> Need to add string replacement utility in feature test framework to support 
> convert from source to sql and ans files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (HAWQ-822) Add string replacement utility in feature test framework to support convert from source to sql and ans files

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo closed HAWQ-822.


> Add string replacement utility in feature test framework to support convert 
> from source to sql and ans files
> 
>
> Key: HAWQ-822
> URL: https://issues.apache.org/jira/browse/HAWQ-822
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> Need to add string replacement utility in feature test framework to support 
> convert from source to sql and ans files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-822) Add string replacement utility in feature test framework to support convert from source to sql and ans files

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo updated HAWQ-822:
-
Fix Version/s: 2.0.0

> Add string replacement utility in feature test framework to support convert 
> from source to sql and ans files
> 
>
> Key: HAWQ-822
> URL: https://issues.apache.org/jira/browse/HAWQ-822
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> Need to add string replacement utility in feature test framework to support 
> convert from source to sql and ans files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HAWQ-822) Add string replacement utility in feature test framework to support convert from source to sql and ans files

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo reopened HAWQ-822:
--

Reopening to add the fix version.

> Add string replacement utility in feature test framework to support convert 
> from source to sql and ans files
> 
>
> Key: HAWQ-822
> URL: https://issues.apache.org/jira/browse/HAWQ-822
> Project: Apache HAWQ
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> Need to add string replacement utility in feature test framework to support 
> convert from source to sql and ans files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HAWQ-835) Cannot retrieve tuple from temp table created in function

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo reopened HAWQ-835:
--

Reopening to add the fix version.

> Cannot retrieve tuple from temp table created in function
> -
>
> Key: HAWQ-835
> URL: https://issues.apache.org/jira/browse/HAWQ-835
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core, Query Execution
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> With function which create temp table and insert tuple into it, if the 
> function is run multiple times, it might cannot retrieve tuple from the temp 
> table in the second run of the function and so on.
> Here are the steps to reproduce:
> Step 1: prepare schema and data
> {noformat}
> CREATE TABLE t(pid int, points double precision[]);
> COPY t (pid, points) FROM stdin DELIMITER '|';
> 1 | {14.23, 1.71, 2.43, 15.6, 127, 2.8, 3.0600, 0.2800, 2.29, 5.64, 1.04, 
> 3.92, 1065}
> 2 | {13.2, 1.78, 2.14, 11.2, 1, 2.65, 2.76, 0.26, 1.28, 4.38, 1.05, 3.49, 
> 1050}
> 3 | {13.16, 2.36,  2.67, 18.6, 101, 2.8,  3.24, 0.3, 2.81, 5.6799, 1.03, 
> 3.17, 1185}
> 4 | {14.37, 1.95, 2.5, 16.8, 113, 3.85, 3.49, 0.24, 2.18, 7.8, 0.86, 3.45, 
> 1480}
> 5 | {13.24, 2.59, 2.87, 21, 118, 2.8, 2.69, 0.39, 1.82, 4.32, 1.04, 2.93, 735}
> 6 | {14.2, 1.76, 2.45, 15.2, 112, 3.27, 3.39, 0.34, 1.97, 6.75, 1.05, 2.85, 
> 1450}
> \.
> {noformat}
> Step 2: run kmeans
> {noformat}
> with q as
> (
> select
> 1 as num_clusters,
>*
> from
> madlib.kmeanspp(
>  't',
>  'points',
>  3,
>  'madlib.squared_dist_norm2',
>  'madlib.avg',
>  30,
>  0.001,
>  1.0
>  )
> )
> select q1.*
> from q as q1, (select * from q) as q2
> where q1.num_clusters = q2.num_clusters;
> ERROR:  Kmeans error: No valid initial centroids given.
> CONTEXT:  SQL statement "SELECT  madlib.kmeans(  $1 ,  $2 , 
> madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 , NULL,  $5 ),  $4 ,  $6 ,  $7 
> ,  $8 )"
> PL/pgSQL function "kmeanspp" line 4 at assignment
> {noformat}
> Step 3: further investigation shows that it cannot retrieve tuple from temp 
> table
> {noformat}
> with q as
> (
> select
> 1 as num_clusters,
>*
> from
> madlib.kmeanspp(
>  't',
>  'points',
>  3,
>  'madlib.squared_dist_norm2',
>  'madlib.avg',
>  30,
>  0.001,
>  1.0
>  )
> )
> select q1.*
> from q as q1, (select * from q) as q2
> where q1.num_clusters = q2.num_clusters;
> WARNING:  --- kmeanspp debug begin ---
> WARNING:  --- kmeanspp seeding begin ---
> WARNING:  --- kmeanspp seed summary ---
> WARNING:  ### kmeanspp_seeding: 1. use all source data
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 2. create temp schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 0 before create tmp schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 140497 after create tmp 
> schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 3. generate centroids
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 3. generate centroids
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 140497 after create tmp table
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: pg_temp._madlib_kmeanspp_args has 1 record XXX
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: pg_temp._madlib_kmeanspp_args table begin 

[jira] [Closed] (HAWQ-835) Cannot retrieve tuple from temp table created in function

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo closed HAWQ-835.


> Cannot retrieve tuple from temp table created in function
> -
>
> Key: HAWQ-835
> URL: https://issues.apache.org/jira/browse/HAWQ-835
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core, Query Execution
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> With function which create temp table and insert tuple into it, if the 
> function is run multiple times, it might cannot retrieve tuple from the temp 
> table in the second run of the function and so on.
> Here are the steps to reproduce:
> Step 1: prepare schema and data
> {noformat}
> CREATE TABLE t(pid int, points double precision[]);
> COPY t (pid, points) FROM stdin DELIMITER '|';
> 1 | {14.23, 1.71, 2.43, 15.6, 127, 2.8, 3.0600, 0.2800, 2.29, 5.64, 1.04, 
> 3.92, 1065}
> 2 | {13.2, 1.78, 2.14, 11.2, 1, 2.65, 2.76, 0.26, 1.28, 4.38, 1.05, 3.49, 
> 1050}
> 3 | {13.16, 2.36,  2.67, 18.6, 101, 2.8,  3.24, 0.3, 2.81, 5.6799, 1.03, 
> 3.17, 1185}
> 4 | {14.37, 1.95, 2.5, 16.8, 113, 3.85, 3.49, 0.24, 2.18, 7.8, 0.86, 3.45, 
> 1480}
> 5 | {13.24, 2.59, 2.87, 21, 118, 2.8, 2.69, 0.39, 1.82, 4.32, 1.04, 2.93, 735}
> 6 | {14.2, 1.76, 2.45, 15.2, 112, 3.27, 3.39, 0.34, 1.97, 6.75, 1.05, 2.85, 
> 1450}
> \.
> {noformat}
> Step 2: run kmeans
> {noformat}
> with q as
> (
> select
> 1 as num_clusters,
>*
> from
> madlib.kmeanspp(
>  't',
>  'points',
>  3,
>  'madlib.squared_dist_norm2',
>  'madlib.avg',
>  30,
>  0.001,
>  1.0
>  )
> )
> select q1.*
> from q as q1, (select * from q) as q2
> where q1.num_clusters = q2.num_clusters;
> ERROR:  Kmeans error: No valid initial centroids given.
> CONTEXT:  SQL statement "SELECT  madlib.kmeans(  $1 ,  $2 , 
> madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 , NULL,  $5 ),  $4 ,  $6 ,  $7 
> ,  $8 )"
> PL/pgSQL function "kmeanspp" line 4 at assignment
> {noformat}
> Step 3: further investigation shows that it cannot retrieve tuple from temp 
> table
> {noformat}
> with q as
> (
> select
> 1 as num_clusters,
>*
> from
> madlib.kmeanspp(
>  't',
>  'points',
>  3,
>  'madlib.squared_dist_norm2',
>  'madlib.avg',
>  30,
>  0.001,
>  1.0
>  )
> )
> select q1.*
> from q as q1, (select * from q) as q2
> where q1.num_clusters = q2.num_clusters;
> WARNING:  --- kmeanspp debug begin ---
> WARNING:  --- kmeanspp seeding begin ---
> WARNING:  --- kmeanspp seed summary ---
> WARNING:  ### kmeanspp_seeding: 1. use all source data
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 2. create temp schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 0 before create tmp schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 140497 after create tmp 
> schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 3. generate centroids
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 3. generate centroids
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 140497 after create tmp table
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: pg_temp._madlib_kmeanspp_args has 1 record XXX
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: pg_temp._madlib_kmeanspp_args table begin 1
> CONTEXT:  SQL statement "SELECT 

[jira] [Updated] (HAWQ-835) Cannot retrieve tuple from temp table created in function

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo updated HAWQ-835:
-
Fix Version/s: 2.0.0

> Cannot retrieve tuple from temp table created in function
> -
>
> Key: HAWQ-835
> URL: https://issues.apache.org/jira/browse/HAWQ-835
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core, Query Execution
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> With function which create temp table and insert tuple into it, if the 
> function is run multiple times, it might cannot retrieve tuple from the temp 
> table in the second run of the function and so on.
> Here are the steps to reproduce:
> Step 1: prepare schema and data
> {noformat}
> CREATE TABLE t(pid int, points double precision[]);
> COPY t (pid, points) FROM stdin DELIMITER '|';
> 1 | {14.23, 1.71, 2.43, 15.6, 127, 2.8, 3.0600, 0.2800, 2.29, 5.64, 1.04, 
> 3.92, 1065}
> 2 | {13.2, 1.78, 2.14, 11.2, 1, 2.65, 2.76, 0.26, 1.28, 4.38, 1.05, 3.49, 
> 1050}
> 3 | {13.16, 2.36,  2.67, 18.6, 101, 2.8,  3.24, 0.3, 2.81, 5.6799, 1.03, 
> 3.17, 1185}
> 4 | {14.37, 1.95, 2.5, 16.8, 113, 3.85, 3.49, 0.24, 2.18, 7.8, 0.86, 3.45, 
> 1480}
> 5 | {13.24, 2.59, 2.87, 21, 118, 2.8, 2.69, 0.39, 1.82, 4.32, 1.04, 2.93, 735}
> 6 | {14.2, 1.76, 2.45, 15.2, 112, 3.27, 3.39, 0.34, 1.97, 6.75, 1.05, 2.85, 
> 1450}
> \.
> {noformat}
> Step 2: run kmeans
> {noformat}
> with q as
> (
> select
> 1 as num_clusters,
>*
> from
> madlib.kmeanspp(
>  't',
>  'points',
>  3,
>  'madlib.squared_dist_norm2',
>  'madlib.avg',
>  30,
>  0.001,
>  1.0
>  )
> )
> select q1.*
> from q as q1, (select * from q) as q2
> where q1.num_clusters = q2.num_clusters;
> ERROR:  Kmeans error: No valid initial centroids given.
> CONTEXT:  SQL statement "SELECT  madlib.kmeans(  $1 ,  $2 , 
> madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 , NULL,  $5 ),  $4 ,  $6 ,  $7 
> ,  $8 )"
> PL/pgSQL function "kmeanspp" line 4 at assignment
> {noformat}
> Step 3: further investigation shows that it cannot retrieve tuple from temp 
> table
> {noformat}
> with q as
> (
> select
> 1 as num_clusters,
>*
> from
> madlib.kmeanspp(
>  't',
>  'points',
>  3,
>  'madlib.squared_dist_norm2',
>  'madlib.avg',
>  30,
>  0.001,
>  1.0
>  )
> )
> select q1.*
> from q as q1, (select * from q) as q2
> where q1.num_clusters = q2.num_clusters;
> WARNING:  --- kmeanspp debug begin ---
> WARNING:  --- kmeanspp seeding begin ---
> WARNING:  --- kmeanspp seed summary ---
> WARNING:  ### kmeanspp_seeding: 1. use all source data
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 2. create temp schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 0 before create tmp schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 140497 after create tmp 
> schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 3. generate centroids
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 3. generate centroids
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 140497 after create tmp table
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: pg_temp._madlib_kmeanspp_args has 1 record XXX
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: pg_temp._madlib_kmeanspp_args table begin 1
> CONTEXT: 

[jira] [Resolved] (HAWQ-835) Cannot retrieve tuple from temp table created in function

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo resolved HAWQ-835.
--
Resolution: Fixed

> Cannot retrieve tuple from temp table created in function
> -
>
> Key: HAWQ-835
> URL: https://issues.apache.org/jira/browse/HAWQ-835
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core, Query Execution
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> With function which create temp table and insert tuple into it, if the 
> function is run multiple times, it might cannot retrieve tuple from the temp 
> table in the second run of the function and so on.
> Here are the steps to reproduce:
> Step 1: prepare schema and data
> {noformat}
> CREATE TABLE t(pid int, points double precision[]);
> COPY t (pid, points) FROM stdin DELIMITER '|';
> 1 | {14.23, 1.71, 2.43, 15.6, 127, 2.8, 3.0600, 0.2800, 2.29, 5.64, 1.04, 
> 3.92, 1065}
> 2 | {13.2, 1.78, 2.14, 11.2, 1, 2.65, 2.76, 0.26, 1.28, 4.38, 1.05, 3.49, 
> 1050}
> 3 | {13.16, 2.36,  2.67, 18.6, 101, 2.8,  3.24, 0.3, 2.81, 5.6799, 1.03, 
> 3.17, 1185}
> 4 | {14.37, 1.95, 2.5, 16.8, 113, 3.85, 3.49, 0.24, 2.18, 7.8, 0.86, 3.45, 
> 1480}
> 5 | {13.24, 2.59, 2.87, 21, 118, 2.8, 2.69, 0.39, 1.82, 4.32, 1.04, 2.93, 735}
> 6 | {14.2, 1.76, 2.45, 15.2, 112, 3.27, 3.39, 0.34, 1.97, 6.75, 1.05, 2.85, 
> 1450}
> \.
> {noformat}
> Step 2: run kmeans
> {noformat}
> with q as
> (
> select
> 1 as num_clusters,
>*
> from
> madlib.kmeanspp(
>  't',
>  'points',
>  3,
>  'madlib.squared_dist_norm2',
>  'madlib.avg',
>  30,
>  0.001,
>  1.0
>  )
> )
> select q1.*
> from q as q1, (select * from q) as q2
> where q1.num_clusters = q2.num_clusters;
> ERROR:  Kmeans error: No valid initial centroids given.
> CONTEXT:  SQL statement "SELECT  madlib.kmeans(  $1 ,  $2 , 
> madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 , NULL,  $5 ),  $4 ,  $6 ,  $7 
> ,  $8 )"
> PL/pgSQL function "kmeanspp" line 4 at assignment
> {noformat}
> Step 3: further investigation shows that it cannot retrieve tuple from temp 
> table
> {noformat}
> with q as
> (
> select
> 1 as num_clusters,
>*
> from
> madlib.kmeanspp(
>  't',
>  'points',
>  3,
>  'madlib.squared_dist_norm2',
>  'madlib.avg',
>  30,
>  0.001,
>  1.0
>  )
> )
> select q1.*
> from q as q1, (select * from q) as q2
> where q1.num_clusters = q2.num_clusters;
> WARNING:  --- kmeanspp debug begin ---
> WARNING:  --- kmeanspp seeding begin ---
> WARNING:  --- kmeanspp seed summary ---
> WARNING:  ### kmeanspp_seeding: 1. use all source data
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 2. create temp schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 0 before create tmp schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 140497 after create tmp 
> schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 3. generate centroids
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 3. generate centroids
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 140497 after create tmp table
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: pg_temp._madlib_kmeanspp_args has 1 record XXX
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: pg_temp._madlib_kmeanspp_args table begin 1
> CONTEXT:  

[jira] [Closed] (HAWQ-869) Add regression test for less tuple is inserted issue in prepared statement

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo closed HAWQ-869.


> Add regression test for less tuple is inserted issue in prepared statement
> --
>
> Key: HAWQ-869
> URL: https://issues.apache.org/jira/browse/HAWQ-869
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> To make sure we don't regress in less tuple is inserted issue in prepared 
> statement, we need to add feature test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-869) Add regression test for less tuple is inserted issue in prepared statement

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo updated HAWQ-869:
-
Fix Version/s: 2.0.0

> Add regression test for less tuple is inserted issue in prepared statement
> --
>
> Key: HAWQ-869
> URL: https://issues.apache.org/jira/browse/HAWQ-869
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> To make sure we don't regress in less tuple is inserted issue in prepared 
> statement, we need to add feature test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HAWQ-869) Add regression test for less tuple is inserted issue in prepared statement

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo reopened HAWQ-869:
--

Reopening to add the fix version.

> Add regression test for less tuple is inserted issue in prepared statement
> --
>
> Key: HAWQ-869
> URL: https://issues.apache.org/jira/browse/HAWQ-869
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> To make sure we don't regress in less tuple is inserted issue in prepared 
> statement, we need to add feature test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HAWQ-869) Add regression test for less tuple is inserted issue in prepared statement

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo resolved HAWQ-869.
--
Resolution: Fixed

> Add regression test for less tuple is inserted issue in prepared statement
> --
>
> Key: HAWQ-869
> URL: https://issues.apache.org/jira/browse/HAWQ-869
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
>
> To make sure we don't regress in less tuple is inserted issue in prepared 
> statement, we need to add feature test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-830) Wrong result in CTE query due to CTE is treated as init plan by planner and evaluated multiple times

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo updated HAWQ-830:
-
Affects Version/s: 2.0.0

> Wrong result in CTE query due to CTE is treated as init plan by planner and 
> evaluated multiple times
> 
>
> Key: HAWQ-830
> URL: https://issues.apache.org/jira/browse/HAWQ-830
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Optimizer
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
>
> In CTE query, if the CTE itself is referenced multiple times, it should be 
> evaluated only once and then be used multiple time. However, it is treated as 
> init plan and evaluated multiple times in hawq 1.x and 2.0. This has two 
> issues here:
> 1. If the query in CTE is "volatile" (i.e., select volatile function) or has 
> side effect (create/drop object in database), it may generate wrong result
> 2. The performance of the query is not so efficient since the query in CTE is 
> evaluated multiple times.
> Here is the steps to reproduce:
> 1) in hawq, CTE is treated as init plan and evaluated 2 times. Thus, the 
> result is incorrect
> {noformat}
> WITH r AS (SELECT random())
> SELECT r1.*, r2.*
> FROM r AS r1, r AS r2;
>   random   |  random
> ---+---
>  0.519145511090755 | 0.751198637764901
> (1 row)
> EXPLAIN
> WITH r AS (SELECT random())
> SELECT r1.*, r2.*
> FROM r AS r1, r AS r2;
>   QUERY PLAN
> --
>  Nested Loop  (cost=0.04..0.77 rows=20 width=16)
>->  Result  (cost=0.01..0.02 rows=1 width=0)
>  InitPlan
>->  Result  (cost=0.00..0.01 rows=1 width=0)
>->  Materialize  (cost=0.03..0.09 rows=6 width=8)
>  ->  Result  (cost=0.01..0.02 rows=1 width=0)
>InitPlan
>  ->  Result  (cost=0.00..0.01 rows=1 width=0)
>  Settings:  default_hash_table_bucket_number=6
>  Optimizer status: legacy query optimizer
> (10 rows)
> {noformat}
> 2) in postgres, CTE is treated as CTE scan and evaluated 1 time. Thus, the 
> result is i
> {noformat}
> WITH r AS (SELECT random())
> SELECT r1.*, r2.*
> FROM r AS r1, r AS r2;
>   random   |  random
> ---+---
>  0.989214501809329 | 0.989214501809329
> (1 row)
> EXPLAIN
> WITH r AS (SELECT random())
> SELECT r1.*, r2.*
> FROM r AS r1, r AS r2;
> QUERY PLAN
> --
>  Nested Loop  (cost=0.01..0.06 rows=1 width=16)
>CTE r
>  ->  Result  (cost=0.00..0.01 rows=1 width=0)
>->  CTE Scan on r r1  (cost=0.00..0.02 rows=1 width=8)
>->  CTE Scan on r r2  (cost=0.00..0.02 rows=1 width=8)
> (5 rows){noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (HAWQ-800) Less tuple is inserted due to data locality information is not refreshed and dispatched in prepared statement

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo closed HAWQ-800.


> Less tuple is inserted due to data locality information is not refreshed and 
> dispatched in prepared statement
> -
>
> Key: HAWQ-800
> URL: https://issues.apache.org/jira/browse/HAWQ-800
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core, Dispatcher, Query Execution, Resource Manager
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
> Attachments: proba.out, proba.sql, proba_execute.out, 
> proba_execute.sql
>
>
> In either explicit (SQL) or implicit (UDF) prepared statement, there is less 
> tuple inserted if we run a prepared "insert into t select * from t" plan 
> multiple times in a transaction.
> Below is a simple case to reproduce this issue. For a more complicated 
> example, you may refer to attached proba_execute and proba.
> 1. There should be 8 tuples, however there is only 4 in hawq 2.0
> {noformat}
> drop table if exists t;
> DROP TABLE
> create table t (id int);
> CREATE TABLE
> insert into t values (1);
> INSERT 0 1
> CREATE OR REPLACE FUNCTION f_load()
> RETURNS TEXT
> LANGUAGE plpgsql
> AS
> $body$
> DECLARE
> l_rec RECORD;
> l_itm RECORD;
> BEGIN
> FOR l_rec IN ( SELECT generate_series(1, 3) AS id )
> LOOP
> INSERT INTO t SELECT * FROM t;
> END LOOP;
> RETURN 'done';
> END;
> $body$
> ;
> CREATE FUNCTION
> SELECT f_load();
>  f_load
> 
>  done
> (1 row)
> SELECT * FROM t;
>  id
> 
>   1
>   1
>   1
>   1
> (4 rows)
> {noformat}
> 2. There are 8 tuples as expected in hawq 1.x
> {noformat}
> drop table if exists t;
> DROP TABLE
> create table t (id int);
> psql:temp.sql:3: NOTICE:  Table doesn't have 'DISTRIBUTED BY' clause -- Using 
> column named 'id' as the Greenplum Database data distribution key for this 
> table.
> HINT:  The 'DISTRIBUTED BY' clause determines the distribution of data. Make 
> sure column(s) chosen are the optimal data distribution key to minimize skew.
> CREATE TABLE
> insert into t values (1);
> INSERT 0 1
> CREATE OR REPLACE FUNCTION f_load()
> RETURNS TEXT
> LANGUAGE plpgsql
> AS
> $body$
> DECLARE
> l_rec RECORD;
> l_itm RECORD;
> BEGIN
> FOR l_rec IN ( SELECT generate_series(1, 3) AS id )
> LOOP
> INSERT INTO t SELECT * FROM t;
> END LOOP;
> RETURN 'done';
> END;
> $body$
> ;
> CREATE FUNCTION
> SELECT f_load();
>  f_load
> 
>  done
> (1 row)
> SELECT * FROM t;
>  id
> 
>   1
>   1
>   1
>   1
>   1
>   1
>   1
>   1
> (8 rows)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-876) Add the support for initFile option of gpdiff.pl in hawq googletest framework.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354560#comment-15354560
 ] 

ASF GitHub Bot commented on HAWQ-876:
-

Github user paul-guo- commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/758#discussion_r68886608
  
--- Diff: src/test/feature/lib/sql_util.cpp ---
@@ -108,8 +111,24 @@ void SQLUtility::execSQLFile(const string ,
   conn->setOutputFile(outFileAbsPath);
   EXPECT_EQ(0, conn->runSQLFile(newSqlFile).getLastStatus());
   conn->resetOutput();
-  EXPECT_FALSE(conn->checkDiff(ansFileAbsPath, outFileAbsPath, true));
-  if (conn->checkDiff(ansFileAbsPath, outFileAbsPath, true) == false) {
+
+  // initFile if any
+  string initFileAbsPath;
+  if (!initFile.empty()) {
+initFileAbsPath = testRootPath + "/" + initFile;
+if (!std::ifstream(initFileAbsPath))
+  ASSERT_TRUE(false) << initFileAbsPath << " doesn't exist";
+fp = splitFilePath(initFileAbsPath);
+// double check to avoid empty fileBaseName
+if (fp.fileBaseName.empty())
+  ASSERT_TRUE(false) << initFileAbsPath << " is invalid";
+  } else {
+initFileAbsPath = "";
+  }
+
+  bool is_sql_ans_diff = conn->checkDiff(ansFileAbsPath, outFileAbsPath, 
true, initFileAbsPath);
+  EXPECT_FALSE(is_sql_ans_diff);
+  if (is_sql_ans_diff == false) {
--- End diff --

I think we can remove the else part, right?

 } else {
EXPECT_FALSE(true);
  }


> Add the support for initFile option of gpdiff.pl in hawq googletest framework.
> --
>
> Key: HAWQ-876
> URL: https://issues.apache.org/jira/browse/HAWQ-876
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Lei Chang
>
> Hawq googletest framework depends on hawq tool gpdiff.pl. The tool provides 
> an enhanced diff comparison between sql output file and expected_out file. It
> supports a useful option "-gpd_init ' which defines some directives 
> for comparison. This option is quite useful for some complex sql tests.  
> Current hawq googletest work does not use this option. We need to add this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #758: HAWQ-876. Add the support for initFile opt...

[paul-guo-]

Github user paul-guo- commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/758#discussion_r68886608
  
--- Diff: src/test/feature/lib/sql_util.cpp ---
@@ -108,8 +111,24 @@ void SQLUtility::execSQLFile(const string ,
   conn->setOutputFile(outFileAbsPath);
   EXPECT_EQ(0, conn->runSQLFile(newSqlFile).getLastStatus());
   conn->resetOutput();
-  EXPECT_FALSE(conn->checkDiff(ansFileAbsPath, outFileAbsPath, true));
-  if (conn->checkDiff(ansFileAbsPath, outFileAbsPath, true) == false) {
+
+  // initFile if any
+  string initFileAbsPath;
+  if (!initFile.empty()) {
+initFileAbsPath = testRootPath + "/" + initFile;
+if (!std::ifstream(initFileAbsPath))
+  ASSERT_TRUE(false) << initFileAbsPath << " doesn't exist";
+fp = splitFilePath(initFileAbsPath);
+// double check to avoid empty fileBaseName
+if (fp.fileBaseName.empty())
+  ASSERT_TRUE(false) << initFileAbsPath << " is invalid";
+  } else {
+initFileAbsPath = "";
+  }
+
+  bool is_sql_ans_diff = conn->checkDiff(ansFileAbsPath, outFileAbsPath, 
true, initFileAbsPath);
+  EXPECT_FALSE(is_sql_ans_diff);
+  if (is_sql_ans_diff == false) {
--- End diff --

I think we can remove the else part, right?

 } else {
EXPECT_FALSE(true);
  }


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (HAWQ-800) Less tuple is inserted due to data locality information is not refreshed and dispatched in prepared statement

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo updated HAWQ-800:
-
Affects Version/s: 2.0.0

> Less tuple is inserted due to data locality information is not refreshed and 
> dispatched in prepared statement
> -
>
> Key: HAWQ-800
> URL: https://issues.apache.org/jira/browse/HAWQ-800
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core, Dispatcher, Query Execution, Resource Manager
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
> Fix For: 2.0.0
>
> Attachments: proba.out, proba.sql, proba_execute.out, 
> proba_execute.sql
>
>
> In either explicit (SQL) or implicit (UDF) prepared statement, there is less 
> tuple inserted if we run a prepared "insert into t select * from t" plan 
> multiple times in a transaction.
> Below is a simple case to reproduce this issue. For a more complicated 
> example, you may refer to attached proba_execute and proba.
> 1. There should be 8 tuples, however there is only 4 in hawq 2.0
> {noformat}
> drop table if exists t;
> DROP TABLE
> create table t (id int);
> CREATE TABLE
> insert into t values (1);
> INSERT 0 1
> CREATE OR REPLACE FUNCTION f_load()
> RETURNS TEXT
> LANGUAGE plpgsql
> AS
> $body$
> DECLARE
> l_rec RECORD;
> l_itm RECORD;
> BEGIN
> FOR l_rec IN ( SELECT generate_series(1, 3) AS id )
> LOOP
> INSERT INTO t SELECT * FROM t;
> END LOOP;
> RETURN 'done';
> END;
> $body$
> ;
> CREATE FUNCTION
> SELECT f_load();
>  f_load
> 
>  done
> (1 row)
> SELECT * FROM t;
>  id
> 
>   1
>   1
>   1
>   1
> (4 rows)
> {noformat}
> 2. There are 8 tuples as expected in hawq 1.x
> {noformat}
> drop table if exists t;
> DROP TABLE
> create table t (id int);
> psql:temp.sql:3: NOTICE:  Table doesn't have 'DISTRIBUTED BY' clause -- Using 
> column named 'id' as the Greenplum Database data distribution key for this 
> table.
> HINT:  The 'DISTRIBUTED BY' clause determines the distribution of data. Make 
> sure column(s) chosen are the optimal data distribution key to minimize skew.
> CREATE TABLE
> insert into t values (1);
> INSERT 0 1
> CREATE OR REPLACE FUNCTION f_load()
> RETURNS TEXT
> LANGUAGE plpgsql
> AS
> $body$
> DECLARE
> l_rec RECORD;
> l_itm RECORD;
> BEGIN
> FOR l_rec IN ( SELECT generate_series(1, 3) AS id )
> LOOP
> INSERT INTO t SELECT * FROM t;
> END LOOP;
> RETURN 'done';
> END;
> $body$
> ;
> CREATE FUNCTION
> SELECT f_load();
>  f_load
> 
>  done
> (1 row)
> SELECT * FROM t;
>  id
> 
>   1
>   1
>   1
>   1
>   1
>   1
>   1
>   1
> (8 rows)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (HAWQ-850) Planner supports refineCachedPlan interface.

[Hubert Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hubert Zhang closed HAWQ-850.
-
Resolution: Fixed

> Planner supports refineCachedPlan interface.
> 
>
> Key: HAWQ-850
> URL: https://issues.apache.org/jira/browse/HAWQ-850
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Core
>Reporter: Hubert Zhang
>Assignee: Hubert Zhang
>
> in PBE(prepare bind execution) cases, plan will be cached, and only be 
> generated once in postgres.
> But in HAWQ since resource is dynamic or elastic, the plan maybe changed 
> during the different execution of PBE. Also, the file split allocation result 
> which is stored in plan is dynamically too, so the split-vseg mapping may 
> also be changed.
> For example, We call a plpgsql function which do "insert into t select * from 
> t".
> in Prepare we cache the plan, with fixed split-vseg mapping(e.g. 10 blocks), 
> but after insert, there would be 20 blocks so the old  split-vseg mapping is 
> invalid. We need to update it. Also when we call this function again and 
> again, the data size of t may request more resource to handle it than the 
> first call, so we should recalculate the entire plan with the new vseg number.
> This function is implemented in a interface called refineCachedPlan.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-876) Add the support for initFile option of gpdiff.pl in hawq googletest framework.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354456#comment-15354456
 ] 

ASF GitHub Bot commented on HAWQ-876:
-

Github user paul-guo- commented on the issue:

https://github.com/apache/incubator-hawq/pull/758
  
Merged before xunzhang commented.


> Add the support for initFile option of gpdiff.pl in hawq googletest framework.
> --
>
> Key: HAWQ-876
> URL: https://issues.apache.org/jira/browse/HAWQ-876
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Lei Chang
>
> Hawq googletest framework depends on hawq tool gpdiff.pl. The tool provides 
> an enhanced diff comparison between sql output file and expected_out file. It
> supports a useful option "-gpd_init ' which defines some directives 
> for comparison. This option is quite useful for some complex sql tests.  
> Current hawq googletest work does not use this option. We need to add this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #758: HAWQ-876. Add the support for initFile opt...

[paul-guo-]

Github user paul-guo- closed the pull request at:

https://github.com/apache/incubator-hawq/pull/758


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-876) Add the support for initFile option of gpdiff.pl in hawq googletest framework.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354457#comment-15354457
 ] 

ASF GitHub Bot commented on HAWQ-876:
-

Github user paul-guo- closed the pull request at:

https://github.com/apache/incubator-hawq/pull/758


> Add the support for initFile option of gpdiff.pl in hawq googletest framework.
> --
>
> Key: HAWQ-876
> URL: https://issues.apache.org/jira/browse/HAWQ-876
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Lei Chang
>
> Hawq googletest framework depends on hawq tool gpdiff.pl. The tool provides 
> an enhanced diff comparison between sql output file and expected_out file. It
> supports a useful option "-gpd_init ' which defines some directives 
> for comparison. This option is quite useful for some complex sql tests.  
> Current hawq googletest work does not use this option. We need to add this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #758: HAWQ-876. Add the support for initFile option of ...

[paul-guo-]

Github user paul-guo- commented on the issue:

https://github.com/apache/incubator-hawq/pull/758
  
Merged before xunzhang commented.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Closed] (HAWQ-835) Cannot retrieve tuple from temp table created in function

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo closed HAWQ-835.


> Cannot retrieve tuple from temp table created in function
> -
>
> Key: HAWQ-835
> URL: https://issues.apache.org/jira/browse/HAWQ-835
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core, Query Execution
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
>
> With function which create temp table and insert tuple into it, if the 
> function is run multiple times, it might cannot retrieve tuple from the temp 
> table in the second run of the function and so on.
> Here are the steps to reproduce:
> Step 1: prepare schema and data
> {noformat}
> CREATE TABLE t(pid int, points double precision[]);
> COPY t (pid, points) FROM stdin DELIMITER '|';
> 1 | {14.23, 1.71, 2.43, 15.6, 127, 2.8, 3.0600, 0.2800, 2.29, 5.64, 1.04, 
> 3.92, 1065}
> 2 | {13.2, 1.78, 2.14, 11.2, 1, 2.65, 2.76, 0.26, 1.28, 4.38, 1.05, 3.49, 
> 1050}
> 3 | {13.16, 2.36,  2.67, 18.6, 101, 2.8,  3.24, 0.3, 2.81, 5.6799, 1.03, 
> 3.17, 1185}
> 4 | {14.37, 1.95, 2.5, 16.8, 113, 3.85, 3.49, 0.24, 2.18, 7.8, 0.86, 3.45, 
> 1480}
> 5 | {13.24, 2.59, 2.87, 21, 118, 2.8, 2.69, 0.39, 1.82, 4.32, 1.04, 2.93, 735}
> 6 | {14.2, 1.76, 2.45, 15.2, 112, 3.27, 3.39, 0.34, 1.97, 6.75, 1.05, 2.85, 
> 1450}
> \.
> {noformat}
> Step 2: run kmeans
> {noformat}
> with q as
> (
> select
> 1 as num_clusters,
>*
> from
> madlib.kmeanspp(
>  't',
>  'points',
>  3,
>  'madlib.squared_dist_norm2',
>  'madlib.avg',
>  30,
>  0.001,
>  1.0
>  )
> )
> select q1.*
> from q as q1, (select * from q) as q2
> where q1.num_clusters = q2.num_clusters;
> ERROR:  Kmeans error: No valid initial centroids given.
> CONTEXT:  SQL statement "SELECT  madlib.kmeans(  $1 ,  $2 , 
> madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 , NULL,  $5 ),  $4 ,  $6 ,  $7 
> ,  $8 )"
> PL/pgSQL function "kmeanspp" line 4 at assignment
> {noformat}
> Step 3: further investigation shows that it cannot retrieve tuple from temp 
> table
> {noformat}
> with q as
> (
> select
> 1 as num_clusters,
>*
> from
> madlib.kmeanspp(
>  't',
>  'points',
>  3,
>  'madlib.squared_dist_norm2',
>  'madlib.avg',
>  30,
>  0.001,
>  1.0
>  )
> )
> select q1.*
> from q as q1, (select * from q) as q2
> where q1.num_clusters = q2.num_clusters;
> WARNING:  --- kmeanspp debug begin ---
> WARNING:  --- kmeanspp seeding begin ---
> WARNING:  --- kmeanspp seed summary ---
> WARNING:  ### kmeanspp_seeding: 1. use all source data
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 2. create temp schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 0 before create tmp schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 140497 after create tmp 
> schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 3. generate centroids
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 3. generate centroids
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 140497 after create tmp table
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: pg_temp._madlib_kmeanspp_args has 1 record XXX
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: pg_temp._madlib_kmeanspp_args table begin 1
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 

[jira] [Resolved] (HAWQ-835) Cannot retrieve tuple from temp table created in function

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo resolved HAWQ-835.
--
Resolution: Fixed

Fixed along with HAWQ-800.

> Cannot retrieve tuple from temp table created in function
> -
>
> Key: HAWQ-835
> URL: https://issues.apache.org/jira/browse/HAWQ-835
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core, Query Execution
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
>
> With function which create temp table and insert tuple into it, if the 
> function is run multiple times, it might cannot retrieve tuple from the temp 
> table in the second run of the function and so on.
> Here are the steps to reproduce:
> Step 1: prepare schema and data
> {noformat}
> CREATE TABLE t(pid int, points double precision[]);
> COPY t (pid, points) FROM stdin DELIMITER '|';
> 1 | {14.23, 1.71, 2.43, 15.6, 127, 2.8, 3.0600, 0.2800, 2.29, 5.64, 1.04, 
> 3.92, 1065}
> 2 | {13.2, 1.78, 2.14, 11.2, 1, 2.65, 2.76, 0.26, 1.28, 4.38, 1.05, 3.49, 
> 1050}
> 3 | {13.16, 2.36,  2.67, 18.6, 101, 2.8,  3.24, 0.3, 2.81, 5.6799, 1.03, 
> 3.17, 1185}
> 4 | {14.37, 1.95, 2.5, 16.8, 113, 3.85, 3.49, 0.24, 2.18, 7.8, 0.86, 3.45, 
> 1480}
> 5 | {13.24, 2.59, 2.87, 21, 118, 2.8, 2.69, 0.39, 1.82, 4.32, 1.04, 2.93, 735}
> 6 | {14.2, 1.76, 2.45, 15.2, 112, 3.27, 3.39, 0.34, 1.97, 6.75, 1.05, 2.85, 
> 1450}
> \.
> {noformat}
> Step 2: run kmeans
> {noformat}
> with q as
> (
> select
> 1 as num_clusters,
>*
> from
> madlib.kmeanspp(
>  't',
>  'points',
>  3,
>  'madlib.squared_dist_norm2',
>  'madlib.avg',
>  30,
>  0.001,
>  1.0
>  )
> )
> select q1.*
> from q as q1, (select * from q) as q2
> where q1.num_clusters = q2.num_clusters;
> ERROR:  Kmeans error: No valid initial centroids given.
> CONTEXT:  SQL statement "SELECT  madlib.kmeans(  $1 ,  $2 , 
> madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 , NULL,  $5 ),  $4 ,  $6 ,  $7 
> ,  $8 )"
> PL/pgSQL function "kmeanspp" line 4 at assignment
> {noformat}
> Step 3: further investigation shows that it cannot retrieve tuple from temp 
> table
> {noformat}
> with q as
> (
> select
> 1 as num_clusters,
>*
> from
> madlib.kmeanspp(
>  't',
>  'points',
>  3,
>  'madlib.squared_dist_norm2',
>  'madlib.avg',
>  30,
>  0.001,
>  1.0
>  )
> )
> select q1.*
> from q as q1, (select * from q) as q2
> where q1.num_clusters = q2.num_clusters;
> WARNING:  --- kmeanspp debug begin ---
> WARNING:  --- kmeanspp seeding begin ---
> WARNING:  --- kmeanspp seed summary ---
> WARNING:  ### kmeanspp_seeding: 1. use all source data
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 2. create temp schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 0 before create tmp schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 140497 after create tmp 
> schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 3. generate centroids
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 3. generate centroids
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 140497 after create tmp table
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: pg_temp._madlib_kmeanspp_args has 1 record XXX
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: pg_temp._madlib_kmeanspp_args table begin 1
> CONTEXT:  

[jira] [Updated] (HAWQ-835) Cannot retrieve tuple from temp table created in function

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo updated HAWQ-835:
-
Affects Version/s: 2.0.0

> Cannot retrieve tuple from temp table created in function
> -
>
> Key: HAWQ-835
> URL: https://issues.apache.org/jira/browse/HAWQ-835
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core, Query Execution
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
>
> With function which create temp table and insert tuple into it, if the 
> function is run multiple times, it might cannot retrieve tuple from the temp 
> table in the second run of the function and so on.
> Here are the steps to reproduce:
> Step 1: prepare schema and data
> {noformat}
> CREATE TABLE t(pid int, points double precision[]);
> COPY t (pid, points) FROM stdin DELIMITER '|';
> 1 | {14.23, 1.71, 2.43, 15.6, 127, 2.8, 3.0600, 0.2800, 2.29, 5.64, 1.04, 
> 3.92, 1065}
> 2 | {13.2, 1.78, 2.14, 11.2, 1, 2.65, 2.76, 0.26, 1.28, 4.38, 1.05, 3.49, 
> 1050}
> 3 | {13.16, 2.36,  2.67, 18.6, 101, 2.8,  3.24, 0.3, 2.81, 5.6799, 1.03, 
> 3.17, 1185}
> 4 | {14.37, 1.95, 2.5, 16.8, 113, 3.85, 3.49, 0.24, 2.18, 7.8, 0.86, 3.45, 
> 1480}
> 5 | {13.24, 2.59, 2.87, 21, 118, 2.8, 2.69, 0.39, 1.82, 4.32, 1.04, 2.93, 735}
> 6 | {14.2, 1.76, 2.45, 15.2, 112, 3.27, 3.39, 0.34, 1.97, 6.75, 1.05, 2.85, 
> 1450}
> \.
> {noformat}
> Step 2: run kmeans
> {noformat}
> with q as
> (
> select
> 1 as num_clusters,
>*
> from
> madlib.kmeanspp(
>  't',
>  'points',
>  3,
>  'madlib.squared_dist_norm2',
>  'madlib.avg',
>  30,
>  0.001,
>  1.0
>  )
> )
> select q1.*
> from q as q1, (select * from q) as q2
> where q1.num_clusters = q2.num_clusters;
> ERROR:  Kmeans error: No valid initial centroids given.
> CONTEXT:  SQL statement "SELECT  madlib.kmeans(  $1 ,  $2 , 
> madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 , NULL,  $5 ),  $4 ,  $6 ,  $7 
> ,  $8 )"
> PL/pgSQL function "kmeanspp" line 4 at assignment
> {noformat}
> Step 3: further investigation shows that it cannot retrieve tuple from temp 
> table
> {noformat}
> with q as
> (
> select
> 1 as num_clusters,
>*
> from
> madlib.kmeanspp(
>  't',
>  'points',
>  3,
>  'madlib.squared_dist_norm2',
>  'madlib.avg',
>  30,
>  0.001,
>  1.0
>  )
> )
> select q1.*
> from q as q1, (select * from q) as q2
> where q1.num_clusters = q2.num_clusters;
> WARNING:  --- kmeanspp debug begin ---
> WARNING:  --- kmeanspp seeding begin ---
> WARNING:  --- kmeanspp seed summary ---
> WARNING:  ### kmeanspp_seeding: 1. use all source data
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 2. create temp schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 0 before create tmp schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 140497 after create tmp 
> schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 3. generate centroids
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 3. generate centroids
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 140497 after create tmp table
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: pg_temp._madlib_kmeanspp_args has 1 record XXX
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: pg_temp._madlib_kmeanspp_args table begin 1
> CONTEXT:  SQL statement "SELECT 

[jira] [Commented] (HAWQ-876) Add the support for initFile option of gpdiff.pl in hawq googletest framework.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354437#comment-15354437
 ] 

ASF GitHub Bot commented on HAWQ-876:
-

Github user xunzhang commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/758#discussion_r68878643
  
--- Diff: src/test/feature/lib/sql_util.cpp ---
@@ -108,8 +111,24 @@ void SQLUtility::execSQLFile(const string ,
   conn->setOutputFile(outFileAbsPath);
   EXPECT_EQ(0, conn->runSQLFile(newSqlFile).getLastStatus());
   conn->resetOutput();
-  EXPECT_FALSE(conn->checkDiff(ansFileAbsPath, outFileAbsPath, true));
-  if (conn->checkDiff(ansFileAbsPath, outFileAbsPath, true) == false) {
+
+  // initFile if any
+  string initFileAbsPath;
+  if (!initFile.empty()) {
+initFileAbsPath = testRootPath + "/" + initFile;
+if (!std::ifstream(initFileAbsPath))
+  ASSERT_TRUE(false) << initFileAbsPath << " doesn't exist";
+fp = splitFilePath(initFileAbsPath);
+// double check to avoid empty fileBaseName
+if (fp.fileBaseName.empty())
+  ASSERT_TRUE(false) << initFileAbsPath << " is invalid";
+  } else {
+initFileAbsPath = "";
+  }
+
+  bool is_sql_ans_diff = conn->checkDiff(ansFileAbsPath, outFileAbsPath, 
true, initFileAbsPath);
+  EXPECT_FALSE(is_sql_ans_diff);
+  if (is_sql_ans_diff == false) {
--- End diff --

why you need to check here? You have already check it above: 
`EXPECT_FALSE(is_sql_ans_diff);`, I think the if-else clause is redundant.


> Add the support for initFile option of gpdiff.pl in hawq googletest framework.
> --
>
> Key: HAWQ-876
> URL: https://issues.apache.org/jira/browse/HAWQ-876
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Lei Chang
>
> Hawq googletest framework depends on hawq tool gpdiff.pl. The tool provides 
> an enhanced diff comparison between sql output file and expected_out file. It
> supports a useful option "-gpd_init ' which defines some directives 
> for comparison. This option is quite useful for some complex sql tests.  
> Current hawq googletest work does not use this option. We need to add this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #758: HAWQ-876. Add the support for initFile opt...

[xunzhang]

Github user xunzhang commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/758#discussion_r68878643
  
--- Diff: src/test/feature/lib/sql_util.cpp ---
@@ -108,8 +111,24 @@ void SQLUtility::execSQLFile(const string ,
   conn->setOutputFile(outFileAbsPath);
   EXPECT_EQ(0, conn->runSQLFile(newSqlFile).getLastStatus());
   conn->resetOutput();
-  EXPECT_FALSE(conn->checkDiff(ansFileAbsPath, outFileAbsPath, true));
-  if (conn->checkDiff(ansFileAbsPath, outFileAbsPath, true) == false) {
+
+  // initFile if any
+  string initFileAbsPath;
+  if (!initFile.empty()) {
+initFileAbsPath = testRootPath + "/" + initFile;
+if (!std::ifstream(initFileAbsPath))
+  ASSERT_TRUE(false) << initFileAbsPath << " doesn't exist";
+fp = splitFilePath(initFileAbsPath);
+// double check to avoid empty fileBaseName
+if (fp.fileBaseName.empty())
+  ASSERT_TRUE(false) << initFileAbsPath << " is invalid";
+  } else {
+initFileAbsPath = "";
+  }
+
+  bool is_sql_ans_diff = conn->checkDiff(ansFileAbsPath, outFileAbsPath, 
true, initFileAbsPath);
+  EXPECT_FALSE(is_sql_ans_diff);
+  if (is_sql_ans_diff == false) {
--- End diff --

why you need to check here? You have already check it above: 
`EXPECT_FALSE(is_sql_ans_diff);`, I think the if-else clause is redundant.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-876) Add the support for initFile option of gpdiff.pl in hawq googletest framework.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354436#comment-15354436
 ] 

ASF GitHub Bot commented on HAWQ-876:
-

Github user xunzhang commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/758#discussion_r68878520
  
--- Diff: src/test/feature/lib/sql_util.cpp ---
@@ -108,8 +111,24 @@ void SQLUtility::execSQLFile(const string ,
   conn->setOutputFile(outFileAbsPath);
   EXPECT_EQ(0, conn->runSQLFile(newSqlFile).getLastStatus());
   conn->resetOutput();
-  EXPECT_FALSE(conn->checkDiff(ansFileAbsPath, outFileAbsPath, true));
-  if (conn->checkDiff(ansFileAbsPath, outFileAbsPath, true) == false) {
+
+  // initFile if any
+  string initFileAbsPath;
+  if (!initFile.empty()) {
+initFileAbsPath = testRootPath + "/" + initFile;
+if (!std::ifstream(initFileAbsPath))
+  ASSERT_TRUE(false) << initFileAbsPath << " doesn't exist";
+fp = splitFilePath(initFileAbsPath);
+// double check to avoid empty fileBaseName
+if (fp.fileBaseName.empty())
+  ASSERT_TRUE(false) << initFileAbsPath << " is invalid";
+  } else {
+initFileAbsPath = "";
+  }
+
+  bool is_sql_ans_diff = conn->checkDiff(ansFileAbsPath, outFileAbsPath, 
true, initFileAbsPath);
+  EXPECT_FALSE(is_sql_ans_diff);
+  if (is_sql_ans_diff == false) {
--- End diff --

`if (!is_sql_ans_diff) {`


> Add the support for initFile option of gpdiff.pl in hawq googletest framework.
> --
>
> Key: HAWQ-876
> URL: https://issues.apache.org/jira/browse/HAWQ-876
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Lei Chang
>
> Hawq googletest framework depends on hawq tool gpdiff.pl. The tool provides 
> an enhanced diff comparison between sql output file and expected_out file. It
> supports a useful option "-gpd_init ' which defines some directives 
> for comparison. This option is quite useful for some complex sql tests.  
> Current hawq googletest work does not use this option. We need to add this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-873) Improve checking time for travis CI

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354435#comment-15354435
 ] 

ASF GitHub Bot commented on HAWQ-873:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/756


> Improve checking time for travis CI
> ---
>
> Key: HAWQ-873
> URL: https://issues.apache.org/jira/browse/HAWQ-873
> Project: Apache HAWQ
>  Issue Type: Improvement
>Reporter: hongwu
>Assignee: hongwu
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #756: HAWQ-873. improve checking time for .travi...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/756


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-537) 'make distdir' can't handle filenames with spaces in them

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354422#comment-15354422
 ] 

ASF GitHub Bot commented on HAWQ-537:
-

Github user radarwave commented on the issue:

https://github.com/apache/incubator-hawq/pull/489
  
@cjcjameson Most looks good to me.
Please use this template file 'GNUMakefile.in' to generate 'GNUMakefile' 
and test the run result. They should be check-in together.  Thanks.


> 'make distdir' can't handle filenames with spaces in them
> -
>
> Key: HAWQ-537
> URL: https://issues.apache.org/jira/browse/HAWQ-537
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Build
>Reporter: Tom Meyer
>Assignee: Lei Chang
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #489: HAWQ-537. Accept filenames with spaces

[radarwave]

Github user radarwave commented on the issue:

https://github.com/apache/incubator-hawq/pull/489
  
@cjcjameson Most looks good to me.
Please use this template file 'GNUMakefile.in' to generate 'GNUMakefile' 
and test the run result. They should be check-in together.  Thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (HAWQ-878) Add googletest cases for the ao snappy compression support.

[Paul Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Guo updated HAWQ-878:
--
Summary: Add googletest cases for the ao snappy compression support.  (was: 
Add ao snappy googletest cases.)

> Add googletest cases for the ao snappy compression support.
> ---
>
> Key: HAWQ-878
> URL: https://issues.apache.org/jira/browse/HAWQ-878
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Lei Chang
>
> I've added ao snappy support in hawq before. Generally I should have 
> integrated feature with test code in one commit, however I'm not familiar 
> with this tradition before, thus I'm opening a new JIRA to integrate the test 
> case this time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HAWQ-878) Add ao snappy googletest cases.

[Paul Guo (JIRA)]

Paul Guo created HAWQ-878:
-

 Summary: Add ao snappy googletest cases.
 Key: HAWQ-878
 URL: https://issues.apache.org/jira/browse/HAWQ-878
 Project: Apache HAWQ
  Issue Type: Bug
Reporter: Paul Guo
Assignee: Lei Chang


I've added ao snappy support in hawq before. Generally I should have integrated 
feature with test code in one commit, however I'm not familiar with this 
tradition before, thus I'm opening a new JIRA to integrate the test case this 
time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (HAWQ-869) Add regression test for less tuple is inserted issue in prepared statement

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo closed HAWQ-869.


> Add regression test for less tuple is inserted issue in prepared statement
> --
>
> Key: HAWQ-869
> URL: https://issues.apache.org/jira/browse/HAWQ-869
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
>
> To make sure we don't regress in less tuple is inserted issue in prepared 
> statement, we need to add feature test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HAWQ-869) Add regression test for less tuple is inserted issue in prepared statement

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo resolved HAWQ-869.
--
Resolution: Fixed

> Add regression test for less tuple is inserted issue in prepared statement
> --
>
> Key: HAWQ-869
> URL: https://issues.apache.org/jira/browse/HAWQ-869
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
>
> To make sure we don't regress in less tuple is inserted issue in prepared 
> statement, we need to add feature test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HAWQ-869) Add regression test for less tuple is inserted issue in prepared statement

[Ruilong Huo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruilong Huo updated HAWQ-869:
-
Affects Version/s: 2.0.0

> Add regression test for less tuple is inserted issue in prepared statement
> --
>
> Key: HAWQ-869
> URL: https://issues.apache.org/jira/browse/HAWQ-869
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Tests
>Affects Versions: 2.0.0
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
>
> To make sure we don't regress in less tuple is inserted issue in prepared 
> statement, we need to add feature test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-876) Add the support for initFile option of gpdiff.pl in hawq googletest framework.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354406#comment-15354406
 ] 

ASF GitHub Bot commented on HAWQ-876:
-

Github user xunzhang commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/758#discussion_r68876831
  
--- Diff: src/test/feature/lib/hawq_config.cpp ---
@@ -151,7 +151,7 @@ bool HawqConfig::isMultinodeMode() {
   psql.getQueryResult("select hostname from gp_segment_configuration");
   std::vector table = result.getRows();
   std::unordered_set hostnameMap;
-  for (int i = 0; i < table.size(); i++) {
+  for (unsigned int i = 0; i < table.size(); i++) {
--- End diff --

use `size_t` instead


> Add the support for initFile option of gpdiff.pl in hawq googletest framework.
> --
>
> Key: HAWQ-876
> URL: https://issues.apache.org/jira/browse/HAWQ-876
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Lei Chang
>
> Hawq googletest framework depends on hawq tool gpdiff.pl. The tool provides 
> an enhanced diff comparison between sql output file and expected_out file. It
> supports a useful option "-gpd_init ' which defines some directives 
> for comparison. This option is quite useful for some complex sql tests.  
> Current hawq googletest work does not use this option. We need to add this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #758: HAWQ-876. Add the support for initFile opt...

[xunzhang]

Github user xunzhang commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/758#discussion_r68876831
  
--- Diff: src/test/feature/lib/hawq_config.cpp ---
@@ -151,7 +151,7 @@ bool HawqConfig::isMultinodeMode() {
   psql.getQueryResult("select hostname from gp_segment_configuration");
   std::vector table = result.getRows();
   std::unordered_set hostnameMap;
-  for (int i = 0; i < table.size(); i++) {
+  for (unsigned int i = 0; i < table.size(); i++) {
--- End diff --

use `size_t` instead


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-871) HAWQ CHECK looking for wrong YARN HA parameters

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354405#comment-15354405
 ] 

ASF GitHub Bot commented on HAWQ-871:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/755


> HAWQ CHECK looking for wrong YARN HA parameters
> ---
>
> Key: HAWQ-871
> URL: https://issues.apache.org/jira/browse/HAWQ-871
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Command Line Tools
>Reporter: Radar Lei
>Assignee: Radar Lei
>
> When YARN HA is enabled in HDP, the following four params don't actually 
> exist. However, hawq check has been hard-coded to look for them, causing a 
> failure. 
> yarn.resourcemanager.address.rm1
> yarn.resourcemanager.address.rm2
> yarn.resourcemanager.scheduler.address.rm1
> yarn.resourcemanager.scheduler.address.rm2 
> **Correct params:**
> yarn.resourcemanager.hostname.rm1
> yarn.resourcemanager.hostname.rm2
> yarn.resourcemanager.scheduler.address (apparently, there is no rm1/rm2 
> specified for scheduler)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HAWQ-877) Need to stop building when building error occurs in gpfdist directory

[Ming LI (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ming LI resolved HAWQ-877.
--
Resolution: Fixed

> Need to stop building when building error occurs in gpfdist directory
> -
>
> Key: HAWQ-877
> URL: https://issues.apache.org/jira/browse/HAWQ-877
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Build
>Reporter: Ming LI
>Assignee: Lei Chang
>
> When building gpfdist report error "'openssl/ssl.h' file not found", it 
> doesn't stop building, so the last error we can see it not the right root 
> error. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-871) HAWQ CHECK looking for wrong YARN HA parameters

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354397#comment-15354397
 ] 

ASF GitHub Bot commented on HAWQ-871:
-

Github user yaoj2 commented on the issue:

https://github.com/apache/incubator-hawq/pull/755
  
LGTM


> HAWQ CHECK looking for wrong YARN HA parameters
> ---
>
> Key: HAWQ-871
> URL: https://issues.apache.org/jira/browse/HAWQ-871
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Command Line Tools
>Reporter: Radar Lei
>Assignee: Radar Lei
>
> When YARN HA is enabled in HDP, the following four params don't actually 
> exist. However, hawq check has been hard-coded to look for them, causing a 
> failure. 
> yarn.resourcemanager.address.rm1
> yarn.resourcemanager.address.rm2
> yarn.resourcemanager.scheduler.address.rm1
> yarn.resourcemanager.scheduler.address.rm2 
> **Correct params:**
> yarn.resourcemanager.hostname.rm1
> yarn.resourcemanager.hostname.rm2
> yarn.resourcemanager.scheduler.address (apparently, there is no rm1/rm2 
> specified for scheduler)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-871) HAWQ CHECK looking for wrong YARN HA parameters

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354388#comment-15354388
 ] 

ASF GitHub Bot commented on HAWQ-871:
-

Github user linwen commented on the issue:

https://github.com/apache/incubator-hawq/pull/755
  
LGTM +1


> HAWQ CHECK looking for wrong YARN HA parameters
> ---
>
> Key: HAWQ-871
> URL: https://issues.apache.org/jira/browse/HAWQ-871
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Command Line Tools
>Reporter: Radar Lei
>Assignee: Radar Lei
>
> When YARN HA is enabled in HDP, the following four params don't actually 
> exist. However, hawq check has been hard-coded to look for them, causing a 
> failure. 
> yarn.resourcemanager.address.rm1
> yarn.resourcemanager.address.rm2
> yarn.resourcemanager.scheduler.address.rm1
> yarn.resourcemanager.scheduler.address.rm2 
> **Correct params:**
> yarn.resourcemanager.hostname.rm1
> yarn.resourcemanager.hostname.rm2
> yarn.resourcemanager.scheduler.address (apparently, there is no rm1/rm2 
> specified for scheduler)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #755: HAWQ-871. HAWQ CHECK looking for wrong YARN HA pa...

[linwen]

Github user linwen commented on the issue:

https://github.com/apache/incubator-hawq/pull/755
  
LGTM +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #757: HAWQ-877. Need to stop building when building err...

[paul-guo-]

Github user paul-guo- commented on the issue:

https://github.com/apache/incubator-hawq/pull/757
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-877) Need to stop building when building error occurs in gpfdist directory

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354384#comment-15354384
 ] 

ASF GitHub Bot commented on HAWQ-877:
-

Github user paul-guo- commented on the issue:

https://github.com/apache/incubator-hawq/pull/757
  
+1


> Need to stop building when building error occurs in gpfdist directory
> -
>
> Key: HAWQ-877
> URL: https://issues.apache.org/jira/browse/HAWQ-877
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Build
>Reporter: Ming LI
>Assignee: Lei Chang
>
> When building gpfdist report error "'openssl/ssl.h' file not found", it 
> doesn't stop building, so the last error we can see it not the right root 
> error. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-876) Add the support for initFile option of gpdiff.pl in hawq googletest framework.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354216#comment-15354216
 ] 

ASF GitHub Bot commented on HAWQ-876:
-

Github user ztao1987 commented on the issue:

https://github.com/apache/incubator-hawq/pull/758
  
+1


> Add the support for initFile option of gpdiff.pl in hawq googletest framework.
> --
>
> Key: HAWQ-876
> URL: https://issues.apache.org/jira/browse/HAWQ-876
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Lei Chang
>
> Hawq googletest framework depends on hawq tool gpdiff.pl. The tool provides 
> an enhanced diff comparison between sql output file and expected_out file. It
> supports a useful option "-gpd_init ' which defines some directives 
> for comparison. This option is quite useful for some complex sql tests.  
> Current hawq googletest work does not use this option. We need to add this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #758: HAWQ-876. Add the support for initFile option of ...

[ztao1987]

Github user ztao1987 commented on the issue:

https://github.com/apache/incubator-hawq/pull/758
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-876) Add the support for initFile option of gpdiff.pl in hawq googletest framework.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354212#comment-15354212
 ] 

ASF GitHub Bot commented on HAWQ-876:
-

Github user wengyanqing commented on the issue:

https://github.com/apache/incubator-hawq/pull/758
  
LGTM


> Add the support for initFile option of gpdiff.pl in hawq googletest framework.
> --
>
> Key: HAWQ-876
> URL: https://issues.apache.org/jira/browse/HAWQ-876
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Lei Chang
>
> Hawq googletest framework depends on hawq tool gpdiff.pl. The tool provides 
> an enhanced diff comparison between sql output file and expected_out file. It
> supports a useful option "-gpd_init ' which defines some directives 
> for comparison. This option is quite useful for some complex sql tests.  
> Current hawq googletest work does not use this option. We need to add this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #758: HAWQ-876. Add the support for initFile option of ...

[wengyanqing]

Github user wengyanqing commented on the issue:

https://github.com/apache/incubator-hawq/pull/758
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-876) Add the support for initFile option of gpdiff.pl in hawq googletest framework.

[Paul Guo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354209#comment-15354209
 ] 

Paul Guo commented on HAWQ-876:
---

I'm requesting the feature since I'm adding AO snappy tests into hawq code and 
those tests need it.

> Add the support for initFile option of gpdiff.pl in hawq googletest framework.
> --
>
> Key: HAWQ-876
> URL: https://issues.apache.org/jira/browse/HAWQ-876
> Project: Apache HAWQ
>  Issue Type: Bug
>Reporter: Paul Guo
>Assignee: Lei Chang
>
> Hawq googletest framework depends on hawq tool gpdiff.pl. The tool provides 
> an enhanced diff comparison between sql output file and expected_out file. It
> supports a useful option "-gpd_init ' which defines some directives 
> for comparison. This option is quite useful for some complex sql tests.  
> Current hawq googletest work does not use this option. We need to add this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #756: HAWQ-873. improve checking time for .travis.yml

[yaoj2]

Github user yaoj2 commented on the issue:

https://github.com/apache/incubator-hawq/pull/756
  
Cool +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #758: HAWQ-876. Add the support for initFile opt...

[paul-guo-]

GitHub user paul-guo- opened a pull request:

https://github.com/apache/incubator-hawq/pull/758

HAWQ-876. Add the support for initFile option of gpdiff.pl in hawq go…

…ogletest framework.

This patch also removes several annoying warnings.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/paul-guo-/incubator-hawq snappy

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/758.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #758


commit 7d2b2f901db6c90c4636c88494cd8fbabc765770
Author: Paul Guo 
Date:   2016-06-28T10:09:23Z

HAWQ-876. Add the support for initFile option of gpdiff.pl in hawq 
googletest framework.

This patch also removes several annoying warnings.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #490: HAWQ-543. Remove references to bootstrap_t...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/490


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-877) Need to stop building when building error occurs in gpfdist directory

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352779#comment-15352779
 ] 

ASF GitHub Bot commented on HAWQ-877:
-

Github user liming01 commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/757#discussion_r68733952
  
--- Diff: src/bin/gpfdist/Makefile ---
@@ -60,9 +60,11 @@ mkgpfdist: mkdir
 
for file in $(GPFDISTFILES); do \
( $(CC) $(INCLUDES) $(CFLAGS) $(LIBS) -c $(code_dir)$${file}); \
--- End diff --

Get it. But I have no test env for retest this write, I will keep it 
unchanged because the previous writing was already verified. 


> Need to stop building when building error occurs in gpfdist directory
> -
>
> Key: HAWQ-877
> URL: https://issues.apache.org/jira/browse/HAWQ-877
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Build
>Reporter: Ming LI
>Assignee: Lei Chang
>
> When building gpfdist report error "'openssl/ssl.h' file not found", it 
> doesn't stop building, so the last error we can see it not the right root 
> error. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #757: HAWQ-877. Need to stop building when build...

[liming01]

Github user liming01 commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/757#discussion_r68733952
  
--- Diff: src/bin/gpfdist/Makefile ---
@@ -60,9 +60,11 @@ mkgpfdist: mkdir
 
for file in $(GPFDISTFILES); do \
( $(CC) $(INCLUDES) $(CFLAGS) $(LIBS) -c $(code_dir)$${file}); \
--- End diff --

Get it. But I have no test env for retest this write, I will keep it 
unchanged because the previous writing was already verified. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-877) Need to stop building when building error occurs in gpfdist directory

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352759#comment-15352759
 ] 

ASF GitHub Bot commented on HAWQ-877:
-

Github user paul-guo- commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/757#discussion_r68732483
  
--- Diff: src/bin/gpfdist/Makefile ---
@@ -60,9 +60,11 @@ mkgpfdist: mkdir
 
for file in $(GPFDISTFILES); do \
( $(CC) $(INCLUDES) $(CFLAGS) $(LIBS) -c $(code_dir)$${file}); \
--- End diff --

Following way is cleaner.
$cmd || exit 1


> Need to stop building when building error occurs in gpfdist directory
> -
>
> Key: HAWQ-877
> URL: https://issues.apache.org/jira/browse/HAWQ-877
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Build
>Reporter: Ming LI
>Assignee: Lei Chang
>
> When building gpfdist report error "'openssl/ssl.h' file not found", it 
> doesn't stop building, so the last error we can see it not the right root 
> error. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-877) Need to stop building when building error occurs in gpfdist directory

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352761#comment-15352761
 ] 

ASF GitHub Bot commented on HAWQ-877:
-

Github user paul-guo- commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/757#discussion_r68732512
  
--- Diff: src/bin/gpfdist/Makefile ---
@@ -60,9 +60,11 @@ mkgpfdist: mkdir
 
for file in $(GPFDISTFILES); do \
( $(CC) $(INCLUDES) $(CFLAGS) $(LIBS) -c $(code_dir)$${file}); \
+   if [[ $$? -ne 0 ]]; then exit 1; fi \
done
# link
$(CC) $(CFLAGS) -o $(code_dir)gpfdist $(OBJS)  $(LIBS)
+   if [[ $$? -ne 0 ]]; then exit 1; fi
--- End diff --

This line code change is not necessary.


> Need to stop building when building error occurs in gpfdist directory
> -
>
> Key: HAWQ-877
> URL: https://issues.apache.org/jira/browse/HAWQ-877
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Build
>Reporter: Ming LI
>Assignee: Lei Chang
>
> When building gpfdist report error "'openssl/ssl.h' file not found", it 
> doesn't stop building, so the last error we can see it not the right root 
> error. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-877) Need to stop building when building error occurs in gpfdist directory

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352668#comment-15352668
 ] 

ASF GitHub Bot commented on HAWQ-877:
-

Github user xunzhang commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/757#discussion_r68725485
  
--- Diff: src/bin/gpfdist/Makefile ---
@@ -60,9 +60,11 @@ mkgpfdist: mkdir
 
for file in $(GPFDISTFILES); do \
( $(CC) $(INCLUDES) $(CFLAGS) $(LIBS) -c $(code_dir)$${file}); \
+   if [[ $$? -ne 0 ]]; then exit 1; fi \
--- End diff --

indent?


> Need to stop building when building error occurs in gpfdist directory
> -
>
> Key: HAWQ-877
> URL: https://issues.apache.org/jira/browse/HAWQ-877
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Build
>Reporter: Ming LI
>Assignee: Lei Chang
>
> When building gpfdist report error "'openssl/ssl.h' file not found", it 
> doesn't stop building, so the last error we can see it not the right root 
> error. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HAWQ-875) Upgrade HAWQ version to 2.0.1.0

[Radar Lei (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Radar Lei reassigned HAWQ-875:
--

Assignee: Radar Lei  (was: Lei Chang)

> Upgrade HAWQ version to 2.0.1.0
> ---
>
> Key: HAWQ-875
> URL: https://issues.apache.org/jira/browse/HAWQ-875
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Build
>Reporter: Radar Lei
>Assignee: Radar Lei
>
> Since we already have a branch for HAWQ 2.0.0.0, maybe we should start to  
> use a new version number '2.0.1.0'.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-835) Cannot retrieve tuple from temp table created in function

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352541#comment-15352541
 ] 

ASF GitHub Bot commented on HAWQ-835:
-

Github user huor closed the pull request at:

https://github.com/apache/incubator-hawq/pull/754


> Cannot retrieve tuple from temp table created in function
> -
>
> Key: HAWQ-835
> URL: https://issues.apache.org/jira/browse/HAWQ-835
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Core, Query Execution
>Reporter: Ruilong Huo
>Assignee: Ruilong Huo
>
> With function which create temp table and insert tuple into it, if the 
> function is run multiple times, it might cannot retrieve tuple from the temp 
> table in the second run of the function and so on.
> Here are the steps to reproduce:
> Step 1: prepare schema and data
> {noformat}
> CREATE TABLE t(pid int, points double precision[]);
> COPY t (pid, points) FROM stdin DELIMITER '|';
> 1 | {14.23, 1.71, 2.43, 15.6, 127, 2.8, 3.0600, 0.2800, 2.29, 5.64, 1.04, 
> 3.92, 1065}
> 2 | {13.2, 1.78, 2.14, 11.2, 1, 2.65, 2.76, 0.26, 1.28, 4.38, 1.05, 3.49, 
> 1050}
> 3 | {13.16, 2.36,  2.67, 18.6, 101, 2.8,  3.24, 0.3, 2.81, 5.6799, 1.03, 
> 3.17, 1185}
> 4 | {14.37, 1.95, 2.5, 16.8, 113, 3.85, 3.49, 0.24, 2.18, 7.8, 0.86, 3.45, 
> 1480}
> 5 | {13.24, 2.59, 2.87, 21, 118, 2.8, 2.69, 0.39, 1.82, 4.32, 1.04, 2.93, 735}
> 6 | {14.2, 1.76, 2.45, 15.2, 112, 3.27, 3.39, 0.34, 1.97, 6.75, 1.05, 2.85, 
> 1450}
> \.
> {noformat}
> Step 2: run kmeans
> {noformat}
> with q as
> (
> select
> 1 as num_clusters,
>*
> from
> madlib.kmeanspp(
>  't',
>  'points',
>  3,
>  'madlib.squared_dist_norm2',
>  'madlib.avg',
>  30,
>  0.001,
>  1.0
>  )
> )
> select q1.*
> from q as q1, (select * from q) as q2
> where q1.num_clusters = q2.num_clusters;
> ERROR:  Kmeans error: No valid initial centroids given.
> CONTEXT:  SQL statement "SELECT  madlib.kmeans(  $1 ,  $2 , 
> madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 , NULL,  $5 ),  $4 ,  $6 ,  $7 
> ,  $8 )"
> PL/pgSQL function "kmeanspp" line 4 at assignment
> {noformat}
> Step 3: further investigation shows that it cannot retrieve tuple from temp 
> table
> {noformat}
> with q as
> (
> select
> 1 as num_clusters,
>*
> from
> madlib.kmeanspp(
>  't',
>  'points',
>  3,
>  'madlib.squared_dist_norm2',
>  'madlib.avg',
>  30,
>  0.001,
>  1.0
>  )
> )
> select q1.*
> from q as q1, (select * from q) as q2
> where q1.num_clusters = q2.num_clusters;
> WARNING:  --- kmeanspp debug begin ---
> WARNING:  --- kmeanspp seeding begin ---
> WARNING:  --- kmeanspp seed summary ---
> WARNING:  ### kmeanspp_seeding: 1. use all source data
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 2. create temp schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 0 before create tmp schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 140497 after create tmp 
> schema
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 3. generate centroids
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: 3. generate centroids
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: temp schema id = 140497 after create tmp table
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### kmeanspp_seeding: pg_temp._madlib_kmeanspp_args has 1 record XXX
> CONTEXT:  SQL statement "SELECT madlib.kmeanspp_seeding( $1 ,  $2 ,  $3 ,  $4 
> , NULL,  $5 )"
> PL/pgSQL function "kmeanspp" line 10 at SQL statement
> WARNING:  ### 

[GitHub] incubator-hawq pull request #:

[liming01]

Github user liming01 commented on the pull request:


https://github.com/apache/incubator-hawq/commit/5a8622707749cfe98c61188ab212a61c83778998#commitcomment-18036114
  
Hi Paul,

I put all these links in one line at the page header, so only the most 
useful links are listed. If you need add more details, we should at them in the 
middle content of README.md.  Thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-873) Improve checking time for travis CI

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352437#comment-15352437
 ] 

ASF GitHub Bot commented on HAWQ-873:
-

GitHub user xunzhang opened a pull request:

https://github.com/apache/incubator-hawq/pull/756

HAWQ-873. improve checking time for .travis.yml

From 25min -> 17min: 
https://travis-ci.org/xunzhang/incubator-hawq/builds/140718129

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xunzhang/incubator-hawq HAWQ-873

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/756.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #756


commit 47f82ea0ed0ebd37d7737a15bc006fc4a333a446
Author: xunzhang 
Date:   2016-06-28T05:53:04Z

HAWQ-873. improve checking time for .travis.yml




> Improve checking time for travis CI
> ---
>
> Key: HAWQ-873
> URL: https://issues.apache.org/jira/browse/HAWQ-873
> Project: Apache HAWQ
>  Issue Type: Improvement
>Reporter: hongwu
>Assignee: hongwu
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #756: HAWQ-873. improve checking time for .travis.yml

[xunzhang]

Github user xunzhang commented on the issue:

https://github.com/apache/incubator-hawq/pull/756
  
cc @liming01 @yaoj2 @radarwave 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (HAWQ-873) Improve checking time for travis CI

[hongwu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hongwu reassigned HAWQ-873:
---

Assignee: hongwu  (was: Lei Chang)

> Improve checking time for travis CI
> ---
>
> Key: HAWQ-873
> URL: https://issues.apache.org/jira/browse/HAWQ-873
> Project: Apache HAWQ
>  Issue Type: Improvement
>Reporter: hongwu
>Assignee: hongwu
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)