[jira] [Commented] (HAWQ-426) Memory leak for query and prepared statement due to introduction of query resource parameters
[ https://issues.apache.org/jira/browse/HAWQ-426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15155648#comment-15155648 ] ASF GitHub Bot commented on HAWQ-426: - GitHub user huor opened a pull request: https://github.com/apache/incubator-hawq/pull/361 HAWQ-426. Fix memory leak for query and prepared statement due to introduction of query resource parameters The query resource parameters is kept in planned statement. So, the resolution is to make query resource parameters using the same memory context as planned statement. Thus, they can be freed at the same time. From implementation standpoint, we do a deep copy of query resource parameters in copy function of plannedstmt so that they use the same memory context. You can merge this pull request into a Git repository by running: $ git pull https://github.com/huor/incubator-hawq huor_pbe Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/361.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #361 commit aa7be29bfb4af1b20ca3a224226d8b77208464e3 Author: Ruilong Huo Date: 2016-02-20T14:34:21Z HAWQ-426. Fix memory leak for query and prepared statement due to introduction of query resource parameters > Memory leak for query and prepared statement due to introduction of query > resource parameters > - > > Key: HAWQ-426 > URL: https://issues.apache.org/jira/browse/HAWQ-426 > Project: Apache HAWQ > Issue Type: Bug > Components: Core, Query Execution >Affects Versions: 2.0.0-beta-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.0.0 > > > During resource allocation for query and prepared statement, we cache the > query resource parameters so that query resource can be allocated for > multiple query execution after query preparation. Currently, query resource > parameters is kept in CacheMemoryContext or TopMemoryContext which may lead > to memory leak. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HAWQ-426) Memory leak for query and prepared statement due to introduction of query resource parameters
[ https://issues.apache.org/jira/browse/HAWQ-426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15155647#comment-15155647 ] Ruilong Huo edited comment on HAWQ-426 at 2/20/16 2:42 PM: --- The query resource parameters is kept in planned statement. So, the resolution is to make query resource parameters using the same memory context as planned statement. Thus, they can be freed at the same time. From implementation standpoint, we do a deep copy of query resource parameters in copy function of plannedstmt so that they use the same memory context. was (Author: huor): The query resource parameters is kept in planned statement. So, the resolution is to make query resource parameters using the same memory context as planned statement. Thus, they can be freed at the same time. From implementation standpoint, we do a deep copy of query resource parameters in copy function of plannedstmt so that they use the same memory context. > Memory leak for query and prepared statement due to introduction of query > resource parameters > - > > Key: HAWQ-426 > URL: https://issues.apache.org/jira/browse/HAWQ-426 > Project: Apache HAWQ > Issue Type: Bug > Components: Core, Query Execution >Affects Versions: 2.0.0-beta-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.0.0 > > > During resource allocation for query and prepared statement, we cache the > query resource parameters so that query resource can be allocated for > multiple query execution after query preparation. Currently, query resource > parameters is kept in CacheMemoryContext or TopMemoryContext which may lead > to memory leak. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HAWQ-426) Memory leak for query and prepared statement due to introduction of query resource parameters
[ https://issues.apache.org/jira/browse/HAWQ-426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15155647#comment-15155647 ] Ruilong Huo edited comment on HAWQ-426 at 2/20/16 2:40 PM: --- The query resource parameters is kept in planned statement. So, the resolution is to make query resource parameters using the same memory context as planned statement. Thus, they can be freed at the same time. From implementation standpoint, we do a deep copy of query resource parameters in copy function of plannedstmt so that they use the same memory context. was (Author: huor): The query resource parameters is kept in planned statement. The resolution is to make query resource parameters using the same memory context as planned statement. Thus, they can be freed at the same time. > Memory leak for query and prepared statement due to introduction of query > resource parameters > - > > Key: HAWQ-426 > URL: https://issues.apache.org/jira/browse/HAWQ-426 > Project: Apache HAWQ > Issue Type: Bug > Components: Core, Query Execution >Affects Versions: 2.0.0-beta-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.0.0 > > > During resource allocation for query and prepared statement, we cache the > query resource parameters so that query resource can be allocated for > multiple query execution after query preparation. Currently, query resource > parameters is kept in CacheMemoryContext or TopMemoryContext which may lead > to memory leak. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-426) Memory leak for query and prepared statement due to introduction of query resource parameters
[ https://issues.apache.org/jira/browse/HAWQ-426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15155647#comment-15155647 ] Ruilong Huo commented on HAWQ-426: -- The query resource parameters is kept in planned statement. The resolution is to make query resource parameters using the same memory context as planned statement. Thus, they can be freed at the same time. > Memory leak for query and prepared statement due to introduction of query > resource parameters > - > > Key: HAWQ-426 > URL: https://issues.apache.org/jira/browse/HAWQ-426 > Project: Apache HAWQ > Issue Type: Bug > Components: Core, Query Execution >Affects Versions: 2.0.0-beta-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.0.0 > > > During resource allocation for query and prepared statement, we cache the > query resource parameters so that query resource can be allocated for > multiple query execution after query preparation. Currently, query resource > parameters is kept in CacheMemoryContext or TopMemoryContext which may lead > to memory leak. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-426) Fix memory leak for query and prepared statement due to introduction of query resource parameters
[ https://issues.apache.org/jira/browse/HAWQ-426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo updated HAWQ-426: - Affects Version/s: 2.0.0-beta-incubating > Fix memory leak for query and prepared statement due to introduction of query > resource parameters > - > > Key: HAWQ-426 > URL: https://issues.apache.org/jira/browse/HAWQ-426 > Project: Apache HAWQ > Issue Type: Bug > Components: Core, Query Execution >Affects Versions: 2.0.0-beta-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.0.0 > > > During resource allocation for query and prepared statement, we cache the > query resource parameters so that query resource can be allocated for > multiple query execution after query preparation. Currently, query resource > parameters is kept in CacheMemoryContext or TopMemoryContext which may lead > to memory leak. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-426) Fix memory leak for query and prepared statement due to introduction of query resource parameters
[ https://issues.apache.org/jira/browse/HAWQ-426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo reassigned HAWQ-426: Assignee: Ruilong Huo (was: George Caragea) > Fix memory leak for query and prepared statement due to introduction of query > resource parameters > - > > Key: HAWQ-426 > URL: https://issues.apache.org/jira/browse/HAWQ-426 > Project: Apache HAWQ > Issue Type: Bug > Components: Core, Query Execution >Affects Versions: 2.0.0-beta-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.0.0 > > > During resource allocation for query and prepared statement, we cache the > query resource parameters so that query resource can be allocated for > multiple query execution after query preparation. Currently, query resource > parameters is kept in CacheMemoryContext or TopMemoryContext which may lead > to memory leak. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-426) Fix memory leak for query and prepared statement due to introduction of query resource parameters
[ https://issues.apache.org/jira/browse/HAWQ-426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo updated HAWQ-426: - Fix Version/s: 2.0.0 > Fix memory leak for query and prepared statement due to introduction of query > resource parameters > - > > Key: HAWQ-426 > URL: https://issues.apache.org/jira/browse/HAWQ-426 > Project: Apache HAWQ > Issue Type: Bug > Components: Core, Query Execution >Affects Versions: 2.0.0-beta-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.0.0 > > > During resource allocation for query and prepared statement, we cache the > query resource parameters so that query resource can be allocated for > multiple query execution after query preparation. Currently, query resource > parameters is kept in CacheMemoryContext or TopMemoryContext which may lead > to memory leak. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-426) Memory leak for query and prepared statement due to introduction of query resource parameters
[ https://issues.apache.org/jira/browse/HAWQ-426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo updated HAWQ-426: - Summary: Memory leak for query and prepared statement due to introduction of query resource parameters (was: Fix memory leak for query and prepared statement due to introduction of query resource parameters) > Memory leak for query and prepared statement due to introduction of query > resource parameters > - > > Key: HAWQ-426 > URL: https://issues.apache.org/jira/browse/HAWQ-426 > Project: Apache HAWQ > Issue Type: Bug > Components: Core, Query Execution >Affects Versions: 2.0.0-beta-incubating >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.0.0 > > > During resource allocation for query and prepared statement, we cache the > query resource parameters so that query resource can be allocated for > multiple query execution after query preparation. Currently, query resource > parameters is kept in CacheMemoryContext or TopMemoryContext which may lead > to memory leak. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-426) Fix memory leak for query and prepared statement due to introduction of query resource parameters
[ https://issues.apache.org/jira/browse/HAWQ-426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo updated HAWQ-426: - Description: During resource allocation for query and prepared statement, we cache the query resource parameters so that query resource can be allocated for multiple query execution after query preparation. Currently, query resource parameters is kept in CacheMemoryContext or TopMemoryContext which may lead to memory leak. (was: During resource allocation for PBE, we cache the query resource parameters so that query resource can be allocated for multiple query execution after query preparation. Currently, query resource parameters is kept in CacheMemoryContext or TopMemoryContext which may lead to memory leak.) > Fix memory leak for query and prepared statement due to introduction of query > resource parameters > - > > Key: HAWQ-426 > URL: https://issues.apache.org/jira/browse/HAWQ-426 > Project: Apache HAWQ > Issue Type: Bug > Components: Core, Query Execution >Reporter: Ruilong Huo >Assignee: George Caragea > > During resource allocation for query and prepared statement, we cache the > query resource parameters so that query resource can be allocated for > multiple query execution after query preparation. Currently, query resource > parameters is kept in CacheMemoryContext or TopMemoryContext which may lead > to memory leak. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-426) Fix memory leak for query and prepared statement due to introduction of query resource parameters
Ruilong Huo created HAWQ-426: Summary: Fix memory leak for query and prepared statement due to introduction of query resource parameters Key: HAWQ-426 URL: https://issues.apache.org/jira/browse/HAWQ-426 Project: Apache HAWQ Issue Type: Bug Components: Core, Query Execution Reporter: Ruilong Huo Assignee: George Caragea During resource allocation for PBE, we cache the query resource parameters so that query resource can be allocated for multiple query execution after query preparation. Currently, query resource parameters is kept in CacheMemoryContext or TopMemoryContext which may lead to memory leak. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-425) In Yarn mode, segment can start without hawq_rm_memory_limit_perseg and hawq_rm_nvcore_limit_perseg configured
Lin Wen created HAWQ-425: Summary: In Yarn mode, segment can start without hawq_rm_memory_limit_perseg and hawq_rm_nvcore_limit_perseg configured Key: HAWQ-425 URL: https://issues.apache.org/jira/browse/HAWQ-425 Project: Apache HAWQ Issue Type: Bug Components: Resource Manager Reporter: Lin Wen Assignee: Lei Chang In Yarn mode, memory and cpu quota is reported by yarn node manager. So it is no need to configure hawq_rm_memory_limit_perseg and hawq_rm_nvcore_limit_perseg in hawk-site.xml in yarn mode. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
