[jira] [Created] (HAWQ-1260) Remove temp tables after hawq restart
Paul Guo created HAWQ-1260: -- Summary: Remove temp tables after hawq restart Key: HAWQ-1260 URL: https://issues.apache.org/jira/browse/HAWQ-1260 Project: Apache HAWQ Issue Type: Bug Components: Command Line Tools Reporter: Paul Guo Assignee: Ed Espino Sometimes hawq encounters errors so have to restart (e.g. oom-kill, debug), useless temp tables are left on hdfs and catalog. It seems that one of the solution is to remove the pg_temp_* schema automatically after hawq restart. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #1074: HAWQ-1249. Don't do ACL checks on segment...
Github user paul-guo- commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/1074#discussion_r94894516 --- Diff: src/backend/catalog/aclchk.c --- @@ -224,16 +224,17 @@ restrict_and_check_grant(bool is_grant, AclMode avail_goptions, bool all_privs, * If we found no grant options, consider whether to issue a hard error. * Per spec, having any privilege at all on the object will get you by * here. +* Superusers or on QE bypass all permission checking. */ - if (avail_goptions == ACL_NO_RIGHTS) + if (avail_goptions == ACL_NO_RIGHTS && GP_ROLE_EXECUTE != Gp_role) --- End diff -- I think you should change to "Gp_role != GP_ROLE_EXECUTE" This is more human-understandable and follows the code convention. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1074: HAWQ-1249. Don't do ACL checks on segment...
Github user paul-guo- commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/1074#discussion_r94894709 --- Diff: src/backend/catalog/aclchk.c --- @@ -3006,8 +3007,8 @@ pg_database_aclmask(Oid db_oid, Oid roleid, Oid ownerId; cqContext *pcqCtx; - /* Superusers or on QE bypass all permission checking. */ - if (GP_ROLE_EXECUTE == Gp_role || superuser_arg(roleid)) --- End diff -- By the way, I know many people like to write "GP_ROLE_EXECUTE == Gp_role" similar code to avoid typo like "GP_role = GP_ROLE_EXECUTE" but gcc will warn against this kind of error. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1074: HAWQ-1249. Don't do ACL checks on segment...
Github user paul-guo- commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/1074#discussion_r94894833 --- Diff: src/backend/executor/execMain.c --- @@ -1912,45 +1912,10 @@ InitPlan(QueryDesc *queryDesc, int eflags) * rangetable here --- subplan RTEs will be checked during * ExecInitSubPlan(). */ - if (operation != CMD_SELECT || - (Gp_role != GP_ROLE_EXECUTE && -!(shouldDispatch && cdbpathlocus_querysegmentcatalogs))) --- End diff -- Are you sure these parts of code could be removed? Please double-check to see whether those conditions still apply, following the comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-1209) docs - create an overview page for HAWQ
[ https://issues.apache.org/jira/browse/HAWQ-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802991#comment-15802991 ] ASF GitHub Bot commented on HAWQ-1209: -- Github user dyozie commented on a diff in the pull request: https://github.com/apache/incubator-hawq-docs/pull/80#discussion_r94880323 --- Diff: admin/setuphawqopenv.html.md.erb --- @@ -0,0 +1,81 @@ +--- +title: Introducing the HAWQ Operating Environment +--- + +Before invoking operations on a HAWQ cluster, you must set up your HAWQ environment. This set up is required for both administrative and non-administrative HAWQ users. + +## Procedure: Setting Up Your HAWQ Operating Environment + +HAWQ installs a script that you can use to set up your HAWQ cluster environment. The `greenplum_path.sh` script, located in your HAWQ root install directory, sets `$PATH` and other environment variables to find HAWQ files. Most importantly, `greenplum_path.sh` sets the `$GPHOME` environment variable to point to the root directory of the HAWQ installation. If you installed HAWQ from a product distribution, the HAWQ root is typically `/usr/local/hawq`. If you built HAWQ from source or downloaded the tarball, you will have selected an install root directory on your own. + +Perform the following steps to set up your HAWQ operating environment: + +1. Log in to the HAWQ \system as the desired user. For example: + +``` shell +$ ssh gpadmin@ +gpadmin@master$ +``` + +Or, if you are already logged in to \ as a different user, switch to the desired user. For example: + +``` shell +gpadmin@master$ su - +Password: +hawq-user@master$ +``` + +2. Set up your HAWQ operating environment by sourcing the `greenplum_path.sh` file. For HAWQ deployments installed from a product distribution: + +``` shell +hawq-node$ source /usr/local/hawq/greenplum_path.sh +``` + +If you built HAWQ from source or downloaded the tarball, substitute the path to the installed or extracted `greenplum_path.sh` file \(for example `/opt/hawq-2.1.0.0/greenplum_path.sh`\). + + +3. Edit your `.bash_profile` or other shell initialization file to source `greenplum_path.sh` on login. For example, add: + +``` shell +source /usr/local/hawq/greenplum_path.sh --- End diff -- This and a few of the following command blocks are omitting the shell prompt. > docs - create an overview page for HAWQ > --- > > Key: HAWQ-1209 > URL: https://issues.apache.org/jira/browse/HAWQ-1209 > Project: Apache HAWQ > Issue Type: Improvement > Components: Documentation >Reporter: Lisa Owen >Assignee: David Yozie >Priority: Minor > > create a "landing" topic that ties together the components in the HAWQ > operating environment and the relevant documentation contents/sections. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1209) docs - create an overview page for HAWQ
[ https://issues.apache.org/jira/browse/HAWQ-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802985#comment-15802985 ] ASF GitHub Bot commented on HAWQ-1209: -- Github user dyozie commented on a diff in the pull request: https://github.com/apache/incubator-hawq-docs/pull/80#discussion_r94880036 --- Diff: admin/setuphawqopenv.html.md.erb --- @@ -0,0 +1,81 @@ +--- +title: Introducing the HAWQ Operating Environment +--- + +Before invoking operations on a HAWQ cluster, you must set up your HAWQ environment. This set up is required for both administrative and non-administrative HAWQ users. + +## Procedure: Setting Up Your HAWQ Operating Environment + +HAWQ installs a script that you can use to set up your HAWQ cluster environment. The `greenplum_path.sh` script, located in your HAWQ root install directory, sets `$PATH` and other environment variables to find HAWQ files. Most importantly, `greenplum_path.sh` sets the `$GPHOME` environment variable to point to the root directory of the HAWQ installation. If you installed HAWQ from a product distribution, the HAWQ root is typically `/usr/local/hawq`. If you built HAWQ from source or downloaded the tarball, you will have selected an install root directory on your own. + +Perform the following steps to set up your HAWQ operating environment: + +1. Log in to the HAWQ \system as the desired user. For example: + +``` shell +$ ssh gpadmin@ +gpadmin@master$ +``` + +Or, if you are already logged in to \ as a different user, switch to the desired user. For example: + +``` shell +gpadmin@master$ su - +Password: +hawq-user@master$ +``` + +2. Set up your HAWQ operating environment by sourcing the `greenplum_path.sh` file. For HAWQ deployments installed from a product distribution: --- End diff -- I think it's fine to remove "For HAWQ deployments installed from a product distribution:" here. > docs - create an overview page for HAWQ > --- > > Key: HAWQ-1209 > URL: https://issues.apache.org/jira/browse/HAWQ-1209 > Project: Apache HAWQ > Issue Type: Improvement > Components: Documentation >Reporter: Lisa Owen >Assignee: David Yozie >Priority: Minor > > create a "landing" topic that ties together the components in the HAWQ > operating environment and the relevant documentation contents/sections. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1209) docs - create an overview page for HAWQ
[ https://issues.apache.org/jira/browse/HAWQ-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802983#comment-15802983 ] ASF GitHub Bot commented on HAWQ-1209: -- Github user dyozie commented on a diff in the pull request: https://github.com/apache/incubator-hawq-docs/pull/80#discussion_r94879877 --- Diff: admin/setuphawqopenv.html.md.erb --- @@ -0,0 +1,81 @@ +--- +title: Introducing the HAWQ Operating Environment +--- + +Before invoking operations on a HAWQ cluster, you must set up your HAWQ environment. This set up is required for both administrative and non-administrative HAWQ users. + +## Procedure: Setting Up Your HAWQ Operating Environment + +HAWQ installs a script that you can use to set up your HAWQ cluster environment. The `greenplum_path.sh` script, located in your HAWQ root install directory, sets `$PATH` and other environment variables to find HAWQ files. Most importantly, `greenplum_path.sh` sets the `$GPHOME` environment variable to point to the root directory of the HAWQ installation. If you installed HAWQ from a product distribution, the HAWQ root is typically `/usr/local/hawq`. If you built HAWQ from source or downloaded the tarball, you will have selected an install root directory on your own. + +Perform the following steps to set up your HAWQ operating environment: + +1. Log in to the HAWQ \system as the desired user. For example: + --- End diff -- Has this convention been used elsewhere? It reads a bit awkwardly. Maybe just say "Login to a HAWQ node as the desired user."? > docs - create an overview page for HAWQ > --- > > Key: HAWQ-1209 > URL: https://issues.apache.org/jira/browse/HAWQ-1209 > Project: Apache HAWQ > Issue Type: Improvement > Components: Documentation >Reporter: Lisa Owen >Assignee: David Yozie >Priority: Minor > > create a "landing" topic that ties together the components in the HAWQ > operating environment and the relevant documentation contents/sections. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1209) docs - create an overview page for HAWQ
[ https://issues.apache.org/jira/browse/HAWQ-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802969#comment-15802969 ] ASF GitHub Bot commented on HAWQ-1209: -- Github user dyozie commented on a diff in the pull request: https://github.com/apache/incubator-hawq-docs/pull/80#discussion_r94879489 --- Diff: admin/RunningHAWQ.html.md.erb --- @@ -2,21 +2,36 @@ title: Running a HAWQ Cluster --- -This section provides information for system administrators and database superusers responsible for administering a HAWQ system. - -This guide provides information and instructions for configuring, maintaining and using a HAWQ system. This guide is intended for system and database administrators responsible for managing a HAWQ system. - -This guide assumes knowledge of Linux/UNIX system administration, database management systems, database administration, and structured query language \(SQL\). Because HAWQ is based on PostgreSQL 8.2.15, this guide assumes some familiarity with PostgreSQL. This guide calls out similarities between HAWQ and PostgreSQL features throughout. It contains the topics: - -* Managing HAWQ Using Ambari -* Starting and Stopping HAWQ -* Expanding a Cluster -* Removing a Node -* Backing Up and Restoring HAWQ -* High Availability in HAWQ -* Master Mirroring -* HAWQ Filespaces and High Availability Enabled HDFS -* Understanding the Fault Tolerance Service -* Recommended Monitoring and Maintenance Tasks -* Routine System Maintenance Tasks -* Monitoring a HAWQ System +Your HAWQ deployment environment is a collection of systems, processes, and data, and the users that operate on these systems, processes, and data. This section provides information for system administrators and database superusers responsible for administering a HAWQ deployment. + +You should have some knowledge of Linux/UNIX system administration, database management systems, database administration, and structured query language \(SQL\) to administer a HAWQ cluster. Because HAWQ is based on PostgreSQL 8.2.15, you should also have some familiarity with PostgreSQL. The HAWQ documentation calls out similarities between HAWQ and PostgreSQL features throughout. + +## HAWQ Users + +HAWQ supports users with both administrative and operating privileges. The HAWQ administrator may choose to manage the HAWQ cluster using either Ambari or the command line. [Managing HAWQ Using Ambari](../admin/ambari-admin.html) provides Ambari-specific HAWQ cluster administration procedures. [Starting and Stopping HAWQ](startstop.html), [Expanding a Cluster](ClusterExpansion.html), and [Removing a Node](ClusterShrink.html) describe specific command-line-managed HAWQ cluster administration procedures. Other topics in this guide are applicable to both Ambari- and command-line-managed HAWQ clusters. + +The default HAWQ admininstrator user is named `gpadmin`. The HAWQ admin may choose to assign administrative and/or operating HAWQ privileges to additional users. Refer to [Configuring Client Authentication](../clientaccess/client_auth.html) and [Managing Roles and Privileges](../clientaccess/roles_privs.html) for additional information on HAWQ user configuration. + +## HAWQ Deployment Systems + +A typical HAWQ deployment will include single HDFS and HAWQ master and standby nodes and multiple HAWQ segment and HDFS data nodes. The HAWQ cluster may also include systems running PXF and other Hadoop services. Refer to [HAWQ Architecture](../overview/HAWQArchitecture.html) and [Select HAWQ Host Machines](../install/select-hosts.html) for information about the different systems in a HAWQ deployment and how they are configured. + + +## HAWQ Databases + +[Creating and Managing Databases](../ddl/ddl-database.html) and [Creating and Managing Tables](../ddl/ddl-table.html) identify specific HAWQ database and table creation and access information. + +You manage HAWQ databases at the command line using the [psql](../reference/cli/client_utilities/psql.html) utility, an interactive front-end to the HAWQ PostgreSQL database. Configuring client access to HAWQ databases and tables may require information related to [Establishing a Database Session](../clientaccess/g-establishing-a-database-session.html). + +[HAWQ Database Drivers and APIs](../clientaccess/g-database-application-interfaces.html) identifies supported HAWQ database drivers and APIs for additional client access methods. + +## HAWQ Data + +HAWQ internal data resides in HDFS. You may require access to data in different formats and locations in your data lake. You can use HAWQ to query both internal and this external data: + +- [Managing Data with HAWQ](../datamgmt/dml.html)
[jira] [Commented] (HAWQ-1209) docs - create an overview page for HAWQ
[ https://issues.apache.org/jira/browse/HAWQ-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802964#comment-15802964 ] ASF GitHub Bot commented on HAWQ-1209: -- Github user dyozie commented on a diff in the pull request: https://github.com/apache/incubator-hawq-docs/pull/80#discussion_r94879281 --- Diff: admin/RunningHAWQ.html.md.erb --- @@ -2,21 +2,36 @@ title: Running a HAWQ Cluster --- -This section provides information for system administrators and database superusers responsible for administering a HAWQ system. - -This guide provides information and instructions for configuring, maintaining and using a HAWQ system. This guide is intended for system and database administrators responsible for managing a HAWQ system. - -This guide assumes knowledge of Linux/UNIX system administration, database management systems, database administration, and structured query language \(SQL\). Because HAWQ is based on PostgreSQL 8.2.15, this guide assumes some familiarity with PostgreSQL. This guide calls out similarities between HAWQ and PostgreSQL features throughout. It contains the topics: - -* Managing HAWQ Using Ambari -* Starting and Stopping HAWQ -* Expanding a Cluster -* Removing a Node -* Backing Up and Restoring HAWQ -* High Availability in HAWQ -* Master Mirroring -* HAWQ Filespaces and High Availability Enabled HDFS -* Understanding the Fault Tolerance Service -* Recommended Monitoring and Maintenance Tasks -* Routine System Maintenance Tasks -* Monitoring a HAWQ System +Your HAWQ deployment environment is a collection of systems, processes, and data, and the users that operate on these systems, processes, and data. This section provides information for system administrators and database superusers responsible for administering a HAWQ deployment. + +You should have some knowledge of Linux/UNIX system administration, database management systems, database administration, and structured query language \(SQL\) to administer a HAWQ cluster. Because HAWQ is based on PostgreSQL 8.2.15, you should also have some familiarity with PostgreSQL. The HAWQ documentation calls out similarities between HAWQ and PostgreSQL features throughout. + +## HAWQ Users + +HAWQ supports users with both administrative and operating privileges. The HAWQ administrator may choose to manage the HAWQ cluster using either Ambari or the command line. [Managing HAWQ Using Ambari](../admin/ambari-admin.html) provides Ambari-specific HAWQ cluster administration procedures. [Starting and Stopping HAWQ](startstop.html), [Expanding a Cluster](ClusterExpansion.html), and [Removing a Node](ClusterShrink.html) describe specific command-line-managed HAWQ cluster administration procedures. Other topics in this guide are applicable to both Ambari- and command-line-managed HAWQ clusters. + +The default HAWQ admininstrator user is named `gpadmin`. The HAWQ admin may choose to assign administrative and/or operating HAWQ privileges to additional users. Refer to [Configuring Client Authentication](../clientaccess/client_auth.html) and [Managing Roles and Privileges](../clientaccess/roles_privs.html) for additional information on HAWQ user configuration. + +## HAWQ Deployment Systems + +A typical HAWQ deployment will include single HDFS and HAWQ master and standby nodes and multiple HAWQ segment and HDFS data nodes. The HAWQ cluster may also include systems running PXF and other Hadoop services. Refer to [HAWQ Architecture](../overview/HAWQArchitecture.html) and [Select HAWQ Host Machines](../install/select-hosts.html) for information about the different systems in a HAWQ deployment and how they are configured. + + +## HAWQ Databases + +[Creating and Managing Databases](../ddl/ddl-database.html) and [Creating and Managing Tables](../ddl/ddl-table.html) identify specific HAWQ database and table creation and access information. + +You manage HAWQ databases at the command line using the [psql](../reference/cli/client_utilities/psql.html) utility, an interactive front-end to the HAWQ PostgreSQL database. Configuring client access to HAWQ databases and tables may require information related to [Establishing a Database Session](../clientaccess/g-establishing-a-database-session.html). + +[HAWQ Database Drivers and APIs](../clientaccess/g-database-application-interfaces.html) identifies supported HAWQ database drivers and APIs for additional client access methods. + +## HAWQ Data + +HAWQ internal data resides in HDFS. You may require access to data in different formats and locations in your data lake. You can use HAWQ to query both internal and this external data: + --- End diff -- The lead-in sentence to this
[jira] [Commented] (HAWQ-1209) docs - create an overview page for HAWQ
[ https://issues.apache.org/jira/browse/HAWQ-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802954#comment-15802954 ] ASF GitHub Bot commented on HAWQ-1209: -- Github user dyozie commented on a diff in the pull request: https://github.com/apache/incubator-hawq-docs/pull/80#discussion_r94878929 --- Diff: admin/RunningHAWQ.html.md.erb --- @@ -2,21 +2,36 @@ title: Running a HAWQ Cluster --- -This section provides information for system administrators and database superusers responsible for administering a HAWQ system. - -This guide provides information and instructions for configuring, maintaining and using a HAWQ system. This guide is intended for system and database administrators responsible for managing a HAWQ system. - -This guide assumes knowledge of Linux/UNIX system administration, database management systems, database administration, and structured query language \(SQL\). Because HAWQ is based on PostgreSQL 8.2.15, this guide assumes some familiarity with PostgreSQL. This guide calls out similarities between HAWQ and PostgreSQL features throughout. It contains the topics: - -* Managing HAWQ Using Ambari -* Starting and Stopping HAWQ -* Expanding a Cluster -* Removing a Node -* Backing Up and Restoring HAWQ -* High Availability in HAWQ -* Master Mirroring -* HAWQ Filespaces and High Availability Enabled HDFS -* Understanding the Fault Tolerance Service -* Recommended Monitoring and Maintenance Tasks -* Routine System Maintenance Tasks -* Monitoring a HAWQ System +Your HAWQ deployment environment is a collection of systems, processes, and data, and the users that operate on these systems, processes, and data. This section provides information for system administrators and database superusers responsible for administering a HAWQ deployment. + +You should have some knowledge of Linux/UNIX system administration, database management systems, database administration, and structured query language \(SQL\) to administer a HAWQ cluster. Because HAWQ is based on PostgreSQL 8.2.15, you should also have some familiarity with PostgreSQL. The HAWQ documentation calls out similarities between HAWQ and PostgreSQL features throughout. + +## HAWQ Users + +HAWQ supports users with both administrative and operating privileges. The HAWQ administrator may choose to manage the HAWQ cluster using either Ambari or the command line. [Managing HAWQ Using Ambari](../admin/ambari-admin.html) provides Ambari-specific HAWQ cluster administration procedures. [Starting and Stopping HAWQ](startstop.html), [Expanding a Cluster](ClusterExpansion.html), and [Removing a Node](ClusterShrink.html) describe specific command-line-managed HAWQ cluster administration procedures. Other topics in this guide are applicable to both Ambari- and command-line-managed HAWQ clusters. + +The default HAWQ admininstrator user is named `gpadmin`. The HAWQ admin may choose to assign administrative and/or operating HAWQ privileges to additional users. Refer to [Configuring Client Authentication](../clientaccess/client_auth.html) and [Managing Roles and Privileges](../clientaccess/roles_privs.html) for additional information on HAWQ user configuration. + +## HAWQ Deployment Systems + +A typical HAWQ deployment will include single HDFS and HAWQ master and standby nodes and multiple HAWQ segment and HDFS data nodes. The HAWQ cluster may also include systems running PXF and other Hadoop services. Refer to [HAWQ Architecture](../overview/HAWQArchitecture.html) and [Select HAWQ Host Machines](../install/select-hosts.html) for information about the different systems in a HAWQ deployment and how they are configured. + + +## HAWQ Databases + +[Creating and Managing Databases](../ddl/ddl-database.html) and [Creating and Managing Tables](../ddl/ddl-table.html) identify specific HAWQ database and table creation and access information. + +You manage HAWQ databases at the command line using the [psql](../reference/cli/client_utilities/psql.html) utility, an interactive front-end to the HAWQ PostgreSQL database. Configuring client access to HAWQ databases and tables may require information related to [Establishing a Database Session](../clientaccess/g-establishing-a-database-session.html). + --- End diff -- I'd remove the postgresql reference here - it makes it sound like we just use a standard (unmodified) postgresql db. > docs - create an overview page for HAWQ > --- > > Key: HAWQ-1209 > URL: https://issues.apache.org/jira/browse/HAWQ-1209 > Project: Apache HAWQ > Issue Type: Improvement > Components: Documentation >Reporter: Lisa
[jira] [Commented] (HAWQ-1209) docs - create an overview page for HAWQ
[ https://issues.apache.org/jira/browse/HAWQ-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802929#comment-15802929 ] ASF GitHub Bot commented on HAWQ-1209: -- Github user dyozie commented on a diff in the pull request: https://github.com/apache/incubator-hawq-docs/pull/80#discussion_r94877959 --- Diff: admin/RunningHAWQ.html.md.erb --- @@ -2,21 +2,36 @@ title: Running a HAWQ Cluster --- -This section provides information for system administrators and database superusers responsible for administering a HAWQ system. - -This guide provides information and instructions for configuring, maintaining and using a HAWQ system. This guide is intended for system and database administrators responsible for managing a HAWQ system. - -This guide assumes knowledge of Linux/UNIX system administration, database management systems, database administration, and structured query language \(SQL\). Because HAWQ is based on PostgreSQL 8.2.15, this guide assumes some familiarity with PostgreSQL. This guide calls out similarities between HAWQ and PostgreSQL features throughout. It contains the topics: - -* Managing HAWQ Using Ambari -* Starting and Stopping HAWQ -* Expanding a Cluster -* Removing a Node -* Backing Up and Restoring HAWQ -* High Availability in HAWQ -* Master Mirroring -* HAWQ Filespaces and High Availability Enabled HDFS -* Understanding the Fault Tolerance Service -* Recommended Monitoring and Maintenance Tasks -* Routine System Maintenance Tasks -* Monitoring a HAWQ System +Your HAWQ deployment environment is a collection of systems, processes, and data, and the users that operate on these systems, processes, and data. This section provides information for system administrators and database superusers responsible for administering a HAWQ deployment. + +You should have some knowledge of Linux/UNIX system administration, database management systems, database administration, and structured query language \(SQL\) to administer a HAWQ cluster. Because HAWQ is based on PostgreSQL 8.2.15, you should also have some familiarity with PostgreSQL. The HAWQ documentation calls out similarities between HAWQ and PostgreSQL features throughout. + +## HAWQ Users + +HAWQ supports users with both administrative and operating privileges. The HAWQ administrator may choose to manage the HAWQ cluster using either Ambari or the command line. [Managing HAWQ Using Ambari](../admin/ambari-admin.html) provides Ambari-specific HAWQ cluster administration procedures. [Starting and Stopping HAWQ](startstop.html), [Expanding a Cluster](ClusterExpansion.html), and [Removing a Node](ClusterShrink.html) describe specific command-line-managed HAWQ cluster administration procedures. Other topics in this guide are applicable to both Ambari- and command-line-managed HAWQ clusters. + +The default HAWQ admininstrator user is named `gpadmin`. The HAWQ admin may choose to assign administrative and/or operating HAWQ privileges to additional users. Refer to [Configuring Client Authentication](../clientaccess/client_auth.html) and [Managing Roles and Privileges](../clientaccess/roles_privs.html) for additional information on HAWQ user configuration. + +## HAWQ Deployment Systems + +A typical HAWQ deployment will include single HDFS and HAWQ master and standby nodes and multiple HAWQ segment and HDFS data nodes. The HAWQ cluster may also include systems running PXF and other Hadoop services. Refer to [HAWQ Architecture](../overview/HAWQArchitecture.html) and [Select HAWQ Host Machines](../install/select-hosts.html) for information about the different systems in a HAWQ deployment and how they are configured. + + +## HAWQ Databases + +[Creating and Managing Databases](../ddl/ddl-database.html) and [Creating and Managing Tables](../ddl/ddl-table.html) identify specific HAWQ database and table creation and access information. + --- End diff -- Change "identify specific HAWQ database and ..." to "describe HAWQ extensions database and table creation commands." or something similar. > docs - create an overview page for HAWQ > --- > > Key: HAWQ-1209 > URL: https://issues.apache.org/jira/browse/HAWQ-1209 > Project: Apache HAWQ > Issue Type: Improvement > Components: Documentation >Reporter: Lisa Owen >Assignee: David Yozie >Priority: Minor > > create a "landing" topic that ties together the components in the HAWQ > operating environment and the relevant documentation contents/sections. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1209) docs - create an overview page for HAWQ
[ https://issues.apache.org/jira/browse/HAWQ-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802920#comment-15802920 ] ASF GitHub Bot commented on HAWQ-1209: -- Github user dyozie commented on a diff in the pull request: https://github.com/apache/incubator-hawq-docs/pull/80#discussion_r94877686 --- Diff: admin/RunningHAWQ.html.md.erb --- @@ -2,21 +2,36 @@ title: Running a HAWQ Cluster --- -This section provides information for system administrators and database superusers responsible for administering a HAWQ system. - -This guide provides information and instructions for configuring, maintaining and using a HAWQ system. This guide is intended for system and database administrators responsible for managing a HAWQ system. - -This guide assumes knowledge of Linux/UNIX system administration, database management systems, database administration, and structured query language \(SQL\). Because HAWQ is based on PostgreSQL 8.2.15, this guide assumes some familiarity with PostgreSQL. This guide calls out similarities between HAWQ and PostgreSQL features throughout. It contains the topics: - -* Managing HAWQ Using Ambari -* Starting and Stopping HAWQ -* Expanding a Cluster -* Removing a Node -* Backing Up and Restoring HAWQ -* High Availability in HAWQ -* Master Mirroring -* HAWQ Filespaces and High Availability Enabled HDFS -* Understanding the Fault Tolerance Service -* Recommended Monitoring and Maintenance Tasks -* Routine System Maintenance Tasks -* Monitoring a HAWQ System +Your HAWQ deployment environment is a collection of systems, processes, and data, and the users that operate on these systems, processes, and data. This section provides information for system administrators and database superusers responsible for administering a HAWQ deployment. + +You should have some knowledge of Linux/UNIX system administration, database management systems, database administration, and structured query language \(SQL\) to administer a HAWQ cluster. Because HAWQ is based on PostgreSQL 8.2.15, you should also have some familiarity with PostgreSQL. The HAWQ documentation calls out similarities between HAWQ and PostgreSQL features throughout. + +## HAWQ Users + +HAWQ supports users with both administrative and operating privileges. The HAWQ administrator may choose to manage the HAWQ cluster using either Ambari or the command line. [Managing HAWQ Using Ambari](../admin/ambari-admin.html) provides Ambari-specific HAWQ cluster administration procedures. [Starting and Stopping HAWQ](startstop.html), [Expanding a Cluster](ClusterExpansion.html), and [Removing a Node](ClusterShrink.html) describe specific command-line-managed HAWQ cluster administration procedures. Other topics in this guide are applicable to both Ambari- and command-line-managed HAWQ clusters. + +The default HAWQ admininstrator user is named `gpadmin`. The HAWQ admin may choose to assign administrative and/or operating HAWQ privileges to additional users. Refer to [Configuring Client Authentication](../clientaccess/client_auth.html) and [Managing Roles and Privileges](../clientaccess/roles_privs.html) for additional information on HAWQ user configuration. + +## HAWQ Deployment Systems + +A typical HAWQ deployment will include single HDFS and HAWQ master and standby nodes and multiple HAWQ segment and HDFS data nodes. The HAWQ cluster may also include systems running PXF and other Hadoop services. Refer to [HAWQ Architecture](../overview/HAWQArchitecture.html) and [Select HAWQ Host Machines](../install/select-hosts.html) for information about the different systems in a HAWQ deployment and how they are configured. + --- End diff -- Change "will include" to "includes" . (In general try to avoid using 'will' and keep things in the present.) > docs - create an overview page for HAWQ > --- > > Key: HAWQ-1209 > URL: https://issues.apache.org/jira/browse/HAWQ-1209 > Project: Apache HAWQ > Issue Type: Improvement > Components: Documentation >Reporter: Lisa Owen >Assignee: David Yozie >Priority: Minor > > create a "landing" topic that ties together the components in the HAWQ > operating environment and the relevant documentation contents/sections. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1209) docs - create an overview page for HAWQ
[ https://issues.apache.org/jira/browse/HAWQ-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802908#comment-15802908 ] ASF GitHub Bot commented on HAWQ-1209: -- Github user dyozie commented on a diff in the pull request: https://github.com/apache/incubator-hawq-docs/pull/80#discussion_r94877232 --- Diff: admin/RunningHAWQ.html.md.erb --- @@ -2,21 +2,36 @@ title: Running a HAWQ Cluster --- -This section provides information for system administrators and database superusers responsible for administering a HAWQ system. - -This guide provides information and instructions for configuring, maintaining and using a HAWQ system. This guide is intended for system and database administrators responsible for managing a HAWQ system. - -This guide assumes knowledge of Linux/UNIX system administration, database management systems, database administration, and structured query language \(SQL\). Because HAWQ is based on PostgreSQL 8.2.15, this guide assumes some familiarity with PostgreSQL. This guide calls out similarities between HAWQ and PostgreSQL features throughout. It contains the topics: - -* Managing HAWQ Using Ambari -* Starting and Stopping HAWQ -* Expanding a Cluster -* Removing a Node -* Backing Up and Restoring HAWQ -* High Availability in HAWQ -* Master Mirroring -* HAWQ Filespaces and High Availability Enabled HDFS -* Understanding the Fault Tolerance Service -* Recommended Monitoring and Maintenance Tasks -* Routine System Maintenance Tasks -* Monitoring a HAWQ System +Your HAWQ deployment environment is a collection of systems, processes, and data, and the users that operate on these systems, processes, and data. This section provides information for system administrators and database superusers responsible for administering a HAWQ deployment. + +You should have some knowledge of Linux/UNIX system administration, database management systems, database administration, and structured query language \(SQL\) to administer a HAWQ cluster. Because HAWQ is based on PostgreSQL 8.2.15, you should also have some familiarity with PostgreSQL. The HAWQ documentation calls out similarities between HAWQ and PostgreSQL features throughout. + +## HAWQ Users + +HAWQ supports users with both administrative and operating privileges. The HAWQ administrator may choose to manage the HAWQ cluster using either Ambari or the command line. [Managing HAWQ Using Ambari](../admin/ambari-admin.html) provides Ambari-specific HAWQ cluster administration procedures. [Starting and Stopping HAWQ](startstop.html), [Expanding a Cluster](ClusterExpansion.html), and [Removing a Node](ClusterShrink.html) describe specific command-line-managed HAWQ cluster administration procedures. Other topics in this guide are applicable to both Ambari- and command-line-managed HAWQ clusters. + +The default HAWQ admininstrator user is named `gpadmin`. The HAWQ admin may choose to assign administrative and/or operating HAWQ privileges to additional users. Refer to [Configuring Client Authentication](../clientaccess/client_auth.html) and [Managing Roles and Privileges](../clientaccess/roles_privs.html) for additional information on HAWQ user configuration. + --- End diff -- Change "on" to "about" or just lose "on HAWQ user configuration" entirely. > docs - create an overview page for HAWQ > --- > > Key: HAWQ-1209 > URL: https://issues.apache.org/jira/browse/HAWQ-1209 > Project: Apache HAWQ > Issue Type: Improvement > Components: Documentation >Reporter: Lisa Owen >Assignee: David Yozie >Priority: Minor > > create a "landing" topic that ties together the components in the HAWQ > operating environment and the relevant documentation contents/sections. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1209) docs - create an overview page for HAWQ
[ https://issues.apache.org/jira/browse/HAWQ-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802900#comment-15802900 ] ASF GitHub Bot commented on HAWQ-1209: -- Github user dyozie commented on a diff in the pull request: https://github.com/apache/incubator-hawq-docs/pull/80#discussion_r94876821 --- Diff: admin/RunningHAWQ.html.md.erb --- @@ -2,21 +2,36 @@ title: Running a HAWQ Cluster --- -This section provides information for system administrators and database superusers responsible for administering a HAWQ system. - -This guide provides information and instructions for configuring, maintaining and using a HAWQ system. This guide is intended for system and database administrators responsible for managing a HAWQ system. - -This guide assumes knowledge of Linux/UNIX system administration, database management systems, database administration, and structured query language \(SQL\). Because HAWQ is based on PostgreSQL 8.2.15, this guide assumes some familiarity with PostgreSQL. This guide calls out similarities between HAWQ and PostgreSQL features throughout. It contains the topics: - -* Managing HAWQ Using Ambari -* Starting and Stopping HAWQ -* Expanding a Cluster -* Removing a Node -* Backing Up and Restoring HAWQ -* High Availability in HAWQ -* Master Mirroring -* HAWQ Filespaces and High Availability Enabled HDFS -* Understanding the Fault Tolerance Service -* Recommended Monitoring and Maintenance Tasks -* Routine System Maintenance Tasks -* Monitoring a HAWQ System +Your HAWQ deployment environment is a collection of systems, processes, and data, and the users that operate on these systems, processes, and data. This section provides information for system administrators and database superusers responsible for administering a HAWQ deployment. + --- End diff -- I think the lead sentence could use some cleanup - Your HAWQ deployment environment is a collection of systems, processes, data, and the users that operate on those systems, processes, and data. Also, I know the original text mentioned database superusers but that seems a bit weird to call them out. I think we could just stick with "system administrators" here. > docs - create an overview page for HAWQ > --- > > Key: HAWQ-1209 > URL: https://issues.apache.org/jira/browse/HAWQ-1209 > Project: Apache HAWQ > Issue Type: Improvement > Components: Documentation >Reporter: Lisa Owen >Assignee: David Yozie >Priority: Minor > > create a "landing" topic that ties together the components in the HAWQ > operating environment and the relevant documentation contents/sections. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #1074: HAWQ-1249. Don't do ACL checks on segment...
GitHub user wcl14 opened a pull request: https://github.com/apache/incubator-hawq/pull/1074 HAWQ-1249. Don't do ACL checks on segments You can merge this pull request into a Git repository by running: $ git pull https://github.com/wcl14/incubator-hawq HAWQ-1249 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/1074.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1074 commit cb4c6ca871d46d859b82531b7fd8541ca9cd4693 Author: Chunling WangDate: 2017-01-05T10:49:59Z HAWQ-1249. Don't do ACL checks on segments --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1071: HAWQ-1242. hawq-site.xml default content ...
Github user jiny2 closed the pull request at: https://github.com/apache/incubator-hawq/pull/1071 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1071: HAWQ-1242. hawq-site.xml default content has wro...
Github user jiny2 commented on the issue: https://github.com/apache/incubator-hawq/pull/1071 In fact some gucs are not used by reading from hawq-site.xml, for example, hawq_master_directory. But for standby_addr_host, it is useful for RM as RM switches its heart-beat to standby server when master server is not achievable. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Closed] (HAWQ-1242) hawq-site.xml default content has wrong guc variable names
[ https://issues.apache.org/jira/browse/HAWQ-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Jin closed HAWQ-1242. > hawq-site.xml default content has wrong guc variable names > -- > > Key: HAWQ-1242 > URL: https://issues.apache.org/jira/browse/HAWQ-1242 > Project: Apache HAWQ > Issue Type: Bug > Components: Core >Reporter: Yi Jin >Assignee: Yi Jin > Fix For: 2.2.0.0-incubating > > > Reported by Paul Guo > I happened to check the log and found some confusing logs. > Have not looked into this yet. Anyone sawq this before? > 2016-12-02 09:28:05.942583 > GMT,,,p366111,th4549860160,,,seg-1,"LOG","0","NOTE: > Recognized configuration: > hawq_dfs_url=localhost:8020/hawq_default""processXMLNode","kvproperties.c",134, > .. > 2016-12-02 09:28:05.948051 > GMT,,,p366111,th4549860160,,,seg-1,"LOG","42704","unrecognized > configuration parameter > ""hawq_standby_address_host""""set_config_option","guc.c",10041, > 2016-12-02 09:28:05.948203 > GMT,,,p366111,th4549860160,,,seg-1,"LOG","42704","unrecognized > configuration parameter > ""hawq_dfs_url""""set_config_option","guc.c",10041, > 2016-12-02 09:28:05.948349 > GMT,,,p366111,th4549860160,,,seg-1,"LOG","42704","unrecognized > configuration parameter > ""hawq_master_directory""""set_config_option","guc.c",10041, > 2016-12-02 09:28:05.948493 > GMT,,,p366111,th4549860160,,,seg-1,"LOG","42704","unrecognized > configuration parameter > ""hawq_segment_directory""""set_config_option","guc.c",10041, -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HAWQ-1242) hawq-site.xml default content has wrong guc variable names
[ https://issues.apache.org/jira/browse/HAWQ-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Jin resolved HAWQ-1242. -- Resolution: Fixed Fix Version/s: 2.2.0.0-incubating > hawq-site.xml default content has wrong guc variable names > -- > > Key: HAWQ-1242 > URL: https://issues.apache.org/jira/browse/HAWQ-1242 > Project: Apache HAWQ > Issue Type: Bug > Components: Core >Reporter: Yi Jin >Assignee: Yi Jin > Fix For: 2.2.0.0-incubating > > > Reported by Paul Guo > I happened to check the log and found some confusing logs. > Have not looked into this yet. Anyone sawq this before? > 2016-12-02 09:28:05.942583 > GMT,,,p366111,th4549860160,,,seg-1,"LOG","0","NOTE: > Recognized configuration: > hawq_dfs_url=localhost:8020/hawq_default""processXMLNode","kvproperties.c",134, > .. > 2016-12-02 09:28:05.948051 > GMT,,,p366111,th4549860160,,,seg-1,"LOG","42704","unrecognized > configuration parameter > ""hawq_standby_address_host""""set_config_option","guc.c",10041, > 2016-12-02 09:28:05.948203 > GMT,,,p366111,th4549860160,,,seg-1,"LOG","42704","unrecognized > configuration parameter > ""hawq_dfs_url""""set_config_option","guc.c",10041, > 2016-12-02 09:28:05.948349 > GMT,,,p366111,th4549860160,,,seg-1,"LOG","42704","unrecognized > configuration parameter > ""hawq_master_directory""""set_config_option","guc.c",10041, > 2016-12-02 09:28:05.948493 > GMT,,,p366111,th4549860160,,,seg-1,"LOG","42704","unrecognized > configuration parameter > ""hawq_segment_directory""""set_config_option","guc.c",10041, -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-1258) segment resource manager does not switch back when it cannot resolve standby host name
[ https://issues.apache.org/jira/browse/HAWQ-1258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Jin reassigned HAWQ-1258: Assignee: Yi Jin (was: Ed Espino) > segment resource manager does not switch back when it cannot resolve standby > host name > -- > > Key: HAWQ-1258 > URL: https://issues.apache.org/jira/browse/HAWQ-1258 > Project: Apache HAWQ > Issue Type: Bug > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > Fix For: 2.2.0.0-incubating > > > When segment resource manager finds master resource manager is not available, > it should switch to standby server, however, when the standby server cannot > be resolved, it does not switch back to original master server side. And it > also should avoid switching to standby if there is no standby which is set as > value 'none'. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-1258) segment resource manager does not switch back when it cannot resolve standby host name
Yi Jin created HAWQ-1258: Summary: segment resource manager does not switch back when it cannot resolve standby host name Key: HAWQ-1258 URL: https://issues.apache.org/jira/browse/HAWQ-1258 Project: Apache HAWQ Issue Type: Bug Components: Resource Manager Reporter: Yi Jin Assignee: Ed Espino Fix For: 2.2.0.0-incubating When segment resource manager finds master resource manager is not available, it should switch to standby server, however, when the standby server cannot be resolved, it does not switch back to original master server side. And it also should avoid switching to standby if there is no standby which is set as value 'none'. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #1073: HAWQ-1248. Merge Dockerfiles for HAWQ Dev into H...
Github user paul-guo- commented on the issue: https://github.com/apache/incubator-hawq/pull/1073 Not an issue of License. Hawq code has included libhdfs3 already. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1073: HAWQ-1248. Merge Dockerfiles for HAWQ Dev into H...
Github user guofengrichard commented on the issue: https://github.com/apache/incubator-hawq/pull/1073 Installation of libhdfs3 has been removed for CentOS 6. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1073: HAWQ-1248. Merge Dockerfiles for HAWQ Dev into H...
Github user stanlyxiang commented on the issue: https://github.com/apache/incubator-hawq/pull/1073 Richard, other improvements could be made in the future, but at least remove the installation of libhdfs3 in the dockerfile. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---