rawlinp commented on a change in pull request #4037: Docs/influx fix URL: https://github.com/apache/trafficcontrol/pull/4037#discussion_r340199081
########## File path: docs/source/admin/traffic_ops.rst ########## @@ -0,0 +1,708 @@ +.. +.. +.. Licensed under the Apache License, Version 2.0 (the "License"); +.. you may not use this file except in compliance with the License. +.. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +.. See the License for the specific language governing permissions and +.. limitations under the License. +.. + +.. role:: bash(code) + :language: bash + +*********** +Traffic Ops +*********** +Traffic Ops is quite possible the single most complex and most important Traffic Control component. It has many different configuration options that affect a wide range of other components and their interactions. + +.. _to-install: + +Installing +========== + +System Requirements +------------------- +The user must have the following for a successful minimal install: + +- CentOS 7+ +- Two machines - physical or virtual -, each with at least two (v)CPUs, 4GB of RAM, and 20 GB of disk space +- Access to CentOS Base and EPEL :manpage:`yum(8)` repositories +- Access to `The Comprehensive Perl Archive Network (CPAN) <http://www.cpan.org/>`_ + +Guide +----- +#. Install PostgreSQL Database. For a production install it is best to install PostgreSQL on its own server/virtual machine. + + .. seealso:: For more information on installing PostgreSQL, see `their documentation <https://www.postgresql.org/docs/>`_. + + .. code-block:: shell + :caption: Example PostgreSQL Install Procedure + + yum update -y + yum install -y https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/pgdg-centos96-9.6-3.noarch.rpm + yum install -y postgresql96-server + su - postgres -c '/usr/pgsql-9.6/bin/initdb -A md5 -W' #-W forces the user to provide a superuser (postgres) password + + +#. Edit :file:`/var/lib/pgsql/9.6/data/pg_hba.conf` to allow the Traffic Ops instance to access the PostgreSQL server. For example, if the IP address of the machine to be used as the Traffic Ops host is ``192.0.2.1`` add the line ``host all all 192.0.2.1/32 md5`` to the appropriate section of this file. + +#. Edit the :file:`/var/lib/pgsql/9.6/data/postgresql.conf` file to add the appropriate listen_addresses or ``listen_addresses = '*'``, set ``timezone = 'UTC'``, and start the database + + .. code-block:: shell + :caption: Starting PostgreSQL with :manpage:`systemd(1)` + + systemctl enable postgresql-9.6 + systemctl start postgresql-9.6 + systemctl status postgresql-9.6 # Prints the status of the PostgreSQL service, to prove it's running + + +#. Build a :file:`traffic_ops-{version string}.rpm` file using the instructions under the :ref:`dev-building` page - or download a pre-built release from `the Apache Continuous Integration server <https://builds.apache.org/view/S-Z/view/TrafficControl/>`_. + +#. Install a PostgreSQL client on the Traffic Ops host + + .. code-block:: shell + :caption: Installing PostgreSQL Client from a Hosted Source + + yum install -y https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/pgdg-centos96-9.6-3.noarch.rpm + +#. Install the Traffic Ops RPM. The Traffic Ops RPM file should have been built in an earlier step. + + .. code-block:: shell + :caption: Installing a Generated Traffic Ops RPM + + yum install -y ./dist/traffic_ops-3.0.0-xxxx.yyyyyyy.el7.x86_64.rpm + + .. note:: This will install the PostgreSQL client, ``psql`` as a dependency. + +#. Login to the Database from the Traffic Ops machine. At this point you should be able to login from the Traffic Ops (hostname ``to`` in the example) host to the PostgreSQL (hostname ``pg`` in the example) host + + .. code-block:: psql + :caption: Example Login to Traffic Ops Database from Traffic Ops Server + + to-# psql -h pg -U postgres + Password for user postgres: + psql (9.6.3) + Type "help" for help. + + postgres=# + + +#. Create the user and database. By default, Traffic Ops will expect to connect as the ``traffic_ops`` user to the ``traffic_ops`` database. + + .. code-block:: console + :caption: Creating the Traffic Ops User and Database + + to-# psql -U postgres -h pg -c "CREATE USER traffic_ops WITH ENCRYPTED PASSWORD 'tcr0cks';" + Password for user postgres: + CREATE ROLE + to-# createdb traffic_ops --owner traffic_ops -U postgres -h pg + Password: + to-# + +#. Now, run the following command as the root user (or with :manpage:`sudo(8)`): :file:`/opt/traffic_ops/install/bin/postinstall`. The :program:`postinstall` script will first get all required Perl packages from :abbr:`CPAN (The Comprehensive Perl Archive Network)`. This may take a while, expect up to 30 minutes on the first install. If there are any prompts in this phase, please just answer with the defaults (some :abbr:`CPAN (The Comprehensive Perl Archive Network)` installs can prompt for install questions). When this phase is complete, you will see ``Complete! Modules were installed into /opt/traffic_ops/app/local``. Some additional files will be installed, and then it will proceed with the next phase of the install, where it will ask you about the local environment for your CDN. Please make sure you remember all your answers and verify that the database answers match the information previously used to create the database. + + .. code-block:: console + :caption: Example Output + + to-# /opt/traffic_ops/install/bin/postinstall + ... + + ===========/opt/traffic_ops/app/conf/production/database.conf=========== + Database type [Pg]: + Database type: Pg + Database name [traffic_ops]: + Database name: traffic_ops + Database server hostname IP or FQDN [localhost]: pg + Database server hostname IP or FQDN: pg + Database port number [5432]: + Database port number: 5432 + Traffic Ops database user [traffic_ops]: + Traffic Ops database user: traffic_ops + Password for Traffic Ops database user: + Re-Enter Password for Traffic Ops database user: + Writing json to /opt/traffic_ops/app/conf/production/database.conf + Database configuration has been saved + ===========/opt/traffic_ops/app/db/dbconf.yml=========== + Database server root (admin) user [postgres]: + Database server root (admin) user: postgres + Password for database server admin: + Re-Enter Password for database server admin: + Download Maxmind Database? [yes]: + Download Maxmind Database?: yes + ===========/opt/traffic_ops/app/conf/cdn.conf=========== + Generate a new secret? [yes]: + Generate a new secret?: yes + Number of secrets to keep? [10]: + Number of secrets to keep?: 10 + Not setting up ldap + ===========/opt/traffic_ops/install/data/json/users.json=========== + Administration username for Traffic Ops [admin]: + Administration username for Traffic Ops: admin + Password for the admin user: + Re-Enter Password for the admin user: + Writing json to /opt/traffic_ops/install/data/json/users.json + ===========/opt/traffic_ops/install/data/json/openssl_configuration.json=========== + Do you want to generate a certificate? [yes]: + Country Name (2 letter code): US + State or Province Name (full name): CO + Locality Name (eg, city): Denver + Organization Name (eg, company): Super CDN, Inc + Organizational Unit Name (eg, section): + Common Name (eg, your name or your server's hostname): + RSA Passphrase: + Re-Enter RSA Passphrase: + ===========/opt/traffic_ops/install/data/json/profiles.json=========== + Traffic Ops url [https://localhost]: + Traffic Ops url: https://localhost + Human-readable CDN Name. (No whitespace, please) [kabletown_cdn]: blue_cdn + Human-readable CDN Name. (No whitespace, please): blue_cdn + DNS sub-domain for which your CDN is authoritative [cdn1.kabletown.net]: blue-cdn.supercdn.net + DNS sub-domain for which your CDN is authoritative: blue-cdn.supercdn.net + Writing json to /opt/traffic_ops/install/data/json/profiles.json + Downloading Maxmind data + --2017-06-11 15:32:41-- http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz + Resolving geolite.maxmind.com (geolite.maxmind.com)... 2400:cb00:2048:1::6810:262f, 2400:cb00:2048:1::6810:252f, 104.16.38.47, ... + Connecting to geolite.maxmind.com (geolite.maxmind.com)|2400:cb00:2048:1::6810:262f|:80... connected. + + ... much SQL output skipped + + Starting Traffic Ops + Restarting traffic_ops (via systemctl): [ OK ] + Waiting for Traffic Ops to restart + Success! Postinstall complete. + + + + .. table:: Explanation of the information that needs to be provided: + + +----------------------------------------------------+------------------------------------------------------------------------------------------------+ + | Field | Description | + +====================================================+================================================================================================+ + | Database type | This requests the type of database to be used. Answer with the default - 'Pg' to indicate a | + | | PostgreSQL database. | + +----------------------------------------------------+------------------------------------------------------------------------------------------------+ + | Database name | The name of the database Traffic Ops uses to store the configuration information. | + +----------------------------------------------------+------------------------------------------------------------------------------------------------+ + | Database server hostname IP or FQDN | The hostname of the database server (``pg`` in the example). | + +----------------------------------------------------+------------------------------------------------------------------------------------------------+ + | Database port number | The database port number. The default value, 5432, should be correct unless you changed it | + | | during the setup. | + +----------------------------------------------------+------------------------------------------------------------------------------------------------+ + | Traffic Ops database user | The username Traffic Ops will use to read/write from the database. | + +----------------------------------------------------+------------------------------------------------------------------------------------------------+ + | Password for Traffic Ops | The password for the database user that Traffic Ops uses. | + +----------------------------------------------------+------------------------------------------------------------------------------------------------+ + | Database server root (admin) user name | Privileged database user that has permission to create the database and user for Traffic Ops...| + +----------------------------------------------------+------------------------------------------------------------------------------------------------+ + | Database server root (admin) user password | The password for the privileged database user. | + +----------------------------------------------------+------------------------------------------------------------------------------------------------+ + | Traffic Ops URL | The URL to connect to this instance of Traffic Ops, usually :samp:`https://{Traffic Ops host}` | + +----------------------------------------------------+------------------------------------------------------------------------------------------------+ + | Human-readable CDN Name | The name of the first CDN which Traffic Ops will be manage. | + +----------------------------------------------------+------------------------------------------------------------------------------------------------+ + | DNS sub-domain for which your CDN is authoritative | The DNS domain that will be delegated to this Traffic Control CDN. | + +----------------------------------------------------+------------------------------------------------------------------------------------------------+ + | Administration username for Traffic Ops | The Administration (highest privilege) Traffic Ops user to create. Use this user to login | + | | for the first time and create other users. | + +----------------------------------------------------+------------------------------------------------------------------------------------------------+ + | Password for the admin user | The password for the administrative Traffic Ops user. | + +----------------------------------------------------+------------------------------------------------------------------------------------------------+ + +Upgrading +========= +To upgrade from older Traffic Ops versions, stop the service, use :manpage:`yum(8)` to upgrade to the latest available Traffic Ops package, and use the :ref:`admin <database-management>` tool to perform the database upgrade. + +.. tip:: In order to upgrade to the latest version of Traffic Ops, please be sure that you have first upgraded to the latest available minor or patch version of your current release. For example, if your current Traffic Ops version is 3.0.0 and version 3.1.0 is available, you must first upgrade to 3.1.0 before proceeding to upgrade to 4.0.0. (Specifically, this means running all migrations, :file:`traffic_ops/app/db/seeds.sql`, and :file:`traffic_ops/app/db/patches.sql` for the latest of your current major version - which should be handled by the :program:`admin` tool). The latest migration available before the release of 4.0.0 (pending at the time of this writing) was :file:`traffic_ops/app/db/migrations/20180814000625_remove_capabilities_for_reseed.sql`, so be sure that migrations up to this point have been run before attempting to upgrade Traffic Ops. + +.. seealso:: :ref:`database-management` for more details about :program:`admin`. + +.. code-block:: shell + :caption: Sample Script for Upgrading Traffic Ops + + systemctl stop traffic_ops + yum upgrade traffic_ops + pushd /opt/traffic_ops/app/ + ./db/admin --env production upgrade + popd + +After this completes, see Guide_ for instructions on running the :program:`postinstall` script. Once the :program:`postinstall` script, has finished, run the following command as the root user (or with :manpage:`sudo(8)`): ``systemctl start traffic_ops`` to start the service. + +Running +======= +Currently, Traffic Ops consists of two programs, as it is in the middle of a transition from one code-base to another. It is not recommended that either one be run on its own. Also, while this section contains instructions for running each piece manually, the only truly supported method is via :manpage:`systemd(8)`, e.g. :bash:`systemctl start traffic_ops` (this method also starts both programs properly and uses their default configuration file locations). + +.. program:: traffic_ops-perl + +Legacy Perl Script +------------------ +The old code base was written in Perl, and is executed as a Perl script. When Installing_ Traffic Ops, the `Hypnotoad <https://mojolicious.org/perldoc/Mojo/Server/Hypnotoad>`_ program should be installed at :file:`/opt/traffic_ops/app/local/bin/hypnotoad`. To run Traffic Ops: + +#. Change directories to :file:`/opt/traffic_ops/app/` - it may not seem like it, but this step is important. *Do* **not** *skip it.* +#. Set the ``PERL5LIB`` environment variable to **exactly** ``/opt/traffic_ops_extensions/private/lib:/opt/traffic_ops/app/lib:/opt/traffic_ops/app/lib/perl5:$PERL5LIB``. That last portion is especially important, to preserve the paths to system libraries - it should expand to whatever ``PERL5LIB`` was prior to this change. +#. Run the command :bash:`/opt/traffic_ops/app/local/bin/hypnotoad script/cdn` + +The script takes no options other than the ones accepted by `Hypnotoad <https://mojolicious.org/perldoc/Mojo/Server/Hypnotoad>`_ itself, and it just expects to gets its configuration from the standard file locations (covered in Configuring_). + +.. envvar:: MOJO_MODE + + This sets the "mode" of the Traffic Ops Mojolicious application. Effectively, this chooses the set of configuration files it will consult. The default value is "development", and the possible values are: + + - development + - integration + - production + - test + +.. program:: traffic_ops + +traffic_ops_golang +------------------ +``traffic_ops_golang [--version] [--plugins] --cfg CONFIG_PATH --dbcfg DB_CONFIG_PATH --riakcfg TRAFFIC_VAULT_CONFIG_PATH`` + +.. option:: --cfg CONFIG_PATH + + This **mandatory** command line flag specifies the absolute or relative path to the configuration file to be used by Traffic Ops - `cdn.conf`_. + +.. option:: --dbcfg DB_CONFIG_PATH + + This **mandatory** command line flag specifies the absolute or relative path to a configuration file used by Traffic Ops to establish connections to the PostgreSQL database - `database.conf`_ + +.. option:: --plugins + + List the installed plugins and exit. + + .. note:: This only accounts for the plugins for the Go version, extensions to the `Legacy Perl Script`_ are not accounted for. + +.. option:: --riakcfg TRAFFIC_VAULT_CONFIG_PATH + + This **mandatory** command line flag specifies the absolute or relative path to a configuration file used by Traffic Ops to establish connections to Traffic Vault - `riak.conf`_ + + .. impl-detail:: The name of this flag is derived from the current database used in the implementation of Traffic Vault - `Riak KV <https://riak.com/products/riak-kv/index.html>`_. + +.. option:: --version + + Print version information and exit. + +Configuring +=========== +The main :program:`traffic_ops_golang` binary and the `Legacy Perl Script`_ use the same set of configuration files (mostly). + +Configuration Files +------------------- + +cdn.conf +"""""""" +This file deals with the configuration parameters of running Traffic Ops itself. It is a JSON-format set of options and their respective values. For the `Legacy Perl Script`_ to work with this file, it must be in its default location at :file:`/opt/traffic_ops/app/conf/cdn.conf`, but `traffic_ops_golang`_ will use whatever file is specified by its :option:`--cfg` option. The keys of the file are described below. + +:geniso: This object contains configuration options for system ISO generation. + + :iso_root_path: Sets the filesystem path to the root of the ISO generation directory. For default installations, this should usually be set to :file:`/opt/traffic_ops/app/public`. + + .. seealso:: :ref:`tp-tools-generate-iso` + +:hypnotoad: This is a group of options used nearly exclusively by the `Legacy Perl Script`_ (as the name implies). + + .. seealso:: For a complete description of this section of the configuration, consult `The Mojolicious Cookbook's section on Hypnotoad <https://mojolicious.org/perldoc/Mojolicious/Guides/Cookbook#Hypnotoad>`_. + + :group: Sets the UNIX group as which the `Legacy Perl Script`_ will run. By default this should be the ``trafops`` group which is created when the Traffic Ops RPM is installed. + :heartbeat_timeout: From the Hypnotoad documentation: "The maximum amount of time in seconds before a worker without a heartbeat will be stopped gracefully". + :listen: This must always be an array containing a single string. This very odd string is apparently a URL. The scheme of the URL specifies the protocol Traffic Ops will use to listen for incoming connections (`traffic_ops_golang`_ ignores that and always uses HTTPS). The host of the URL tells Traffic Ops on what hostname to listen (`traffic_ops_golang`_ understands this as the hostname of the `Legacy Perl Script`_ server), and likewise the port sets the port on which it listens (the port is only used for the `Legacy Perl Script`_ - `traffic_ops_golang`_ uses the setting of ``traffic_ops_golang.port``). The "cert" query parameter sets the location of the SSL certificate to use for encrypting connections, while the "key" query parameter is the certificate's corresponding private key. The default configuration file also has the "verify" query parameter which sets the TLS verification mode - `0` for no verification of certificates, `1` to verify them (`traffic_ops_golang`_ ignores this and uses ``traffic_ops_golang.insecure`` instead). Finally, a list of TLS ciphers to offer to incoming clients of the `Legacy Perl Script`_ delimited by commas can be given in the "ciphers" query parameter. + :pid_file: When the `Legacy Perl Script`_ starts as a daemon, it will write its :abbr:`PID (Process ID)` to this file - so be sure that ``user``/``group`` can properly access it! The default setting is :file:`/var/run/traffic_ops.pid`. + :user: Sets the UNIX user as whom the `Legacy Perl Script`_ will run. By default this should be the ``trafops`` user who is created when the Traffic Ops RPM is installed. + :workers: This is used only by the `Legacy Perl Script`_, and has no known purpose. Though, one could guess that it's the number of concurrent HTTP server workers allowed. Review comment: I believe this is the number of Mojolicious worker processes that will be spawned to handle HTTP requests for TO-Perl. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
