Documentation editorial update This is a first editorial pass through things (grammar, spelling, etc.) as well as putting to page a lot of the style guide suggestions. But it does also include some section reworkings, and more substantial reordering of the step-by-step examples (e.g. the commented-as-clunky Security Options guides).
Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/9fbd4201 Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/9fbd4201 Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/9fbd4201 Branch: refs/heads/master Commit: 9fbd4201766e98435f94eb4ce29b2e99ef64a284 Parents: 39f9ded Author: Jon Sime <js...@omniti.com> Authored: Wed Oct 22 09:24:08 2014 -0700 Committer: James Peach <jpe...@apache.org> Committed: Wed Oct 22 09:24:08 2014 -0700 ---------------------------------------------------------------------- doc/admin/cluster-howto.en.rst | 111 ++-- doc/admin/configuring-cache.en.rst | 213 +++--- doc/admin/configuring-traffic-server.en.rst | 22 +- doc/admin/event-logging-formats.en.rst | 228 +++++-- doc/admin/explicit-proxy-caching.en.rst | 34 +- doc/admin/faqs.en.rst | 187 +++--- doc/admin/forward-proxy.en.rst | 146 +++-- doc/admin/getting-started.en.rst | 155 ++--- doc/admin/hierachical-caching.en.rst | 137 ++-- doc/admin/http-proxy-caching.en.rst | 657 +++++++++++-------- doc/admin/index.en.rst | 114 ++-- doc/admin/monitoring-traffic.en.rst | 54 +- doc/admin/performance-tuning.en.rst | 30 +- doc/admin/reverse-proxy-http-redirects.en.rst | 190 +++--- doc/admin/security-options.en.rst | 268 ++++---- doc/admin/session-protocol.en.rst | 50 +- doc/admin/traffic-server-error-messages.en.rst | 99 +-- doc/admin/transparent-proxy.en.rst | 69 +- doc/admin/working-log-files.en.rst | 574 ++++++++-------- doc/reference/commands/index.en.rst | 20 +- .../configuration/logs_xml.config.en.rst | 6 +- 21 files changed, 1873 insertions(+), 1491 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/trafficserver/blob/9fbd4201/doc/admin/cluster-howto.en.rst ---------------------------------------------------------------------- diff --git a/doc/admin/cluster-howto.en.rst b/doc/admin/cluster-howto.en.rst index 3a3a186..1c00616 100644 --- a/doc/admin/cluster-howto.en.rst +++ b/doc/admin/cluster-howto.en.rst @@ -5,22 +5,20 @@ Traffic Server Cluster .. Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file - distributed with this work for additional information - regarding copyright ownership. The ASF licenses this file - to you under the Apache License, Version 2.0 (the - "License"); you may not use this file except in compliance - with the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - KIND, either express or implied. See the License for the - specific language governing permissions and limitations - under the License. + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. Traffic Server scales from a single node to multiple nodes that form a cluster allowing you to improve system performance and reliability. @@ -74,44 +72,45 @@ If a node fails or is shut down and removed, Traffic Server removes references to the missing node on all nodes in the cluster. Full clustering recommends a dedicated network interface for cluster -communication to get better performance. +communication to achieve better performance. Enabling Clustering Mode ======================== -Before you put a node into a cluster, please make sure the following -things are in order: +Before a node is added into a cluster, please ensure that the following +conditions are being met: -- You are using the same operation system on all nodes: +- All nodes are running the same operating system: - - Using the same distribution, e.g.: RHEL 5.5 - - Have same kernel, e.g.: 2.6.18-194.17.1.el5 - - The same architecture, e.g.: ``x86_64`` + - The same distribution, e.g.: RHEL 5.5 + - The same kernel, e.g.: 2.6.18-194.17.1.el5 + - The same architecture, e.g.: x86_64 -- You have the same version of Traffic Server installed -- The same hardware -- On the same switch or same VLAN. +- All nodes have the same version of Traffic Server installed +- All nodes are composed of the same hardware +- All nodes are on the same switch or same VLAN. Traffic Server does not apply the clustering mode change to all the nodes in the cluster. You must change the clustering mode on each node individually. You may following these instructions: -1. Setup the same cluster name, with :ts:cv:`proxy.config.proxy_name`, e.g. MyCluster. +1. Setup the same cluster name, with :ts:cv:`proxy.config.proxy_name`, e.g. ``MyCluster``. -2. Set :ts:cv:`proxy.local.cluster.type` to ``1``, to enable cluster mode. The - following values of this configuration are valid +2. Set :ts:cv:`proxy.local.cluster.type` to ``1`` to enable cluster mode. The + following values of this configuration are valid: ================= ================================================== Value Description ================= ================================================== -1 full-clustering mode -2 management-only mode -3 no clustering (*default*) +1 Full-Clustering mode +2 Management-Only mode +3 No clustering (*default*) ================= ================================================== -3. Setup a :ts:cv:`proxy.config.cluster.ethernet_interface`, e.g.: ``eth0``. - This should be replaced by your real interface; we recommends a - dedicated interface here. Refer to :ts:cv:`proxy.local.cluster.type` for a full description. +3. Configure :ts:cv:`proxy.config.cluster.ethernet_interface`, e.g.: ``eth0``. + This should be replaced with the node's real interface. We recommends a + dedicated physical interface here. Refer to :ts:cv:`proxy.local.cluster.type` + for a full description. 4. Enable configuration changes:: @@ -125,10 +124,13 @@ Value Description restart after the change of :ts:cv:`proxy.local.cluster.type` and :ts:cv:`proxy.config.cluster.ethernet_interface` have taken place. -Traffic Server will join the cluster in about 10 seconds, and you can -run :option:`traffic_line -r` `proxy.process.cluster.nodes` to check the hosts -in the cluster, or check out the ``cluster.config`` in the configuration -directory. This configuration is generated by the system, and should not be +Traffic Server will join the cluster in about 10 seconds. To verify the hosts in the +cluster, you can run:: + + traffic_line -r proxy.process.cluster.nodes + +Cluster node status is also tracked in ``cluster.config`` in the configuration +directory. This configuration is generated by the system and should not be edited. It contains a list of the machines that are currently members of the cluster, for example:: @@ -137,34 +139,34 @@ cluster, for example:: 127.1.2.4:80 127.1.2.5:80 -After successfully joining of a cluster, all changes of global -configurations on any node, will take effect on **all** nodes. This means you -can make changes on any cluster node member, and they are automatically -distributed to all members. +After successfully joining a cluster, all changes of global configurations +performed on any node in that cluster will take effect on **all** nodes, removing +the need to manually duplicate configuration changes across each node individually. Deleting Nodes from a Cluster ============================= -To delete a node from the Traffic Server cluster, just roll back -:ts:cv:`proxy.local.cluster.type` to the default value 3 and reload. +To delete a node from the Traffic Server cluster, return the setting +:ts:cv:`proxy.local.cluster.type` to the default value ``3`` and reload. Common issues for Cluster setup =============================== -1. The Cluster member auto discovery is build upon from multi-casting of the UDP, - so imposible to setup where multi-casting is not avaliable, such as AWS EC2. +1. The Cluster member auto discovery is built upon multi-casting UDP, and as such + is impossible to setup where multi-casting is not avaliable, such as AWS EC2. -2. The Cluster will depend on some internal ports: 8088 8089 and 8086, you should - make sure the firewall will not ban them. +2. The Cluster will depend on ports 8088, 8089, and 8086. These ports must not be + blocked by any network configurations or firewalls on the network used for + internal cluster communication. -Performance tweak for busy Cluster -================================== +Performance Tuning for Busy Clusters +==================================== -Starting from v3.2.0, Apache Traffic Server can handle multiple internal -cluster connections, and we can tweak the number of Cluster threads. Each -of the thread will keep one connection to all of peering cluster machines. +Beginning with version 3.2.0, Apache Traffic Server can handle multiple internal +cluster connections and the number of Cluster Threads is configurable. Each +of the threads will keep one connection open to all peering cluster nodes. -Increasing Cluster threads +Increasing Cluster Threads -------------------------- In the cluster environment, the current performance of the cluster threads @@ -172,6 +174,7 @@ will consume the same cpu usage as a normal network thread. It's reasonable to keep roughly the same number of cluster threads as network threads. For example, if you are running a system with 10 network processing threads, you can set the number of cluster threads by modifying -:ts:cv:`proxy.config.cluster.threads` to ``10``. E.g.:: +:ts:cv:`proxy.config.cluster.threads` to ``10``:: traffic_line -s proxy.config.cluster.threads -v 10 + http://git-wip-us.apache.org/repos/asf/trafficserver/blob/9fbd4201/doc/admin/configuring-cache.en.rst ---------------------------------------------------------------------- diff --git a/doc/admin/configuring-cache.en.rst b/doc/admin/configuring-cache.en.rst index 4df5bc0..dc009d2 100644 --- a/doc/admin/configuring-cache.en.rst +++ b/doc/admin/configuring-cache.en.rst @@ -9,7 +9,7 @@ Configuring the Cache regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance - with the License. You may obtain a copy of the License at + with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 @@ -21,7 +21,7 @@ Configuring the Cache under the License. The Traffic Server cache consists of a high-speed object database called -the **object store** that indexes objects according to URLs and their +the *object store* that indexes objects according to URLs and their associated headers. .. toctree:: @@ -31,7 +31,7 @@ The Traffic Server Cache ======================== The Traffic Server cache consists of a high-speed object database called -the **object store**. The object store indexes objects according to URLs +the *object store*. The object store indexes objects according to URLs and associated headers. This enables Traffic Server to store, retrieve, and serve not only web pages, but also parts of web pages - which provides optimum bandwidth savings. Using sophisticated object @@ -53,11 +53,11 @@ You can perform the following cache configuration tasks: - Change the total amount of disk space allocated to the cache: refer to `Changing Cache Capacity`_. - Partition the cache by reserving cache disk space for specific - protocols and origin servers/domains: refer to `Partitioning the Cache`_. -- Delete all data in the cache: refer to `Clearing the Cache`_. + protocols and origin servers/domains; refer to `Partitioning the Cache`_. +- Delete all data in the cache; refer to `Clearing the Cache`_. - Override cache directives for a requested domain name, regex on a url, - hostname or ip, with extra filters for time, port, method of the request - (and more). ATS can be configured to never cache; always cache; + hostname or ip, with extra filters for time, port, method of the request, + and more. ATS can be configured to never cache, always cache, ignore no-cache directives, etc. These are configured in :file:`cache.config`. The RAM Cache @@ -69,28 +69,37 @@ and reduces load on disks, especially during temporary traffic peaks. You can configure the RAM cache size to suit your needs, as described in :ref:`changing-the-size-of-the-ram-cache` below. -The RAM cache supports two cache eviction algorithms, a regular **LRU** -(*Least Recently Used*) and the more advanced **CLFUS** (*Clocked Least -Frequently Used by Size*, which balances recentness, frequency and size -to maximize hit rate -- similar to a most frequently used algorithm). -The default is to use **CLFUS**, and this is controlled via +The RAM cache supports two cache eviction algorithms, a regular *LRU* +(Least Recently Used) and the more advanced *CLFUS* (Clocked Least +Frequently Used by Size; which balances recentness, frequency, and size +to maximize hit rate, similar to a most frequently used algorithm). +The default is to use *CLFUS*, and this is controlled via :ts:cv:`proxy.config.cache.ram_cache.algorithm`. -Both the **LRU** and **CLFUS** RAM caches support a configuration to increase -scan resistance. In a typical **LRU**, if you request all possible objects in +Both the *LRU* and *CLFUS* RAM caches support a configuration to increase +scan resistance. In a typical *LRU*, if you request all possible objects in sequence, you will effectively churn the cache on every request. The option :ts:cv:`proxy.config.cache.ram_cache.use_seen_filter` can be set to add some resistance against this problem. -In addition, **CLFUS** also supports compressing in the RAM cache itself. +In addition, *CLFUS* also supports compressing in the RAM cache itself. This can be useful for content which is not compressed by itself (e.g. -images). This should not be confused with *Content-Encoding: gzip*, this +images). This should not be confused with ``Content-Encoding: gzip``, this feature is only thereto save space internally in the RAM cache itself. As such, it is completely transparent to the User-Agent. The RAM cache compression is enabled with the option -:ts:cv:`proxy.config.cache.ram_cache.compress`. The default is 0, which means -no compression. Other possible values are 1 for **fastlz**, 2 for **libz** and -3 for **liblzma**. +:ts:cv:`proxy.config.cache.ram_cache.compress`. + +Possible values are: + +======= ============================= +Value Meaning +======= ============================= +0 No compression (*default*) +1 *fastlz* compression +2 *libz* compression +3 *liblzma* compression +======= ============================= .. _changing-the-size-of-the-ram-cache: @@ -114,17 +123,15 @@ its previous value. To change the RAM cache size: -1. Stop Traffic Server. -2. Set the variable :ts:cv:`proxy.config.cache.ram_cache.size` - to specify the size of the RAM cache. The default value of -1 means +#. Stop Traffic Server. +#. Set the variable :ts:cv:`proxy.config.cache.ram_cache.size` + to specify the size of the RAM cache. The default value of ``-1`` means that the RAM cache is automatically sized at approximately 1MB per gigabyte of disk. -3. Restart Traffic Server. If you increase the RAM cache to a size of +#. Restart Traffic Server. If you increase the RAM cache to a size of 1GB or more, then restart with the :program:`trafficserver` command (refer to :ref:`start-traffic-server`). -Â - Changing Cache Capacity ======================= @@ -136,27 +143,25 @@ Increasing Cache Capacity ------------------------- To increase the total amount of disk space allocated to the cache on -existing disks or to add new disks to a Traffic Server node, follow the -steps below: +existing disks, or to add new disks to a Traffic Server node: -1. Stop Traffic Server. -2. Add hardware, if necessary. -3. Edit :file:`storage.config` to increase the amount of disk space allocated +#. Stop Traffic Server. +#. Add hardware, if necessary. +#. Edit :file:`storage.config` to increase the amount of disk space allocated to the cache on existing disks or describe the new hardware you are adding. -4. Restart Traffic Server. +#. Restart Traffic Server. Reducing Cache Capacity ----------------------- To reduce the total amount of disk space allocated to the cache on an -existing disk or to remove disks from a Traffic Server node, follow the -steps below: +existing disk, or to remove disks from a Traffic Server node: -1. Stop Traffic Server. -2. Remove hardware, if necessary. -3. Edit :file:`storage.config` to reduce the amount of disk space allocated +#. Stop Traffic Server. +#. Remove hardware, if necessary. +#. Edit :file:`storage.config` to reduce the amount of disk space allocated to the cache on existing disks or delete the reference to the hardware you're removing. -4. Restart Traffic Server. +#. Restart Traffic Server. .. important:: In :file:`storage.config`, a formatted or raw disk must be at least 128 MB. @@ -177,15 +182,15 @@ Creating Cache Partitions for Specific Protocols You can create separate volumes for your cache that vary in size to store content according to protocol. This ensures that a certain amount of disk space is always available for a particular protocol. Traffic -Server currently supports the **http** partition type for HTTP objects. +Server currently supports the ``http`` partition type for HTTP objects. .. XXX: but not https? To partition the cache according to protocol: -1. Enter a line in the :file:`volume.config` file for +#. Enter a line in the :file:`volume.config` file for each volume you want to create -2. Restart Traffic Server. +#. Restart Traffic Server. Making Changes to Partition Sizes and Protocols ----------------------------------------------- @@ -203,17 +208,19 @@ note the following: recreated, even if the size and protocol type remain the same. - When you add new disks to your Traffic Server node, volume sizes specified in percentages will increase proportionately. -- A lot of changes to volume sizes might result in disk fragmentation, - which affects performance and hit rate. You should clear the cache +- Substantial changes to volume sizes can result in disk fragmentation, + which affects performance and cache hit rate. You should clear the cache before making many changes to cache volume sizes (refer to `Clearing the Cache`_). Partitioning the Cache According to Origin Server or Domain ----------------------------------------------------------- +.. XXX: rewrite to remove repetitious single-v-multiple points; break out global partition note for clarify; fix up plurality + After you have partitioned the cache according to size and protocol, you can assign the volumes you created to specific origin servers and/or -domains. You can assign a volumes to a single origin server or to -multiple origin servers. However, if a volumes is assigned to multiple +domains. You can assign a volume to a single origin server or to +multiple origin servers. However, if a volume is assigned to multiple origin servers, then there is no guarantee on the space available in the volumes for each origin server. Content is stored in the volumes according to popularity. In addition to assigning volumes to specific @@ -225,7 +232,7 @@ then Traffic Server will run in proxy-only mode. .. note:: - You do *not* need to stop Traffic Server before you assign + You do not need to stop Traffic Server before you assign volumes to particular hosts or domains. However, this type of configuration is time-consuming and can cause a spike in memory usage. Therefore, it's best to configure partition assignment during periods of @@ -233,20 +240,20 @@ then Traffic Server will run in proxy-only mode. To partition the cache according to hostname and domain: -1. Configure the cache volumes according to size and protocol, as +#. Configure the cache volumes according to size and protocol, as described in `Creating Cache Partitions for Specific Protocols`_. -2. Create a separate volume based on protocol for each host and domain, +#. Create a separate volume based on protocol for each host and domain, as well as an additional generic partition to use for content that does not belong to these origin servers or domains. The volumes do not need to be the same size. -3. Enter a line in the :file:`hosting.config` file to - allocate the volume(s) used for each origin server and/or domain -4. Assign a generic volume to use for content that does not belong to +#. Enter a line in the :file:`hosting.config` file to + allocate the volume(s) used for each origin server and/or domain. +#. Assign a generic volume to use for content that does not belong to any of the origin servers or domains listed in the file. If all volumes for a particular origin server become corrupt, then Traffic Server will also use the generic volume to store content for that origin server as per :file:`hosting.config`. -5. Run the command :option:`traffic_line -x` to apply the configuration +#. Run the command :option:`traffic_line -x` to apply the configuration changes. Configuring the Cache Object Size Limit @@ -256,10 +263,10 @@ By default, Traffic Server allows objects of any size to be cached. You can change the default behavior and specify a size limit for objects in the cache via the steps below: -1. Set :ts:cv:`proxy.config.cache.max_doc_size` - to specify the maximum size allowed for objects in the cache in - bytes. ``0`` (zero) if you do not want a size limit. -2. Run the command :option:`traffic_line -x` to apply the configuration +#. Set :ts:cv:`proxy.config.cache.max_doc_size` + to specify the maximum size in bytes allowed for objects in the cache. + A setting of ``0`` (zero) will permit cache objects to be unlimited in size. +#. Run the command :option:`traffic_line -x` to apply the configuration changes. .. _clearing-the-cache: @@ -269,13 +276,13 @@ Clearing the Cache When you clear the cache, you remove all data from the entire cache - including data in the host database. You should clear the cache before -performing certain cache configuration tasks, such as partitioning. You +performing certain cache configuration tasks such as partitioning. You cannot clear the cache when Traffic Server is running. To clear the cache: -1. Stop Traffic Server (refer to :ref:`Stopping Traffic Server <stop-traffic-server>`) -2. Enter the following command to clear the cache: :: +#. Stop Traffic Server (see :ref:`stop-traffic-server`) +#. Enter the following command to clear the cache:: traffic_server -Cclear @@ -283,7 +290,7 @@ To clear the cache: host database. Traffic Server does not prompt you to confirm the deletion. -3. Restart Traffic Server (refer to :ref:`Starting Traffic Server <start-traffic-server>`). +#. Restart Traffic Server (see :ref:`start-traffic-server`). Removing an Object From the Cache ================================= @@ -294,6 +301,11 @@ cache and is successfully removed, then Traffic Server responds with a ``200 OK`` HTTP message; otherwise, a ``404 File Not Found`` message is returned. +.. note:: + + By default, the PURGE request method is only processed if received on + the localhost interface. + In the following example, Traffic Server is running on the domain ``example.com`` and you want to remove the image ``remove_me.jpg`` from cache. Because by default we do not permit ``PURGE`` requests from @@ -314,12 +326,12 @@ any other IP, we connect to the daemon via localhost: :: < Connection: keep-alive The next time Traffic Server receives a request for the removed object, -it will contact the origin server to retrieve it (i.e., it has been -purged from the Traffic Server cache). +it will contact the origin server to retrieve a new copy, which will replace +the previously cached version in Traffic Server. -Note: The procedure above only removes an object from a *specific* -Traffic Server cache. Users may still see the old (removed) content if -it was cached by intermediary caches or by the end-users' web browser. +This procedure only removes an object from a specific Traffic Server cache. +Users may still see the old (removed) content if it was cached by intermediary +caches or by the end-users' web browser. .. _inspecting-the-cache: @@ -328,14 +340,16 @@ Inspecting the Cache Traffic Server provides a Cache Inspector utility that enables you to view, delete, and invalidate URLs in the cache (HTTP only). The Cache -Inspector utility is a powerful tool that's capable of deleting *all* -the objects in your cache; therefore, make sure that only authorized -administrators are allowed to access this utility, see :ref:`controlling-client-access-to-cache` and the ``@src_ip`` option in :file:`remap.config`. +Inspector utility is a powerful tool that's capable of deleting all +the objects in your cache. Therefore, make sure that only authorized +administrators are allowed to access this utility through proper use +of the ``@src_ip`` option in :file:`remap.config` and the instructions +detailed in :ref:`controlling-client-access-to-cache`. Accessing the Cache Inspector Utility ------------------------------------- -To access the Cache Inspector utility, follow the steps below: +To access the Cache Inspector utility: #. Set :ts:cv:`proxy.config.http_ui_enabled` to ``1``. #. To access the cache inspector in reverse proxy mode, you must add a @@ -345,45 +359,46 @@ To access the Cache Inspector utility, follow the steps below: map http://yourhost.com/myCI/ http://{cache} @action=allow @src_ip=172.28.56.1-172.28.56.254 -#. From the Traffic Server ``bin`` directory, enter the following - command to re-read the configuration file: ``traffic_line -x`` +#. Reload the Traffic Server configuration by running :option:`traffic_line -x`. #. Open your web browser and configure it to use your Traffic Server as a proxy server. Type the following URL:: http://yourhost/myCI/ -#. The Cache page opens. - -Using the Cache Page --------------------- - -The **Cache page** provides several options that enable you to view and -delete the contents of your cache: - -- Click **Lookup url** to search for a particular URL in the cache. - When Traffic Server finds the URL in the cache, it displays details - about the object that corresponds to the URL (such as the header - length and the number of alternates). From the display page, you can - delete the URL from the cache. -- Click **Delete url** to delete a particular URL or list of URLs from - the cache. Traffic Server indicates if a delete is successful. -- Click **Regex lookup** to search for URLs that match one or more - regular expressions. From the display page, you can delete the URLs - listed. For example, enter the following to search for all URLs that - end in html and are prefixed with ``http://www.dianes.com``: - ``http://www.dianes.com/.*\.html$`` -- Click **Regex delete** to delete all URLs that match a specified - regular expression. For example, enter the following to delete all - HTTP URLs that end in ``html``: ``http://.*\.html$`` -- Click **Regex invalidate** to invalidate URLs that match a specified - regular expression. When you invalidate a URL, Traffic Server marks - the object that corresponds to the URL as stale in the cache. Traffic - Server then contacts the origin server to check if the object is - still fresh (revalidates) before serving it from the cache. + You will now be presented with the Cache Inspector interface. + +Using the Cache Inspector Utility +--------------------------------- + +The Cache Inspector Utility provides several options that enable you to view and +delete the contents of your cache. + +Lookup URL + Search for a particular URL in the cache. When Traffic Server finds the URL + in the cache, it will display details of the object that corresponds to the + URL (e.g. header length and number of alternates). The option to delete the + URL from the cache will be presented. + +Delete URL + Delete the object from the cache which corresponds to the given URL. Success + or failure will be indicated after a delete has been attempted. + +Regex Lookup + Search URLs within the cache using one or more regular expressions. + +Regex Delete + Deletes all objects from the cache which match the provided regular + expressions. + +Regex Invalidate + Marks any objects in the cache which match the given regular expressions as + stale. Traffic Server will contact the relevant origin server(s) to confirm + the validity and freshness of the cached object, updating the cached object + if necessary. .. note:: Only one administrator should delete and invalidate cache - entries from the Cache page at any point in time. Changes made by + entries from the Cache Inspector at any point in time. Changes made by multiple administrators at the same time can lead to unpredictable results. http://git-wip-us.apache.org/repos/asf/trafficserver/blob/9fbd4201/doc/admin/configuring-traffic-server.en.rst ---------------------------------------------------------------------- diff --git a/doc/admin/configuring-traffic-server.en.rst b/doc/admin/configuring-traffic-server.en.rst index 2fe0443..0e23a7c 100644 --- a/doc/admin/configuring-traffic-server.en.rst +++ b/doc/admin/configuring-traffic-server.en.rst @@ -1,4 +1,3 @@ - .. _configuring-traffic-server: Configuring Traffic Server @@ -41,7 +40,7 @@ To view a configuration setting, enter the following command:: traffic_line -r var -where ``var`` is the variable associated with the configuration +where *var* is the variable associated with the configuration option. For a list of variables, refer to :ref:`configuration-variables`. Change Configuration Options in Traffic Line @@ -52,8 +51,8 @@ command:: traffic_line -s var -v value -where ``var`` is the variable associated with the configuration option -and ``value`` is the value you want to use. For a list of the +where *var* is the variable associated with the configuration option +and *value* is the value you want to use. For a list of the variables, see :ref:`configuration-variables`. Configure Traffic Server Using Configuration Files @@ -61,14 +60,13 @@ Configure Traffic Server Using Configuration Files As an alternative to using Traffic Line or Traffic Shell, you can change Traffic Server configuration options by manually editing specific -variables in the :file:`records.config` file. -After modifying the :file:`records.config` file, -Traffic Server must reread the configuration files: enter the Traffic -Line command :option:`traffic_line -x`. You may need to restart Traffic Server -to apply some of the configuration changes. +variables in :file:`records.config`. + +Traffic Server must reread the configuration files for any changes to take effect. +This is done with :option:`traffic_line -x`. Some configuration changes require a +full restart of Traffic Server. -The following is a sample portion of the -:file:`records.config` file: +The following is a sample portion of :file:`records.config`: .. figure:: ../static/images/admin/records.jpg :align: center @@ -76,7 +74,7 @@ The following is a sample portion of the Sample records.config file -In addition to the :file:`records.config` file, +In addition to :file:`records.config`, Traffic Server provides other configuration files that are used to configure specific features. You can manually edit all configuration files as described in :ref:`configuration-file-reference`. http://git-wip-us.apache.org/repos/asf/trafficserver/blob/9fbd4201/doc/admin/event-logging-formats.en.rst ---------------------------------------------------------------------- diff --git a/doc/admin/event-logging-formats.en.rst b/doc/admin/event-logging-formats.en.rst index b81a4e7..50c5b0b 100644 --- a/doc/admin/event-logging-formats.en.rst +++ b/doc/admin/event-logging-formats.en.rst @@ -5,20 +5,20 @@ Event Logging Formats .. Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file - distributed with this work for additional information - regarding copyright ownership. The ASF licenses this file - to you under the Apache License, Version 2.0 (the - "License"); you may not use this file except in compliance - with the License. You may obtain a copy of the License at - + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - KIND, either express or implied. See the License for the - specific language governing permissions and limitations - under the License. + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. This document provides a reference for all the different logging formats Traffic Server supports. @@ -37,103 +37,146 @@ Custom Logging Fields The following list describes Traffic Server custom logging fields. +.. _cqh: + ``{HTTP header field name}cqh`` Logs the information in the requested field of the client request HTTP header. For example, ``%<{Accept-Language}cqh>`` logs the ``Accept-Language:`` field in client request headers. +.. _pqh: + ``{HTTP header field name}pqh`` Logs the information in the requested field of the proxy request HTTP header. For example, ``%<{Authorization}pqh>`` logs the ``Authorization:`` field in proxy request headers. +.. _psh: + ``{HTTP header field name}psh`` Logs the information in the requested field of the proxy response HTTP header. For example, ``%<{Retry-After}psh>`` logs the ``Retry-After:`` field in proxy response headers. +.. _ssh: + ``{HTTP header field name}ssh`` Logs the information in the requested field of the server response HTTP header. For example, ``%<{Age}ssh>`` logs the ``Age:`` field in server response headers. +.. _caun: + ``caun`` The client authenticated username; result of the RFC931/ident lookup of the client username. +.. _cfsc: + ``cfsc`` The client finish status code; specifies whether the client request to Traffic Server was successfully completed (``FIN``) or interrupted (``INTR``). +.. _chi: + ``chi`` The IP address of the client's host machine. +.. _chih: + ``chih`` The IP address of the client's host machine in hexadecimal. +.. _chp: + ``chp`` The port number of the client's host machine. +.. _cps: + ``cps`` Client Protocol Stack, the output would be the conjunction of protocol names in the stack spliced with '+', such as "TLS+SPDY". +.. _cqbl: + ``cqbl`` The client request transfer length; the body length in the client request to Traffic Server (in bytes). +.. _cqhl: + ``cqhl`` The client request header length; the header length in the client request to Traffic Server. +.. _cqhm: + ``cqhm`` The HTTP method in the client request to Traffic Server: ``GET``, ``POST``, and so on (subset of ``cqtx``). +.. _cqhv: + ``cqhv`` The client request HTTP version. +.. _cqtd: + ``cqtd`` The client request timestamp. Specifies the date of the client request in the format yyyy-mm-dd, where yyyy is the 4-digit year, mm is the 2-digit month, and dd is the 2-digit day. +.. _cqtn: + ``cqtn`` The client request timestamp; date and time of the client's request (in the Netscape timestamp format). +.. _cqtq: + ``cqtq`` The client request timestamp, with millisecond resolution. +.. _cqts: + ``cqts`` The client-request timestamp in Squid format; the time of the client request since January 1, 1970 UTC. Time is expressed in seconds, with millisecond resolution. +.. _cqtt: + ``cqtt`` The client request timestamp. The time of the client request in the format hh:mm:ss, where hh is the two-digit hour in 24-hour format, mm is the two-digit minutes value, and ss is the 2-digit seconds value (for example, 16:01:19). +.. _cqtx: + ``cqtx`` The full HTTP client request text, minus headers; for example, :: GET http://www.company.com HTTP/1.0 In reverse proxy mode, Traffic Server logs the rewritten/mapped URL - (according to the rules in the - :file:`remap.config` file), _not_ the pristine/unmapped URL. + (according to the rules in :file:`remap.config`), _not_ the + pristine/unmapped URL. + +.. _cqu: ``cqu`` The universal resource identifier (URI) of the request from client to Traffic Server (subset of ``cqtx`` ). In reverse proxy mode, Traffic Server logs the rewritten/mapped URL - (according to the rules in the - :file:`remap.config` file), - _not_ the pristine/unmapped URL. + (according to the rules in :file:`remap.config`), _not_ the + pristine/unmapped URL. + +.. _cquc: ``cquc`` The client request canonical URL. This differs from ``cqu`` in that @@ -144,6 +187,8 @@ The following list describes Traffic Server custom logging fields. See `cquuc`_. +.. _cqup: + ``cqup`` The client request URL path; specifies the argument portion of the URL (everything after the host). For example, if the URL is @@ -152,6 +197,8 @@ The following list describes Traffic Server custom logging fields. See `cquup`_. +.. _cqus: + ``cqus`` The client request URL scheme. @@ -173,50 +220,76 @@ The following list describes Traffic Server custom logging fields. The client request unmapped URL host. This field records a URL's host before it is remapped (reverse proxy mode). +.. _crat: + ``crat`` The Retry-After time in seconds, if specified by the origin server. +.. _crc: + ``crc`` The cache result code; specifies how the cache responded to the request (``HIT``, ``MISS``, and so on). +.. _csscl: + ``csscl`` The cached response length (in bytes) from origin server to Traffic Server. +.. _csshl: + ``csshl`` The cached header length in the origin server response to Traffic Server (in bytes). +.. _csshv: + ``csshv`` The cached server response HTTP version (1.0, 1.1, etc.). +.. _csssc: + ``csssc`` The cached HTTP response status code from origin server to Traffic Server. +.. _cwr: + ``cwr`` The cache write result (``-``, ``WL_MISS``, ``INTR```, ``ERR`` or ``FIN``) +.. _cwtr: + ``cwtr`` The cache write transform result +.. _fsiz: + ``fsiz`` The size of the file (*n* bytes) as seen by the origin server. +.. _pfsc: + ``pfsc`` The proxy finish status code; specifies whether the Traffic Server request to the origin server was successfully completed (``FIN``), interrupted (``INTR``) or timed out (``TIMEOUT``). +.. _phn: + ``phn`` The hostname of the Traffic Server that generated the log entry in collated log files. +.. _phi: + ``phi`` The IP of the Traffic Server that generated the log entry in collated log files. +.. _phr: + ``phr`` The proxy hierarchy route; the route Traffic Server used to retrieve the object. @@ -231,75 +304,110 @@ The following list describes Traffic Server custom logging fields. ``pitag`` The plugin tag for the transaction. This is set for plugin driven transactions via :c:func:`TSHttpConnectWithPluginId`. +.. _pqbl: + ``pqbl`` The proxy request transfer length; the body length in Traffic Server's request to the origin server. +.. _pqhl: + ``pqhl`` The proxy request header length; the header length in Traffic Server's request to the origin server. +.. _pqsi: + ``pqsi`` The proxy request server IP address (0 on cache hits and parent-ip for requests to parent proxies). +.. _pqsn: + ``pqsn`` The proxy request server name; the name of the server that fulfilled the request. +.. _pscl: + ``pscl`` The length of the Traffic Server response to the client (in bytes). +.. _psct: + ``psct`` The content type of the document from server response header: (for example, ``img/gif`` ). +.. _pshl: + ``pshl`` The header length in Traffic Server's response to the client. +.. _psql: + ``psql`` The proxy response transfer length in Squid format (includes header and content length). +.. _pssc: + ``pssc`` The HTTP response status code from Traffic Server to the client. +.. _shi: + ``shi`` The IP address resolved from the DNS name lookup of the host in the request. For hosts with multiple IP addresses, this field records the IP address resolved from that particular DNS lookup. This can be misleading for cached documents. For example: if the - first request was a cache miss and came from **``IP1``** for server - **``S``** and the second request for server **``S``** resolved to - **``IP2``** but came from the cache, then the log entry for the - second request will show **``IP2``**. + first request was a cache miss and came from *IP1* for server + *S* and the second request for server *S* resolved to + *IP2* but came from the cache, then the log entry for the + second request will show *IP2*. + +.. _shn: ``shn`` The hostname of the origin server. +.. _sscl: + ``sscl`` The response length (in bytes) from origin server to Traffic Server. +.. _sshl: + ``sshl`` - The header length in the origin server response to Traffic Server - (in bytes). + The header length (in bytes) in the origin server response to Traffic Server. + +.. _sshv: ``sshv`` The server response HTTP version (1.0, 1.1, etc.). +.. _sssc: + ``sssc`` The HTTP response status code from origin server to Traffic Server. +.. _ttms: + ``ttms`` The time Traffic Server spends processing the client request; the number of milliseconds between the time the client establishes the connection with Traffic Server and the time Traffic Server sends the last byte of the response back to the client. +.. _ttmsh: + ``ttmsh`` Same as ``ttms`` but in hexadecimal. +.. _ttmsf: + ``ttmsf`` The time Traffic Server spends processing the client request as a fractional number of seconds. Time is specified in millisecond @@ -311,6 +419,8 @@ The following list describes Traffic Server custom logging fields. displays 1.5 while the ``ttms`` field displays 1500 and the ``tts`` field displays 1. +.. _tts: + ``tts`` The time Traffic Server spends processing the client request; the number of seconds between the time at which the client establishes @@ -333,20 +443,20 @@ Squid Logging Formats The following is a list of the Squid logging fields and the corresponding logging field symbols. -================== ============= -Squid Field Symbols -================== ============= -``time`` ``cqts`` -``elapsed`` ``ttms`` -``client`` ``chi`` -``action/code`` ``crc/pssc`` -``size`` ``psql`` -``method`` ``cqhm`` -``url`` ``cquc`` -``ident`` ``caun`` -``hierarchy/from`` ``phr/pqsn`` -``content`` ``psct`` -================== ============= +============== ============= +Squid Field Symbols +============== ============= +time `cqts`_ +elapsed `ttms`_ +client `chi`_ +action/code `crc`_/`pssc`_ +size `psql`_ +method `cqhm`_ +url `cquc`_ +ident `caun`_ +hierarchy/from `phr`_/`pqsn`_ +content `psct`_ +============== ============= Netscape Common Logging Formats ------------------------------- @@ -357,12 +467,12 @@ corresponding Traffic Server logging field symbols. =============== ============= Netscape Common Field Symbols =============== ============= -``host`` ``chi`` -``usr`` ``caun`` -``[time]`` ``[cqtn]`` -``"req"`` ``"cqtx"`` -``s1`` ``pssc`` -``c1`` ``pscl`` +host `chi`_ +usr `caun`_ +[time] [`cqtn`_] +"req" "`cqtx`_" +s1 `pssc`_ +c1 `pscl`_ =============== ============= Netscape Extended Logging Formats @@ -374,21 +484,21 @@ corresponding Traffic Server logging field symbols. ================= ============= Netscape Extended Field Symbols ================= ============= -``host`` ``chi`` -``usr`` ``caun`` -``[time]`` ``[cqtn]`` -``"req"`` ``"cqtx"`` -``s1`` ``pssc`` -``c1`` ``pscl`` -``s2`` ``sssc`` -``c2`` ``sscl`` -``b1`` ``cqbl`` -``b2`` ``pqbl`` -``h1`` ``cqhl`` -``h2`` ``pshl`` -``h3`` ``pqhl`` -``h4`` ``sshl`` -``xt`` ``tts`` +host `chi`_ +usr `caun`_ +[time] [`cqtn`_] +"req" "`cqtx`_" +s1 `pssc`_ +c1 `pscl`_ +s2 `sssc`_ +c2 `sscl`_ +b1 `cqbl`_ +b2 `pqbl`_ +h1 `cqhl`_ +h2 `pshl`_ +h3 `pqhl`_ +h4 `sshl`_ +xt `tts`_ ================= ============= Netscape Extended-2 Logging Formats http://git-wip-us.apache.org/repos/asf/trafficserver/blob/9fbd4201/doc/admin/explicit-proxy-caching.en.rst ---------------------------------------------------------------------- diff --git a/doc/admin/explicit-proxy-caching.en.rst b/doc/admin/explicit-proxy-caching.en.rst index 60341b4..86dd503 100644 --- a/doc/admin/explicit-proxy-caching.en.rst +++ b/doc/admin/explicit-proxy-caching.en.rst @@ -5,20 +5,20 @@ Explicit Proxy Caching .. Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file - distributed with this work for additional information - regarding copyright ownership. The ASF licenses this file - to you under the Apache License, Version 2.0 (the - "License"); you may not use this file except in compliance - with the License. You may obtain a copy of the License at - + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - KIND, either express or implied. See the License for the - specific language governing permissions and limitations - under the License. + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. .. toctree:: :maxdepth: 2 @@ -42,9 +42,9 @@ To manually configure a browser to send HTTP requests to Traffic Server, clients must provide the following information: - The fully-qualified hostname or IP address of the Traffic Server node -- The Traffic Server proxy server port (port 8080) +- The Traffic Server proxy server port (by default, 8080) -In addition, clients can specify *not* to use Traffic Server for certain +In addition, clients can specify not to use Traffic Server for certain sites - in such cases, requests to the listed sites go directly to the origin server. The procedures for manual configuration vary among browser versions; refer to specific browser documentation for complete @@ -57,13 +57,13 @@ from manually-configured browsers. Using a PAC File ================ -A **PAC file** is a specialized JavaScript function definition that a +A *PAC file* is a specialized JavaScript function definition that a browser calls to determine how requests are handled. Clients must specify (in their browser settings) the URL from which the PAC file is loaded. You can store a PAC file on Traffic Server (or on any server in your network) and then provide the URL for this file to your clients. -If you want to store a PAC file on the Traffic Server system, then you +If you want to store a PAC file on the Traffic Server system, you must perform the following configuration: - Either copy an existing PAC file into the Traffic Server ``config`` http://git-wip-us.apache.org/repos/asf/trafficserver/blob/9fbd4201/doc/admin/faqs.en.rst ---------------------------------------------------------------------- diff --git a/doc/admin/faqs.en.rst b/doc/admin/faqs.en.rst index cc19c29..d6dd259 100644 --- a/doc/admin/faqs.en.rst +++ b/doc/admin/faqs.en.rst @@ -1,4 +1,4 @@ -.. _admin-faqs: +.. _faqs: FAQ and Troubleshooting Tips **************************** @@ -59,9 +59,9 @@ Can Traffic Server cache Java applets, JavaScript programs, or other application Yes, Traffic Server can store and serve Java applets, JavaScript programs, VBScripts, and other executable objects from its cache according to the freshness and cacheability rules for HTTP objects. -Traffic Server does not execute the applets, scripts, or programs, -however - these objects run only when the client system (ie, the one -that sent the request) loads them. +Traffic Server does not execute the applets, scripts, or programs. +These objects run entirely client-side (the system which originated +the request for the objects), and do not execute on the server. In Squid- and Netscape-format log files, what do the cache result codes mean? ----------------------------------------------------------------------------- @@ -71,9 +71,9 @@ This is described in detail in the :ref:`log-formats-squid-format` documentation What is recorded by the ``cqtx`` field in a custom log file? ------------------------------------------------------------ -- In **forward proxy mode**, the cqtx field records the complete client +- In *forward proxy mode*, the ``cqtx`` field records the complete client request in the log file (for example, ``GET http://www.company.com HTTP/1.0``). -- In **reverse proxy mode**, the cqtx field records the hostname or IP +- In *reverse proxy mode*, the ``cqtx`` field records the hostname or IP address of the origin server because Traffic Server first remaps the request as per map rules in the :file:`remap.config` file. @@ -88,7 +88,7 @@ to compare the ``ttl`` value set by the name server with the ``ttl`` value set by Traffic Server, and then use either the lower or the higher value. -see :ts:cv:`proxy.config.hostdb.ttl_mode` for more info +Refer to :ts:cv:`proxy.config.hostdb.ttl_mode` for more info. Can you improve the look of your custom response pages by using images, animated .gifs, and Java applets? --------------------------------------------------------------------------------------------------------- @@ -105,14 +105,14 @@ Can Traffic Server run in forward proxy and reverse proxy modes at the same time --------------------------------------------------------------------------------- Yes. When you enable reverse proxy mode, Traffic Server remaps incoming -requests according to the map rules in the :file:`remap.config` file. All -other requests that do not match a map rule are simply served in forward +requests according to the map rules in :file:`remap.config`. All +other requests that do not match a map rule are served in forward proxy mode. If you want to run in reverse proxy only mode (wherein Traffic Server -does *not* serve requests that fail to match a map rule), then you must +does not serve requests that fail to match a map rule), then you must set the configuration variable :ts:cv:`proxy.config.url_remap.remap_required` -to ``1`` in the :file:`records.config` file. +to ``1`` in :file:`records.config`. How do I enable forward proxy mode ---------------------------------- @@ -325,24 +325,23 @@ blank no server connection Support for HTTP Expect: Header ------------------------------- -Traffic Server currently does not handle request Expect: headers +Traffic Server currently does not handle Expect: request headers according to the HTTP/1.1 spec. -Note that clients such as cURL automatically send Expect: for POST +Clients such as cURL automatically send Expect: for POST requests with large POST bodies, with a 1 second timeout if a 100 Continue response is not received. To avoid the timeout when using cURL -as a client to Traffic Server, you can turn off the Expect: header as -follows:: +as a client to Traffic Server, you can turn off the Expect: header:: curl -H"Expect:" http://www.example.com/ -C (libcurl):: +Or with the C (libcurl) library from within your own applications:: struct curl_slist *header_list=NULL; header_list = curl_slist_append(header_list, "Expect:"); curl_easy_setopt(my_curlp, CURLOPT_HTTPHEADER, header_list); -php:: +Or with the PHP cURL library:: curl_setopt($ch, CURLOPT_HTTPHEADER, array('Expect:')); @@ -364,28 +363,31 @@ You are unable to execute Traffic Line commands Traffic Line commands do not execute under the following conditions: -- **When the traffic_manager process is not running** Check to see - if the :program:`traffic_manager` process is running by entering the - following command: ``pgrep -l traffic_manager`` +**When the traffic_manager process is not running** + Check to see if the :program:`traffic_manager` process is running by entering the + following command:: - If the :program:`traffic_manager` process is not running, then enter the - following command from the Traffic Server ``bin`` directory to start it: - ``./traffic_manager`` + pgrep -l traffic_manager - .. this is wrong + If the :program:`traffic_manager` process is not running, then enter the + following command from the Traffic Server ``bin`` directory to start it:: - You should always start and stop Traffic Server with the - :program:`trafficserver start`` and :program:`trafficserver stop` commands to ensure - that all the processes start and stop correctly. For more information, - refer to :ref:`getting-started`. + ./traffic_manager -- **When you are not executing the command from $TSHome/bin** If the Traffic Server - ``bin`` directory is not in your path, then prepend the Traffic Line - commands with ``./`` (for example, ``./traffic_line -h``). +.. XXX: this is wrong -- **When multiple Traffic Server installations are present and you are not - executing the Traffic Line command from the active Traffic Server path - specified in ``/etc/trafficserver``** + You should always start and stop Traffic Server with the + :program:`trafficserver start`` and :program:`trafficserver stop` commands to ensure + that all the processes start and stop correctly. For more information, + refer to :ref:`getting-started`. + +**When you are not executing the command from $TSHome/bin** + If the Traffic Server ``bin`` directory is not in your path, then prepend the + Traffic Line commands with ``./`` (for example, ``./traffic_line -h``). + +**When multiple Traffic Server installations are present and you are not +executing the Traffic Line command from the active Traffic Server path +specified in ``/etc/trafficserver``** You observe inconsistent behavior when one node obtains an object from another node in the cluster @@ -397,7 +399,7 @@ problems, but differences of more than a few minutes can affect Traffic Server operation. You should run a clock synchronization daemon such as xntpd. To obtain -the latest version of xntpd, go to ``http://www.eecis.udel.edu/~ntp/`` +the latest version of xntpd, go to `<http://www.eecis.udel.edu/~ntp/>`_. Web browsers display an error document with a 'data missing' message -------------------------------------------------------------------- @@ -411,7 +413,7 @@ A message similar to the following might display in web browsers: :: This is a Web browser issue and not a problem specific to (or caused by) Traffic Server. Because Web browsers maintain a separate local cache in memory and/or disk on the client system, messages about documents that -have expired from cache refer to the browser's local cache and *not* +have expired from cache refer to the browser's local cache and not to the Traffic Server cache. There is no Traffic Server message or condition that can cause such messages to appear in a web browser. @@ -437,8 +439,8 @@ read the name resolution file: then you must add a name server entry for ``127.0.0.1`` or ``0.0.0.0`` in the :manpage:`resolv.conf(5)` file. - Check that the Traffic Server user account has permission to read the - /etc/resolv.conf file. If it does not, then change the file - permissions to ``rw-r--r--`` (``644``) + :manpage:`resolv.conf(5)` file. If it does not, then change the file + permissions to ``rw-r--r--`` (``644``). 'Maximum document size exceeded' message in the system log file --------------------------------------------------------------- @@ -465,11 +467,11 @@ The following messages may appear in the system log file: :: Feb 20 23:53:58 louis traffic_manager[4414]: ERROR ==> [drainIncomingChannel] Unknown message: 'GET http://www.ip.pt/ HTTP/1.0' These error messages indicate that a browser is sending HTTP requests to -one of the Traffic Server cluster ports - either ``rsport`` (default +one of the Traffic Server cluster ports, either ``rsport`` (default port 8088) or ``mcport`` (default port 8089). Traffic Server discards -the request; this error does not cause any Traffic Server problems. The +these requests. This error does not cause any Traffic Server problems. The misconfigured browser must be reconfigured to use the correct proxy -port. Traffic Server clusters work best when configured to use a +port. Traffic Server clusters should ideally be configured to use a separate network interface and cluster on a private subnet, so that client machines have no access to the cluster ports. @@ -487,77 +489,84 @@ starting :program:`traffic_manager` or performing any health checks. The it has been stopped with the option:`trafficserver stop` command. Without this static control, Traffic Server would restart automatically upon system reboot. The ``no_cop`` control keeps Traffic Server off until it -is explicitly restarted with the :: +is explicitly restarted with: :: trafficserver start -command. - - Warning in the system log file when manually editing vaddrs.config ------------------------------------------------------------------ -If you manually edit the vaddrs.config file as a non-root user, then +If you manually edit :file:`vaddrs.config` as a non-root user, then Traffic Server issues a warning message in the system log file similar to the following:: WARNING: interface is ignored: Operation not permitted -You can safely ignore this message; Traffic Server *does* apply your +You can safely ignore this message as Traffic Server will still apply your configuration edits. Traffic Server is running but no log files are created ------------------------------------------------------ Traffic Server only writes event log files when there is information to -record. If Traffic Server is idle, then it's possible/probable that no -log files exist. In addition: - -Make sure you're looking in the correct directory. By default, Traffic -Server creates log files in the logs directory. Check the location of -log files by checking the value of the variable -proxy.config.log.logfile_dir in the records.config file. Check that the -log directory has read/write permissions for the Traffic Server user -account. If the log directory does not have the correct permissions, -then the traffic_server process is unable to open or create log files. -Check that logging is enabled by checking the value of the -proxy.config.log.logging_enabled variable in the records.config file. -Check that a log format is enabled. In the records.config file, select -the standard or custom format by editing variables in the Logging Config -section. +record. If Traffic Server is idle, then it's possible that no log files +exist. + +If Traffic Server is not idle, and you still do not see log files being +generated, verify the following: + +- Make sure you're looking in the correct directory. By default, Traffic + Server creates log files in the ``logs`` directory. This can be modified + by changing :ts:cv:`proxy.config.log.logfile_dir` in :file:`records.config`. + +- Check that the log directory has read/write permissions for the Traffic + Server user account. If the log directory does not have the correct + permissions, then the :program:`traffic_server` process will be unable to + open or create log files. + +- Check that logging is enabled by checking the value of the + :ts:cv:`proxy.config.log.logging_enabled` variable in :file:`records.config`. + +- Check that a log format is enabled. In :file:`records.config`, select + the standard or custom format by editing variables in the Logging Config + section. Traffic Server shows an error indicating too many network connections --------------------------------------------------------------------- -By default, Traffic Server supports 8000 network connections: half of +By default, Traffic Server supports 8000 network connections. Half of this number is allocated for client connections and the remaining half -is for origin server connections. A **connection throttle event** occurs -when client or origin server connections reach 90% of half the -configured limit (3600 by default). When a connection throttle event +is for origin server connections. A *connection throttle event* occurs +when either client or origin server connections reach 90% of half the +configured total limit (3600 by default). When a connection throttle event occurs, Traffic Server continues processing all existing connections but will not accept new client connection requests until the connection count falls below the limit. Connection throttle events can occur under the following conditions: -- If there is a **connection spike** (e.g., if thousands of client - requests all reach Traffic Server at the same time). Such events are - typically transient and require no corrective action. -- If there is a **service overload** (e.g., if client requests - continuously arrive faster than Traffic Server can service them). - Service overloads often indicate network problems between Traffic - Server and origin servers. Conversely, it may indicate that Traffic - Server needs more memory, CPU, cache disks, or other resources to - handle the client load. - -If necessary, you can reset the maximum number of connections supported -by Traffic Server by editing the value of the -:ts:cv:`proxy.config.net.connections_throttle` configuration variable in -the records.config file. Do not increase the connection throttle limit -unless the system has adequate memory to handle the client connections -required. A system with limited RAM might need a throttle limit lower -than the default value. Do not set this variable below the minimum value -of 100. +Connection Spike + Too many client requests (enough to exceed your configured maximum connections) + all reach Traffic Server at the same time. Such events are typically transient + and require no corrective action if your connection limits are already + configured appropriately for your Traffic Server and origin resources. + +Service Overload + Client requests are arriving at a rate faster than that which Traffic + Server can service them. This may indicate network problems between Traffic + Server and origin servers or that Traffic Server may require more memory, CPU, + cache disks, or other resources to handle the client load. + +If necessary, you can adjust the maximum number of connections supported +by Traffic Server by editing :ts:cv:`proxy.config.net.connections_throttle` in +:file:`records.config`. + +.. note:: + + Do not increase the connection throttle limit unless the system has adequate + memory to handle the client connections required. A system with limited RAM + might need a throttle limit lower than the default value. Do not set this + variable below the minimum value of ``100``. Low memory symptoms ------------------- @@ -582,9 +591,9 @@ load on Traffic Server. Connection timeouts with the origin server ------------------------------------------ -Certain origin servers take longer than 30 seconds to post HTTP -requests, which results in connection timeouts with Traffic Server. To -prevent such connection timeouts, you must change the value of the -configuration variable proxy.config.http.connect_attempts_timeout in -the records.config file to 60 seconds or more. +By default, Traffic Server will timeout after 30 seconds when contacting +origin servers. If you cannot avoid such timeouts by otherwise addressing the +performance on your origin servers, you may adjust the origin connection timeout +in Traffic Server by changing :ts:cv:`proxy.config.http.connect_attempts_timeout` +in :file:`records.config` to a larger value. http://git-wip-us.apache.org/repos/asf/trafficserver/blob/9fbd4201/doc/admin/forward-proxy.en.rst ---------------------------------------------------------------------- diff --git a/doc/admin/forward-proxy.en.rst b/doc/admin/forward-proxy.en.rst index 03c3d77..8420b1b 100644 --- a/doc/admin/forward-proxy.en.rst +++ b/doc/admin/forward-proxy.en.rst @@ -5,76 +5,120 @@ Forward Proxy .. Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file - distributed with this work for additional information - regarding copyright ownership. The ASF licenses this file - to you under the Apache License, Version 2.0 (the - "License"); you may not use this file except in compliance - with the License. You may obtain a copy of the License at - + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - KIND, either express or implied. See the License for the - specific language governing permissions and limitations - under the License. -The Apache Traffic Server is a general purpose *proxy*. As such it can -also be used as forward proxy. + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +The Apache Traffic Server is a general purpose proxy, configurable as both a +reverse and forward proxy. + +A forward proxy can be used as a central tool in your infrastructure +to access the web and it may be combined with a cache to lower your overall +bandwidth usage. Forward proxies act as a gatekeeper between client browsers +on your local network and all (or some, at your configuration's discretion) +web sites accessed by those clients. The forward proxy will receive the +HTTP requests, perform any filtering or request alteration rules you +establish, and when appropriate forward the request on to its destination +website. The response will return through your proxy, where it may optionally +be cached and/or modified, and then returned to the original client. + +There are two modes in which your forward proxy may operate: + +Forward Proxy + Each client must be configured explicitly to use the forward proxy. Client + browsers will be aware of the fact they are using a proxy and will form their + HTTP requests appropriately. This results in the initial HTTP command being + issued with fully qualified URIs that contain the destination hostname:: -A forward proxy is can be used as a central tool in your infrastructure -to access the web. In combination with a cache that means overall -reduced bandwidth usage. + GET http://example.com/index.php?id=123 HTTP/1.1 -If your forward proxy is not also configured as :ref:`transparent-proxy` -your clients will have to be configured to actually use it. +Transparent Proxy + The use of a transparent proxy is typically done in concert with network + routing rules which redirect all outbound HTTP traffic through your proxy. + Clients will behave, and form their HTTP requests, as if they are contacting + the remote site directly, and will not be aware of the existence of a proxy + server in between themselves and the remote servers. HTTP requests will be + generated per their usual form, with only paths in the command and a + separate Host request header:: -The main difference between a forward and a transparent proxy is that -User Agents *know* that they are accessing a proxy, thus forming their -requests like so: :: + GET /index?id=123 HTTP/1.1 + Host: example.com - GET http://example.com/index.php?id=1337 HTTP/1.1 +Apache Traffic Server may be configured to operate as both a forward and +a transparent proxy simultaneously. -This request, then is translated by the proxy to:: +Proxy Configuration +=================== - GET /index?id=1337 HTTP/1.1 - Host: example.com +Configuring basic forward proxy operation in Traffic Server is quite simple +and straightforward. -Apache Traffic Server offers two ways to User Agents: They can either be -pointed directly to the default ``8080`` port. Alternatively, they can -be pointed to the more dynamic :ts:cv:`proxy.config.url_remap.default_to_server_pac` +1. Permit Traffic Server to process requests for hosts not explicitly configured + in the remap rules, by modifying :ts:cv:`proxy.config.url_remap.remap_required` + in :file:`records.config`:: -This port will then serve a JavaScript like configuration that User -Agents can use to determine where to send their requests to. + CONFIG proxy.config.url_remap.remap_required INT 0 -Configuration -============= +2. *Optional*: If Traffic Server will be operating strictly as a forward proxy, + you will want to disable reverse proxy support by modifying + :ts:cv:`proxy.config.reverse_proxy.enabled` in :file:`records.config`:: -In order to configure Apache Traffic Server as forward proxy you will -have to edit :file:`records.config` and set + CONFIG proxy.config.reverse_proxy.enabled INT 0 -- :ts:cv:`proxy.config.url_remap.remap_required` to ``0`` +You may also want to consider some of these configuration options: -If your proxy is serving as *pure* forward proxy, you will also want to -set +- Setting :ts:cv:`proxy.config.http.no_dns_just_forward_to_parent` determines which + host will be used for DNS resolution. -- :ts:cv:`proxy.config.reverse_proxy.enabled` to ``0`` +- Proxy Authentication can be enabled or disabled with + :ts:cv:`proxy.config.http.forward.proxy_auth_to_parent` should you also be + employing a proxy cache. -Other configuration variables to consider: +- The client request header X-Forwarded-For may be toggled with + :ts:cv:`proxy.config.http.insert_squid_x_forwarded_for`. -- :ts:cv:`proxy.config.http.no_dns_just_forward_to_parent` -- :ts:cv:`proxy.config.http.forward.proxy_auth_to_parent` -- :ts:cv:`proxy.config.http.insert_squid_x_forwarded_for` +Client Configuration +==================== + +If you are operating your proxy in transparent mode, your clients should require +no special proxy-related configuration. + +If you are operating in explicit forward proxy mode, without automatic routing +rules on your network to direct all outbound traffic through the proxy, your +client browsers will need to be directed to the proxy. This may be accomplished +in two different ways. + +Clients may be configured to use the default ``8080`` port on your Traffic Server +host as a proxy. This will result in all requests from that client browser being +issued through the single forward proxy as configured. + +Alternatively, you may configure clients to use the more dynamic +:ts:cv:`proxy.config.url_remap.default_to_server_pac` port, which will deliver a +Javascript based PAC configuration to the client. This permits you to configure +rules on which proxy servers are used for various types of requests, or to allow +some requests to bypass proxies all together. Security Considerations ======================= -It's important to note that once your Apache Traffic Server is -configured as forward proxy it will indiscriminately accept proxy -requests from anyone. That means, if it's reachable on the internet, you -have configured an *Open Proxy*. Most of the time, this is *not* what -you want, so you'll have to make sure it's either only reachable within -your NAT or is secured by firewall rules that permit only those clients -to access it which you want to it to access. +It's important to note that once your Apache Traffic Server is configured as a +forward proxy it will indiscriminately accept proxy requests from anyone. If it +is reachable from the Internet, then you have configured an *Open Proxy*. + +This is generally not desirable, as it will permit anyone to potentially use +your network as the source of traffic to sites of their choosing. To avoid +this, you'll have to make sure your proxy server is either only reachable from +within your private network or is secured by firewall rules that permit only +those you wish to have access to the proxy. http://git-wip-us.apache.org/repos/asf/trafficserver/blob/9fbd4201/doc/admin/getting-started.en.rst ---------------------------------------------------------------------- diff --git a/doc/admin/getting-started.en.rst b/doc/admin/getting-started.en.rst index bdb68d8..c03c756 100644 --- a/doc/admin/getting-started.en.rst +++ b/doc/admin/getting-started.en.rst @@ -1,4 +1,3 @@ - .. _getting-started: Getting Started @@ -6,46 +5,47 @@ Getting Started .. Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file - distributed with this work for additional information - regarding copyright ownership. The ASF licenses this file - to you under the Apache License, Version 2.0 (the - "License"); you may not use this file except in compliance - with the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - KIND, either express or implied. See the License for the - specific language governing permissions and limitations - under the License. + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. .. toctree:: :maxdepth: 2 -Before you start +Before You Start ================ Before you get started with Traffic Server you may have to decide which version you want to use. Traffic Server follows the `Semantic Versioning -<http://semver.org>`_ guidelines, in summary +<http://semver.org>`_ guidelines. -A version is made of a version-triplet: ``MAJOR.MINOR.PATCH`` +A complete version number is made of a version-triplet: ``MAJOR.MINOR.PATCH``. As of v4.0.0, there are no longer any development (or unstable) releases. -All releases are considered stable and ready for production use, releases +All releases are considered stable and ready for production use. Releases within a major version are always upgrade compatible. More details are -available on the `Wiki page -<https://cwiki.apache.org/confluence/display/TS/New+Release+Processes>`_. - -Sometimes we speak of trunk, master or HEAD, all of which are used -interchangeably: trunk or master or sometimes TIP or HEAD, refer to the -latest code in a Git Version Control System. Master is always kept releasable, -and compatible with the current major release version. Incompatible changes -are sometimes committed on a next-major release branch, for example we have -the ``5.0.x`` branch where changes incompatible with 4.x are managed. +available on the `New Release Processes +<https://cwiki.apache.org/confluence/display/TS/New+Release+Processes>`_ wiki +page. + +Sometimes we speak of *trunk*, *master* or *HEAD*, all of which are used +interchangeably. Trunk and master, or sometimes TIP or HEAD, refer to the +latest code in a Git version control system (also referred to as a *repository* +or *Git repo*). Master is always kept releasable, and compatible with the +current major release version. Incompatible changes are sometimes committed on +a next-major release branch; for example, we have the ``5.0.x`` branch where +changes incompatible with 4.x are managed. If your distribution does not come with a prepackaged Traffic Server, please go to `downloads </downloads>`_ to choose the version that you @@ -53,20 +53,23 @@ consider most appropriate for yourself. If you want to really be on the bleeding edge you can clone our `git repository <https://git-wip-us.apache.org/repos/asf/trafficserver.git>`_. -Please note that while we do have a `GitHub -Mirror <https://github.com/apache/trafficserver>`_ that you can also use -to submit pull requests, it may not be entirely up-to-date. +.. note:: + + We do also have a `GitHub Mirror <https://github.com/apache/trafficserver>`_ + that you may use to submit pull requests. However, it may not be + entirely up-to-date, and you should always refer to our official project + Git repository for the very latest state of the source code. Building Traffic Server ======================= In order to build Traffic Server from source you will need the following -(development) packages: +development tools and libraries installed: - pkgconfig - libtool - gcc (>= 4.3 or clang > 3.0) -- make (GNU Make!) +- GNU make - openssl - tcl - expat @@ -75,64 +78,67 @@ In order to build Traffic Server from source you will need the following - flex (for TPROXY) - hwloc - lua -- curses -- curl (both for :program:`traffic_top`) +- curses (for :program:`traffic_top`) +- curl (for :program:`traffic_top`) -if you're building from a git clone, you'll also need +If you're building from a git clone, you'll also need: - git - autoconf - automake -We will show-case a build from git:: +The following instructions demonstrate building a fresh Traffic Server from +Git sources. - git clone https://git-wip-us.apache.org/repos/asf/trafficserver.git +#. Clone the official Git repository for Traffic Server. :: -Next, we ``cd trafficserver`` and run:: + git clone https://git-wip-us.apache.org/repos/asf/trafficserver.git - autoreconf -if +#. Change your work directory to the newly-cloned local repository and run autoreconf. :: -This will generate a ``configure`` file from ``configure.ac``, so now we -can run that:: + cd trafficserver/ + autoreconf -if - ./configure --prefix=/opt/ats +#. A ``configure`` script will be generated from ``configure.ac`` which may now + be used to configure the source tree for your build. :: -Note well, that by default Traffic Server uses the user ``nobody``, as -well as user's primary group as Traffic Server user. If you want to -change that, you can override it here:: + ./configure --prefix=/opt/ats - ./configure --prefix=/opt/ats --with-user=tserver + By default, Traffic Server will be built to use the ``nobody`` user and group. + You may change this with the ``--with-user`` argument to ``configure``:: -If dependencies are not in standard paths (``/usr/local`` or ``/usr``), -you need to pass options to ``configure`` to account for that:: + ./configure --prefix=/opt/ats --with-user=tserver - ./configure --prefix=/opt/ats --with-user=tserver --with-lua=/opt/csw + If dependencies are not in standard paths (``/usr/local`` or ``/usr``), + you may need to pass options to ``configure`` to account for that:: -Most ``configure`` path-options accept a format of -``"INCLUDE_PATH:LIBRARY_PATH"``:: + ./configure --prefix=/opt/ats --with-lua=/opt/csw - ./configure --prefix=/opt/ats --with-user=tserver --with-lua=/opt/csw \ - --with-pcre=/opt/csw/include:/opt/csw/lib/amd64 + Most ``configure`` path-options accept a format of "*INCLUDE_PATH*:*LIBRARY_PATH*":: -We can run ``make`` to build the project. We highly recommend to run -``make check`` to verify the build's general sanity:: + ./configure --prefix=/opt/ats --with-pcre=/opt/csw/include:/opt/csw/lib/amd64 - make - make check +#. Once the source tree has been configured, you may proceed on to building with + the generated Makefiles. The ``make check`` command may be used to perform + sanity checks on the resulting build, prior to installation, and it is + recommended that you use this. :: -We can finally run ``make install`` to install (you may have to switch -to root to do this):: + make + make check - sudo make install +#. With the source built and checked, you may now install all of the binaries, + header files, documentation, and other artifacts to their final locations on + your system. :: -We also recommend to run a regression test. Please note that this will -only work successfully with the default ``layout``:: + sudo make install - cd /opt/ats - sudo bin/traffic_server -R 1 +#. Finally, it is recommended that you run the regression test suite. Please note + that the regression tests will only be successful with the default layout. :: -After you have installed Traffic Server on your system, you can do any -of the following: + cd /opt/ats + sudo bin/traffic_server -R 1 + +You are now ready to configure and run your Traffic Server installation. .. _start-traffic-server: @@ -140,24 +146,19 @@ Start Traffic Server ==================== To start Traffic Server manually, issue the ``trafficserver`` command, -passing in the attribute ``start``. This command starts all the +passing in the subcommand ``start``. This command starts all the processes that work together to process Traffic Server requests as well -as manage, control, and monitor the health of the Traffic Server system. - -To run the ``trafficserver start`` command, e.g.:: +as manage, control, and monitor the health of the Traffic Server system. :: bin/trafficserver start -At this point your server is up and running in the default configuration -of a :ref:`reverse-proxy-and-http-redirects`. - .. _start-straffic-line: Start Traffic Line ================== Traffic Line provides a quick way of viewing Traffic Server statistics -and configuring the Traffic Server system via command-line interface. To +and configuring the Traffic Server system via a command-line interface. To execute individual commands or script multiple commands, refer to :program:`traffic_line`. @@ -172,7 +173,7 @@ For a list of :program:`traffic_line` commands, enter:: Please note that :program:`traffic_line`, while a fine tool for an administrator, is a poor choice for automation, especially that of monitoring. See our chapter on :ref:`monitoring-traffic` -for how to do that better. +for how to do that more efficiently and effectively. .. _stop-traffic-server: @@ -183,7 +184,7 @@ To stop Traffic Server, always use the :program:`trafficserver` command, passing in the attribute ``stop``. This command stops all the Traffic Server processes (:program:`traffic_manager`, :program:`traffic_server`, and :program:`traffic_cop`). Do not manually stop processes, as this can lead to -unpredictable results.:: +unpredictable results. :: bin/trafficserver stop
