With the new setup, I'm happy to report that the downtime, the 30 second downtime we had with balancer plugin, is gone.
Hobin On Sat, Jan 12, 2019 at 2:34 PM Hobin Yoon <[email protected]> wrote: > dest_ip did the magic. Thanks Miles! Here is my setup for the record. > > On child node: (It's ip address is 100.100.100.1) > records.config > CONFIG proxy.config.http.parent_proxy_routing_enable INT 1 > parent.config: > dest_domain=100.100.100.1 scheme=http parent="100.100.200.1:80, > 100.100.200.1:80" round_robin=consistent_hash go_direct=false > > On parent nodes (100.100.200.1 and 100.100.200.2): > records.config > CONFIG proxy.config.http.parent_proxy_routing_enable INT 0 > remap.config > map / http://(origin_ip) > > Hobin > > On Sat, Jan 12, 2019 at 12:15 PM Miles Libbey <[email protected]> wrote: > >> A loop would indicate its not matching parent.config then. I've never >> used an IP in a map. In >> >> https://docs.trafficserver.apache.org/en/8.0.x/admin-guide/files/parent.config.en.html >> it has a >> dest_ip >> field that would seem to make more sense than dest_domain|host for >> that map rule. >> >> You'll also need a map http://52.91.93.206/ http://someorigiin on your >> parent... >> miles >> >> On Fri, Jan 11, 2019 at 10:49 PM Hobin Yoon <[email protected]> wrote: >> > >> > Now, I'm getting >> > >> > Cycle Prohibited >> > ________________________________ >> > Description: Could not process your request for the document because it >> would cause an HTTP proxy cycle. Please check the URL and your browser's >> proxy settings. >> > >> > I think it's because remap.config has the same ip addresses ... I tried >> to comment out the map rule, but got the same error... >> > >> > On child node: (It's ip address is 52.91.93.206) >> > records.config >> > CONFIG proxy.config.http.parent_proxy_routing_enable INT 1 >> > CONFIG proxy.config.http.cache.http INT 0 >> > remap.config: >> > map http://52.91.93.206 http://52.91.93.206 >> > parent.config: >> > dest_domain=52.91.93.206 scheme=http parent="18.212.133.3:80, >> 34.207.101.159:80" round_robin=consistent_hash go_direct=false >> > >> > On parent nodes: >> > records.config >> > CONFIG proxy.config.http.parent_proxy_routing_enable INT 0 >> > CONFIG proxy.config.http.cache.http INT 1 >> > >> > Hobin >> > >> > On Fri, Jan 11, 2019 at 5:31 PM Miles Libbey <[email protected]> >> wrote: >> >> >> >> First thing I'd check is proxy.config.http.parent_proxy_routing_enable >> >> on the children :) >> >> >> >> On Fri, Jan 11, 2019 at 2:15 PM Hobin Yoon <[email protected]> >> wrote: >> >> > >> >> > It is disabled. I just tried with both on and off, but none of them >> worked. What bothers me is ATS doesn't seem to be reading parent.config at >> all since I don't get any error when I intentionally made a typo in >> parent.config. >> >> > >> >> > Hobin >> >> > >> >> > On Fri, Jan 11, 2019 at 4:38 PM Leif Hedstrom <[email protected]> >> wrote: >> >> >> >> >> >> >> >> >> >> >> >> On Jan 11, 2019, at 12:54 PM, Hobin Yoon <[email protected]> >> wrote: >> >> >> >> >> >> Ok, I added the rule in remap.config. Now, the requests are going >> to "origin". But, they bypasses parent nodes; I don't see any requests on >> either of the parent nodes. >> >> >> >> >> >> Here is my system setup for test. "child" is a pure load balancer >> and doesn't cache any data items. parent1 and parent2 do cache. >> >> >> Client (web browser) - child (ATS)--- parent1 (ATS) --- origin >> (Apache2) >> >> >> └- parent2 (ATS) -┘ >> >> >> >> >> >> Here is the configuration in "child" node. >> >> >> >> >> >> remap.config >> >> >> map / http://origin >> >> >> >> >> >> parent.config >> >> >> dest_domain=origin scheme=http parent="parent1:80;parent2:80" >> round_robin=consistent_hash >> >> >> >> >> >> >> >> >> >> >> >> Do you have “pristine” host headers enabled? The mapping in parent >> selection happens on the “remapped” URL, but I’m pretty sure it’d be >> different if you have pristine host headers disabled. >> >> >> >> >> >> That much said, I’d run with a diagnostics (-T) on “parent”. >> >> >> >> >> >> — Leif >> >> >> >> >> >> >> >> >> >> >> >> records.config >> >> >> CONFIG proxy.config.http.parent_proxy_routing_enable INT 1 >> >> >> >> >> >> Hobin >> >> >> >> >> >> On Fri, Jan 11, 2019 at 1:08 PM Leif Hedstrom <[email protected]> >> wrote: >> >> >>> >> >> >>> >> >> >>> >> >> >>> On Jan 11, 2019, at 10:00 AM, Hobin Yoon <[email protected]> >> wrote: >> >> >>> >> >> >>> Should you still specify the mapping in remap.config when you have >> the mapping in parent.config? >> >> >>> >> >> >>> >> >> >>> Yes. Unless you run an open proxy, and then you have to turn off >> “require remap”. Not recommended for almost all cases. >> >> >>> >> >> >>> — leif >> >> >>> >> >> >>> >> >> >>> >> >> >>> On Fri, Jan 11, 2019 at 11:33 AM Randall Meyer < >> [email protected]> wrote: >> >> >>>> >> >> >>>> >> >> >>>> Looking at your remap.config in that repo, I see that all the >> mappings are all commented out. Was that intentional? >> >> >>>> On Thursday, January 10, 2019, 8:28:53 PM PST, Hobin Yoon < >> [email protected]> wrote: >> >> >>>> >> >> >>>> >> >> >>>> Yes, they are also ATS nodes and they worked fine when requests >> were made to them directly. >> >> >>>> >> >> >>>> I uploaded the config files here, if you could take a look: >> https://gitlab.com/hobinyoon/trafficserver-config >> >> >>>> >> >> >>>> Hobin >> >> >>>> >> >> >>>> >> >> >>>> >> >> >>>> On Thu, Jan 10, 2019 at 7:31 PM Miles Libbey <[email protected]> >> wrote: >> >> >>>> >> >> >>>> Typically that error means the request doesn't match a rule in >> >> >>>> remap.config. Is the error coming from the first hop or one of the >> >> >>>> 123.123.123.[1-4] nodes? That is, if 123.123.123.[1-4] are ATS >> nodes, >> >> >>>> are they configured to accept the requests they are getting? >> >> >>>> >> >> >>>> On Thu, Jan 10, 2019 at 2:51 PM Hobin Yoon <[email protected]> >> wrote: >> >> >>>> > >> >> >>>> > With parent.config >> >> >>>> > dest_domain=. scheme=http parent="123.123.123.1:80, >> 123.123.123.2:80" round_robin=consistent_hash go_direct=false >> >> >>>> > >> >> >>>> > I'm getting >> >> >>>> > >> >> >>>> > Not Found on Accelerator >> >> >>>> > ________________________________ >> >> >>>> > Description: Your request on the specified host was not found. >> Check the location and try again. >> >> >>>> > >> >> >>>> > I must be missing something ... >> >> >>>> > >> >> >>>> > Hobin >> >> >>>> > >> >> >>>> > On Thu, Jan 10, 2019 at 1:18 PM Hobin Yoon <[email protected]> >> wrote: >> >> >>>> >> >> >> >>>> >> Alan, that is the only map rule we have with the varying >> number of cache nodes. During the down time, ATS doesn't return "HTTP/1.1 >> 200 OK" for the requests. I didn't check what it returned. >> >> >>>> >> >> >> >>>> >> Miles, I'll check out parent plugin! >> >> >>>> >> >> >> >>>> >> Hobin >> >> >>>> >> >> >> >>>> >> On Thu, Jan 10, 2019 at 1:13 PM Alan Carroll < >> [email protected]> wrote: >> >> >>>> >>> >> >> >>>> >>> It could be an artifact of reloading plugin configurations if >> you have a lot of remap rules with plugins, although internally ATS should >> do the load and then swap the configuration. During the down time, does ATS >> process any traffic, or there is traffic but no caching? >> >> >>>> >>> >> >> >>>> >>> On Thu, Jan 10, 2019 at 10:53 AM Miles Libbey < >> [email protected]> wrote: >> >> >>>> >>>> >> >> >>>> >>>> We don't experience downtime when using traffic_ctl config >> reload (we >> >> >>>> >>>> use that ~daily). >> >> >>>> >>>> >> >> >>>> >>>> We don't use the balancer plugin. Instead, we use >> parent.config >> >> >>>> >>>> ( >> https://docs.trafficserver.apache.org/en/8.0.x/admin-guide/files/parent.config.en.html >> ) >> >> >>>> >>>> to achieve the same consistent hash. Your config would >> translate to >> >> >>>> >>>> >> >> >>>> >>>> remap.config >> >> >>>> >>>> map / http://127.0.0.1 @plugin=cachekey.so >> >> >>>> >>>> @pparam=--include-params=p0,p1 @pparam=--sort-params=true >> >> >>>> >>>> >> >> >>>> >>>> parent.config >> >> >>>> >>>> dest_domain=. scheme=http >> >> >>>> >>>> parent="123.123.123.1:80,23.123.123.2:80,123.123.123.3:80, >> 123.123.123.4:80" >> >> >>>> >>>> round_robin=consistent_hash go_direct=false >> >> >>>> >>>> dest_domain=. scheme=https >> >> >>>> >>>> parent="123.123.123.1:443,23.123.123.2:443,123.123.123.3:443 >> ,123.123.123.4:443" >> >> >>>> >>>> round_robin=consistent_hash go_direct=false >> >> >>>> >>>> >> >> >>>> >>>> miles >> >> >>>> >>>> >> >> >>>> >>>> On Wed, Jan 9, 2019 at 10:53 PM Hobin Yoon < >> [email protected]> wrote: >> >> >>>> >>>> > >> >> >>>> >>>> > Hi, >> >> >>>> >>>> > >> >> >>>> >>>> > We are noticing there is quite a bit of delay when we >> reload the config with traffic_ctl config reload. The delay is up to about >> 30 seconds, during which period we don't get any caching. We are using >> consistency hashing plugin. The number of nodes changes dynamically between >> 5 to 30. >> >> >>>> >>>> > >> >> >>>> >>>> > Here is an example balancer (consistent hash) >> configuration in remap.config. >> >> >>>> >>>> > >> >> >>>> >>>> > map / http://127.0.0.1 @plugin=cachekey.so >> @pparam=--include-params=p0,p1 @pparam=--sort-params=true >> @plugin=balancer.so @pparam=--policy=hash,key @pparam=123.123.123.1 >> @pparam=123.123.123.2 @pparam=123.123.123.3 @pparam=123.123.123.4 ... >> >> >>>> >>>> > >> >> >>>> >>>> > Is this downtime normal? How do you guys avoid the service >> downtime while reconfiguring the cache nodes in the cluster? >> >> >>>> >>>> > >> >> >>>> >>>> > Hobin >> >> >>>> >>>> > >> >> >>>> >>>> > >> >> >>>> >>>> > >> >> >>>> >>>> > >> >> >>>> >>>> > >> >> >>>> >>> >> >> >>>> >>> >> >> >>>> >>> >> >> >>>> >>> -- >> >> >>>> >>> Beware the fisherman who's casting out his line in to a dried >> up riverbed. >> >> >>>> >>> Oh don't try to tell him 'cause he won't believe. Throw some >> bread to the ducks instead. >> >> >>>> >>> It's easier that way. - Genesis : Duke : VI 25-28 >> >> >>> >> >> >>> >> >> >> >> >
