Yes, but one of the nice things about CAS was that we could drop new service definitions into a running instance and have them picked up automatically. Anything that requires us to actually restart our service get us involved in a formal change management process and restricts us to weekend early mornings. :(
The thing that seems inconsistent to me in this case, is that the bad service definition doesn't tank an already running instance. The faulty definition just doesn't get incorporated into the running config. This seems much preferable to breaking everything should the system be restarted for some reason, IMO. So, I guess I understand the way it is, but must it be that way? Should it be that way? On Mon, May 01, 2017 at 07:20:00PM -0600, Bryan Wooten wrote: >Ok, If you are manually editing the JSON service registry or using any tool >(home grown or provided by Apereo) you MUST carefully validate the final >JSON file syntax. > >I never make a change during normal hours. > >We are HA with 4 CAS servers. I make the change on one server. (They are >all standalone no rsync or cross mounts for any config file) I then restart >that one server and verify is comes back up and starts servicing tickets. >(All CAS servers are behind a Citirix Netscalar Load Balancer). So there is >no outage. > >After I confirm the change has the correct JSON syntax I then push the >change to the other servers. > >Owning SSO is terrifying. We can never have an outage for 60k users and >500 servers. Never. Or my life gets bad. > >So yeah, it is manual and I take great care and someday I hope to put in >place an automated system. > >Until then I live with a little stress..... > >On Mon, May 1, 2017 at 6:29 PM, Baron Fujimoto <ba...@hawaii.edu> wrote: > >> So this happened. We are using CAS 5.0.3.1 with JSON service registry >> files. We inadvertantly created a service registration with an error in >> the regex for the serviceID (improperly escaped "\\." as "\."). Although >> we didn't notice it at the time, this was logged with the error/warnings >> >> ERROR [org.apereo.cas.services.AbstractResourceBasedServiceRegistryDao] - >> <Error reading configuration file Bad_serviceID-20170328162739.json> >> java.lang.IllegalArgumentException: org.hjson.ParseException: Expected >> valid escape sequence at 3:30 >> ... >> WARN [org.apereo.cas.services.AbstractResourceBasedServiceRegistryDao] - >> <Could not load service definition from file /home/cas/cas/services/Bad_ >> serviceID-20170328162739.json> >> WARN [org.apereo.cas.services.AbstractResourceBasedServiceRegistryDao] - >> <1 errors encountered when loading service definitions. New definitions are >> not loaded until errors are corrected> >> >> But CAS continued on working in general. However, some time later, the CAS >> service was restarted after an OS reboot. Upon restart, the same errors >> are noted, but there was crucial difference >> >> INFO [org.apereo.cas.services.DefaultServicesManagerImpl] - <Loaded 0 >> services from JsonServiceRegistryDao.> >> >> None of the other valid service definitions were loaded, and thus our CAS >> was effectively broken for everyone by this one bad definition. >> >> Is there a way to configure CAS to just ignore the bad definitions rather >> than just fail completely in situations like this? While we probably should >> have caught this sooner, it definitely turned out to be a time bomb for >> us. Is there a good reason for this behavior that we're missing? >> >> Aloha, >> -baron >> -- >> Baron Fujimoto <ba...@hawaii.edu> :: UH Information Technology Services >> minutas cantorum, minutas balorum, minutas carboratum desendus pantorum >> >> -- >> - CAS gitter chatroom: https://gitter.im/apereo/cas >> - CAS mailing list guidelines: https://apereo.github.io/cas/ >> Mailing-Lists.html >> - CAS documentation website: https://apereo.github.io/cas >> - CAS project website: https://github.com/apereo/cas >> --- >> You received this message because you are subscribed to the Google Groups >> "CAS Community" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to cas-user+unsubscr...@apereo.org. >> To view this discussion on the web visit https://groups.google.com/a/ >> apereo.org/d/msgid/cas-user/20170502002914.mfo3dcbcul3oo5k5% >> 40combobulate.mgt.hawaii.edu. >> > >-- >- CAS gitter chatroom: https://gitter.im/apereo/cas >- CAS mailing list guidelines: https://apereo.github.io/cas/Mailing-Lists.html >- CAS documentation website: https://apereo.github.io/cas >- CAS project website: https://github.com/apereo/cas >--- >You received this message because you are subscribed to the Google Groups "CAS >Community" group. >To unsubscribe from this group and stop receiving emails from it, send an >email to cas-user+unsubscr...@apereo.org. >To view this discussion on the web visit >https://groups.google.com/a/apereo.org/d/msgid/cas-user/CAG9x2GVK0zMcb2Swb0KhKK6n%3D4pG%3DiZFnu8aRLtdxb%2BtJN7%2B3Q%40mail.gmail.com. -- Baron Fujimoto <ba...@hawaii.edu> :: UH Information Technology Services minutas cantorum, minutas balorum, minutas carboratum desendus pantorum -- - CAS gitter chatroom: https://gitter.im/apereo/cas - CAS mailing list guidelines: https://apereo.github.io/cas/Mailing-Lists.html - CAS documentation website: https://apereo.github.io/cas - CAS project website: https://github.com/apereo/cas --- You received this message because you are subscribed to the Google Groups "CAS Community" group. To unsubscribe from this group and stop receiving emails from it, send an email to cas-user+unsubscr...@apereo.org. To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/cas-user/20170502234855.3tewdzzz6buc4mjh%40combobulate.mgt.hawaii.edu.
