You should be able to easily prove your theory with the access logs. If you grep for all of the "validate" calls, cut for the ST field, sort, uniq -c (I'm sure Linux ninjas could tell you how do this with just sed or a Perl regex) you should be able to see how frequently the same ST is being validated.
Cheers, Scott On Tue, Jul 21, 2015 at 4:58 PM, Bryan Wooten <bryan.woo...@utah.edu> wrote: > Thanks all for the feed back. > > With the hints provided I am fairly confident I have “proven” that the > issue is created by a client re-using an ST. I am still not clear why the > client is doing this or why there was such a spike last week. > > While I pouring over log files a co-worker was busy doing some mods to > the 3.3.x CAS java client. He extended a class such that a validation > error sent a redirect back to the browser to go get another ticket (the > CAS10 validation filter). > > I am not sure how he is preventing an endless loop though. > > Is this a feature that is desirable in the client? I am sure he wouldn’t > mind sharing with the java client devs. If so how should he submit the > change? > > Cheers, > > Bryan > > From: Marvin Addison <marvin.addi...@gmail.com> > Reply-To: "cas-dev@lists.jasig.org" <cas-dev@lists.jasig.org> > Date: Tuesday, July 21, 2015 at 7:23 AM > To: "cas-dev@lists.jasig.org" <cas-dev@lists.jasig.org> > Subject: Re: [cas-dev] Diagnosing Service Ticket validation errors > > Sorry if I am spamming this list but I am desperate. >> > > Yeah, this is a support issue, but we'll cut you some slack ;) > > >> On July 14th we got over 2000 of these errors out of about 30k >> successful logins. This led to (thanks ITIL >> ) awareness up to the VP level. I am under the gun to find a “solution” >> before the start of school August 24th. >> > > I think a ~7% ticket validation failure rate is something of legitimate > concern. Do you see validation failures on this order of magnitude on a > regular basis, or did you just have a peak on that day? > > I have turned up log level to debug on the CAS servers. I see >> successful validations in the logs, but not unsuccessful validations. >> > > Ticket validation failures are indeed logged, both in audit and in the > validator components. Here are some randomly-chosen audit events from our > log for today: > > > 2015-07-21T09:06:02.252|ST-567525-EK0BvF9xYKDqDSHQUCTV-cas2|audit:unknown|SERVICE_TICKET_VALIDATE_FAILED|198.82.164.189 > > 2015-07-21T09:01:33.503|ST-567442-71agV1hMMND2CC4ntePf-cas2|audit:unknown|SERVICE_TICKET_VALIDATE_FAILED|198.82.164.171 > > 2015-07-21T09:00:59.650|null|audit:unknown|SERVICE_TICKET_VALIDATE_FAILED|128.173.56.37 > > 2015-07-21T08:55:42.543|ST-71780-WlfObEPXfSOfnYdWZlzp-cas1|audit:unknown|SERVICE_TICKET_VALIDATE_FAILED|198.82.169.7 > > 2015-07-21T08:51:00.075|AAHnrNAKBfKATlQQH7UKhnKXNdebx13zB0yXtKeDauD1CWNJ1o30W0QV/|audit:unknown|SERVICE_TICKET_VALIDATE_FAILED|198.82.162.156 > > Now if I understand how CAS works, there can only be 3 reasons an ST >> won’t validate: it is being reused, it has timed out or it does not exist / >> is corrupted. >> > > Correct. So if you don't have any record of ticket validation failure, > what evidence do you have that validation is failing? > > >> Can someone point me to the method(s) that does the validation? >> > > Several places within the following method you could add debug logging, > but you can see there's already quite a bit: > > > https://github.com/Jasig/cas/blob/3.5.x/cas-server-core/src/main/java/org/jasig/cas/CentralAuthenticationServiceImpl.java#L338 > > Here is a diagram of our infrastructure: >> >> >> https://www.lucidchart.com/invitations/accept/da009b9d-e55f-4f95-9301-e6bd23d508ab >> > > I'll take a closer look at the diagram once I get some more information > on how you're identifying ticket validation failures. You should be getting > logging on the CAS server, and the fact that you are apparently not getting > that suggests a client problem. > > M > > -- > You are currently subscribed to cas-dev@lists.jasig.org as: > bwoo...@acs.utah.edu > To unsubscribe, change settings or access archives, see > http://www.ja-sig.org/wiki/display/JSG/cas-dev > > -- > You are currently subscribed to cas-dev@lists.jasig.org as: > scott.battag...@gmail.com > To unsubscribe, change settings or access archives, see > http://www.ja-sig.org/wiki/display/JSG/cas-dev > > -- You are currently subscribed to cas-dev@lists.jasig.org as: arch...@mail-archive.com To unsubscribe, change settings or access archives, see http://www.ja-sig.org/wiki/display/JSG/cas-dev
