On Thu, Sep 27, 2012 at 10:38 AM, Nico Williams wrote:
>> The above incident is a single misbehaving client suddenly doing about
>> 600 requests / minute for around 30 minutes. During this window no one
>> else could get a KDC response before the client timed out.
>
> The client is not misbehavin
On Thu, Sep 27, 2012 at 10:17 AM, Jack Neely wrote:
> This definitely seems to explain the lag in responses I've noticed
> during a kprop operation. Usually I get a response in under a second,
> but if I hit my KDC during when its receiving a kprop it can be 4 or 5
> seconds.
Perfect. It's very
On Wed, Sep 26, 2012 at 04:04:25PM -0500, Nico Williams wrote:
> On Wed, Sep 26, 2012 at 1:25 PM, Jack Neely wrote:
> > After spending some quality time with my logs, I do about 1.3 million
> > kerberos requests a day or 960/min on average. The incident that took
> > out the kerberos servers with
On Tue, Sep 25, 2012 at 2:02 PM, Jack Neely wrote:
> My network engineers tell me that the firewall in one DC had 8000
> concurrent connections from the offending IP address to the KDCs and
> 4000 in the second DC. (Oddly, the DC with only 1 slave.) The KDCs
> weren't able to handle other reque
On Tue, Sep 25, 2012 at 2:08 PM, Russ Allbery wrote:
> We were quite concerned when we first looked at putting Kerberos KDCs
> behind a hardware firewall because of that session limit. Our firewalls
> have a 100,000 UDP session limit and a fairly quick timeout.
Ideally you just disable the con
On Wed, Sep 26, 2012 at 1:25 PM, Jack Neely wrote:
> After spending some quality time with my logs, I do about 1.3 million
> kerberos requests a day or 960/min on average. The incident that took
> out the kerberos servers with an additional 600 hits/min (from the krb
> logs) doesn't even make a s
On Tue, Sep 25, 2012 at 02:08:29PM -0700, Russ Allbery wrote:
> Jack Neely writes:
>
> > Thanks for reading between the lines. I don't have evidence that my
> > KDCs were overloaded, yet I got quite a few cannot reach KDC errors and
> > a logins stopped working everywhere.
>
> > The slaves are
On Tue, Sep 25, 2012 at 09:55:21PM -0700, Frank Cusack wrote:
> On Tue, Sep 25, 2012 at 2:02 PM, Jack Neely wrote:
>
> > My network engineers tell me that the firewall in one DC had 8000
> > concurrent connections from the offending IP address to the KDCs and
> > 4000 in the second DC. (Oddly, t
Frank Cusack writes:
> On Tue, Sep 25, 2012 at 2:08 PM, Russ Allbery wrote:
>> We were quite concerned when we first looked at putting Kerberos KDCs
>> behind a hardware firewall because of that session limit. Our firewalls
>> have a 100,000 UDP session limit and a fairly quick timeout.
> Idea
On Tue, Sep 25, 2012 at 4:41 PM, Nico Williams wrote:
> https://github.com/krb5/krb5/commit/b858e776ece87756202d4c646931d35bd407e3ea
That's likely to apply cleanly to recent releases, FYI. If you
cherry-pick this patch you might also want to cherry-pick the
surrounding ones too:
9e182bcee06362d
On Tue, Sep 25, 2012 at 3:56 PM, Glenn Machin wrote:
> A performance issue we have seen has to do when a KDC has a heavy load
> and cannot provide a response within 1 sec.
That's almost certainly due to a locking bug that was recently fixed
in the master
branch:
https://github.com/krb5/krb5/comm
Jack Neely writes:
> Thanks for reading between the lines. I don't have evidence that my
> KDCs were overloaded, yet I got quite a few cannot reach KDC errors and
> a logins stopped working everywhere.
> The slaves are HP G7 blades with 12GB of RAM and a 6 core Intel Xeon. 2
> servers in one D
On Tue, Sep 25, 2012 at 01:49:55PM -0700, Russ Allbery wrote:
> Jack Neely writes:
>
> > Has anyone done any rate limiting or throttling of Kerberos requests?
> > I've had several situations where I had a load of 600 requests / minute
> > against my 3 kerberos slaves that caused degradation of pe
A performance issue we have seen has to do when a KDC has a heavy load
and cannot provide a response within 1 sec.
The Kerberos client libraries apparently expect a response within a sec
and if they don't get it they move on to the next KDC in the list for
the realm and so on for both udp and tc
Jack Neely writes:
> Has anyone done any rate limiting or throttling of Kerberos requests?
> I've had several situations where I had a load of 600 requests / minute
> against my 3 kerberos slaves that caused degradation of performance for
> everyone else. Always from misbehaving tools or applica
Greetings,
Has anyone done any rate limiting or throttling of Kerberos requests?
I've had several situations where I had a load of 600 requests / minute
against my 3 kerberos slaves that caused degradation of performance for
everyone else. Always from misbehaving tools or applications.
What sugg
16 matches
Mail list logo
