All,
I'm working on changing our underlying java memcached client to the
"spy" memcached client. Things have gone well in local testing,
however, in broader testing, our applications are outright dying due
to the following stack trace:
2007-12-11 20:25:15,239 ERROR [Memcached IO over {MemcachedConnection
(very-long-list-of-servers)}]
java.nio.channels.CancelledKeyException
at sun.nio.ch.SelectionKeyImpl.ensureValid
(SelectionKeyImpl.java:55)
at sun.nio.ch.SelectionKeyImpl.readyOps
(SelectionKeyImpl.java:69)
at java.nio.channels.SelectionKey.isReadable
(SelectionKey.java:271)
at net.spy.memcached.MemcachedConnection.handleIO
(MemcachedConnection.java:262)
at net.spy.memcached.MemcachedConnection.handleIO
(MemcachedConnection.java:180)
at net.spy.memcached.MemcachedClient.run
(MemcachedClient.java:730)
I can reproduce this by restarting the memcached servers while there
is memcache IO in progress. It appears that in our environment (jdk
1.5.0_11 on debian linux), a failed connection/read from one of the
memcached servers is being interpreted as a CancelledKeyException
(uncaught, bubbles all the way out of the JVM) vs. an IOException
(Which appears to be caught by the code and retried).
I'm tempted to hack in an exception handler to deal with this
exception type but I am not sure if that is appropriate, or if I am
missing something more fundamental. I'm not a java programmer by
trade so I don't know if I'm missing something silly - Can anyone
comment on my misfortunes? :-)
Thanks!
-Andy
