I ran into a curious performance issue yesterday, when several students
were testing a digital collection interface we have developed. When all
the students accessed the same search results page at the same time (a
"canned" search page), the application would freeze up for several
minutes. The search results page contained 10 thumbnails, all of which
were served up by Fedora via a disseminator.
After several hours of targeted load testing and debugging, I narrowed
down the source of the problem to communication between Fedora and a
backend web service that delivers the thumbnails. I started digging
through the source code, and found fedora.common.http.WebClient:
public class WebClient {
/** Seconds to wait before a connection is established. */
public int TIMEOUT_SECONDS = 20;
/** Seconds to wait while waiting for data over the socket
(SO_TIMEOUT). */
public int SOCKET_TIMEOUT_SECONDS = 120;
/** Maxiumum http connections per host */
public int MAX_CONNECTIONS_PER_HOST = 5;
/** Maxiumum total http connections */
public int MAX_TOTAL_CONNECTIONS = 20;
/** Whether to automatically follow HTTP redirects. */
public boolean FOLLOW_REDIRECTS = true;
/**
* Maximum number of redirects to follow per request if
FOLLOW_REDIRECTS is
* true.
*/
public int MAX_REDIRECTS = 3;
...
}
Suspecting that Fedora had run out of connections for making requests to
my backend web service, I manually changed WebClient.java to bump up
MAX_CONNECTIONS_PER_HOST to 256, and MAX_TOTAL_CONNECTIONS to 256,
recompiled, then put the changed class file in place, restarted Tomcat,
and re-executed my load tests. The problem disappeared.
I'd like to request the following features be added to a future release
of Fedora:
1. Make the internal web client parameters configurable in fedora.fcfg
(TIMEOUT_SECONDS, SOCKET_TIMEOUT_SECONDS, MAX_CONNECTIONS_PER_HOST,
FOLLOW_REDIRECTS, MAX_REDIRECTS)
2. Throw a meaningful exception that can be captured in the logs
whenever one of the above constraints is exceeded.
If this sounds reasonable, I'll open up a feature request in JIRA.
thanks,
-- Scott
--
Scott Prater
Library, Instructional, and Research Applications (LIRA)
Division of Information Technology (DoIT)
University of Wisconsin - Madison
[email protected]
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Fedora-commons-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/fedora-commons-developers
