Sam - thanks for the input; I think you can ignore this. There was
something funky in our Java layer which has now apparently vanished (I
updated the component to a newer version) - it must have been including
some overhead in our system beyond just the XCC traffic and deserialization.
Now I have the job of chasing down that 300 ms! But I don't think it's
in any of the ML code.
-Mike
On 04/14/2011 09:46 PM, Michael Sokolov wrote:
On 4/14/2011 4:41 PM, Sam Neth wrote:
By "cold" do you mean immediately following a server restart? Is
each "cold" run a new Java process? If so, you probably have some
class-loader activity in your measurement. A fair bit of the
difference is probably an authentication round-trip. Might also be
some penalty for DNS resolution. Just a few thoughts.
No, the java process is not restarting - I'm running the queries
through a webapp running in a jetty service which stays resident in
memory - running the whole time. I think the XCC Session object may
even be persistent across requests.
Sometimes, I did restart the ML server in order to guarantee its
caches were flushed. More often, I just used new query terms of
roughly the same frequency in my corpus so as to get uncached result sets.
Can you provide some more information and/or source code to clarify
how you're running the test?
Yes - but perhaps off list? The java part, in particular, is embedded
in a large application. To pursue this I think I will need to isolate
the java part to a smaller program.
What OS/JDK versions are you running?
Linux (Fedora 11) / JDK 1.6.0_20
Are client and server on the same machine?
No, they are on the same 100MBps LAN, though
Thanks for the follow-up; I'll get back in touch tomorrow (it's
getting late here)
,Sam Neth
Lead Engineer
MarkLogic Corporation
On Apr 14, 2011, at 12:52 PM, Mike Sokolov wrote:
I'm working on optimizing some queries on one of our MarkLogic sites
using cq's profiling feature and also looking at query times in our
logs, and they seem oddly out of sync.
The times in our logs are based on wrapping calls to XCC
Session.submitRequest() between calls to Java
System.currentTimeMillis(), so presumably they include some network
time
and serialization/deserialization that may be different from what's
shown in cq.
However, the difference is much larger for cold queries (first time
run)
then it is for warmed queries, which seems odd since you would expect
that kind of difference (packaging and transmission) to be the same for
warm and cold queries.
A typical set of numbers might be:
cold, cq: 110 ms
warm, cq 10 ms
cold, xcc: 440 ms
warm, xcc: 14 ms
This result is consistent across many queries: it's the rule, so far as
I can tell.
We're using a slightly older XCC (version 4.1.3, I think, and the
server
version is 4.1.6). I do plan to upgrade, but this was so odd that I
wanted to make sure I wasn't overlooking something known that could
cause this kind of discrepancy. Any ideas?
--
Michael Sokolov
Engineering Director
www.ifactory.com <http://www.ifactory.com>
_______________________________________________
General mailing list
[email protected]
http://developer.marklogic.com/mailman/listinfo/general
_______________________________________________
General mailing list
[email protected]
http://developer.marklogic.com/mailman/listinfo/general
_______________________________________________
General mailing list
[email protected]
http://developer.marklogic.com/mailman/listinfo/general
_______________________________________________
General mailing list
[email protected]
http://developer.marklogic.com/mailman/listinfo/general
