Please find the test code below - I believe it doesn't do anything too
stupid :-) If anything jumps out, please let me know.
queries.txt is a text file with one query per line - it is a mix of SQL and
XPath queries. The query tester can take upto 2 parameters. The first is
the # of threads to run, the second is the # of queries to run per thread.
The tester defaults to a total of 100 queries, so if the second parameter
isn't given, the # of queries run per thread = 100 / # threads.
-Dave
---
package jackrabbitquerytester;
import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.net.MalformedURLException;
import java.rmi.NotBoundException;
import java.rmi.RemoteException;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CountDownLatch;
import javax.jcr.Credentials;
import javax.jcr.Repository;
import javax.jcr.RepositoryException;
import javax.jcr.Session;
import javax.jcr.SimpleCredentials;
import javax.jcr.query.Query;
import javax.jcr.query.QueryManager;
import javax.jcr.query.QueryResult;
import java.util.Properties;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import org.apache.jackrabbit.rmi.client.ClientAdapterFactory;
import org.apache.jackrabbit.rmi.remote.RemoteRepository;
/**
*
* @author djohnson
*/
public class Main {
/** Creates a new instance of Main */
public Main() {
}
static private class QueryRunner implements Runnable {
private Session session;
private List<String> queries = new ArrayList();
private CountDownLatch cdl;
public QueryRunner(CountDownLatch cdl) throws RepositoryException {
session = getJackrabbitSession();
this.cdl = cdl;
}
public void run() {
try {
int count = 0;
long start = System.currentTimeMillis();
try {
for(String query : queries) {
try {
long qstart=System.currentTimeMillis();
QueryManager qm = session.getWorkspace
().getQueryManager();
String type = Query.SQL;
if(query.startsWith("/")) type = Query.XPATH;
Query q = qm.createQuery(query, type);
QueryResult qr = q.execute();
count++;
System.out.println("\t" + query + "\t" + (
System.currentTimeMillis()-qstart));
} catch (RepositoryException ex) {
ex.printStackTrace();
}
}
long time = System.currentTimeMillis() - start;
System.out.println("Processed " + count + " queries in "
+ time + " ms");
} finally {
cdl.countDown();
}
} finally{
session.logout();
}
}
public void addQuery(String query) {
queries.add(query);
}
}
/**
* @param args the command line arguments
*/
public static void main(String[] args) throws FileNotFoundException,
IOException, RepositoryException, InterruptedException {
FileReader fr = new FileReader("queries.txt");
BufferedReader br = new BufferedReader(fr);
int count = 0;
String line;
long start = System.currentTimeMillis();
System.out.println("Starting " + start);
// defaults for the # of threads (runners) and the
// # of queries per thread
int runnerCount = 50;
int queryCount = 2;
if(args.length > 0) {
runnerCount = Integer.parseInt(args[0]);
queryCount = 100 / runnerCount;
}
if(args.length == 2) {
queryCount = Integer.parseInt(args[1]);
}
CountDownLatch cdl = new CountDownLatch(runnerCount);
List<QueryRunner> runners = new ArrayList();
for(int i = 0 ;i < runnerCount; i++) {
runners.add(new QueryRunner(cdl));
}
while(true) {
// walk through each query runner and add a query to it
for(QueryRunner qr: runners) {
qr.addQuery(br.readLine());
}
// For now only put 100 queries in each query runner
count++;
if(count >= queryCount) break;
}
System.out.println("starting threads");
for(QueryRunner qr: runners) {
new Thread(qr).start();
}
System.out.println("threads started!");
cdl.await();
System.out.println("threads done took " +
(System.currentTimeMillis()-start)
+ " ms.");
}
static public Session getJackrabbitSession() throws RepositoryException
{
Properties env = new Properties();
env.put(Context.INITIAL_CONTEXT_FACTORY, "
org.jnp.interfaces.NamingContextFactory");
env.put(Context.PROVIDER_URL, "jnp://localhost:1099");
env.put(Context.URL_PKG_PREFIXES, "
org.jboss.naming:org.jnp.interfaces")
;
Session sessionCRX = null;
try {
InitialContext ctx = new InitialContext(env);
ClientAdapterFactory adapter = new ClientAdapterFactory();
RemoteRepository rr;
rr = (RemoteRepository) ctx.lookup
("jnp://localhost:1099/jcrServer");
Repository repository = adapter.getRepository(rr);
Credentials credJBoss = new SimpleCredentials("username",
"password"
.toCharArray());
return repository.login(credJBoss);
} catch (NamingException ex) {
ex.printStackTrace();
}
return null;
}
}
On 3/13/07, Marcel Reutegger <[EMAIL PROTECTED]> wrote:
David Johnson wrote:
> This is related to two ongoing list threads - one on synchronization and
> the
> other on query performance.
>
> As I have mentioned in previous posts, I have been running a variety of
> query tests. I am using a suite of 100 queries and running them against
> Jackrabbit in several different threading scenarios - i.e., I change the
#
> of threads used to run sub-sets of the 100 queries. To be clear - if I
run
> a single thread case, it will run all 100 queries, one after the
other. If
> I run 2 threads - one thread will run 50 queries, while the other thread
> will run the other 50 queries. In all cases, the 100 queries are the
same,
> the only thing that changes is the number of threads used to run them.
> Also, in all tests, the repository is read only - nothing is making any
> writes to the repository.
>
> Here are some results:
>
> 1 thread: 100 queries in 41139 ms
> 2 threads: 50 queries in 37828 ms, 50 queries in 38622 ms - total time
for
> all threads to complete 38960 ms
> 4 threads: 25 queries in 25895 ms, 25 queries in 28034 ms, 25 queries in
> 32335 ms, 25 queries in 32391 ms - total time 32801 ms
> 10 threads: 10 queries in 18733 ms, 10 queries in 19894 ms, ... , 10
> queries
> in 33798 ms, 10 queries in 34924 ms - total time 35286 ms
> 25 threads: 4 queries in 2413 ms, 4 queries in 11725 ms, 4 queries in
18294
> ms, ... , 4 queries in 36059 ms, 4 queries in 36222 ms
>
> Some details on the box that I am running these tests on: it is a dual
Xeon
> running Linux - /proc/cpuinfo shows 4 processors, so I am assuming it is
a
> dual core. I am running Jackrabbit 1.2.3 with the Bundle Persistence
> Manager.
>
> I am not sure what the numbers above are really saying, although they
don't
> really look right :-) We have a multi-user use case - large web site
with
> many ongoing reads, occasional writes. I am using the multiple threads
to
> "test" multiple users. I am hoping that the developers with more
> understanding of the internals can explain what's going on above.
>
> I am wondering if I am hitting the synchronization issue that is being
> discussed in other posts? Thoughts?
that's difficult to tell just from the timings and numbers of threads. do
you
mind sharing your tests classes and post them to the list? we'd also need
some
information on the content you have in your workspace.
regards
marcel
