FYI (if it is of any interest), we just hacked CloudSolrServer locally
to support routing of realtime-get requests. Limitations are:
- Only "id"-parameter and not "ids"-parameter supported in realtime-get
requests.
- Only schemas with uniqueKey on field named "id" and only "id"-field of
type string supported.
We did this to be able to start performance tests on our own system
building on SolrCloud. The performance of our own system is dependent on
being able to do realtime-gets from the client (our system), because we
often do updates of documents very quickly after they have been indexed
for the first time (and we run with soft-commit = 1 sec - we cant wait
for that). We use the "version control" (for optimistic locking) and
"unique key constraint where you fail instead of overwrite if document
already exists"
(http://wiki.apache.org/solr/Per%20Steffensen/Update%20semantics) in our
highly concurrent performance test, so that will also be tested wrt
performance.
What we did in CloudSolrServer was:
* Added the following to the requst method between the "if (collection
== null)" statement and the "LBHttpSolrServer.Req req = new
LBHttpSolrServer.Req(request, urlList);" statement:
List<String> urlList = new ArrayList<String>();
if (reqParams.get(CommonParams.QT) != null &&
reqParams.get(CommonParams.QT).equals("/get")) {
String id = reqParams.get("id");
int hash = hash(id);
String shardId = getShard(hash, collection, cloudState);
ZkCoreNodeProps leaderProps = null;
try {
leaderProps = new ZkCoreNodeProps(zkStateReader.getLeaderProps(
collection, shardId));
} catch (InterruptedException ie) {
throw new SolrServerException(ie);
}
String fullUrl =
ensureUrlHasProtocolIdentifier(leaderProps.getCoreUrl());
urlList.add(fullUrl);
} else {
<stuff that was already in request between the "if (collection ==
null)" statement and the "LBHttpSolrServer.Req req = new
LBHttpSolrServer.Req(request, urlList);" statement
}
* Added the follow helper-methods (stolen from
DistributedUpdateProcessor etc.)
private String ensureUrlHasProtocolIdentifier(String url) {
if (!url.startsWith("http://";) && !url.startsWith("https://";)) {
url = "http://"; + url;
}
return url;
}
private String getShard(int hash, String collection, CloudState
cloudState) {
return cloudState.getShard(hash, collection);
}
private int hash(String id) {
BytesRef indexedId = new BytesRef();
UnicodeUtil.UTF16toUTF8(id, 0, id.length(), indexedId);
return Hash.murmurhash3_x86_32(indexedId.bytes, indexedId.offset,
indexedId.length, 0);
}
It seems to work for us, but we look very much forward to the "real"
solution.
Regards, Per Steffensen
Per Steffensen skrev:
Right, you can't yet even with CloudSolrServer - but I think it will
be done soon - certainly before the 4 release anyway.
Ok, I will cross my fingers for it to be done soon. Thanks for your
kind help.
Regards, Steff
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
