These all sound like good ideas for solr2. The ability to handle changes
on the fly easily across many machines would be an awesome place to
reach. Dynamically changing field schema/stopword stuff is also a great
feature to have.
I think the key to reaching those goals is to really join the solr open
source process and push at it a piece at a time. I think almost all of
the features you have talked about make sense for the future of solr in
one way or another. Take advantage of solrs popularity and I think
eventually we can have more people helping make these goals a reality.
It might not happen as fast as you'd like (already having all this Ocean
code), but I think the long run returns will be much higher.
You already have a great start with the patches and wiki info, but I
think I am suggesting leaving the Ocean part out a little more, and
maybe tackling these issues more form a Lucene/Solr perspective, using
the Ocean ideas and code as leverage to move forward faster.
- Mark
Jason Rutherglen wrote:
That may be the way to go, however there are many issues that you will
probably run into, the same ones I ran into that made the integration
difficult as mentioned. I could be wrong however. Most "realtime
search" is something on the order of a 5-10 second delay and there is
no transaction log. I guess I made the goal of Ocean to be a
replacement for SQL databases, 0 second delay, transaction log
reliability, easy scalability, easy to add heavily customized queries
(span, payload, custom similarities, etc), and maximum uptime. I
wanted to concentrate a lot on the grid computing part and leave the
rest of system as generic as possible so that what I consider to be
application specific code is not part of the search server. For
example I consider custom Analyzers to be application specific code.
I want those dynamically loaded so I don't have to reboot 100 servers
if I make a change to one. Add a stop word? Well then don't have to
reboot 100 servers. Synonyms? The list does on and on. Search is
fluid and dynamic.
On Fri, Sep 5, 2008 at 10:07 AM, Noble Paul നോബിള് नोब्ळ्
<[EMAIL PROTECTED]> wrote:
On Fri, Sep 5, 2008 at 7:23 PM, Mark Miller <[EMAIL PROTECTED]> wrote:
IMO, you are underestimating the difficulty of integrating Ocean with Solr's
current API's.
OK. you are right. Actually ,I did not mean the ocean integration. I
am mostly interested in the Realtime search part. If we take one baby
step at a time it may be easy for us . We can add features one by
one, but why don't we start with realtime search? (this sounds like an
immediately useful feature to an average solr user).
Also, Jason has already mentioned that Ocean is much more than just realtime
search. Adding realtime search to something like solr 1.5 is a different
goal than possibly integrating the Ocean work that has been done / is
planned, which seems like a very large scope project and if done would
certainly seem to merit a 2.0 change in its own right.
Still seems large and nebulous to me at the moment...just like solr 2. They
go well together in my mind <g>
Noble Paul നോബിള് नोब्ळ् wrote:
Postponing Ocean Integration towards 2.0 is not a good idea. First of
all we do not know when 2.0 is going to happen. delaying such a good
feature till 2.0 is wasting time.
My assumption was that Actually realtime search may have nothing to do
with the core itself . It may be fine with a Pluggable
SolrIndexSearcherFactory/SolrIndexWriterFactory . Ocean can have a
unified reader-writer which may choose to implement both in one class.
A total rewrite has its own problems. Achieving consensus on how
things should change is time consuming. So it will keep getting
delayed. If with a few changes we can start the integration, that is
the best way forward . Eventually , we can slowly , evolve to a
better design. But, the design need not be as important as the feature
itself.
On Fri, Sep 5, 2008 at 6:46 PM, Yonik Seeley <[EMAIL PROTECTED]> wrote:
On Fri, Sep 5, 2008 at 9:03 AM, Jason Rutherglen
<[EMAIL PROTECTED]> wrote:
Ok, SOLR 2 can be a from the ground up rewrite?
Sort-of... I think that's up for discussion at this point, but enough
should change that keeping Java APIs back compatible is not a priority
(just my opinion of course). Supporting the current main search and
update interfaces and migrating most of the handlers shouldn't be that
difficult. We should be able to provide relatively painless back
compatibility for the 95% of Solr users that don't do any custom
Java.... and the others hopefully won't mind migrating their stuff to
get the cool new features :-)
As far as SolrCore goes... I agree it's probably best to not do
pluggability at that level.
The way that Lucene has evolved, and may evolve (and how we want Solr
to evolve), it seems like we want more of a combo
IndexReader/IndexWriter interface. It also needs (optional)
optimistic concurrency... that was also assumed in the discussions
about bailey.
-Yonik
--
--Noble Paul
