Plugin of course? I don't want my intranet site pinging anyone :)
-- Sent from my Palm Pre
Evan Prodromou wrote:
Craig Andrews wrote:
So... how can we implement federated, cross OMB search? I have some crazy
ideas, but I'm interested in what others think first.
(Ideally, search would search OMB microblogs, not just StatusNet instances.)
There are three ways this can work that I can think of.
Peer-to-peer search, like Gnutella. When you do a global
search on a site, it asks all the sites it knows about (through
subscriptions or whatever) for their results for the search
term. They maybe ask their peers for the search term, out to
some arbitrary depth. At an outer limit, we could demand that all
sites respond to the request, either by keeping them in a master list
somewhere, or some other weird way.
This would perform terribly -- a matter of minutes or hours. It
wouldn't get all notices, and it would be hard for the site
where the search originated to present the results in any reasonable
way. As far as I'm concerned, distributed peer-to-peer search is not a
problem we should have to solve.
Centralized search, pull model. A centralized search
engine provides the UI for search (and maybe an API that OMB sites can
embed into their sites, viz. Collecta's API). The search engine
discovers OMB servers on its own and polls the public feed or
individuals' feeds at regular intervals. It may use SUP to make this
crawling more efficient; it may even use PuSH or RSSCloud to do
push-mode subscriptions. We could also support subscriptions to the
public feed through OMB.
The great part about this is that we (statusnet developers) don't have
to do anything. The bad part is that it's hard to run a search engine.
You have to have a lot of resources and smarts.
Centralized search, push model. We could configure
StatusNet to, by default, ping or push public notices to one or more
centralized search engines. For status.net hosted sites, we ping
pingomatic, Google Blog Search, and weblogs.com by default. We could
probably add these and others to lib/common.php for pings by default.
We could do the same thing with the public XMPP feed (identi.ca feeds
most search engines using XMPP, but very few StatusNet sites are
XMPP-enabled). Pings are a little inefficient; we could use PuSH,
RSSCloud, or a custom notification system.
We could also run a big weblogs.com-style "reflector" for all sites.
So, every StatusNet site (and other OMB or even non-OMB microblogging
sites) could ping "reflector.status.net" or whatever, and that site
could in turn ping lots of other search services. We could even host a
Free Network Service for search (although I would still want to have
the reflector service.)
Of the three, I think #3 makes the most sense. I think we should add
some default ping targets in 0.9.0, and try to get a reflector working
for status.net ASAP.
-Evan
--
Evan Prodromou
CEO, StatusNet, Inc.
[email protected] - http://identi.ca/evan - +1-514-554-3826
_______________________________________________
StatusNet-dev mailing list
[email protected]
http://lists.status.net/mailman/listinfo/statusnet-dev
