[htdig] Remote Request?

2001-01-02 Thread Mike Paradis 

In my application, I could use some sort of remote search request.

In other words, just having the database remote is not good enough. It takes quite a
while to actually go through the big database over a network connection of course.

If I could just send a small network request and receive the results, that would be 
much
more efficient.

Does anyone know of a way of doing this?


Mike




To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
List archives:  http://www.htdig.org/mail/menu.html
FAQ:http://www.htdig.org/FAQ.html

Re: [htdig] Remote Request?

2001-01-02 Thread Geoff Hutchison 

On Tue, 2 Jan 2001, Mike Paradis wrote:

 If I could just send a small network request and receive the results,
 that would be much more efficient.

I'm not quite sure that I follow you exactly. But I think you're asking
for something along the lines of the ExternalTransport code in the 3.2
branch. You could, for example write up some sort of Perl (or other)
script to access the database and then rewrite the URLs afterwards.

In this fashion, you'd probably want to index on the DB server itself and
then transfer the ht://Dig databases if you wanted.

Am I following you, or did you have something else in mind?

--
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/



To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
List archives:  http://www.htdig.org/mail/menu.html
FAQ:http://www.htdig.org/FAQ.html

Re: [htdig] Remote Request?

2001-01-02 Thread Geoff Hutchison 


Based on what you say, you'll probably want to have some sort of 
"wrapper" CGI. This can take the user's query through the search 
server and then call htsearch on the index servers using something 
like ssh to execute htsearch remotely. (I would say rsh since more 
people are familiar with using rsh in this fashion, but it's 
insecure.)

--
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

At 10:46 AM -0600 1/2/01, htdighelp wrote:
  In this fashion, you'd probably want to index on the DB server
  itself and then transfer the ht://Dig databases if you wanted.
  
  Am I following you, or did you have something else in mind?

I am using a cluster of servers that do nothing but index 24hrs/day.
Each server contains it's own database of results but is not a web 
server so that it can be searched.

Additional servers are the actual search servers. The search servers 
could contain a local copy of the indexing servers but I would 
rather leave the databases remote.

Right now, I have things set up so that a search server takes the 
input from a user, then uses an NFS connection to the particular 
index server's database. The problem, as I see it right now, is that 
this type of method will have network limitations in speed as the 
databases have to be checked completely, and, from a remote 
connection.

A much better solution would be if the search server could send a 
query to the index server which could quickly process the request 
and send back the results only. Would this not speed things up? The 
only network traffic would be the request and the result this way 
rather than the entire database file reads that need to happen. 
Since a fellow user mentioned the backlink variable, the results are 
*much* faster already, even over an nfs link.

Does this explain better?

Stand alone htdig servers. Stand alone search servers. Users enter a 
search on a search server, which looks on the htdig server's db 
files for the results.

Mike



To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
List archives:  http://www.htdig.org/mail/menu.html
FAQ:http://www.htdig.org/FAQ.html