I'm forwarding the first of two messages (with attachments) from someone
who has expanded the database support. I haven't the time to review them,
but post them here for others to look at them.
-Geoff
---------- Forwarded message ----------
Date: Mon, 27 Dec 1999 14:23:57 +0100 (MET)
From: Zoran Constantinescu-Fulop <[EMAIL PROTECTED]>
To: Geoff Hutchison <[EMAIL PROTECTED]>
Subject: ht://Dig and MySQL patches ...
Hi,
> You can always get the latest development code through
> <http://www.htdig.org/files/snapshots/> since a snapshot is made every
> week from the CVS tree. It's quite a bit different from the 3.0.x and
> 3.1.x code, so we'd be willing to see patches to 3.1.4 as well.
>
I figured out how to work with CVS :-) It's great! However I
made the patches for the 3.1.4 release and not for the current
snapshot.
I attached a file describing the changes i've made to use MySQL with
ht://Dig. I added the X_db, SQL_db and MySQL_db classes, and the
hierarchy of classes becomes:
X_db
|
`--- Database
|
`------ DB2_db
|
`------ SQL_db
|
`------ MySQL_db
The only MySQL specific code is in the MySQL class, which uses the
API of mysql (library libmysqlclient.a). The SQL code doesn't serialize
the information as the DB2 code does, instead it uses specialized
tables for all information (URLS, URLINFO, WORDS, LINKS and so). The
Perl scripts for creating the SQL tables in the database are found
in htdig3/contrib/sqldb. So I had to change all Database::Put/Get()
calls everywhere to specialized calls. That's what I've put in the
X_db class. To keep the DB2 functionality unchanged, I added a config
variable 'db_type' which I used where the Put/Get methods where called
(if we use SQL then the new functions will be called).
I've attached also a patch with the changes to the 3.1.4 version.
--zoran
-------------------------------
Zoran Constantinescu -o)
[EMAIL PROTECTED] /\\
http://www.idi.ntnu.no/~zoran _\_v
-------------------------------
Sat Dec 25 10:04:32 1999 Zoran Constantinescu <[EMAIL PROTECTED]>
* htlib/Database.h: Class Database is inherited from X_db too.
* htlib/X_db.h: Generic functions for SQL. Added to class Database
to make as small changes to the code as possible.
* htlib/Database.cc: Choose database type from config file! Changed
getDatabaseInstance to use config["db_type"].
* htlib/SQL_db.cc,h: Added code for generic SQL databases.
* htlib/MySQL_db.cc,h: Added code for MySQL databases.
* htlib/HashWord.h: Experimental code for buffering the words from
database in memory (for database speed-ul). Need to be completed.
* htcommon/DocumentRef.h: Added new ReferenceStates and the server
where the document is (docServerID) and its type (docServerType).
* htcommon/DocumentDB.cc,h: Added the external 'config' variable.
The Add() function is changed to call SQL_db::AddDocInfo(). Added
operator[](int) to get documents from database by ID. Modified
code for SQL in CreateSearchDB() and URLs() (do nothing if SQL!?).
* htcommon/defaults.cc: added default values for config parameters:
db_type - database type: (DB2), GDBM, MySQL (! any SQL type database
should contain the "SQL" letters!),
SQL_debug - debug level for logging SQL messages (default 0),
SQL_log_file - log file for SQL messages,
SQL_dbase - connection information for SQL servs "user:pswd@host:dbase"
* htcommon/WordList.cc: Flush() changed to write link informations
to the SQL database (if SQL).
* htfuzzy/Fuzzy.h: Added new getWords(...+wordIDs) function which
returns also a list of the wordsIDs (if we are SQL, we use IDs
much more).
* htfuzzy/Synonym.cc,h: Added code for the new getWords() function.
* htfuzzy/Endings.cc,h: Added code for the new getWords() function.
* htfuzzy/EndingsDB.cc: Changed code to use SQL databases if necessary.
The createRoot() function uses SQL.
* htlib/Makefile.in: Added SQL_db.o MySQL_db.o to the list of
objects.
* htdig/htdig.cc: If SQL, no need to add the list to retriever.
Added options '-x start_url' and '-z limit_urls_to'.
* htdig/Document.cc,h: Added server_type (what kind of web server).
* htdig/Retriever.cc,h: Added SQL support for links. Each link
found in documents will be added to the database.
* htsearch/htsearch.cc: Added SQL support. And add the search to
the database.
* htsearch/WeightWord.h: Added word_id;
* htsearch/ResultMatch.h: Added id and functions setDocID(), getDocID().
* htsearch/Display.cc: Added SQL support. To be checked!
htdig-3.1.4-patch.tar.gz
------------------------------------
To unsubscribe from the htdig3-dev mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
