[htdig] Can I delete these files?
Hi all: Still trying to find out why many known search terms do not show up when searching, I have come to want to try a brand new site index, but I'm running out of server space. I see that the directory /home/myusername/../htdig-3.1.4 takes up a LOT of space. Since the binaries appear in .../htdig/bin/ and the other files are in /conf/, etc., can I delete the entire /htdig-3.1.4 tree if I understand that I'm losing the documentation? Or does anything refer to this directory? Thanks, -- Brian A. Marks Vertical Stores Corp. Grand Prix Store / PartsForYourTruck.com To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
[htdig] how to use htdig with MySQL?
Hi! I need your help! I need to find out how to use htdig with MySQL. I found a patch in this maillist: ftp://sol.ccsf.cc.ca.us/htdig-patches/3.1.4/MySQL.tar.gz Did anybody use it? How reliable is it and how to use it? I will greatly appretiate any help! Maxim To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
[htdig] Compile errors of FreeBSD 3.3
Greetings, I'm just compiled HtDig 3.1.4 on FreeBSD 3.3. It appears to have compiled but during the make there were a lot of error messages like: gcc -o db_load db_load.o err.o getlong.o libdb.a -lc_r /usr/lib/libc.so: warning: this program uses gets(), which is unsafe. /usr/lib/libc.so: WARNING! setkey(3) not present in the system! /usr/lib/libc.so: WARNING! des_setkey(3) not present in the system! /usr/lib/libc.so: WARNING! encrypt(3) not present in the system! /usr/lib/libc.so: WARNING! des_cipher(3) not present in the system! /usr/lib/libc.so: warning: this program uses f_prealloc(), which is stupid. Anyone else have a similar experience? How can I fix it. The current port at the FreeBSD site is 3.1.3. I've also sent a message to the maintainer asking him if there were plans to upgrade the port. /Paul To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
[htdig] Following ./links
I have nosed around the mail archives and have found some discussion on this but I cant seem to get the right combo for this to work. Do I need to patch the HTML.cc with code I saw relating to image maps?
I am trying to index a "slide show" and the forward-backward links are ./nextslide.html type links. I run ./htdig -i -vv -s -c ../conf/dental_search.conf
where the start_url is:
http://www.dent.ucla.edu/ce/online/case_studies/remov_prost/case001.1.html and limit_url is ${start_url} but I get this error when htdig looks at the next link:
url rejected: (level 1)http://www.dent.ucla.edu/ce/online/case_studies/remov_pro
st/case001.2.html
A tag: pos = 2, position = ="./case001.2.html">
I am running 3.1.4
Thanks,
Rob
--
=
Robert Zimmerman [[EMAIL PROTECTED]]
IT Manager
ReadySite LLC
http://readysite.net
Ph:888-356-2992
fax:603-356-6990
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
[htdig] Re: [htdig3-dev] HTDIG SSL Patch
At 2:04 PM +0100 1/21/00, Browny wrote: >We patched htdig 3.1.4 to support openssl. Now you can index http and https >server as well. Thanks. I'm sure plenty of people will get some use out of it. Though the legal picture is changing somewhat, it may be some time before we can include any SSL-based code. The upcoming 3.2.0b1 release supports a mechanism for external transport methods, so this will allow htdig to use legal HTTPS mechanisms (among others). Cheers, -Geoff Hutchison Williams Students Online http://wso.williams.edu/ To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
[htdig] HTDIG SSL Patch
Hi,
We patched htdig 3.1.4 to support openssl. Now you can index http and https
server as well.
Have fun.
SAP IT BSS Internet Services
diff -u --recursive htdig-3.1.4.org/CONFIG htdig-3.1.4/CONFIG
--- htdig-3.1.4.org/CONFIG Fri Dec 10 01:29:30 1999
+++ htdig-3.1.4/CONFIG Wed Jan 19 14:39:56 2000
@@ -8,7 +8,7 @@
# These variables are set by configure
#
# This specifies the root of the directory tree to be used by ht://Dig
-prefix= /opt/www/htdig
+prefix= /opt/htdig
# This specifies the root of the directory tree to be used for programs
# installed by ht://Dig
diff -u --recursive htdig-3.1.4.org/Makefile.config.in htdig-3.1.4/Makefile.config.in
--- htdig-3.1.4.org/Makefile.config.in Fri Dec 10 01:28:21 1999
+++ htdig-3.1.4/Makefile.config.in Wed Jan 19 14:39:35 2000
@@ -24,13 +24,13 @@
SENDMAIL= @SENDMAIL@
DEFINES= -DDEFAULT_CONFIG_FILE=\"$(DEFAULT_CONFIG_FILE)\"
-LIBDIRS= -L../htlib -L../htcommon -L../db/dist -L/usr/lib
+LIBDIRS= -L../htlib -L../htcommon -L../db/dist -L/usr/lib -L/opt/ssl/lib
INCS= -I$(top_srcdir)/htlib -I$(top_srcdir)/htcommon \
- -I../db/dist -I../include
+ -I../db/dist -I../include -I/opt/ssl/include
HTLIBS=../htcommon/libcommon.a \
../htlib/libht.a \
../db/dist/libdb.a
-LIBS= $(HTLIBS) @LIBS@
+LIBS= $(HTLIBS) @LIBS@ -lssl -lcrypto
DIST= @PACKAGE@-@VERSION@
DISTDIR= $(top_srcdir)/../$(DIST)
diff -u --recursive htdig-3.1.4.org/htcommon/DocumentDB.cc
htdig-3.1.4/htcommon/DocumentDB.cc
--- htdig-3.1.4.org/htcommon/DocumentDB.cc Fri Dec 10 01:28:44 1999
+++ htdig-3.1.4/htcommon/DocumentDB.cc Thu Jan 20 10:16:44 2000
@@ -217,7 +217,7 @@
while ((key = dbf->Get_Next()))
{
dbf->Get(key, data);
- if (strncmp(HtURLCodec::instance()->decode(key), "http:", 5) == 0)
+ if (strncmp(HtURLCodec::instance()->decode(key), "http:", 5) == 0 ||
+strncmp(HtURLCodec::instance()->decode(key), "https:", 6) == 0)
{
ref = new DocumentRef;
ref->Deserialize(data);
@@ -284,7 +284,7 @@
while ((coded_key = dbf->Get_Next()))
{
String key = HtURLCodec::instance()->decode(coded_key);
- if (mystrncasecmp(key, "http:", 5) == 0)
+ if (mystrncasecmp(key, "http:", 5) == 0 || mystrncasecmp(key, "https:", 6) ==
+0)
{
DocumentRef *ref = (*this)[key];
if (ref)
diff -u --recursive htdig-3.1.4.org/htcommon/defaults.cc
htdig-3.1.4/htcommon/defaults.cc
--- htdig-3.1.4.org/htcommon/defaults.ccFri Dec 10 01:28:44 1999
+++ htdig-3.1.4/htcommon/defaults.ccThu Jan 20 10:16:37 2000
@@ -37,7 +37,7 @@
{"bad_querystr",""},
{"bad_word_list", "${common_dir}/bad_words"},
{"case_sensitive", "true"},
-{"common_url_parts","http:// http://www. ftp:// ftp://ftp. /pub/
.html .htm .gif .jpg .jpeg /index.html /index.htm .com/ .com mailto:"},
+{"common_url_parts","https:// https://www. http:// http://www.
+ftp:// ftp://ftp. /pub/ .html .htm .gif .jpg .jpeg /index.html /index.htm .com/ .com
+mailto:"},
{"create_image_list", "false"},
{"create_url_list","false"},
{"compression_level", "0"},
diff -u --recursive htdig-3.1.4.org/htdig/Document.cc htdig-3.1.4/htdig/Document.cc
--- htdig-3.1.4.org/htdig/Document.cc Fri Dec 10 01:28:44 1999
+++ htdig-3.1.4/htdig/Document.cc Fri Jan 21 10:33:46 2000
@@ -220,6 +220,7 @@
tm.tm_year += 1900;
tm.tm_yday = 0; // clear these to prevent problems in strftime()
tm.tm_wday = 0;
+ tm.tm_isdst = -1;
if (debug > 2)
{
@@ -328,7 +329,7 @@
return Document_no_host;
}
}
-
+c.assign_ssl(strcmp(url->service(), "https") == 0);
if (c.connect(1) == NOTOK)
{
if (debug)
diff -u --recursive htdig-3.1.4.org/htdig/Images.cc htdig-3.1.4/htdig/Images.cc
--- htdig-3.1.4.org/htdig/Images.cc Fri Dec 10 01:28:44 1999
+++ htdig-3.1.4/htdig/Images.cc Thu Jan 20 10:15:16 2000
@@ -61,7 +61,7 @@
{
String u = url;
URL Url(url);
- if (strcmp(Url.service(), "http") != 0)
+ if (strcmp(Url.service(), "http") != 0 && strcmp(Url.service(), "https") != 0)
return 0;
u.lowercase();
@@ -81,6 +81,7 @@
return 0;
if (c.assign_server(Url.host()) == NOTOK)
return 0;
+ c.assign_ssl(strcmp(Url.service(), "https") == 0);
if (c.connect(1) == NOTOK)
{
diff -u --recursive htdig-3.1.4.org/htdig/Retriever.cc htdig-3.1.4/htdig/Retriever.cc
--- htdig-3.1.4.org/htdig/Retriever.cc Fri Dec 10 01:28:44 1999
+++ htdig-3.1.4/htdig/Retriever.cc Thu Jan 20 16:
Re: [htdig] Problem with HTSEARCH
Justin Richardson wrote: > > Hopefully this is the correct address for HTDIG problems, please accept my > apologies if incorrect. > > Have installed and complied HTDig and all works well, used rundig to > generate a database which seems find. However, when I use the same > HTSEARCH.HTML it returns thousands for spurious characters. Running > HTSEARCH manually from Linux works fine. > > Any ideas? I suspect that your server tries to transmit the contents of the htsearch program "as is" and not execute it as a CGI application. You should therefore check, if your server setup matches the installation of the htsearch CGI appli- cation. On some servers you need to have a file name extension like ".cgi" for CGI applications. On others, you need to install the htsearch application in a special directory (for which you might need system administrator privileges). hth, Torsten -- InWise - Wirtschaftlich-Wissenschaftlicher Internet Service GmbH Waldhofstraße 14Tel: +49-4101-403605 D-25474 EllerbekFax: +49-4101-403606 E-Mail: [EMAIL PROTECTED]Internet: http://www.inwise.de To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
[htdig] Problem with HTSEARCH
Hopefully this is the correct address for HTDIG problems, please accept my apologies if incorrect. Have installed and complied HTDig and all works well, used rundig to generate a database which seems find. However, when I use the same HTSEARCH.HTML it returns thousands for spurious characters. Running HTSEARCH manually from Linux works fine. Any ideas? To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
