htdig@htdig.org
According to [EMAIL PROTECTED]: > I am modifying the perl script that parses pdf's and have come across a > problem. I am indexing sites that use CGI; htdig (I think) seems to cut > off my URL just before the & and unfortunately the information > following the & is the most important. The URL should look like > /cgi-bin/../../..?..=..&..=&.. but $ARGV[2] returns /cgi-bin/../../?..=.. > Could this be coded into htdig? And if so, where can I find the code > to modify it? I have looked at URL.cc and don't see it > coded into there but maybe I am overlooking something. If you're running version 3.1.3, upgrade to 3.1.5. -- Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]> Spinal Cord Research Centre WWW:http://www.scrc.umanitoba.ca/~grdetil Dept. Physiology, U. of Manitoba Phone: (204)789-3766 Winnipeg, MB R3E 3J7 (Canada) Fax:(204)789-3930 To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
htdig@htdig.org
Hello, I am modifying the perl script that parses pdf's and have come across a problem. I am indexing sites that use CGI; htdig (I think) seems to cut off my URL just before the & and unfortunately the information following the & is the most important. The URL should look like /cgi-bin/../../..?..=..&..=&.. but $ARGV[2] returns /cgi-bin/../../?..=.. Could this be coded into htdig? And if so, where can I find the code to modify it? I have looked at URL.cc and don't see it coded into there but maybe I am overlooking something. Thank you in advance, Sheri To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
RE: [htdig] Solaris Compile Problems
That did the trick. I went out to sunfreeware.com and got the 2.95.2 gcc package. Installed it and everything went smooth after that. thanks chad >>> "Budd, S" <[EMAIL PROTECTED]> 07/19/00 03:40AM >>> I have always had to use 2.95.2 gcc -Original Message- From: Chad Phillips [mailto:[EMAIL PROTECTED]] Sent: Monday, July 17, 2000 5:30 PM To: [EMAIL PROTECTED] Subject: [htdig] Solaris Compile Problems I just upgraded my OS from Solairs 2.6 to 8, and now I can't compile HtDig. When I run configure it start fine but then gives me this error: checking for fstream.h... no configure: error: To compile ht://Dig, you will need a C++ library. Try installing libstdc++. I installed packages for gcc and libstdc. libstdc++-2.8.1.1-sol7-sparc-local GCC281 Any ideas would be greatly appreciated. thanks chad To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this. To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
Re: [htdig] start/stop/restart
According to Franck Horlaville: > I'd like to set htdig to crawl only at night. > > Right now I know how to start it at specified times with cron and > tell it to save context before quitting so it restarts where it left > off. > > Current operation chain : > > general "do it forever and log" script >slightly modified rundig.sh > htdig > > I would like to include cron somewhere in there to turn crawling on and off. > > Any suggestions ? You'd need to run htdig with the -l option, and save its process ID somewhere. Then, your cron job to stop crawling would have to kill that process ID. -- Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]> Spinal Cord Research Centre WWW:http://www.scrc.umanitoba.ca/~grdetil Dept. Physiology, U. of Manitoba Phone: (204)789-3766 Winnipeg, MB R3E 3J7 (Canada) Fax:(204)789-3930 To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
Re: [htdig] Solaris Compile Problems
According to Chad Phillips: > I just upgraded my OS from Solairs 2.6 to 8, and now I can't compile HtDig. > When I run configure it start fine but then gives me this error: > checking for fstream.h... no > configure: error: To compile ht://Dig, you will need a C++ library. Try installing >libstdc++. > > I installed packages for gcc and libstdc. > libstdc++-2.8.1.1-sol7-sparc-local > GCC281 > > Any ideas would be greatly appreciated. Does the libstdc++ package include all the header files? That seems to be what configure is failing to find. If the header files are in a non- standard location, perhaps you need to set your CFLAGS environment variable before running ./configure, to point it in the right direction. -- Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]> Spinal Cord Research Centre WWW:http://www.scrc.umanitoba.ca/~grdetil Dept. Physiology, U. of Manitoba Phone: (204)789-3766 Winnipeg, MB R3E 3J7 (Canada) Fax:(204)789-3930 To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
Re: [htdig] Error in ./configure
According to Sigfus Oddsson: > My Linux distribution is Red Hat 6.2, very minimal though - I only installed > what I considered to be the bare minimum of packages. > > This is my sed version: sed-3.02-6 > > Included is my config.status file > > My $SHELL is /bin/bash. ... > > > > > When doing a ./configure on the 3.1.5 version of Ht://Dig I > constantly get > > > > > this error: > > > > > > > > > > sed: file conftest.s1 line 3: Unterminated `s' command > > > > > > > > > > Which results in no Makefile and CONFIG being generated so I can't > 'make' > > > > > the binary. In the attached config.status file... [snip] > sed 's/%@/@@/; s/@%/@@/; s/%g$/@g/; /@g$/s/[\\&%]/\\&/g; > s/@@/%@/; s/@@/@%/; s/@g$/%g/' > conftest.subs <<\CEOF > /^[ ]*VPATH[]*=[^:]*$/d > > s%@SHELL@%/bin/sh%g > s%@CFLAGS@%-O3 -march=pentium -mcpu=pentium -ffast-math -funroll-loops >-fomit-frame-pointer -fforce-mem -fforce-addr > -malign-double -fno-exceptions%g [snip] This seems to be the problem right here. The CFLAGS environment variable contains an embedded newline character, which is throwing off the sed script. You should either rerun ./configure with the CFLAGS variable corrected, if that's how configure got it's CFLAGS setting in the first place, or you can edit config.status to rejoin the two lines above into a single line and then execute config.status to build all the necessary Makefiles. -- Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]> Spinal Cord Research Centre WWW:http://www.scrc.umanitoba.ca/~grdetil Dept. Physiology, U. of Manitoba Phone: (204)789-3766 Winnipeg, MB R3E 3J7 (Canada) Fax:(204)789-3930 To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
[htdig] htdig - searching
Whenever i search for some word (i.e. "frog" or any other word) it doesn't show me any results even though i know "frog" has to be found at least in one file... any ideas why my search results r not finding anything? i tried to add META tags but to no avail... htdig and htmerge seemed to have worked fine but my searches r not finding anything... plz help thx -alan Gilles Detillieux wrote: > According to alan: > > thx... now i run htdig -vvv but nothing appears on the screen yet the program exits > > normally... any idea why i'm not seeing ANYTHING at all? > > Strange. Try "htdig -v -c /path/to/your/config/file.conf", and make > sure your config file makes sense. You should get some sort of output, > unless htdig is dying prematurely. > > -- > Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]> > Spinal Cord Research Centre WWW:http://www.scrc.umanitoba.ca/~grdetil > Dept. Physiology, U. of Manitoba Phone: (204)789-3766 > Winnipeg, MB R3E 3J7 (Canada) Fax:(204)789-3930 > > > To unsubscribe from the htdig mailing list, send a message to > [EMAIL PROTECTED] > You will receive a message to confirm this. To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
Re: [htdig] htdig verbose mode
ok my config file had a minor error which i didnt realize :) thx anyway -alan Gilles Detillieux wrote: > According to alan: > > thx... now i run htdig -vvv but nothing appears on the screen yet the program exits > > normally... any idea why i'm not seeing ANYTHING at all? > > Strange. Try "htdig -v -c /path/to/your/config/file.conf", and make > sure your config file makes sense. You should get some sort of output, > unless htdig is dying prematurely. > > -- > Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]> > Spinal Cord Research Centre WWW:http://www.scrc.umanitoba.ca/~grdetil > Dept. Physiology, U. of Manitoba Phone: (204)789-3766 > Winnipeg, MB R3E 3J7 (Canada) Fax:(204)789-3930 > > > To unsubscribe from the htdig mailing list, send a message to > [EMAIL PROTECTED] > You will receive a message to confirm this. To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
Re: [htdig] Error in ./configure
According to [EMAIL PROTECTED]: > Oh, I'm sorry that I didn't state that. I figured it would appear in the log > file. > > I'm running Linux 2.2.16 on an i386 I've taken a look at the ./configure script, and as near as I can tell, the only thing that would cause that error is an error in the substitution command for the $SHELL variable. What is your setting of $SHELL when you run ./configure? Which distribution of Linux are you running, and what is your sed version? (For an rpm-based distribution, you can find out the sed version with "rpm -qf `which sed`".) Could you send or post your config.status file? It might provide a clue as to what the problem is. > > According to Sigfus Oddsson: > > > When doing a ./configure on the 3.1.5 version of Ht://Dig I constantly get > > > this error: > > > > > > sed: file conftest.s1 line 3: Unterminated `s' command > > > > > > Which results in no Makefile and CONFIG being generated so I can't 'make' > > > the binary. > > > > What is your operating system (& which version)? -- Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]> Spinal Cord Research Centre WWW:http://www.scrc.umanitoba.ca/~grdetil Dept. Physiology, U. of Manitoba Phone: (204)789-3766 Winnipeg, MB R3E 3J7 (Canada) Fax:(204)789-3930 To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
Re: [htdig] htdig verbose mode
According to alan: > thx... now i run htdig -vvv but nothing appears on the screen yet the program exits > normally... any idea why i'm not seeing ANYTHING at all? Strange. Try "htdig -v -c /path/to/your/config/file.conf", and make sure your config file makes sense. You should get some sort of output, unless htdig is dying prematurely. -- Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]> Spinal Cord Research Centre WWW:http://www.scrc.umanitoba.ca/~grdetil Dept. Physiology, U. of Manitoba Phone: (204)789-3766 Winnipeg, MB R3E 3J7 (Canada) Fax:(204)789-3930 To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
Re: [htdig] Htmerge: "Deleted, invalid"
According to David Adams: > I use the standard MIPSpro compiler. The script I use (thanks to my former > collegeaue James Hammick) to setup the Makefile is: > > #!/bin/sh > CFLAGS="-woff all -O2 -mips4 -n32 -DHAVE_ALLOCA_H" ; export CFLAGS > CPPFLAGS="-woff all -O2 -mips4 -n32 -DHAVE_ALLOCA_H" ; export CPPFLAGS > LDFLAGS="-mips4 -L/usr/lib32 -rpath /opt/local/htdig-3.1.5/lib"; > export LDFLAGS > ./configure --prefix=/opt/local/htdig-3.1.5 \ > --with-cgi-bin-dir=/opt/local/htdig-3.1.5/cgi-bin \ > --with-image-dir=/opt/local/htdig-3.1.5/graphics \ > --with-search-dir=/opt/local/htdig-3.1.5/htdocs/sample > > A lot of that is site-specific, and the "-rpath " option is only > needed because the compression library is not in a standard place on the > machine on which htdig is run. > > The "-woff all" option suppresses most warning messages. I will remove it, > recompile htdig and send the result directly to Gilles, it might contain a clue. As Sinclair mentioned, 'you need to have the 2.95.2 gcc and the latest gnu "make".' I don't know that anyone has ever gotten ht://Dig to work with SGI's own compiler. If fact, we got a lot of reports from folks who couldn't even get it to compile. If you're really determined to get to the bottom of this and make it work with the SGI compiler, I wish you well, but I doubt I can help much. I looked at the output you sent me, and didn't really see any red flags pointing to an obvious problem. I know that the Serialize and Deserialize functions for the db.docdb records can be a tad finicky, so that would probably be a place to look. There could also be problems with incorrect assumptions about word sizes, e.g. if the SGI compiler has 64-bit long ints. I'd also look at the db.wordlist records (they're ASCII text) before and after htmerge, to see if htdig is actually telling htmerge to remove some of these documents, or if htmerge is deciding to do so on its own. For the time being, the ht://Dig code hasn't had much of a workout on non-GNU compilers, so it doesn't seem to do well on them. If you can help remedy that, great. If you want to get the package working as quickly and easily as possible, I'd suggest trying the GNU C and C++ compilers. -- Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]> Spinal Cord Research Centre WWW:http://www.scrc.umanitoba.ca/~grdetil Dept. Physiology, U. of Manitoba Phone: (204)789-3766 Winnipeg, MB R3E 3J7 (Canada) Fax:(204)789-3930 To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
Re: [htdig] htdig verbose mode
thx... now i run htdig -vvv but nothing appears on the screen yet the program exits normally... any idea why i'm not seeing ANYTHING at all? thx -alan Gilles Detillieux wrote: > According to alan: > > how do i run htdig in verbose mode in unix? and how do i show all the htdig options > > (i.e. htdig -v -c ...) > > You can use one or more -v options to htdig to select the level of verbosity > you want. A -\? option will show all available options for a given program > in the package. Also have a look at all the on-line documentation at > http://www.htdig.org/ , which will explain the options for each of the > programs in greater detail. > > -- > Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]> > Spinal Cord Research Centre WWW:http://www.scrc.umanitoba.ca/~grdetil > Dept. Physiology, U. of Manitoba Phone: (204)789-3766 > Winnipeg, MB R3E 3J7 (Canada) Fax:(204)789-3930 > > > To unsubscribe from the htdig mailing list, send a message to > [EMAIL PROTECTED] > You will receive a message to confirm this. To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
Re: [htdig] htdig verbose mode
According to alan: > how do i run htdig in verbose mode in unix? and how do i show all the htdig options > (i.e. htdig -v -c ...) You can use one or more -v options to htdig to select the level of verbosity you want. A -\? option will show all available options for a given program in the package. Also have a look at all the on-line documentation at http://www.htdig.org/ , which will explain the options for each of the programs in greater detail. -- Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]> Spinal Cord Research Centre WWW:http://www.scrc.umanitoba.ca/~grdetil Dept. Physiology, U. of Manitoba Phone: (204)789-3766 Winnipeg, MB R3E 3J7 (Canada) Fax:(204)789-3930 To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
[htdig] htdig verbose mode
how do i run htdig in verbose mode in unix? and how do i show all the htdig options (i.e. htdig -v -c ...) thx -alan To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
Re: [htdig] Error in ./configure
According to Sigfus Oddsson: > When doing a ./configure on the 3.1.5 version of Ht://Dig I constantly get > this error: > > sed: file conftest.s1 line 3: Unterminated `s' command > > Which results in no Makefile and CONFIG being generated so I can't 'make' > the binary. What is your operating system (& which version)? -- Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]> Spinal Cord Research Centre WWW:http://www.scrc.umanitoba.ca/~grdetil Dept. Physiology, U. of Manitoba Phone: (204)789-3766 Winnipeg, MB R3E 3J7 (Canada) Fax:(204)789-3930 To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
[htdig] htdig on irix
we have built and are running htdig on irix 6.5 you need to have the 2.95.2 gcc and the latest gnu "make" Regards Sinclair To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
