I'm afraid I can't comment on whether your example
would work - I don't use it on any of my sites, because they all have some
dynamic content, (PHP, JSP etc) which doesn't work with direct access like
this.
In most situations, there is no need for the requests
to leave the local machine. In my case, I need to use a proxy to be able to
reach other machines on the network, but from what you say you probably don't
need this set at all. If you are still getting traffic going out onto the
internet,you might want to experiment with indexing 'http://localhost/" and
using URL aliases to re-insert the proper domain name.
If you are still running for too long, it would
probably be possible to adjust things so that you did your indexing in several
discrete steps, or do it on a separate machine and transfer the databases once
complete.
Hope that helps,
Mike
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Amaltas Bohra
Sent: Wednesday, April 19, 2006 4:38 AM
To: [email protected]
Subject: [htdig] Re: htdig-general digest, Vol 1 #1799 - 2 msgsI have already installed htdig with default configuration settings, and just changed the database_dir and start_url. when I run rundig, it takes long time because rundig tries to index local pages as if they reside on different server and request goes to internet and comes back to my server, this may also be consuming lot of bandwidth.
I just want to index my own website, and make rundig index pages on local filesystem, but as if it is HTTP url.
I dont' konw about prxy settings, but will this work:Amaltas
On 4/19/06, [EMAIL PROTECTED] < [EMAIL PROTECTED]> wrote:Send htdig-general mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.sourceforge.net/lists/listinfo/htdig-general
or, via email, send a message with subject or body 'help' to
[EMAIL PROTECTED]
You can reach the person managing the list at
[EMAIL PROTECTED]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of htdig-general digest..."
Today's Topics:
1. Indexing local website (Amaltas Bohra)
2. RE: Indexing local website ([EMAIL PROTECTED])
--__--__--
Message: 1
Date: Tue, 18 Apr 2006 15:57:59 +0530
From: "Amaltas Bohra" <[EMAIL PROTECTED]>
To: [email protected]
Subject: [htdig] Indexing local website
------=_Part_24292_1398814.1145356079268
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
Hi,
I am new to indexing and htdig. I have two questions:
1) When we index local site (htdig and html documents are in the same
folder), are they fast or do htdig indexes them as if it is a different
website and all the requests come through internet. I have some limits with
my webhost and I cannot run a script for more than 180 seconds.
2) Do htdig have any configuration settings wherein we can put settings so
that it can differentiate between local documents and documents on other
servers.
Thanks
------=_Part_24292_1398814.1145356079268
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
<div>Hi,<br>I am new to indexing and htdig. I have two questions:<br>1) Whe=
n we index local site (htdig and html documents are in the same folder=
), are they fast or do htdig indexes them as if it is a different website a=
nd all the requests come through internet. I have some limit=
s with my webhost and I cannot run a script for more than 180 seconds.
<br>2) Do htdig have any configuration settings wherein we can put settings=
so that it can differentiate between local documents and documents on othe=
r servers.<br><br>Thanks</div>
------=_Part_24292_1398814.1145356079268--
--__--__--
Message: 2
Subject: RE: [htdig] Indexing local website
Date: Tue, 18 Apr 2006 11:45:10 +0100
From: <[EMAIL PROTECTED]>
To: <[email protected]>
This is a multi-part message in MIME format.
------_=_NextPart_001_01C662D5.2A14768C
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
I'm not quite clear from this whether you are currently running htdig,
or just thinking about it.
=20
The following settings are particularly relevant to this situation; if
you have already installed htdig, can you tell us how you have them
set?:
http_proxy: =09
http_proxy_exclude:
local_urls:=20
local_urls_only:=20
server_wait_time:=20
=20
More info on all of the above can be found at:
http://www.htdig.org/confindex.html
=20
Regards,
Mike
=20
_____ =20
From: [EMAIL PROTECTED]
[mailto: [EMAIL PROTECTED]] On Behalf Of Amaltas
Bohra
Sent: Tuesday, April 18, 2006 11:28 AM
To: [email protected]
Subject: [htdig] Indexing local website
=09
=09
Hi,
I am new to indexing and htdig. I have two questions:
1) When we index local site (htdig and html documents are in the
same folder), are they fast or do htdig indexes them as if it is a
different website and all the requests come through internet. I have
some limits with my webhost and I cannot run a script for more than 180
seconds.=20
2) Do htdig have any configuration settings wherein we can put
settings so that it can differentiate between local documents and
documents on other servers.
=09
Thanks
------_=_NextPart_001_01C662D5.2A14768C
Content-Type: text/html;
charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=3DContent-Type content=3D"text/html; =
charset=3Dus-ascii">
<META content=3D"MSHTML 6.00.2800.1522" name=3DGENERATOR></HEAD>
<BODY>
<DIV dir=3Dltr align=3Dleft><FONT face=3DArial color=3D#0000ff =
size=3D2><SPAN=20
class=3D998443910-18042006>I'm not quite clear from this whether you are =
currently=20
running htdig, or just thinking about it.</SPAN></FONT></DIV>
<DIV dir=3Dltr align=3Dleft><FONT face=3DArial color=3D#0000ff =
size=3D2><SPAN=20
class=3D998443910-18042006></SPAN></FONT> </DIV>
<DIV dir=3Dltr align=3Dleft><FONT face=3DArial color=3D#0000ff =
size=3D2><SPAN=20
class=3D998443910-18042006>The following settings are particularly =
relevant to=20
this situation; if you have already installed htdig, can you tell us how =
you=20
have them set?:</SPAN></FONT></DIV><FONT color=3D#0000ff size=3D2><SPAN=20
class=3D998443910-18042006>
<DIV dir=3Dltr align=3Dleft>
<TABLE>
<TBODY>
<TR>
<TD noWrap><FONT face=3DArial>http_proxy: =
</FONT></TD></TR></TBODY></TABLE></DIV>
<DIV dir=3Dltr align=3Dleft><FONT =
face=3DArial>http_proxy_exclude:</FONT></DIV>
<DIV dir=3Dltr align=3Dleft><FONT face=3DArial>local_urls: </FONT></DIV>
<DIV dir=3Dltr align=3Dleft><FONT face=3DArial>local_urls_only: =
</FONT></DIV>
<DIV dir=3Dltr align=3Dleft><FONT face=3DArial>server_wait_time: =
</FONT></DIV>
<DIV dir=3Dltr align=3Dleft><FONT face=3DArial></FONT> </DIV>
<DIV dir=3Dltr align=3Dleft><SPAN class=3D998443910-18042006><FONT =
face=3DArial>More=20
info on all of the above can be found at: <A=20
href="" href="http://www.htdig.org/confindex.html"> http://www.htdig.org/confindex.html">http://www.htdig.org/confind=
ex.html</A></FONT></SPAN></DIV>
<DIV dir=3Dltr align=3Dleft><SPAN class=3D998443910-18042006><FONT=20
face=3DArial></FONT></SPAN> </DIV>
<DIV dir=3Dltr align=3Dleft><SPAN class=3D998443910-18042006><FONT=20
face=3DArial>Regards,</FONT></SPAN></DIV>
<DIV dir=3Dltr align=3Dleft><SPAN class=3D998443910-18042006><FONT=20
face=3DArial>Mike</FONT></SPAN></DIV>
<DIV dir=3Dltr align=3Dleft><SPAN=20
class=3D998443910-18042006></SPAN> </DIV></SPAN></FONT><BR>
<BLOCKQUOTE dir=3Dltr=20
style=3D"PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px =
solid; MARGIN-RIGHT: 0px">
<DIV class=3DOutlookMessageHeader lang=3Den-us dir=3Dltr align=3Dleft>
<HR tabIndex=3D-1>
<FONT face=3DTahoma size=3D2><B>From:</B>=20
[EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] ] <B>On Behalf Of =
</B>Amaltas=20
Bohra<BR><B>Sent:</B> Tuesday, April 18, 2006 11:28 AM<BR><B>To:</B>=20
[email protected]<BR><B>Subject:</B> [htdig] =
Indexing local=20
website<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV>Hi,<BR>I am new to indexing and htdig. I have two =
questions:<BR>1) When=20
we index local site (htdig and html documents are in the same =
folder),=20
are they fast or do htdig indexes them as if it is a different website =
and=20
all the requests come through internet. I have some =
limits with=20
my webhost and I cannot run a script for more than 180 seconds. <BR>2) =
Do=20
htdig have any configuration settings wherein we can put settings so =
that it=20
can differentiate between local documents and documents on other=20
servers.<BR><BR>Thanks</DIV></BLOCKQUOTE></BODY></HTML>
------_=_NextPart_001_01C662D5.2A14768C--
--__--__--
_______________________________________________
ht://Dig general mailing list: <[email protected]>
ht://Dig FAQ: http://htdig.sourceforge.net/FAQ.html
List information (subscribe/unsubscribe, etc.)
https://lists.sourceforge.net/lists/listinfo/htdig-general
End of htdig-general Digest
