The problem is that I only want sites, which are physically in Latvia to be
indexed. DisallowNoMatch, that filters *.lv works just fine, however there
is quite a lot of sites which are registered under .lv domain which are not
physically in Latvia, therefore I have to set up a list of subnets to index
via countries file.
What do you mean by checking at least one site from specified range? Well, I
have one line, which is 159.148.0.0-159.148.255.255 LV
However, when running index I see it adding URL, which is *.lv, but does not
belong to specified IP range (e.g. is 194.192.x.x.)
Now, what I just did:
Still have FollowOutside on
Removed DisallowNoMatch entry from aspseek.conf
Still have Countries /tmp/lv
Still have AllowCountries LV
/tmp/lv is:
159.148.0.0-159.148.255.255 LV
I run 1 instance of index with Server http://www.latnet.lv, which is
159.148.9.90 and see:
....
Adding URL: http://u.extreme-dm.com/robots.txt which is 213.244.183.213, so
I make a conclusion that Countries does not work, or it does not load
/tmp/lv file
N.
-----Original Message-----
From: Alexander F Avdonkin [mailto:[EMAIL PROTECTED]]
Sent: Monday, August 06, 2001 18:44
To: [EMAIL PROTECTED]
Subject: Re: [aseek-users] countries
Nick Ustinov �����(�):
> Well, nothing changes. Isn't it supposed to load countries information
upon
> startup of index from /tmp/lv and store that in countries table for
further
> use? countries table is still empty. Maybe there's something wrong with
the
> file format I use? Can anyone give an example for working countries file?
>
"countries" table is not used now. Don't see on it.
Is your problem that only sites from lv domain are indexed ?
Can you check at least one site with IP address from specified range(s) ?
>
> N.
>
> -----Original Message-----
> From: Alexander F Avdonkin [mailto:[EMAIL PROTECTED]]
> Sent: Monday, August 06, 2001 18:04
> To: Nick Ustinov; [EMAIL PROTECTED]
> Subject: Re: [aseek-users] countries
>
> Nick Ustinov �����(�):
>
> > Hi
> >
> > Maybe you can give me some advice here -- I want to index all .lv sites.
> > There is map of .lv IP addresses available at
http://www.nic.lv/local.net
> > The IPs are masked like xxx.xxx.xxx.xxx/mmm
> >
> > as for aspseek.conf, I have
> >
> > DisallowNoMatch ://[^/]*\.lv/
> > Countries /tmp/lv
> > AllowCountries LV
> >
>
> You should put AllowCountries before DisallowNoMatch, otherwise
> AllowCountries
> will never be reached for hosts from domains other than lv.
>
> >
> > as for /tmp/lv, then it's just one single line for test purposes:
> > 159.148.0.0-159.148.255.255 LV
> >
> > I've about 40 Servers described in aseek.conf; FollowOutside is enabled.
I
> > run index -N 75
> > The system is RH7, 256Mb RAM
> >
> > Would increasing number of threads speed up the process? mysql key
buffer
> > size is set to 64M
> >
> > Nick
> >
> > -----Original Message-----
> > From: Alexander F Avdonkin [mailto:[EMAIL PROTECTED]]
> > Sent: Monday, August 06, 2001 17:39
> > To: [EMAIL PROTECTED]
> > Subject: Re: [aseek-users] countries
> >
> > Nick Ustinov �����(�):
> >
> > > Concerning countries directive -- it doesn't seem to work at all.
Here's
> > > what I do:
> > >
> > > in aspseek.conf:
> > > Countries /tmp/lv
> > > AllowCountries LV
> > >
>
> > > in /tmp/lv:
> > > 159.148.0.0-159.148.255.255 LV
> > >
> > > running index doesn't even update countries table. what am I doing
> wrong?
> > >
> >
> > "countries" table is not updated.
> > Purpose of "AllowCountries" is to enable indexing of hosts whose IP
> > addresses fall in specified range(s).
> > Order of AllowCountries and Disallow(NoMatch) is also important.
> > Tell in details, which hosts fo you want to index and send your config
> file.
> >
> > >
> > > //Nick
