I can give you some reading material, but setting one up from scratch sounds
like it may be outside your current skill set. You can read about it here,
which is somewhat specific to Gentoo Linux:
http://gentoo-wiki.com/HOWTO_Packet_Shaping.
Your best solution might be to go out and get a broadband router that has
traffic shaping, even if most of the other features will never be used. This
way you assign your computers current address to the routers outside interface,
information passes through the routers traffic shaping programs and back out
another port using a new NAT'ed subnet.
It might sound weird, but it will do exactly what your looking for.
----- Original Message ----
From: Harmesh <[EMAIL PROTECTED]>
To: [email protected]
Sent: Wednesday, March 7, 2007 4:14:52 AM
Subject: Re: [SOLVED] memory consumpition by nutch
Hi thanks for the sugestion , but as i am not that much familiar with linux
can u please tell me what is throttling firewall and how to deploy it , more
over i would like to add that i had already change the fetcher.threads.fetch
property to 1 but it was still using the same bandbith.
Sean Dean-3 wrote:
>
> I guess your looking to limit the bandwidth used by the Nutch crawler?
>
> One idea is to set a "very low" thread count for the fetcher, maybe even
> going right down to 1 depending on the effect your looking for. Another
> idea, which would be much more controlled is to setup a throttling
> firewall on the same machine or anywhere in-between it and your gateway.
>
> If you go the thread route, you will need to place the below lines in your
> nutch-site.xml file;
>
> <property>
> <name>fetcher.threads.fetch</name>
> <value>1</value>
> <description>The number of FetcherThreads the fetcher should use.
> This is also determines the maximum number of requests that are
> made at once (each FetcherThread handles one
> connection).</description>
> </property>
>
>
> ----- Original Message ----
> From: Harmesh <[EMAIL PROTECTED]>
> To: [email protected]
> Sent: Wednesday, March 7, 2007 2:01:50 AM
> Subject: memory consumpition by nutch
>
>
> while i am crawling most of the band with is used by nutch crawler is
> there
> any way to optimize it
> --
> View this message in context:
> http://www.nabble.com/memory-consumpition-by-nutch-tf3360441.html#a9347708
> Sent from the Nutch - User mailing list archive at Nabble.com.
>
--
View this message in context:
http://www.nabble.com/memory-consumpition-by-nutch-tf3360441.html#a9349223
Sent from the Nutch - User mailing list archive at Nabble.com.
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
