Re: [Nutch-general] Compression

2007-06-03 Thread Emmanuel JOKE
Thanks but it seems it doesn't work. My data still use 7Go of space on my disk. My command line is: /usr/local/java/bin/java -Xmx512m -Dhadoop.log.dir=/data/sengine/logs - Dhadoop.log.file=hadoop.log - Djava.library.path=/data/sengine/lib/native/Linux-i386-32 -classpath

Re: [Nutch-general] Compression

2007-06-03 Thread Doğacan Güney
Hi, On 6/3/07, Emmanuel JOKE [EMAIL PROTECTED] wrote: Thanks but it seems it doesn't work. My data still use 7Go of space on my disk. My command line is: /usr/local/java/bin/java -Xmx512m -Dhadoop.log.dir=/data/sengine/logs - Dhadoop.log.file=hadoop.log -

Re: [Nutch-general] Compression

2007-06-03 Thread Emmanuel JOKE
yes i did. property nameio.seqfile.compression.type/name valueBLOCK/value /property Any clues ? Hi, On 6/3/07, Emmanuel JOKE [EMAIL PROTECTED] wrote: Thanks but it seems it doesn't work. My data still use 7Go of space on my disk. My command line is: /usr/local/java/bin/java -Xmx512m

Re: [Nutch-general] Compression

2007-06-03 Thread Doğacan Güney
On 6/3/07, Emmanuel JOKE [EMAIL PROTECTED] wrote: yes i did. property nameio.seqfile.compression.type/name valueBLOCK/value /property That's strange. I may be wrong but I don't think this is a native-lib problem. Even if you don't have native libs in place, hadoop will fall back to java

[Nutch-general] Is fetcher.throttle.bandwidth known to work?

2007-06-03 Thread Enzo Michelangeli
In my case (with Nutch 0.8), it seems not: I set it to 500, and the fetcher still saturates the 1.5 Mbit/s link... Is it supposed to work for the total bandwidth, or for each thread? Enzo - This SF.net email is sponsored

Re: [Nutch-general] Is fetcher.throttle.bandwidth known to work?

2007-06-03 Thread Andrzej Bialecki
Enzo Michelangeli wrote: In my case (with Nutch 0.8), it seems not: I set it to 500, and the fetcher still saturates the 1.5 Mbit/s link... Is it supposed to work for the total bandwidth, or for each thread? There's nothing in the current code base to support this, neither there is a config

Re: [Nutch-general] Error with the inject command

2007-06-03 Thread Mathijs Homminga
It looks like the plugins cannot be found I'm getting the same errors as you when I delete the plugins/ subdirectory. Is yours still there? Berlin Brown wrote: Anybody? Still cant figure it out. I even created the crawl/crawldb directory. Nothing. The URLS are just a set of URLS. I am

[Nutch-general] Job Opportunity for Developers, Distributed Java at Fredhopper (Amsterdam)

2007-06-03 Thread Vasil Kokareshkov
TOP SOFTWARE PRODUCT COMPANY OFFERS TALENTED SOFTWARE DEVELOPERS A UNIQUE CHANCE TO TACKLE SCALABILTY AND DISTRIBUTION CHALLENGES Company: Fredhopper, Location: Amsterdam, Position: Full-time Do you have extensive programming experience? Do you like working with cutting-edge technologies? Do

[Nutch-general] Job Opportunity for Developers, Distributed Java at Fredhopper (Amsterdam)

2007-06-03 Thread Vasil Kokareshkov
TOP SOFTWARE PRODUCT COMPANY OFFERS TALENTED SOFTWARE DEVELOPERS A UNIQUE CHANCE TO TACKLE SCALABILTY AND DISTRIBUTION CHALLENGES Company: Fredhopper, Location: Amsterdam, Position: Full-time Do you have extensive programming experience? Do you like working with cutting-edge technologies? Do

Re: [Nutch-general] Is fetcher.throttle.bandwidth known to work?

2007-06-03 Thread Enzo Michelangeli
- Original Message - From: Andrzej Bialecki [EMAIL PROTECTED] Sent: Monday, June 04, 2007 1:31 AM Enzo Michelangeli wrote: In my case (with Nutch 0.8), it seems not: I set it to 500, and the fetcher still saturates the 1.5 Mbit/s link... Is it supposed to work for the total

[Nutch-general] How to enable followRedirects?

2007-06-03 Thread Manoharam Reddy
I get this error message for many URLs. Is there any property to enable redirect requests to be allowed? 2007-06-04 10:00:42,298 INFO httpclient.HttpMethodDirector - Redirect requested but followRedirects is disabled - This

Re: [Nutch-general] How to enable followRedirects?

2007-06-03 Thread Doğacan Güney
On 6/4/07, Manoharam Reddy [EMAIL PROTECTED] wrote: I get this error message for many URLs. Is there any property to enable redirect requests to be allowed? 2007-06-04 10:00:42,298 INFO httpclient.HttpMethodDirector - Redirect requested but followRedirects is disabled This is not a