I ported shell start-up script to standard windows .BAT file (tested in
Windows XP).
Where can i upload it? I need help with testing nutch under native
windows. Should i open bug report and attach .BAT file to it?
> in a nutshell you can't use Ivy or Maven for the Gora dependency,
which is why we are currently stuck with the trunk and can't compile it
without first downloading and compiling GORA locally.
i compiled gora-*-0.2-incubating.jars locally. Where should i put them
to get nutch trunk compiled?
I agree. Nuke crawl command
-1
I don't want to mark release 2.0 as unmaintained. Cassandra backend
works really well for us and fixed performance problems with hadoop
database. Instead of moving it out trunk, recruit more ppl should come
and fix open problems. don't give up.
> I'm glad to hear that there at least 2 people in the community that
do business in their field and proudly use a Nutch-based crawler
together with
> Cassandra to store the data through Gora. That would not have been
possible with Nutch 1.x version.
what about to drop Gora, because it is progr
> The nutchgora branch will still be there, and if there's a desire to
have a nutchcassandra or nutchhbase pure branch, and you have some spare
cycles to help see it come about, we would welcome it.
it needs to be done in more long term strategic way.
1. research what ppl expect from Nutch 2?
can you add NUTCH-1098 to 1.4?
can you package 1.4 with updated hadoop jars? i have problems with
running nutch in local mode. If i run multiple tasks at once, they
delete each other temporary files. Its worth a try if newer hadoop libs
will fix that.
I have problems with running injector in nutch-1.4 on hadoop, same
command with nutch-1.3 works fine. As you can see, list of URLs is
loaded from hdfs correctly Map input records=66906 but no records are on
map ouput. Could it be some problems with broken filtering?
ponto:(crawler)runtime/depl
Let me know if anybody got injector to work in 1.4 branch i have Hadoop
0.20.204.0 and cant make it to insert single url
Environment: Windows XP Home
Reporter: Radim Kolar
Priority: Minor
Its possible to run Nutch in windows without cygwin.
1. Startup script needs to be ported from SH to BAT
2. Because hadoop runs on unix only, we must emulate unix commands to make it
work. Luckily only chmod, bash
[
https://issues.apache.org/jira/browse/NUTCH-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radim Kolar updated NUTCH-1070:
---
Attachment: chmod.c
bash.c
nutch.bat
> Run nutch under nat
[
https://issues.apache.org/jira/browse/NUTCH-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088443#comment-13088443
]
Radim Kolar commented on NUTCH-990:
---
I have this problem too protocol-httpclient f
Environment: Any
Reporter: Radim Kolar
Fix For: 1.4
Attachments: urlnormalizer.patch
Basic URL normalizer lacks 2 important features
Encode space in URL into %20 to unbreak httpclient and possibly others who do
not expect space inside URL
Ability to decode %33 encoding in
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radim Kolar updated NUTCH-1098:
---
Attachment: urlnormalizer.patch
Patch against branch-1.4
> better url-normalizer ba
[
https://issues.apache.org/jira/browse/NUTCH-937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13091740#comment-13091740
]
Radim Kolar commented on NUTCH-937:
---
we should stick with hadoop 0.20.203.0 not CDH
[
https://issues.apache.org/jira/browse/NUTCH-937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13092059#comment-13092059
]
Radim Kolar commented on NUTCH-937:
---
nutch-1.4 contains hadoop-core 0.20.2. If nutch
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radim Kolar updated NUTCH-1098:
---
Attachment: nutch.diff
Updated patch. It also normalizes unprintable % sequences to upper case. Like
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radim Kolar updated NUTCH-1098:
---
Attachment: (was: urlnormalizer.patch)
> better url-normalizer ba
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120912#comment-13120912
]
Radim Kolar commented on NUTCH-1098:
1. Some servers sends spaces in URLs
2. Base
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120966#comment-13120966
]
Radim Kolar commented on NUTCH-1098:
Actually it might be even better to
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126594#comment-13126594
]
Radim Kolar commented on NUTCH-1098:
Patch is good. i will add replace high bit c
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127440#comment-13127440
]
Radim Kolar commented on NUTCH-1098:
I did, but due to lack of time to test
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128264#comment-13128264
]
Radim Kolar commented on NUTCH-1098:
Browsers seems to send spaces in URL enc
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radim Kolar updated NUTCH-1098:
---
Attachment: patch-urlnormalizer.diff
> better url-normalizer ba
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radim Kolar updated NUTCH-1098:
---
Attachment: (was: patch-urlnormalizer.diff)
> better url-normalizer ba
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radim Kolar updated NUTCH-1098:
---
Attachment: (was: nutch.diff)
> better url-normalizer ba
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radim Kolar updated NUTCH-1098:
---
Attachment: patch-urlnormalizer.diff
Do not decode # and / characters during %XX decoding. Unit
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141233#comment-13141233
]
Radim Kolar commented on NUTCH-1098:
1. patch --ignore-whitespace
2. only s
[
https://issues.apache.org/jira/browse/NUTCH-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142544#comment-13142544
]
Radim Kolar commented on NUTCH-1194:
locking should be done in setup/cleanup
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142699#comment-13142699
]
Radim Kolar commented on NUTCH-1098:
a/ Please direct your complains about qualit
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radim Kolar updated NUTCH-1098:
---
Attachment: (was: patch-urlnormalizer.diff)
> better url-normalizer ba
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radim Kolar updated NUTCH-1098:
---
Attachment: patch-with-utf8-encoding.diff
Added support for encoding string to UTF-8 and then URL
[
https://issues.apache.org/jira/browse/NUTCH-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radim Kolar updated NUTCH-1194:
---
Comment: was deleted
(was: locking should be done in setup/cleanup task. Currently if you kill
[
https://issues.apache.org/jira/browse/NUTCH-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radim Kolar updated NUTCH-1070:
---
Attachment: (was: nutch.bat)
> Run nutch under native windows (no cyg
[
https://issues.apache.org/jira/browse/NUTCH-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radim Kolar resolved NUTCH-1070.
Resolution: Won't Fix
> Run nutch under native windows (n
[
https://issues.apache.org/jira/browse/NUTCH-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radim Kolar updated NUTCH-1070:
---
Attachment: (was: bash.c)
> Run nutch under native windows (no cyg
[
https://issues.apache.org/jira/browse/NUTCH-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radim Kolar updated NUTCH-1070:
---
Attachment: (was: chmod.c)
> Run nutch under native windows (no cyg
[
https://issues.apache.org/jira/browse/NUTCH-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13143984#comment-13143984
]
Radim Kolar commented on NUTCH-1070:
i closed it because i removed my patches, i
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radim Kolar updated NUTCH-1098:
---
Attachment: (was: patch-with-utf8-encoding.diff)
> better url-normalizer ba
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radim Kolar resolved NUTCH-1098.
Resolution: Invalid
Attached patch was in improper format.
> better
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144020#comment-13144020
]
Radim Kolar commented on NUTCH-1098:
By removing my patch i also withdraw permis
[
https://issues.apache.org/jira/browse/NUTCH-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radim Kolar closed NUTCH-1070.
--
> Run nutch under native windows (no cyg
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144160#comment-13144160
]
Radim Kolar commented on NUTCH-1098:
If you are so clever and hard working then
[
https://issues.apache.org/jira/browse/NUTCH-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144183#comment-13144183
]
Radim Kolar commented on NUTCH-1098:
Remove my patch from this ticket. I
45 matches
Mail list logo
