[
https://issues.apache.org/jira/browse/NUTCH-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14338211#comment-14338211
]
Chong Li commented on NUTCH-1950:
---------------------------------
I have thought about that and at first we just wanted every new filename to be
unique.
I tried to save the exact 255 characters and 128 characters as the filename
before and the new url was still not human readable because there were a lot of
random characters in it.. and that is the reason why those filenames are so
long....
I think it is a good idea to save the first 32 characters or just save the
domain name, and then plus a unique key.
Thanks for the advice! I will change my solution！
> File name too long when bin/nutch dump
> --------------------------------------
>
> Key: NUTCH-1950
> URL: https://issues.apache.org/jira/browse/NUTCH-1950
> Project: Nutch
> Issue Type: Bug
> Components: segment
> Affects Versions: 1.10
> Reporter: Chong Li
> Priority: Minor
> Fix For: 1.10
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> When bin/dump in version 1.10-trunk, there will be an exception saying "File
> name too long". When crawling, the length of the url may be longer than 255
> bytes and nutch save the file using the url as file name. It can be saved in
> segments but when dumping the files to local file system, the length of the
> filename can not be longer than 255 bytes.
> The FileDumper.java need to be changed to handle such exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
