There is also some discussion on that JIRA considering a checksum strategy
independent of block size. I don't think anything was ever implemented
though, and there would be some drawbacks to that approach. Sorry if this
caused confusion.
--Chris Nauroth
On 5/24/16, 9:55 AM, "Dmitry Sivachenk
> On 24 May 2016, at 19:53, Chris Nauroth wrote:
>
> Hello Dmitry,
>
> To clarify, the intent of MAPREDUCE-5065 was to message the user that
> using different block sizes on source and destination might cause a
> failure to checksum mismatch. The message to the user recommends either
> the -pb
Hello Dmitry,
To clarify, the intent of MAPREDUCE-5065 was to message the user that
using different block sizes on source and destination might cause a
failure to checksum mismatch. The message to the user recommends either
the -pb (preserve block size) or -skipCrc (skip checksum validation) as
p
> On 21 May 2016, at 09:34, Dmitry Sivachenko wrote:
>
>
>> On 21 May 2016, at 02:15, Chris Nauroth wrote:
>>
>> Hello Dmitry,
>>
>> MAPREDUCE-5065 has been included in these branches for a long time. Are
>> you certain that you passed a dfs.blocksize equal to what was used in the
>> source
> On 21 May 2016, at 02:15, Chris Nauroth wrote:
>
> Hello Dmitry,
>
> MAPREDUCE-5065 has been included in these branches for a long time. Are
> you certain that you passed a dfs.blocksize equal to what was used in the
> source files? Did all source files use the same block size?
>
No, I a
Hello Dmitry,
MAPREDUCE-5065 has been included in these branches for a long time. Are
you certain that you passed a dfs.blocksize equal to what was used in the
source files? Did all source files use the same block size?
--Chris Nauroth
On 5/20/16, 3:30 PM, "Dmitry Sivachenko" wrote:
>Hell
Hello,
When I copy files with distcp and -D dfs.blocksize=XXX (hadoop-2.7.2), it fails
with
"Source and target differ in block-size" error despite MAPREDUCE-5065 was
committed 3 years ago.
Is it possible to merge this change to 2.7 / 2.8 branches?
Thanks.
-
