[
https://issues.apache.org/jira/browse/HADOOP-12011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14555804#comment-14555804
]
Kai Zheng commented on HADOOP-12011:
------------------------------------
This is a real dump output:
{noformat}
Erasure coder test settings:
numDataUnits=10 numParityUnits=4 chunkSize=513
erasedDataIndexes=[0, 1] erasedParityIndexes=[0, 1] usingDirectBuffer=true
usingFixedData=true
Testing data chunks
0x00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
0x02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11
0x04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13
0x06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15
0x08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17
0x0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19
0x0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B
0x0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D
0x10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F
0x12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F 20 21
Encoded parity chunks
0x08 5B 6A 39 8D DE 2E 7D 03 50 36 65 24 77 77 24
0xAE FD 49 1A 2B 78 06 55 A5 F6 B7 E4 82 D1 5F 0C
0x0A C3 C2 0B 29 E0 35 FC 0F C6 27 EE 24 ED 94 5D
0x85 4C 6E A7 A6 6F 9C 55 80 49 83 4A AB 62 68 A1
Decoding input chunks
0x00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0A C3 C2 0B 29 E0 35 FC 0F C6 27 EE 24 ED 94 5D
0x85 4C 6E A7 A6 6F 9C 55 80 49 83 4A AB 62 68 A1
0x00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13
0x06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15
0x08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17
0x0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19
0x0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B
0x0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D
0x10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F
0x12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F 20 21
Decoded/recovered chunks
0x08 5B 6A 39 8D DE 2E 7D 03 50 36 65 24 77 77 24
0xAE FD 49 1A 2B 78 06 55 A5 F6 B7 E4 82 D1 5F 0C
0x00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
0x02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11
Erasure coder test settings:
numDataUnits=10 numParityUnits=4 chunkSize=496
erasedDataIndexes=[0, 1] erasedParityIndexes=[0, 1] usingDirectBuffer=true
usingFixedData=true
Testing data chunks
0x00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
0x02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11
0x04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13
0x06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15
0x08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17
0x0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19
0x0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B
0x0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D
0x10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F
0x12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F 20 21
Encoded parity chunks
0x08 5B 6A 39 8D DE 2E 7D 03 50 36 65 24 77 77 24
0xAE FD 49 1A 2B 78 06 55 A5 F6 B7 E4 82 D1 5F 0C
0x0A C3 C2 0B 29 E0 35 FC 0F C6 27 EE 24 ED 94 5D
0x85 4C 6E A7 A6 6F 9C 55 80 49 83 4A AB 62 68 A1
Decoding input chunks
0x00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0A C3 C2 0B 29 E0 35 FC 0F C6 27 EE 24 ED 94 5D
0x85 4C 6E A7 A6 6F 9C 55 80 49 83 4A AB 62 68 A1
0x00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13
0x06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15
0x08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17
0x0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19
0x0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B
0x0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D
0x10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F
0x12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F 20 21
Decoded/recovered chunks
0x08 5B 6A 39 8D DE 2E 7D 03 50 36 65 24 77 77 24
0xAE FD 49 1A 2B 78 06 55 A5 F6 B7 E4 82 D1 5F 0C
0x00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
0x02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11
Erasure coder test settings:
numDataUnits=10 numParityUnits=4 chunkSize=529
erasedDataIndexes=[0, 1] erasedParityIndexes=[0, 1] usingDirectBuffer=true
usingFixedData=true
Testing data chunks
0x00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
0x02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11
0x04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13
0x06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15
0x08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17
0x0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19
0x0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B
0x0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D
0x10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F
0x12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F 20 21
Encoding (numAlnumParityUnitslUnits = 10, numDataUnits = 10)
EncodeMatrix:
1 0 0 0 0 0 0 0 0 0 0 1 0 0
0 0 0 0 0 0 0 0 1 0 0 0 0 0
0 0 0 0 0 1 0 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0 0 0 0 0 1
0 0 0 0 0 0 0 0 0 0 1 0 0 0
0 0 0 0 0 0 0 1 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0 0 0 0 0 0
0 1 dd 98 ad 9d 5d 96 3d aa 8e f4 98 dd
9d ad 96 5d aa 3d f4 8e 3d aa 5d 96 ad 9d
dd 98 47 a7 aa 3d 96 5d 9d ad 98 dd a7 47
Recovering (numAllUnits = 14, numDataUnits = 10, numErased = 4)
- ErasedIndexes = 0 1 10 11 - DecodeIndex = 2 3 4 5 6 7 8 9 12 13
EncodeMatrix:
1 0 0 0 0 0 0 0 0 0 0 1 0 0
0 0 0 0 0 0 0 0 1 0 0 0 0 0
0 0 0 0 0 1 0 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0 0 0 0 0 1
0 0 0 0 0 0 0 0 0 0 1 0 0 0
0 0 0 0 0 0 0 1 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0 0 0 0 0 0
0 1 dd 98 ad 9d 5d 96 3d aa 8e f4 98 dd
9d ad 96 5d aa 3d f4 8e 3d aa 5d 96 ad 9d
dd 98 47 a7 aa 3d 96 5d 9d ad 98 dd a7 47
InvertMatrix:
9e 1f 59 bf 6a c5 23 d3 eb b7
1f 9e bf 59 c5 6a d3 23 b7 eb
1 0 0 0 0 0 0 0 0 0
0 1 0 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0 0
0 0 0 1 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0 0
0 0 0 0 0 1 0 0 0 0
0 0 0 0 0 0 1 0 0 0
0 0 0 0 0 0 0 1 0 0
DecodeMatrix:
9e 1f 59 bf 6a c5 23 d3 eb b7 1f 9e bf 59
c5 6a d3 23 b7 eb 32 7a 65 c8 43 75 d0 6b
6a c5 7a 32 c8 65 75 43 6b d0 c5 6a 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
Encoding (numAlnumParityUnitslUnits = 10, numDataUnits = 10)
EncodeMatrix:
1 0 0 0 0 0 0 0 0 0 0 1 0 0
0 0 0 0 0 0 0 0 1 0 0 0 0 0
0 0 0 0 0 1 0 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0 0 0 0 0 1
0 0 0 0 0 0 0 0 0 0 1 0 0 0
0 0 0 0 0 0 0 1 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0 0 0 0 0 0
0 1 dd 98 ad 9d 5d 96 3d aa 8e f4 98 dd
9d ad 96 5d aa 3d f4 8e 3d aa 5d 96 ad 9d
dd 98 47 a7 aa 3d 96 5d 9d ad 98 dd a7 47
Recovering (numAllUnits = 14, numDataUnits = 10, numErased = 4)
- ErasedIndexes = 0 1 10 11 - DecodeIndex = 2 3 4 5 6 7 8 9 12 13
EncodeMatrix:
1 0 0 0 0 0 0 0 0 0 0 1 0 0
0 0 0 0 0 0 0 0 1 0 0 0 0 0
0 0 0 0 0 1 0 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0 0 0 0 0 1
0 0 0 0 0 0 0 0 0 0 1 0 0 0
0 0 0 0 0 0 0 1 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0 0 0 0 0 0
0 1 dd 98 ad 9d 5d 96 3d aa 8e f4 98 dd
9d ad 96 5d aa 3d f4 8e 3d aa 5d 96 ad 9d
dd 98 47 a7 aa 3d 96 5d 9d ad 98 dd a7 47
InvertMatrix:
9e 1f 59 bf 6a c5 23 d3 eb b7
1f 9e bf 59 c5 6a d3 23 b7 eb
1 0 0 0 0 0 0 0 0 0
0 1 0 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0 0
0 0 0 1 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0 0
0 0 0 0 0 1 0 0 0 0
0 0 0 0 0 0 1 0 0 0
0 0 0 0 0 0 0 1 0 0
DecodeMatrix:
9e 1f 59 bf 6a c5 23 d3 eb b7 1f 9e bf 59
c5 6a d3 23 b7 eb 32 7a 65 c8 43 75 d0 6b
6a c5 7a 32 c8 65 75 43 6b d0 c5 6a 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
Encoding (numAlnumParityUnitslUnits = 10, numDataUnits = 10)
EncodeMatrix:
1 0 0 0 0 0 0 0 0 0 0 1 0 0
0 0 0 0 0 0 0 0 1 0 0 0 0 0
0 0 0 0 0 1 0 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0 0 0 0 0 1
0 0 0 0 0 0 0 0 0 0 1 0 0 0
0 0 0 0 0 0 0 1 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0 0 0
Encoded parity chunks
0x08 5B 6A 39 8D DE 2E 7D 03 50 36 65 24 77 77 24
0xAE FD 49 1A 2B 78 06 55 A5 F6 B7 E4 82 D1 5F 0C
0x0A C3 C2 0B 29 E0 35 FC 0F C6 27 EE 24 ED 94 5D
0x85 4C 6E A7 A6 6F 9C 55 80 49 83 4A AB 62 68 A1
Decoding input chunks
0x00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0A C3 C2 0B 29 E0 35 FC 0F C6 27 EE 24 ED 94 5D
0x85 4C 6E A7 A6 6F 9C 55 80 49 83 4A AB 62 68 A1
0x00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13
0x06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15
0x08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17
0x0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19
0x0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B
0x0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D
0x10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F
0x12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F 20 21
Decoded/recovered chunks
0x08 5B 6A 39 8D DE 2E 7D 03 50 36 65 24 77 77 24
0xAE FD 49 1A 2B 78 06 55 A5 F6 B7 E4 82 D1 5F 0C
0x00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
0x02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11
0 0 0
0 1 dd 98 ad 9d 5d 96 3d aa 8e f4 98 dd
9d ad 96 5d aa 3d f4 8e 3d aa 5d 96 ad 9d
dd 98 47 a7 aa 3d 96 5d 9d ad 98 dd a7 47
Process finished with exit code 0
Recovering (numAllUnits = 14, numDataUnits = 10, numErased = 4)
- ErasedIndexes = 0 1 10 11 - DecodeIndex = 2 3 4 5 6 7 8 9 12 13
EncodeMatrix:
1 0 0 0 0 0 0 0 0 0 0 1 0 0
0 0 0 0 0 0 0 0 1 0 0 0 0 0
0 0 0 0 0 1 0 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0 0 0 0 0 1
0 0 0 0 0 0 0 0 0 0 1 0 0 0
0 0 0 0 0 0 0 1 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0 0 0 0 0 0
0 1 dd 98 ad 9d 5d 96 3d aa 8e f4 98 dd
9d ad 96 5d aa 3d f4 8e 3d aa 5d 96 ad 9d
dd 98 47 a7 aa 3d 96 5d 9d ad 98 dd a7 47
InvertMatrix:
9e 1f 59 bf 6a c5 23 d3 eb b7
1f 9e bf 59 c5 6a d3 23 b7 eb
1 0 0 0 0 0 0 0 0 0
0 1 0 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0 0
0 0 0 1 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0 0
0 0 0 0 0 1 0 0 0 0
0 0 0 0 0 0 1 0 0 0
0 0 0 0 0 0 0 1 0 0
DecodeMatrix:
9e 1f 59 bf 6a c5 23 d3 eb b7 1f 9e bf 59
c5 6a d3 23 b7 eb 32 7a 65 c8 43 75 d0 6b
6a c5 7a 32 c8 65 75 43 6b d0 c5 6a 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
{noformat}
> Allow to dump verbose information to ease debugging in raw erasure coders
> -------------------------------------------------------------------------
>
> Key: HADOOP-12011
> URL: https://issues.apache.org/jira/browse/HADOOP-12011
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Kai Zheng
> Assignee: Kai Zheng
> Fix For: HDFS-7285
>
> Attachments: HADOOP-12011-HDFS-7285-v1.patch
>
>
> While working on native erasure coders, it was found useful to dump key
> information like encode/decode matrix, erasures and etc. for the
> encode/decode call.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
