[
https://issues.apache.org/jira/browse/HADOOP-11887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14626033#comment-14626033
]
Kai Zheng commented on HADOOP-11887:
------------------------------------
Hi [~cmccabe],
I have some understanding about the following two major questions and need to
discuss with you further.
bq. I can see you're using -Drequire.erasurecode, -Derasurecode.prefix, etc.
for the build parameters. This might create confusion, though. Users might ask
if erasure encoding is disabled if they build without the -Derasurecode...
options. Given that the name of the library is ISA-L, maybe we can have
-Drequire.ISAL, -DISAL.prefix, etc.? Seems more straightforward.
I totally got your concern. I'm using the general name {{erasurecode}} instead
of {{isal}} directly because I wish the overall work won't couple with ISA-L
too tightly. In future other native libraries like Jerasure or even hardware
based ones could also be supported as well without too much change. I'm
thinking that the native APIs defined in {{erasure_code.h}} should be general
enough so other native libraries could also be easily mapped to it, thus when
building, other libraries could also be passed to via the mentioned options.
Note {{require.erasurecode}} is used to enable it, and if enabled,
{{erasurecode.prefix}} should be specified to provide the library place; If not
enabled (by default for now), the building should go as normally, and the
result won't contain any erasure code related symbols. The logic is similar to
existing codes like for snappy library.
bq. I think we should avoid adding a new test program for this, and instead add
this functionality to checknative.
You found another place I need to change. Yes I need to add an entry for
erasrue code in the tool. The question here is, I'm wondering if it can serve
the purpose of the new tool here, because executing of {{hadoop checknative}}
may need some configuration or tweak to make it work, the new tool can directly
run just after it's out, so can be used in maven unit tests cleanly. I
understand introducing a new tool just for ONE native test may be too heavy, if
you agree, maybe we could go simple, say, if no native library is available,
the native test program could just exit with a warning message? Do we need more
native tests anyway in future? If so the checking with the new tool may sound
more reasonable.
Would you let know your thoughts? Thanks again.
> Introduce Intel ISA-L erasure coding library for the native support
> -------------------------------------------------------------------
>
> Key: HADOOP-11887
> URL: https://issues.apache.org/jira/browse/HADOOP-11887
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: io
> Reporter: Kai Zheng
> Assignee: Kai Zheng
> Attachments: HADOOP-11887-v1.patch, HADOOP-11887-v2.patch,
> HADOOP-11887-v3.patch, HADOOP-11887-v4.patch
>
>
> This is to introduce Intel ISA-L erasure coding library for the native
> support, via dynamic loading mechanism (dynamic module, like *.so in *nix and
> *.dll on Windows).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
