[
https://issues.apache.org/jira/browse/HADOOP-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112029#comment-16112029
]
Allen Wittenauer commented on HADOOP-14498:
-------------------------------------------
bq. I guess that's what the comment means by "different syntaxes".
Yeah. In the beginning I was trying really really really hard to avoid arrays,
for lots of reasons. One of the big ones that I'm willing to write down was
that not all of the array functions are available in our target bash 3.2.
(e.g., associative arrays, mapfile, etc). Plus backward compatibility with
the raw string format of HADOOP_OPTS + trying to solve the duplicate parameter
problem led to add_param. It's not pretty and I'm not proud of it. But it
works.
Sidenote:
HADOOP_OPTS is probably at this point the biggest hindsight-20-20 mistake in
Hadoop. I don't think people really understand how much of an impact it's had
on literally everything in the system. For example, it's *the* reason that
spaces in file paths are a complete nightmare. HADOOP-13365 is my attempt at
fixing it. I'm not sure if it makes it worse or better though.
bq. It'd be nice to replace that inline loop with a join function.
Yeah, I'd love for someone to take another whack at it. I can't remember what
all I tried before I ended up just settling on the loop. I seem to recall I
had a better way, but it only worked with bash 4.x. I guess we could always
put a version check in there. (There's one or two other places like that
already.)
bq. Or should I just throw it in hadoop-functions.sh?
I've just been throwing everything into hadoop-functions.sh, as it ends up
creating one big API doc at mvn site time. Pretty convenient.
> HADOOP_OPTIONAL_TOOLS not parsed correctly
> ------------------------------------------
>
> Key: HADOOP-14498
> URL: https://issues.apache.org/jira/browse/HADOOP-14498
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 3.0.0-alpha1
> Reporter: Mingliang Liu
> Assignee: Sean Mackrory
> Priority: Critical
> Attachments: HADOOP-14498.001.patch, HADOOP-14498.002.patch
>
>
> # This will make hadoop-azure not show up in the hadoop classpath, though
> both hadoop-aws and hadoop-azure-datalake are in the
> classpath.{code:title=hadoop-env.sh}
> export HADOOP_OPTIONAL_TOOLS="hadoop-azure,hadoop-aws,hadoop-azure-datalake"
> {code}
> # And if we put only hadoop-azure and hadoop-aws, both of them are shown in
> the classpath.
> {code:title=hadoop-env.sh}
> export HADOOP_OPTIONAL_TOOLS="hadoop-azure,hadoop-aws"
> {code}
> This makes me guess that, while parsing the {{HADOOP_OPTIONAL_TOOLS}}, we
> make some assumptions that hadoop tool modules have a single "-" in names,
> and the _hadoop-azure-datalake_ overrides the _hadoop-azure_. Or any other
> assumptions about the {{${project.artifactId\}}}?
> Ping [~aw].
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
