[jira] [Commented] (HDDS-280) Support ozone dist-start-stitching on openbsd/osx
[
https://issues.apache.org/jira/browse/HDDS-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598142#comment-16598142
]
Hudson commented on HDDS-280:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14855 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/14855/])
HDDS-280. Support ozone dist-start-stitching on openbsd/osx. Contributed
(msingh: rev 692736f7cfb72b8932dc2eb4f4faa995dc6521f8)
* (edit) hadoop-ozone/common/pom.xml
* (edit) dev-support/bin/ozone-dist-layout-stitching
* (edit) hadoop-ozone/acceptance-test/pom.xml
* (edit) hadoop-ozone/acceptance-test/dev-support/bin/robot-all.sh
* (edit) hadoop-ozone/acceptance-test/dev-support/bin/robot.sh
* (edit) hadoop-ozone/docs/content/GettingStarted.md
* (edit) hadoop-ozone/acceptance-test/src/test/acceptance/commonlib.robot
* (edit) hadoop-ozone/pom.xml
* (edit)
hadoop-ozone/acceptance-test/src/test/acceptance/basic/ozone-shell.robot
* (edit) dev-support/bin/ozone-dist-tar-stitching
* (edit) hadoop-ozone/acceptance-test/dev-support/bin/robot-dnd-all.sh
> Support ozone dist-start-stitching on openbsd/osx
> -
>
> Key: HDDS-280
> URL: https://issues.apache.org/jira/browse/HDDS-280
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
> Fix For: 0.2.1
>
> Attachments: HDDS-280.001.patch, HDDS-280.002.patch,
> HDDS-280.003.patch
>
>
> {quote}Ozone is creating a symlink during the dist process.
> Using the "ozone" directory as a destination name all the docker-based
> acceptance tests and docker-compose files are more simple as they don't need
> to have the version information in the path.
> But to keep the version specific folder name in the tar file we create a
> symbolic link during the tar creation. With the symbolic link and the
> '–dereference' tar argument we can create the tar file which includes a
> versioned directory (ozone-0.2.1) but we can use the a dist directory without
> the version in the name (hadoop-dist/target/ozone).
> {quote}
> This is the description of the current
> dev-support/bin/ozone-dist-tar-stitching. [~aw] in a comment for HDDS-276
> pointed to the problem that some bsd variants don't support the dereference
> command line options of the ln command.
> The main reason to use this approach is to get a simplified destination name
> without the version (hadoop-dist/target/ozone instead of
> hadoop-dist/target/ozone-0.2.1). It simplifies the docker-compose based
> environments and acceptance tests, therefore I prefer to keep the simplified
> destination name.
> The issue is the tar file creation, if and only if we need the version number
> in the name of the root directory inside of the tar.
> Possible solutions:
> # Use cp target/ozone target/ozone-0.2.1 + tar. It's simple but more slow
> and requires more space.
> # Do the tar distribution from docker all the time in case of 'dereference'
> is not supported. Not very convenient
> # Accept that tar will contain ozone directory and not ozone-0.2.1. This is
> the more simple and can be improved with an additional VERSION file in the
> root of the distribution.
> # (+1) Use hadoop-dist/target/ozone-0.2.1 instead of
> hadoop-dist/target/ozone. This is more complex for the docker based testing
> as we need the explicit names in the compose files (volume:
> ../../../hadoop-dist/target/ozone-0.2.1). The structure is more complex with
> using version in the directory name.
> Please comment your preference.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-280) Support ozone dist-start-stitching on openbsd/osx
[
https://issues.apache.org/jira/browse/HDDS-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596761#comment-16596761
]
Mukul Kumar Singh commented on HDDS-280:
Thanks for working on this [~elek]. The patch looks good to me as well.
+1 for the v3 patch, there is a minor nitpick in acceptance-test/pom.xml:47,
the indentation is off by 1.
I will handle this during commit.
> Support ozone dist-start-stitching on openbsd/osx
> -
>
> Key: HDDS-280
> URL: https://issues.apache.org/jira/browse/HDDS-280
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
> Fix For: 0.2.1
>
> Attachments: HDDS-280.001.patch, HDDS-280.002.patch,
> HDDS-280.003.patch
>
>
> {quote}Ozone is creating a symlink during the dist process.
> Using the "ozone" directory as a destination name all the docker-based
> acceptance tests and docker-compose files are more simple as they don't need
> to have the version information in the path.
> But to keep the version specific folder name in the tar file we create a
> symbolic link during the tar creation. With the symbolic link and the
> '–dereference' tar argument we can create the tar file which includes a
> versioned directory (ozone-0.2.1) but we can use the a dist directory without
> the version in the name (hadoop-dist/target/ozone).
> {quote}
> This is the description of the current
> dev-support/bin/ozone-dist-tar-stitching. [~aw] in a comment for HDDS-276
> pointed to the problem that some bsd variants don't support the dereference
> command line options of the ln command.
> The main reason to use this approach is to get a simplified destination name
> without the version (hadoop-dist/target/ozone instead of
> hadoop-dist/target/ozone-0.2.1). It simplifies the docker-compose based
> environments and acceptance tests, therefore I prefer to keep the simplified
> destination name.
> The issue is the tar file creation, if and only if we need the version number
> in the name of the root directory inside of the tar.
> Possible solutions:
> # Use cp target/ozone target/ozone-0.2.1 + tar. It's simple but more slow
> and requires more space.
> # Do the tar distribution from docker all the time in case of 'dereference'
> is not supported. Not very convenient
> # Accept that tar will contain ozone directory and not ozone-0.2.1. This is
> the more simple and can be improved with an additional VERSION file in the
> root of the distribution.
> # (+1) Use hadoop-dist/target/ozone-0.2.1 instead of
> hadoop-dist/target/ozone. This is more complex for the docker based testing
> as we need the explicit names in the compose files (volume:
> ../../../hadoop-dist/target/ozone-0.2.1). The structure is more complex with
> using version in the directory name.
> Please comment your preference.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-280) Support ozone dist-start-stitching on openbsd/osx
[
https://issues.apache.org/jira/browse/HDDS-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596032#comment-16596032
]
Anu Engineer commented on HDDS-280:
---
[~aw],[~elek] I will commit this shortly. Thanks for the reviews and
contribution.
> Support ozone dist-start-stitching on openbsd/osx
> -
>
> Key: HDDS-280
> URL: https://issues.apache.org/jira/browse/HDDS-280
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
> Fix For: 0.2.1
>
> Attachments: HDDS-280.001.patch, HDDS-280.002.patch
>
>
> {quote}Ozone is creating a symlink during the dist process.
> Using the "ozone" directory as a destination name all the docker-based
> acceptance tests and docker-compose files are more simple as they don't need
> to have the version information in the path.
> But to keep the version specific folder name in the tar file we create a
> symbolic link during the tar creation. With the symbolic link and the
> '–dereference' tar argument we can create the tar file which includes a
> versioned directory (ozone-0.2.1) but we can use the a dist directory without
> the version in the name (hadoop-dist/target/ozone).
> {quote}
> This is the description of the current
> dev-support/bin/ozone-dist-tar-stitching. [~aw] in a comment for HDDS-276
> pointed to the problem that some bsd variants don't support the dereference
> command line options of the ln command.
> The main reason to use this approach is to get a simplified destination name
> without the version (hadoop-dist/target/ozone instead of
> hadoop-dist/target/ozone-0.2.1). It simplifies the docker-compose based
> environments and acceptance tests, therefore I prefer to keep the simplified
> destination name.
> The issue is the tar file creation, if and only if we need the version number
> in the name of the root directory inside of the tar.
> Possible solutions:
> # Use cp target/ozone target/ozone-0.2.1 + tar. It's simple but more slow
> and requires more space.
> # Do the tar distribution from docker all the time in case of 'dereference'
> is not supported. Not very convenient
> # Accept that tar will contain ozone directory and not ozone-0.2.1. This is
> the more simple and can be improved with an additional VERSION file in the
> root of the distribution.
> # (+1) Use hadoop-dist/target/ozone-0.2.1 instead of
> hadoop-dist/target/ozone. This is more complex for the docker based testing
> as we need the explicit names in the compose files (volume:
> ../../../hadoop-dist/target/ozone-0.2.1). The structure is more complex with
> using version in the directory name.
> Please comment your preference.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-280) Support ozone dist-start-stitching on openbsd/osx
[
https://issues.apache.org/jira/browse/HDDS-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16595563#comment-16595563
]
Anu Engineer commented on HDDS-280:
---
+1,[~aw] if you have no further comments, I would like to commit this. I will
wait for the end of the day before committing this patch.
> Support ozone dist-start-stitching on openbsd/osx
> -
>
> Key: HDDS-280
> URL: https://issues.apache.org/jira/browse/HDDS-280
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
> Fix For: 0.2.1
>
> Attachments: HDDS-280.001.patch, HDDS-280.002.patch
>
>
> {quote}Ozone is creating a symlink during the dist process.
> Using the "ozone" directory as a destination name all the docker-based
> acceptance tests and docker-compose files are more simple as they don't need
> to have the version information in the path.
> But to keep the version specific folder name in the tar file we create a
> symbolic link during the tar creation. With the symbolic link and the
> '–dereference' tar argument we can create the tar file which includes a
> versioned directory (ozone-0.2.1) but we can use the a dist directory without
> the version in the name (hadoop-dist/target/ozone).
> {quote}
> This is the description of the current
> dev-support/bin/ozone-dist-tar-stitching. [~aw] in a comment for HDDS-276
> pointed to the problem that some bsd variants don't support the dereference
> command line options of the ln command.
> The main reason to use this approach is to get a simplified destination name
> without the version (hadoop-dist/target/ozone instead of
> hadoop-dist/target/ozone-0.2.1). It simplifies the docker-compose based
> environments and acceptance tests, therefore I prefer to keep the simplified
> destination name.
> The issue is the tar file creation, if and only if we need the version number
> in the name of the root directory inside of the tar.
> Possible solutions:
> # Use cp target/ozone target/ozone-0.2.1 + tar. It's simple but more slow
> and requires more space.
> # Do the tar distribution from docker all the time in case of 'dereference'
> is not supported. Not very convenient
> # Accept that tar will contain ozone directory and not ozone-0.2.1. This is
> the more simple and can be improved with an additional VERSION file in the
> root of the distribution.
> # (+1) Use hadoop-dist/target/ozone-0.2.1 instead of
> hadoop-dist/target/ozone. This is more complex for the docker based testing
> as we need the explicit names in the compose files (volume:
> ../../../hadoop-dist/target/ozone-0.2.1). The structure is more complex with
> using version in the directory name.
> Please comment your preference.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-280) Support ozone dist-start-stitching on openbsd/osx
[
https://issues.apache.org/jira/browse/HDDS-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16595447#comment-16595447
]
genericqa commented on HDDS-280:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 15m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 42s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 18m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
1s{color} | {color:green} The patch generated 0 new + 0 unchanged - 6 fixed =
0 total (was 6) {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
12s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 10s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 19m
31s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
46s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 99m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | HDDS-280 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12937477/HDDS-280.002.patch |
| Optional Tests | asflicense shellcheck shelldocs mvnsite unit |
| uname | Linux 23e4ad73ac65 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / df21e1b |
| maven | version: Apache Maven 3.3.9 |
| shellcheck | v0.4.6 |
| Test Results |
https://builds.apache.org/job/PreCommit-HDDS-Build/880/testReport/ |
| Max. process+thread count | 334 (vs. ulimit of 1) |
| modules | C: hadoop-ozone/acceptance-test . hadoop-ozone/docs U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HDDS-Build/880/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Support ozone dist-start-stitching on openbsd/osx
> -
>
> Key: HDDS-280
> URL: https://issues.apache.org/jira/browse/HDDS-280
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
> Fix For: 0.2.1
>
> Attachments: HDDS-280.001.patch, HDDS-280.002.patch
>
>
> {quote}Ozone is creating a symlink during the dist process.
> Using the "ozone" directory as a destination name all the docker-based
> acceptance tests and docker-compose files are more simple as they don't need
> to have the version information in the path.
> But to keep the version specific folder name in the tar file we create a
> symbolic link during the tar creation. With the symbolic link and the
> '–dereference' tar argument we can create the tar file which inc
[jira] [Commented] (HDDS-280) Support ozone dist-start-stitching on openbsd/osx
[
https://issues.apache.org/jira/browse/HDDS-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16591824#comment-16591824
]
genericqa commented on HDDS-280:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
28s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 15m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 57s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 18m
22s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red} 0m
1s{color} | {color:red} The patch generated 10 new + 3 unchanged - 1 fixed = 13
total (was 4) {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
12s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 46s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 23m
37s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
49s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}108m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | HDDS-280 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12937006/HDDS-280.001.patch |
| Optional Tests | asflicense shellcheck shelldocs mvnsite unit |
| uname | Linux cc2c9975eb40 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 138b0c1 |
| maven | version: Apache Maven 3.3.9 |
| shellcheck | v0.4.6 |
| shellcheck |
https://builds.apache.org/job/PreCommit-HDDS-Build/836/artifact/out/diff-patch-shellcheck.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDDS-Build/836/testReport/ |
| Max. process+thread count | 302 (vs. ulimit of 1) |
| modules | C: hadoop-ozone/acceptance-test . hadoop-ozone/docs U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HDDS-Build/836/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Support ozone dist-start-stitching on openbsd/osx
> -
>
> Key: HDDS-280
> URL: https://issues.apache.org/jira/browse/HDDS-280
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
> Attachments: HDDS-280.001.patch
>
>
> {quote}Ozone is creating a symlink during the dist process.
> Using the "ozone" directory as a destination name all the docker-based
> acceptance tests and docker-compose files are more simple as they don't need
> to have the version information in the path.
> But to keep the version specific folder name in the tar file we create a
> symbolic link during the tar creation. With the symbolic link and the
> '–derefer
[jira] [Commented] (HDDS-280) Support ozone dist-start-stitching on openbsd/osx
[
https://issues.apache.org/jira/browse/HDDS-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16591672#comment-16591672
]
Elek, Marton commented on HDDS-280:
---
Pseudo cluster files are fixed with HDDS-218, acceptance tests are fixed in
this patch + symbolic link creation is removed.
Thanks all the discussions, please review this approach.
> Support ozone dist-start-stitching on openbsd/osx
> -
>
> Key: HDDS-280
> URL: https://issues.apache.org/jira/browse/HDDS-280
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
> Attachments: HDDS-280.001.patch
>
>
> {quote}Ozone is creating a symlink during the dist process.
> Using the "ozone" directory as a destination name all the docker-based
> acceptance tests and docker-compose files are more simple as they don't need
> to have the version information in the path.
> But to keep the version specific folder name in the tar file we create a
> symbolic link during the tar creation. With the symbolic link and the
> '–dereference' tar argument we can create the tar file which includes a
> versioned directory (ozone-0.2.1) but we can use the a dist directory without
> the version in the name (hadoop-dist/target/ozone).
> {quote}
> This is the description of the current
> dev-support/bin/ozone-dist-tar-stitching. [~aw] in a comment for HDDS-276
> pointed to the problem that some bsd variants don't support the dereference
> command line options of the ln command.
> The main reason to use this approach is to get a simplified destination name
> without the version (hadoop-dist/target/ozone instead of
> hadoop-dist/target/ozone-0.2.1). It simplifies the docker-compose based
> environments and acceptance tests, therefore I prefer to keep the simplified
> destination name.
> The issue is the tar file creation, if and only if we need the version number
> in the name of the root directory inside of the tar.
> Possible solutions:
> # Use cp target/ozone target/ozone-0.2.1 + tar. It's simple but more slow
> and requires more space.
> # Do the tar distribution from docker all the time in case of 'dereference'
> is not supported. Not very convenient
> # Accept that tar will contain ozone directory and not ozone-0.2.1. This is
> the more simple and can be improved with an additional VERSION file in the
> root of the distribution.
> # (+1) Use hadoop-dist/target/ozone-0.2.1 instead of
> hadoop-dist/target/ozone. This is more complex for the docker based testing
> as we need the explicit names in the compose files (volume:
> ../../../hadoop-dist/target/ozone-0.2.1). The structure is more complex with
> using version in the directory name.
> Please comment your preference.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-280) Support ozone dist-start-stitching on openbsd/osx
[
https://issues.apache.org/jira/browse/HDDS-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16560398#comment-16560398
]
Allen Wittenauer commented on HDDS-280:
---
There are probably a million ways to get a docker-compose yaml file (or
anything else you'd need, for that matter) built with the version embedded.
(maven-resources-plugin, maven-assembly-plugin, and/or a smarter CMD in the
docker image that does directory detection are the first three that come to
mind)
> Support ozone dist-start-stitching on openbsd/osx
> -
>
> Key: HDDS-280
> URL: https://issues.apache.org/jira/browse/HDDS-280
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Elek, Marton
>Priority: Major
>
> {quote}Ozone is creating a symlink during the dist process.
> Using the "ozone" directory as a destination name all the docker-based
> acceptance tests and docker-compose files are more simple as they don't need
> to have the version information in the path.
> But to keep the version specific folder name in the tar file we create a
> symbolic link during the tar creation. With the symbolic link and the
> '–dereference' tar argument we can create the tar file which includes a
> versioned directory (ozone-0.2.1) but we can use the a dist directory without
> the version in the name (hadoop-dist/target/ozone).
> {quote}
> This is the description of the current
> dev-support/bin/ozone-dist-tar-stitching. [~aw] in a comment for HDDS-276
> pointed to the problem that some bsd variants don't support the dereference
> command line options of the ln command.
> The main reason to use this approach is to get a simplified destination name
> without the version (hadoop-dist/target/ozone instead of
> hadoop-dist/target/ozone-0.2.1). It simplifies the docker-compose based
> environments and acceptance tests, therefore I prefer to keep the simplified
> destination name.
> The issue is the tar file creation, if and only if we need the version number
> in the name of the root directory inside of the tar.
> Possible solutions:
> # Use cp target/ozone target/ozone-0.2.1 + tar. It's simple but more slow
> and requires more space.
> # Do the tar distribution from docker all the time in case of 'dereference'
> is not supported. Not very convenient
> # Accept that tar will contain ozone directory and not ozone-0.2.1. This is
> the more simple and can be improved with an additional VERSION file in the
> root of the distribution.
> # (+1) Use hadoop-dist/target/ozone-0.2.1 instead of
> hadoop-dist/target/ozone. This is more complex for the docker based testing
> as we need the explicit names in the compose files (volume:
> ../../../hadoop-dist/target/ozone-0.2.1). The structure is more complex with
> using version in the directory name.
> Please comment your preference.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-280) Support ozone dist-start-stitching on openbsd/osx
[
https://issues.apache.org/jira/browse/HDDS-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559988#comment-16559988
]
Elek, Marton commented on HDDS-280:
---
Thanks the feedback.
We use docker-compose with pre-created images: build args can't work (AFAIK).
We can use .ENV file, and originally we started to use .ENV file and maven
filtering. But filtering all the acceptance files and give all the parameters
to the robot framework made it more complex.
I understand the differences between priorities, I am looking for a solution
where both are possible: standardized packaging and easy to use dev/test
tooling.
> Support ozone dist-start-stitching on openbsd/osx
> -
>
> Key: HDDS-280
> URL: https://issues.apache.org/jira/browse/HDDS-280
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Elek, Marton
>Priority: Major
>
> {quote}Ozone is creating a symlink during the dist process.
> Using the "ozone" directory as a destination name all the docker-based
> acceptance tests and docker-compose files are more simple as they don't need
> to have the version information in the path.
> But to keep the version specific folder name in the tar file we create a
> symbolic link during the tar creation. With the symbolic link and the
> '–dereference' tar argument we can create the tar file which includes a
> versioned directory (ozone-0.2.1) but we can use the a dist directory without
> the version in the name (hadoop-dist/target/ozone).
> {quote}
> This is the description of the current
> dev-support/bin/ozone-dist-tar-stitching. [~aw] in a comment for HDDS-276
> pointed to the problem that some bsd variants don't support the dereference
> command line options of the ln command.
> The main reason to use this approach is to get a simplified destination name
> without the version (hadoop-dist/target/ozone instead of
> hadoop-dist/target/ozone-0.2.1). It simplifies the docker-compose based
> environments and acceptance tests, therefore I prefer to keep the simplified
> destination name.
> The issue is the tar file creation, if and only if we need the version number
> in the name of the root directory inside of the tar.
> Possible solutions:
> # Use cp target/ozone target/ozone-0.2.1 + tar. It's simple but more slow
> and requires more space.
> # Do the tar distribution from docker all the time in case of 'dereference'
> is not supported. Not very convenient
> # Accept that tar will contain ozone directory and not ozone-0.2.1. This is
> the more simple and can be improved with an additional VERSION file in the
> root of the distribution.
> # (+1) Use hadoop-dist/target/ozone-0.2.1 instead of
> hadoop-dist/target/ozone. This is more complex for the docker based testing
> as we need the explicit names in the compose files (volume:
> ../../../hadoop-dist/target/ozone-0.2.1). The structure is more complex with
> using version in the directory name.
> Please comment your preference.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-280) Support ozone dist-start-stitching on openbsd/osx
[
https://issues.apache.org/jira/browse/HDDS-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559784#comment-16559784
]
Allen Wittenauer commented on HDDS-280:
---
bq. if and only if we need the version number in the name of the root
directory inside of the tar.
Yes, you do.
bq. It simplifies the docker-compose based environments and acceptance tests,
therefore I prefer to keep the simplified destination name.
It sounds like a prioritization problem. distribution standardization is way
more important than testing. Besides, why can't you use a docker build arg to
know what the version is when doing the tests? This way you'll know what the
fully versioned directory is.
> Support ozone dist-start-stitching on openbsd/osx
> -
>
> Key: HDDS-280
> URL: https://issues.apache.org/jira/browse/HDDS-280
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Elek, Marton
>Priority: Major
>
> {quote}Ozone is creating a symlink during the dist process.
> Using the "ozone" directory as a destination name all the docker-based
> acceptance tests and docker-compose files are more simple as they don't need
> to have the version information in the path.
> But to keep the version specific folder name in the tar file we create a
> symbolic link during the tar creation. With the symbolic link and the
> '–dereference' tar argument we can create the tar file which includes a
> versioned directory (ozone-0.2.1) but we can use the a dist directory without
> the version in the name (hadoop-dist/target/ozone).
> {quote}
> This is the description of the current
> dev-support/bin/ozone-dist-tar-stitching. [~aw] in a comment for HDDS-276
> pointed to the problem that some bsd variants don't support the dereference
> command line options of the ln command.
> The main reason to use this approach is to get a simplified destination name
> without the version (hadoop-dist/target/ozone instead of
> hadoop-dist/target/ozone-0.2.1). It simplifies the docker-compose based
> environments and acceptance tests, therefore I prefer to keep the simplified
> destination name.
> The issue is the tar file creation, if and only if we need the version number
> in the name of the root directory inside of the tar.
> Possible solutions:
> # Use cp target/ozone target/ozone-0.2.1 + tar. It's simple but more slow
> and requires more space.
> # Do the tar distribution from docker all the time in case of 'dereference'
> is not supported. Not very convenient
> # Accept that tar will contain ozone directory and not ozone-0.2.1. This is
> the more simple and can be improved with an additional VERSION file in the
> root of the distribution.
> # (+1) Use hadoop-dist/target/ozone-0.2.1 instead of
> hadoop-dist/target/ozone. This is more complex for the docker based testing
> as we need the explicit names in the compose files (volume:
> ../../../hadoop-dist/target/ozone-0.2.1). The structure is more complex with
> using version in the directory name.
> Please comment your preference.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-280) Support ozone dist-start-stitching on openbsd/osx
[
https://issues.apache.org/jira/browse/HDDS-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559758#comment-16559758
]
Anu Engineer commented on HDDS-280:
---
+1, on the VERSION file in the root approach. That is simpler and cleaner.
[~aw] Thoughts ?
> Support ozone dist-start-stitching on openbsd/osx
> -
>
> Key: HDDS-280
> URL: https://issues.apache.org/jira/browse/HDDS-280
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Elek, Marton
>Priority: Major
>
> {quote}Ozone is creating a symlink during the dist process.
> Using the "ozone" directory as a destination name all the docker-based
> acceptance tests and docker-compose files are more simple as they don't need
> to have the version information in the path.
> But to keep the version specific folder name in the tar file we create a
> symbolic link during the tar creation. With the symbolic link and the
> '–dereference' tar argument we can create the tar file which includes a
> versioned directory (ozone-0.2.1) but we can use the a dist directory without
> the version in the name (hadoop-dist/target/ozone).
> {quote}
> This is the description of the current
> dev-support/bin/ozone-dist-tar-stitching. [~aw] in a comment for HDDS-276
> pointed to the problem that some bsd variants don't support the dereference
> command line options of the ln command.
> The main reason to use this approach is to get a simplified destination name
> without the version (hadoop-dist/target/ozone instead of
> hadoop-dist/target/ozone-0.2.1). It simplifies the docker-compose based
> environments and acceptance tests, therefore I prefer to keep the simplified
> destination name.
> The issue is the tar file creation, if and only if we need the version number
> in the name of the root directory inside of the tar.
> Possible solutions:
> # Use cp target/ozone target/ozone-0.2.1 + tar. It's simple but more slow
> and requires more space.
> # Do the tar distribution from docker all the time in case of 'dereference'
> is not supported. Not very convenient
> # Accept that tar will contain ozone directory and not ozone-0.2.1. This is
> the more simple and can be improved with an additional VERSION file in the
> root of the distribution.
> # (+1) Use hadoop-dist/target/ozone-0.2.1 instead of
> hadoop-dist/target/ozone. This is more complex for the docker based testing
> as we need the explicit names in the compose files (volume:
> ../../../hadoop-dist/target/ozone-0.2.1). The structure is more complex with
> using version in the directory name.
> Please comment your preference.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-280) Support ozone dist-start-stitching on openbsd/osx
[
https://issues.apache.org/jira/browse/HDDS-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552463#comment-16552463
]
Elek, Marton commented on HDDS-280:
---
My preference is 3 with additional VERSION file.
> Support ozone dist-start-stitching on openbsd/osx
> -
>
> Key: HDDS-280
> URL: https://issues.apache.org/jira/browse/HDDS-280
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Elek, Marton
>Priority: Major
>
> {quote}Ozone is creating a symlink during the dist process.
> Using the "ozone" directory as a destination name all the docker-based
> acceptance tests and docker-compose files are more simple as they don't need
> to have the version information in the path.
> But to keep the version specific folder name in the tar file we create a
> symbolic link during the tar creation. With the symbolic link and the
> '–dereference' tar argument we can create the tar file which includes a
> versioned directory (ozone-0.2.1) but we can use the a dist directory without
> the version in the name (hadoop-dist/target/ozone).
> {quote}
> This is the description of the current
> dev-support/bin/ozone-dist-tar-stitching. [~aw] in a comment for HDDS-276
> pointed to the problem that some bsd variants don't support the dereference
> command line options of the ln command.
> The main reason to use this approach is to get a simplified destination name
> without the version (hadoop-dist/target/ozone instead of
> hadoop-dist/target/ozone-0.2.1). It simplifies the docker-compose based
> environments and acceptance tests, therefore I prefer to keep the simplified
> destination name.
> The issue is the tar file creation, if and only if we need the version number
> in the name of the root directory inside of the tar.
> Possible solutions:
> # Use cp target/ozone target/ozone-0.2.1 + tar. It's simple but more slow
> and requires more space.
> # Do the tar distribution from docker all the time in case of 'dereference'
> is not supported. Not very convenient
> # Accept that tar will contain ozone directory and not ozone-0.2.1. This is
> the more simple and can be improved with an additional VERSION file in the
> root of the distribution.
> # (+1) Use hadoop-dist/target/ozone-0.2.1 instead of
> hadoop-dist/target/ozone. This is more complex for the docker based testing
> as we need the explicit names in the compose files (volume:
> ../../../hadoop-dist/target/ozone-0.2.1). The structure is more complex with
> using version in the directory name.
> Please comment your preference.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
