This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
The following commit(s) were added to refs/heads/master by this push:
new 8235366 HDDS-3858. Remove support to start Ozone and HDFS datanodes
in the same JVM (#1117)
8235366 is described below
commit 823536604509f69d65880f4224974c331f7418f5
Author: Elek, Márton <e...@users.noreply.github.com>
AuthorDate: Thu Jun 25 12:33:37 2020 +0200
HDDS-3858. Remove support to start Ozone and HDFS datanodes in the same JVM
(#1117)
---
hadoop-hdds/docs/content/beyond/RunningWithHDFS.md | 70 ----------------------
.../docs/content/beyond/RunningWithHDFS.zh.md | 64 --------------------
hadoop-ozone/dist/src/main/compose/ozone-hdfs/.env | 18 ------
.../main/compose/ozone-hdfs/docker-compose.yaml | 70 ----------------------
.../dist/src/main/compose/ozone-hdfs/docker-config | 35 -----------
5 files changed, 257 deletions(-)
diff --git a/hadoop-hdds/docs/content/beyond/RunningWithHDFS.md
b/hadoop-hdds/docs/content/beyond/RunningWithHDFS.md
deleted file mode 100644
index 2bfbc99..0000000
--- a/hadoop-hdds/docs/content/beyond/RunningWithHDFS.md
+++ /dev/null
@@ -1,70 +0,0 @@
----
-title: Running concurrently with HDFS
-linktitle: Runing with HDFS
-weight: 1
-summary: Ozone is designed to run concurrently with HDFS. This page explains
how to deploy Ozone in a exisiting HDFS cluster.
----
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
-Ozone is designed to work with HDFS. So it is easy to deploy ozone in an
-existing HDFS cluster.
-
-The container manager part of Ozone can run inside DataNodes as a pluggable
module
-or as a standalone component. This document describe how can it be started as
-a HDFS datanode plugin.
-
-To activate ozone you should define the service plugin implementation class.
-
-<div class="alert alert-warning" role="alert">
-<b>Important</b>: It should be added to the <b>hdfs-site.xml</b> as the plugin
should
-be activated as part of the normal HDFS Datanode bootstrap.
-</div>
-
-{{< highlight xml >}}
-<property>
- <name>dfs.datanode.plugins</name>
- <value>org.apache.hadoop.ozone.HddsDatanodeService</value>
-</property>
-{{< /highlight >}}
-
-You also need to add the jar file under path /opt/ozone/share/ozone/lib/ to
the classpath:
-
-{{< highlight bash >}}
-export HADOOP_CLASSPATH=/opt/ozone/share/ozone/lib/*.jar
-{{< /highlight >}}
-
-
-
-To start ozone with HDFS you should start the the following components:
-
- 1. HDFS Namenode (from Hadoop distribution)
- 2. HDFS Datanode (from the Hadoop distribution with the plugin on the
- classpath from the Ozone distribution)
- 3. Ozone Manager (from the Ozone distribution)
- 4. Storage Container Manager (from the Ozone distribution)
-
-Please check the log of the datanode whether the HDDS/Ozone plugin is started
or
-not. Log of datanode should contain something like this:
-
-```
-2018-09-17 16:19:24 INFO HddsDatanodeService:158 - Started plug-in
org.apache.hadoop.ozone.web.OzoneHddsDatanodeService@6f94fb9d
-```
-
-<div class="alert alert-warning" role="alert">
-<b>Note:</b> document above is based on Hadoop 3.1.
-</div>
diff --git a/hadoop-hdds/docs/content/beyond/RunningWithHDFS.zh.md
b/hadoop-hdds/docs/content/beyond/RunningWithHDFS.zh.md
deleted file mode 100644
index 981c1e3..0000000
--- a/hadoop-hdds/docs/content/beyond/RunningWithHDFS.zh.md
+++ /dev/null
@@ -1,64 +0,0 @@
----
-title: 与 HDFS 并存运行
-linktitle: Runing with HDFS
-weight: 1
-summary: Ozone 能够与 HDFS 并存运行，本页介绍如何将 Ozone 部署到已有的 HDFS 集群上。
----
-<!---
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
-Ozone 支持与 HDFS 并存工作，所以用户可以轻易的在已有的 HDFS 集群上部署 Ozone。
-
-Ozone 的容器管理组件可以在 HDFS 数据节点上以插件的形式或是独立运行，下文介绍插件运行的方法。
-
-为了在 HDFS 数据节点上启用 Ozone 插件，你需要定义服务插件实现类。
-
-<div class="alert alert-warning" role="alert">
-<b>重要</b>：因为插件在 HDFS 数据节点启动过程中被激活，服务插件实现类的定义需要添加到 <b>hdfs-site.xml</b> 中。
-</div>
-
-{{< highlight xml >}}
-<property>
- <name>dfs.datanode.plugins</name>
- <value>org.apache.hadoop.ozone.HddsDatanodeService</value>
-</property>
-{{< /highlight >}}
-
-此外还需要将 /opt/ozone/share/ozone/lib/ 路径下的 jar 包添加到 Hadoop classpath 下：
-
-{{< highlight bash >}}
-export HADOOP_CLASSPATH=/opt/ozone/share/ozone/lib/*.jar
-{{< /highlight >}}
-
-
-
-让 Ozone 随 HDFS 一同启动的步骤为：
-
- 1. HDFS Namenode（从 Hadoop 中启动）
- 2. HDFS Datanode (从 Hadoop 中启动，需要按照如上配置插件和 classpath）
- 3. Ozone Manager (从 Ozone 中启动）
- 4. Storage Container Manager (从 Ozone 启动）
-
-检查数据节点的日志以确认 HDDS/Ozone 插件是否启动，日志中应当包含以下内容：
-
-```
-2018-09-17 16:19:24 INFO HddsDatanodeService:158 - Started plug-in
org.apache.hadoop.ozone.HddsDatanodeService@6f94fb9d
-```
-
-<div class="alert alert-warning" role="alert">
-<b>注意：</b> 上面的测试是基于 Hadoop 3.1 进行的。
-</div>
diff --git a/hadoop-ozone/dist/src/main/compose/ozone-hdfs/.env
b/hadoop-ozone/dist/src/main/compose/ozone-hdfs/.env
deleted file mode 100644
index df9065c..0000000
--- a/hadoop-ozone/dist/src/main/compose/ozone-hdfs/.env
+++ /dev/null
@@ -1,18 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-HADOOP_VERSION=3
-OZONE_RUNNER_VERSION=${docker.ozone-runner.version}
diff --git a/hadoop-ozone/dist/src/main/compose/ozone-hdfs/docker-compose.yaml
b/hadoop-ozone/dist/src/main/compose/ozone-hdfs/docker-compose.yaml
deleted file mode 100644
index 77939e9..0000000
--- a/hadoop-ozone/dist/src/main/compose/ozone-hdfs/docker-compose.yaml
+++ /dev/null
@@ -1,70 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-version: "3"
-services:
- namenode:
- image: apache/hadoop:${HADOOP_VERSION}
- ports:
- - 9870:9870
- env_file:
- - ./docker-config
- environment:
- ENSURE_NAMENODE_DIR: "/tmp/hadoop-root/dfs/name"
- command: ["hdfs", "namenode"]
- datanode:
- image: apache/hadoop:${HADOOP_VERSION}
- ports:
- - 9864
- volumes:
- - ../..:/opt/ozone
- command: ["hdfs","datanode"]
- environment:
- HADOOP_CLASSPATH: /opt/ozone/share/ozone/lib/*.jar
- env_file:
- - ./docker-config
- om:
- image: apache/ozone-runner:${OZONE_RUNNER_VERSION}
- volumes:
- - ../..:/opt/hadoop
- ports:
- - 9874:9874
- - 9862:9862
- environment:
- ENSURE_OM_INITIALIZED: /data/metadata/om/current/VERSION
- env_file:
- - ./docker-config
- command: ["ozone","om"]
- scm:
- image: apache/ozone-runner:${OZONE_RUNNER_VERSION}
- volumes:
- - ../..:/opt/hadoop
- ports:
- - 9876:9876
- env_file:
- - ./docker-config
- environment:
- ENSURE_SCM_INITIALIZED: /data/metadata/scm/current/VERSION
- command: ["ozone","scm"]
- s3g:
- image: apache/ozone-runner:${OZONE_RUNNER_VERSION}
- volumes:
- - ../..:/opt/hadoop
- ports:
- - 9878:9878
- env_file:
- - ./docker-config
- command: ["ozone","s3g"]
diff --git a/hadoop-ozone/dist/src/main/compose/ozone-hdfs/docker-config
b/hadoop-ozone/dist/src/main/compose/ozone-hdfs/docker-config
deleted file mode 100644
index bcbac49..0000000
--- a/hadoop-ozone/dist/src/main/compose/ozone-hdfs/docker-config
+++ /dev/null
@@ -1,35 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-OZONE-SITE.XML_ozone.om.address=om
-OZONE-SITE.XML_ozone.om.http-address=om:9874
-OZONE-SITE.XML_ozone.scm.names=scm
-OZONE-SITE.XML_ozone.scm.datanode.id.dir=/data
-OZONE-SITE.XML_ozone.scm.block.client.address=scm
-OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
-OZONE-SITE.XML_ozone.scm.client.address=scm
-OZONE-SITE.XML_ozone.replication=1
-OZONE-SITE.XML_hdds.datanode.dir=/data/hdds
-
-HDFS-SITE.XML_dfs.datanode.plugins=org.apache.hadoop.ozone.HddsDatanodeService
-HDFS-SITE.XML_dfs.namenode.rpc-address=namenode:9000
-HDFS-SITE.XML_rpc.metrics.quantile.enable=true
-HDFS-SITE.XML_rpc.metrics.percentiles.intervals=60,300
-
-#Enable this variable to print out all hadoop rpc traffic to the stdout. See
http://byteman.jboss.org/ to define your own instrumentation.
-#BYTEMAN_SCRIPT_URL=https://raw.githubusercontent.com/apache/hadoop/trunk/dev-support/byteman/hadooprpc.btm
-
-no_proxy=om,scm,s3g,kdc,localhost,127.0.0.1
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
