This is an automated email from the ASF dual-hosted git repository.
jialiang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/ambari-website.git
The following commit(s) were added to refs/heads/main by this push:
new c61cec0 AMBARI-26340: Update build Docs for Ambari 3.0 (#25)
c61cec0 is described below
commit c61cec0c8e35b0bd3e0562264025a6527a2ca6c8
Author: jialiang <[email protected]>
AuthorDate: Fri Mar 21 14:03:40 2025 +0800
AMBARI-26340: Update build Docs for Ambari 3.0 (#25)
---
docs/ambari-dev/bigtop-guide.md | 154 ++++++++++++++++
docs/ambari-dev/building-from-source.md | 126 +++++++++++++
docs/ambari-dev/how-to-contribute.md | 22 ++-
docs/ambari-dev/running-tests.md | 109 +++++++++++
docs/quick-start/installation-guide.md | 48 ++++-
docs/release-notes.md | 149 ++++++++-------
sidebars.js | 3 +
src/pages/team/languages.json | 80 +++++----
.../version-3.0.0/ambari-dev/bigtop-guide.md | 154 ++++++++++++++++
.../ambari-dev/building-from-source.md | 125 +++++++++++++
.../version-3.0.0/ambari-dev/how-to-contribute.md | 17 +-
.../version-3.0.0/ambari-dev/running-tests.md | 109 +++++++++++
.../quick-start/installation-guide.md | 48 ++++-
versioned_docs/version-3.0.0/release-notes.md | 199 +++++++++++++++------
versioned_sidebars/version-3.0.0-sidebars.json | 5 +
15 files changed, 1177 insertions(+), 171 deletions(-)
diff --git a/docs/ambari-dev/bigtop-guide.md b/docs/ambari-dev/bigtop-guide.md
new file mode 100644
index 0000000..875468a
--- /dev/null
+++ b/docs/ambari-dev/bigtop-guide.md
@@ -0,0 +1,154 @@
+---
+title: Compiling Components for Ambari Bigtop Stack
+---
+
+# Compiling Components for Ambari Bigtop Stack
+
+## Introduction
+
+Apache Bigtop is designed for infrastructure engineers and data scientists
seeking comprehensive packaging, testing, and configuration of leading
open-source big data components. Bigtop supports a wide range of
components/projects, including but not limited to Hadoop, HBase, and Spark.
This guide specifically focuses on how to compile components for the **Ambari
Bigtop Stack**.
+
+## Use Cases for Apache Bigtop
+
+1. **Simplified Package Building**: Bigtop significantly simplifies the
process of compiling RPM or DEB packages for big data components across
different operating systems through pre-configured Docker images, making it
quick and efficient.
+
+2. **Dependency Management**: Bigtop integrates complex dependencies required
during the compilation process, effectively resolving common compilation errors
and ensuring a smooth compilation experience through patches in the code. This
means users no longer need to worry about official packages failing to compile
or setting up complex compilation environments.
+
+3. **Apache Ambari Support**: Bigtop provides support for Apache Ambari,
allowing users to easily package big data software that is compatible with
Ambari and meets installation requirements.
+
+## Getting Started with Bigtop
+
+This guide uses the official Bigtop 3.3.0 as an example, with CentOS 7 as the
compilation operating system. The same operations apply to other systems and
versions.
+
+### Prerequisites
+
+- Linux environment
+- Docker installed on your system
+- Git
+
+### Step-by-Step Guide
+
+#### 1. Create a Development Directory
+
+```bash
+mkdir ~/dev/
+```
+
+#### 2. Clone Bigtop Repository
+
+```bash
+cd ~/dev/
+git clone https://github.com/apache/bigtop.git
+```
+
+#### 3. Switch to Version 3.3.0
+
+```bash
+cd bigtop
+git checkout release-3.3.0
+```
+
+#### 4. Pull the Bigtop CentOS 7 Compilation Environment Image
+
+```bash
+# If you need to compile for other operating systems or architectures (e.g.,
ARM),
+# you can search for the corresponding Bigtop version in the image repository
+# https://hub.docker.com/r/bigtop/slaves/tags
+docker pull bigtop/slaves:3.3.0-centos-7
+```
+
+#### 5. Launch the Container
+
+**Scenario 1**: If you've previously compiled big data components locally and
have a Maven repository cache, it's best to map this directory to the
container's default Maven download directory to avoid downloading packages
again.
+
+For example, if your local Maven repository directory is `/data/repository`:
+
+```bash
+cd ~/dev/bigtop
+docker run -d -it --network host -v `pwd`:/ws -v
/data/repository:/root/.m2/repository --workdir /ws --name bigtopr
bigtop/slaves:3.3.0-centos-7
+```
+
+**Scenario 2**: If you don't have a Maven cache locally or are unfamiliar with
this, you should still map a directory to the Bigtop container to facilitate
repeated compilations using downloaded Maven cache. Otherwise, when the
container is deleted, your Maven cache will be lost, and dependency downloading
is the most time-consuming stage of recompilation.
+
+```bash
+mkdir -p ~/m2/repository
+cd ~/dev/bigtop
+docker run -d -it --network host -v `pwd`:/ws -v
~/m2/repository:/root/.m2/repository --workdir /ws --name bigtopr
bigtop/slaves:3.3.0-centos-7
+```
+
+#### 6. Modify Maven Repository Settings (Optional)
+
+You can configure Maven to use mirrors that are faster for your location. This
step is optional but can significantly improve download speeds.
+
+1. Enter the container:
+```bash
+docker exec -it bigtopr /bin/bash
+```
+
+2. Edit the Maven settings file:
+```bash
+vi /usr/local/maven/conf/settings.xml
+```
+
+3. Add appropriate mirror repositories based on your location. For example:
+
+```xml
+<mirrors>
+ <mirror>
+ <id>central-mirror</id>
+ <mirrorOf>central</mirrorOf>
+ <name>Central Repository Mirror</name>
+ <url>https://repo1.maven.org/maven2/</url>
+ </mirror>
+ <!-- Add other mirrors as needed -->
+</mirrors>
+```
+
+#### 7. Compile Big Data Components
+
+Enter your running container:
+
+```bash
+docker exec -it bigtopr /bin/bash
+```
+
+Compile components:
+
+```bash
+. /etc/profile.d/bigtop.sh
+./gradlew flink-clean flink-pkg -PparentDir=/usr/bigtop -PpkgSuffix
-PbuildThreads=2C repo
+```
+
+**Explanation of compilation parameters**:
+
+- `-PparentDir=/usr/bigtop`: Changes the default installation path of the
package, making Bigtop-built packages conform to Ambari installation
specifications.
+- `-PpkgSuffix`: Makes the output package include the Bigtop version number
(e.g., hadoop_3_3_0), conforming to Ambari Bigtop service specifications.
+- `-PbuildThreads=2C`: Sets the number of threads for compilation (2 times the
number of CPU cores).
+
+## Parallel Compilation for Improved Performance
+
+A pull request for parallel compilation to speed up the build process has been
submitted to the community and is currently under review. Once merged, all Java
components in Bigtop will be able to compile in parallel, expected to be
available in versions after Bigtop 3.3.1.
+
+Performance comparison for parallel compilation (after all dependencies are
downloaded):
+
+| Component | Time Before | Time After |
+|------------|-------------|------------|
+| Alluxio | 21min | 07:43min |
+| Hive | 05:33min | 03:04min |
+| HBase | 06:18min | 02:55min |
+| Zookeeper | 01:25min | 35s |
+| Livy | 03:29min | 03:12min |
+| Phoenix | 11:23min | 05:32min |
+| Zeppelin | 14:15min | 13:19min |
+| Flink | 36:27min | 14:16min |
+| Hadoop | 50min | 16min |
+
+Example of parallel compilation command:
+
+```bash
+docker run -d -it --network host -v `pwd`:/ws -v
/data/repository:/data/repository --workdir /ws --name bigtop
bigtop/slaves:trunk-centos-7 --cpus 16
+source /etc/profile.d/bigtop.sh
+./gradlew alluxio-clean alluxio-pkg -PcompileThreads=2C
+```
+
+This approach shows a 2-3x improvement in compilation speed, with even more
significant effects during initial compilation (e.g., Hadoop initial
compilation time reduced from 3 hours to 1 hour).
diff --git a/docs/ambari-dev/building-from-source.md
b/docs/ambari-dev/building-from-source.md
new file mode 100644
index 0000000..d8f7b61
--- /dev/null
+++ b/docs/ambari-dev/building-from-source.md
@@ -0,0 +1,126 @@
+---
+title: Building from Source
+---
+
+# Building Apache Ambari from Source
+
+This guide explains how to build Apache Ambari 3.0 and its related subprojects
from source code.
+
+## Prerequisites
+
+Before you begin, ensure you have the following requirements installed:
+
+### System Requirements
+- Operating System: Rocky Linux 8 or 9 (recommended)
+- Python 3 Development Tools (`python3-devel`)
+
+### Java Requirements
+- Ambari Main Project: JDK 17
+- Ambari Metrics: JDK 8
+- Ambari Infra: JDK 8
+
+## Building Ambari Main Project
+
+### 1. Clone the Repository
+```bash
+git clone [email protected]:apache/ambari.git
+cd ambari
+```
+
+### 2. Build Options
+
+#### Build Without RPM
+To build Ambari without creating RPM packages:
+```bash
+mvn -B -T 2C clean install package \
+ -Drat.skip=true \
+ -DskipTests \
+ -Dmaven.test.skip=true \
+ -Dfindbugs.skip=true \
+ -Dcheckstyle.skip=true
+```
+
+#### Build With RPM
+To build Ambari and create RPM packages:
+```bash
+mvn -B -T 2C clean install package rpm:rpm \
+ -Drat.skip=true \
+ -DskipTests \
+ -Dmaven.test.skip=true \
+ -Dfindbugs.skip=true \
+ -Dcheckstyle.skip=true
+```
+
+The RPM packages will be generated at:
+- Ambari Agent:
`ambari/ambari-agent/target/rpm/ambari-agent/RPMS/x86_64/ambari-agent-3.0.0.0-SNAPSHOT.x86_64.rpm`
+- Ambari Server:
`ambari/ambari-server/target/rpm/ambari-server/RPMS/x86_64/ambari-server-3.0.0.0-SNAPSHOT.x86_64.rpm`
+
+## Building Ambari Metrics
+
+:::tip Performance Optimization
+To significantly improve build performance, download binary dependencies
locally before building:
+
+1. Create a local directory for dependencies:
+```bash
+mkdir -p /ws/dl/
+```
+
+2. Download the required binary files:
+```bash
+wget -P /ws/dl/
http://repo.bigtop.apache.org.s3.amazonaws.com/bigtop-stack-binary/3.2.0/centos-7/x86_64/hbase-2.4.13-bin.tar.gz
+wget -P /ws/dl/
http://repo.bigtop.apache.org.s3.amazonaws.com/bigtop-stack-binary/3.2.0/centos-7/x86_64/hadoop-3.3.4.tar.gz
+wget -P /ws/dl/
https://dl.grafana.com/oss/release/grafana-11.1.4.linux-amd64.tar.gz
+wget -P /ws/dl/
http://repo.bigtop.apache.org.s3.amazonaws.com/bigtop-stack-binary/3.2.0/centos-7/x86_64/phoenix-hbase-2.4-5.1.2-bin.tar.gz
+```
+
+3. Modify the `pom.xml` in ambari-metrics project to use local files:
+```xml
+<!-- Update these properties to use local files -->
+<properties>
+ <hbase.tar>file:///ws/dl/hbase-2.4.13-bin.tar.gz</hbase.tar>
+ <hadoop.tar>file:///ws/dl/hadoop-3.3.4.tar.gz</hadoop.tar>
+ <grafana.tar>file:///ws/dl/grafana-11.1.4.linux-amd64.tar.gz</grafana.tar>
+ <phoenix.tar>file:///ws/dl/phoenix-hbase-2.4-5.1.2-bin.tar.gz</phoenix.tar>
+</properties>
+```
+
+This optimization will save significant time during repeated builds by
avoiding large downloads.
+:::
+
+### 1. Clone the Repository
+```bash
+git clone [email protected]:apache/ambari-metrics.git
+cd ambari-metrics
+```
+
+### 2. Build Options
+
+#### Build Without RPM
+To build Ambari Metrics without creating RPM packages:
+```bash
+mvn -T 2C clean install -DskipTests
+```
+
+#### Build With RPM
+To build Ambari Metrics and create RPM packages:
+```bash
+mvn -T 2C clean install -DskipTests -Dbuild-rpm
+```
+
+To locate the generated RPM packages:
+```bash
+find ./ -name "*.rpm"
+```
+
+## Building Ambari Infra
+
+### 1. Clone the Repository
+```bash
+git clone [email protected]:apache/ambari-infra.git
+cd ambari-infra
+```
+
+### 2. Build RPM Package
+```bash
+make rpm
+```
diff --git a/docs/ambari-dev/how-to-contribute.md
b/docs/ambari-dev/how-to-contribute.md
index 1e44e8c..23897ec 100644
--- a/docs/ambari-dev/how-to-contribute.md
+++ b/docs/ambari-dev/how-to-contribute.md
@@ -36,11 +36,21 @@ Repeat these steps for all the branches that needs to be
synced with the remote.
Apache Ambari uses JIRA to track issues including bugs and improvements, and
uses Github pull requests to manage code reviews and code merges. Major design
changes are discussed in JIRA and implementation changes are discussed in pull
requests after a pull request is created.
+:::note Important Changes to JIRA Registration
+* JIRA registration is currently closed to the public
+* To get a JIRA account:
+ 1. Register on [Apache JIRA](https://issues.apache.org/jira)
+ 2. Contact a PMC member to approve your registration
+* Alternatively, you can:
+ 1. Submit your Pull Request first
+ 2. Community members will help create the corresponding JIRA ticket for you
+:::
+
* Find an existing Apache JIRA that the change pertains to
* Do not create a new JIRA if the change is minor and relates to an
existing JIRA; add to the existing discussion and work instead
* Look for existing pull requests that are linked from the JIRA, to
understand if someone is already working on the JIRA
-* If the change is new, then create a new JIRA:
+* If the change is new and you have JIRA access, then create a new JIRA:
* Provide a descriptive Title
* Write a detailed Description. For bug reports, this should ideally
include a short reproduction of the problem. For new features, it may include a
design document.
* Fill the required fields:
@@ -49,11 +59,11 @@ Apache Ambari uses JIRA to track issues including bugs and
improvements, and use
* Blocker: pointless to release without this change as the release
would be unusable to a large minority of users
* Critical: a large minority of users are missing important
functionality without this, and/or a workaround is difficult
* Major: a small minority of users are missing important
functionality without this, and there is a workaround
- * Minor: a niche use case is missing some support, but it does not
affect usage or is easily worked around
- * Trivial: a nice-to-have change but unlikely to be any problem in
practice otherwise
- * Component. Choose the components that are affected by this change.
Choose from Ambari Components
- * Affects Version. For Bugs, assign at least one version that is known
to exhibit the problem or need the change
- * Do not include a patch file; pull requests are used to propose the
actual change.
+
+* If you don't have JIRA access:
+ * Submit your Pull Request first
+ * In the PR description, clearly describe the issue or improvement
+ * A community member will create a JIRA ticket and link it to your PR
### Pull Request
diff --git a/docs/ambari-dev/running-tests.md b/docs/ambari-dev/running-tests.md
new file mode 100644
index 0000000..310988a
--- /dev/null
+++ b/docs/ambari-dev/running-tests.md
@@ -0,0 +1,109 @@
+---
+title: Running Tests
+---
+
+# Running Tests in Apache Ambari
+
+This guide explains how to run different types of tests in Apache Ambari. For
the official test configuration, you can refer to the [Ambari
Jenkinsfile](https://github.com/apache/ambari/blob/trunk/Jenkinsfile).
+
+## Java Tests
+
+### Running All Java Tests
+To run all Java tests for the Ambari Server:
+```bash
+mvn -am test -pl ambari-server \
+ -DskipPythonTests \
+ -Dmaven.test.failure.ignore \
+ -Dmaven.artifact.threads=10 \
+ -Drat.skip \
+ -DskipAdminWebTests=true
+```
+
+### Running Specific Java Tests
+To run a specific Java test class:
+```bash
+mvn -am test -pl ambari-server \
+ -DskipPythonTests \
+ -Dmaven.test.failure.ignore \
+ -Dmaven.artifact.threads=10 \
+ -Drat.skip \
+ -DskipAdminWebTests=true \
+ -Dtest=AmbariServerTest
+```
+
+To run a specific test method within a test class:
+```bash
+mvn -am test -pl ambari-server \
+ -DskipPythonTests \
+ -Dmaven.test.failure.ignore \
+ -Dmaven.artifact.threads=10 \
+ -Drat.skip \
+ -DskipAdminWebTests=true \
+ -Dtest=AmbariServerTest#testMethodName
+```
+
+### Test Parameters Explained
+- `-am`: Also build dependencies
+- `-pl ambari-server`: Only build the ambari-server module
+- `-DskipPythonTests`: Skip Python tests
+- `-Dmaven.test.failure.ignore`: Continue the build even if tests fail
+- `-Dmaven.artifact.threads=10`: Use 10 threads for parallel artifact
resolution
+- `-Drat.skip`: Skip Apache RAT (Release Audit Tool) checks
+- `-DskipAdminWebTests`: Skip admin web interface tests
+- `-Dtest`: Specify which test class or method to run
+
+## Python Tests
+
+### Running All Python Tests
+To run all Python tests:
+```bash
+mvn test -pl ambari-server \
+ -DskipJavaTests \
+ -Dpython.test.mask="*_test.py" \
+ -Dpython.test.skip.pattern="agent_perf.py"
+```
+
+### Running Specific Python Tests
+To run a specific Python test file:
+```bash
+mvn test -pl ambari-server \
+ -DskipJavaTests \
+ -Dpython.test.mask="test_file_name.py"
+```
+
+### Python Test Parameters Explained
+- `-DskipJavaTests`: Skip Java tests
+- `-Dpython.test.mask`: Pattern to match test files to run
+- `-Dpython.test.skip.pattern`: Pattern to match test files to skip
+
+## Integration Tests
+
+### Running Integration Tests
+To run integration tests:
+```bash
+mvn verify -pl ambari-server \
+ -P integration-tests \
+ -DskipPythonTests \
+ -DskipJavaTests
+```
+
+### Integration Test Parameters
+- `-P integration-tests`: Activate the integration-tests profile
+- `-DskipPythonTests`: Skip Python tests
+- `-DskipJavaTests`: Skip Java tests
+
+## Test Reports
+
+Test reports can be found in the following locations after test execution:
+
+### Java Test Reports
+- Unit Tests: `ambari-server/target/surefire-reports/`
+- Integration Tests: `ambari-server/target/failsafe-reports/`
+
+### Python Test Reports
+- Test Results: `ambari-server/target/python-test-results/`
+- Coverage Reports: `ambari-server/target/python-coverage/`
+
+:::tip
+When debugging test failures, check these report directories for detailed test
execution logs and stack traces.
+:::
diff --git a/docs/quick-start/installation-guide.md
b/docs/quick-start/installation-guide.md
index db52d1d..485a50e 100644
--- a/docs/quick-start/installation-guide.md
+++ b/docs/quick-start/installation-guide.md
@@ -2,6 +2,42 @@
This guide covers the installation and setup of Apache Ambari on bare metal,
KVM, Docker, or Vagrant environments.
+## Important: Firewall Configuration
+
+:::tip Before You Begin
+Firewall settings can significantly impact cluster functionality by blocking
necessary communication ports between components. Please review the following
guidelines carefully.
+:::
+
+### Development & Testing Environments
+For development or testing environments, consider disabling the firewall:
+```bash
+systemctl stop firewalld
+systemctl disable firewalld
+```
+
+### Production Environments
+
+#### For System Administrators
+Configure the firewall to allow required Hadoop ecosystem ports based on your
specific component deployment:
+
+| Component | Ports | Purpose |
+|-----------|-------|---------|
+| Ambari Server | 8080, 8440, 8441 | Web UI, Agent communication |
+| Core Hadoop | 8020, 9000, 50070, 50075 | HDFS NameNode, DataNode HTTP |
+| YARN | 8032, 8088, 19888 | ResourceManager, UI, JobHistory |
+| Hive | 9083, 10000 | Metastore, HiveServer2 |
+
+:::note NodeManager Ports
+YARN NodeManagers allocate containers on dynamically assigned ports (default
range: 32768-65535).
+This range can be restricted via `yarn.nodemanager.resource.ports` in
`yarn-site.xml`
+:::
+
+:::info For General Users
+If you're not familiar with advanced network configuration, we recommend:
+- Disabling the firewall during initial setup and cluster testing
+- Consulting with a network security specialist for production deployments
+:::
+
## Prerequisites
Ensure you have a working environment (bare metal, KVM, Docker, or [Vagrant
setup](./environment-setup/vagrant-environment-setup.md)) before proceeding.
@@ -204,16 +240,10 @@ Default credentials:
## Troubleshooting
-1. If you encounter firewall issues, disable it (for development environments
only):
-```bash
-systemctl stop firewalld
-systemctl disable firewalld
-```
-
-2. Ensure proper hostname resolution by configuring `/etc/hosts` on all nodes.
+1. Ensure proper hostname resolution by configuring `/etc/hosts` on all nodes.
-3. For MySQL 8 connection issues, verify the JDBC URL includes the correct SSL
parameters in `ambari.properties`.
+2. For MySQL 8 connection issues, verify the JDBC URL includes the correct SSL
parameters in `ambari.properties`.
-4. Check service logs:
+3. Check service logs:
- Ambari Server: `/var/log/ambari-server/ambari-server.log`
- Ambari Agent: `/var/log/ambari-agent/ambari-agent.log`
diff --git a/docs/release-notes.md b/docs/release-notes.md
index 25f84e9..fdc3390 100644
--- a/docs/release-notes.md
+++ b/docs/release-notes.md
@@ -6,10 +6,22 @@ sidebar_position: 2
## Overview
+:::tip Major Release
Apache Ambari 3.0.0 represents a significant milestone in the project's
development, bringing major improvements to cluster management capabilities,
user experience, and platform support. This release focuses on modernizing the
technology stack, enhancing security features, and improving overall stability
and performance.
+:::
## Release Highlights
+### Major Improvements
+
+- **Apache Bigtop Integration**:
+ - Ambari 3.0.0 now uses Apache Bigtop as the default packaging system for
component management.
+ - Bigtop Stack serves as the default project stack, providing a more
sustainable and community-driven approach to package management.
+
+:::tip Bigtop Component Compilation
+For developers interested in compiling components for the Ambari Bigtop Stack,
we have prepared a **[detailed guide](ambari-dev/bigtop-guide.md)** that walks
through the process step by step.
+:::
+
### Platform Support
- Added support for Rocky Linux 8 and 9
([AMBARI-26133](https://issues.apache.org/jira/browse/AMBARI-26133))
@@ -147,6 +159,7 @@ Apache Ambari 3.0.0 represents a significant milestone in
the project's developm
- Fixed issue with 'supported-refresh-commands' element
([AMBARI-25863](https://issues.apache.org/jira/browse/AMBARI-25863))
- Reduced excess Zookeeper and Hadoop logging
([AMBARI-24140](https://issues.apache.org/jira/browse/AMBARI-24140))
+
- **Bug Fixes**:
- Fixed OceanBase support in Ambari MySQL DDL
([AMBARI-26273](https://issues.apache.org/jira/browse/AMBARI-26273))
- Fixed regex pattern flag position in ambari_jinja2 filters
([AMBARI-26269](https://issues.apache.org/jira/browse/AMBARI-26269))
@@ -166,6 +179,77 @@ Apache Ambari 3.0.0 represents a significant milestone in
the project's developm
- Fixed alter dispatcher
([AMBARI-26240](https://issues.apache.org/jira/browse/AMBARI-26240))
- Corrected spelling mistakes in documentation
([AMBARI-26104](https://issues.apache.org/jira/browse/AMBARI-26104))
+## Acknowledgments
+
+:::tip Our Amazing Contributors 🌟
+Apache Ambari 3.0.0 represents the collective effort of our incredible
community. We are deeply honored to recognize the individuals who have shaped
this milestone release:
+
+<div class="alert alert--success">
+
+### Core Contributors 💫
+These dedicated individuals have made substantial contributions to this
release:
+
+- **jialiang** 🏆
+- **zrain** 🏆
+- **Peng Lu** 🏆
+- **Mohammad Arshad** 🏆
+- **Sandeep Kumar** 🏆
+- **coldless177** 🏆
+- **Vishal Suvagia** 🏆
+- **zhenye zhang** 🏆
+- **yaolei** 🏆
+- **xjmu** 🏆
+- **Himanshu Maurya** 🏆
+- **timyuer** 🏆
+- **yaruyng** 🏆
+- **tongxiaojun** 🏆
+
+### Community Champions 🌠
+Their valuable contributions have helped drive the project forward:
+
+- **Ananya Singh**
+- **rzuo**
+- **Viraj Jasani**
+- **William Horn**
+- **vanshuhassija**
+- **Yu Hou**
+- **basapuram-kumar**
+- **Prabhjyot Singh**
+- **Brahma Reddy Battula**
+- **Basapuram Kumar**
+- **Shreeya Sand**
+- **Bhavik Patel**
+
+### Technical Innovators ⚡
+Their technical expertise has been instrumental:
+
+- **lupeng**
+- **piaolingzxh**
+- **Murali Krishna**
+- **LiJie20190102**
+- **HARSHITH GANDHE**
+- **userhimanshuverma**
+- **Arnout Engelen**
+- **wangda**
+- **Dmytro Sen**
+- **Rich Bowen**
+- **Shubham Sharma**
+- **Weijian Wen**
+- **Will Guo**
+- **guluo**
+- **Peng Lee**
+- **liqinwyyx**
+
+</div>
+
+:::note Special Recognition 🎖
+A special note of appreciation goes to our Release Manager(s), whose tireless
coordination and attention to detail have been crucial in bringing this release
to fruition.
+:::
+
+:::tip Join Our Community! 🤝
+Every contribution, no matter how small, helps make Ambari better. We welcome
new contributors to join our vibrant community!
+:::
+
## Known Issues
- Timeline Service Reader may fail to start if HBase is not installed
([AMBARI-26248](https://issues.apache.org/jira/browse/AMBARI-26248))
@@ -230,68 +314,5 @@ sha256sum --check apache-ambari-3.0.0.tar.gz.sha256
# Verify the PGP signature
gpg --verify apache-ambari-3.0.0.tar.gz.asc apache-ambari-3.0.0.tar.gz
-```
-
-The PGP signatures can be verified using the public keys of the Apache Ambari
Release Managers, which can be found on the [Apache Ambari Download
Page](https://ambari.apache.org/download.html).
-
-## Acknowledgments
-Apache Ambari 3.0.0 is the result of significant contributions from the Apache
Ambari community. We would like to thank all the contributors who have helped
make this release possible:
-
-- jialiang
-- zrain
-- Peng Lu
-- Mohammad Arshad
-- yaruyng
-- tongxiaojun
-- Sandeep Kumar
-- coldless177
-- Vishal Suvagia
-- zhenye zhang
-- yaolei
-- xjmu
-- Himanshu Maurya
-- timyuer
-- Ananya Singh
-- rzuo
-- Viraj Jasani
-- William Horn
-- vanshuhassija
-- Yu Hou
-- basapuram-kumar
-- Prabhjyot Singh
-- Brahma Reddy Battula
-- Basapuram Kumar
-- Shreeya Sand
-- Bhavik Patel
-- lupeng
-- piaolingzxh
-- Murali Krishna
-- LiJie20190102
-- HARSHITH GANDHE
-- userhimanshuverma
-- Arnout Engelen
-- wangda
-- Dmytro Sen
-- Rich Bowen
-- Shubham Sharma
-- Weijian Wen
-- Will Guo
-- guluo
-- Peng Lee
-- liqinwyyx
-- and many others
-
-Special thanks to our Release Manager(s) for their dedication and hard work in
preparing this release.
-
-## Community
-
-We welcome your feedback and contributions to Apache Ambari:
-
-- [Mailing Lists](https://ambari.apache.org/mail-lists.html)
-- [Issue Tracker](https://issues.apache.org/jira/projects/AMBARI)
-- [GitHub Repository](https://github.com/apache/ambari)
-
-## License
-
-Apache Ambari is released under the [Apache License, Version
2.0](https://www.apache.org/licenses/LICENSE-2.0).
+```
\ No newline at end of file
diff --git a/sidebars.js b/sidebars.js
index c571cba..7a9c9b2 100644
--- a/sidebars.js
+++ b/sidebars.js
@@ -130,12 +130,15 @@ const sidebars = {
collapsed: false,
link: {type: 'doc', id: 'ambari-dev/index'},
items: [
+ "ambari-dev/building-from-source",
+ "ambari-dev/running-tests",
"ambari-dev/development-process-for-new-major-features",
"ambari-dev/ambari-code-layout",
"ambari-dev/apache-ambari-jira",
"ambari-dev/coding-guidelines-for-ambari",
"ambari-dev/how-to-commit",
"ambari-dev/how-to-contribute",
+ "ambari-dev/bigtop-guide",
"ambari-dev/code-review-guidelines",
"ambari-dev/releasing-ambari",
"ambari-dev/admin-view-ambari-admin-development",
diff --git a/src/pages/team/languages.json b/src/pages/team/languages.json
index 0858890..1c0eef9 100644
--- a/src/pages/team/languages.json
+++ b/src/pages/team/languages.json
@@ -5,13 +5,20 @@
"tip": "(In no particular order)"
},
"pmc": [
-
+ {
+ "apacheId": "arshad",
+ "avatarUrl": "https://avatars.githubusercontent.com/u/3433835?v=4";,
+ "email": "[email protected]",
+ "gitUrl": "https://github.com/arshadmohammad";,
+ "githubId": "arshadmohammad",
+ "name": "Mohammad Arshad"
+ },
{
"apacheId": "atri",
"avatarUrl": "https://avatars.githubusercontent.com/u/1724131?v=4";,
"email": "[email protected]",
"gitUrl": "https://github.com/atris";,
- "githubId": "atri",
+ "githubId": "atris",
"name": "Atri Sharma"
},
{
@@ -27,7 +34,7 @@
"avatarUrl": "https://avatars.githubusercontent.com/u/938524?v=4";,
"email": "[email protected]",
"gitUrl": "https://github.com/ddraj";,
- "githubId": "ddas",
+ "githubId": "ddraj",
"name": "Devaraj Das"
},
{
@@ -53,8 +60,7 @@
"gitUrl": "https://github.com/iwasakims";,
"githubId": "iwasakims",
"name": "Masatake Iwasaki"
- }
- ,
+ },
{
"apacheId": "junhe",
"avatarUrl": "https://avatars.githubusercontent.com/u/17865359?v=4";,
@@ -62,19 +68,15 @@
"gitUrl": "https://github.com/JunHe77";,
"githubId": "JunHe77",
"name": "Jun He"
- }
-
- ,
+ },
{
"apacheId": "masatana",
"avatarUrl": "https://avatars.githubusercontent.com/u/240015?v=4";,
"email": "[email protected]",
- "gitUrl": "https://github.com/masahirotanaka";,
- "githubId": "masahirotanaka",
+ "gitUrl": "https://github.com/masatana";,
+ "githubId": "masatana",
"name": "Masahiro Tanaka"
- }
-
- ,
+ },
{
"apacheId": "mithmatt",
"avatarUrl": "https://avatars.githubusercontent.com/u/5906984?v=4";,
@@ -82,9 +84,7 @@
"gitUrl": "https://github.com/mithmatt";,
"githubId": "mithmatt",
"name": "Matt"
- }
-
- ,
+ },
{
"apacheId": "rvs",
"avatarUrl": "https://avatars.githubusercontent.com/u/34680?v=4";,
@@ -100,8 +100,7 @@
"gitUrl": "https://github.com/sekikn";,
"githubId": "sekikn",
"name": "Kengo Seki"
- }
- ,
+ },
{
"apacheId": "umamahesh",
"avatarUrl": "https://avatars.githubusercontent.com/u/1719507?v=4";,
@@ -109,8 +108,7 @@
"gitUrl": "https://github.com/umamaheswararao";,
"githubId": "umamaheswararao",
"name": "Uma Maheswara Rao G"
- }
- ,
+ },
{
"apacheId": "vgogate",
"avatarUrl": "https://avatars.githubusercontent.com/u/913821?v=4";,
@@ -118,8 +116,7 @@
"gitUrl": "https://github.com/vgogate";,
"githubId": "vgogate",
"name": "Suhas"
- }
- ,
+ },
{
"apacheId": "vishalsuvagia",
"avatarUrl": "https://avatars.githubusercontent.com/u/7812238?v=4";,
@@ -127,8 +124,7 @@
"gitUrl": "https://github.com/vishalsuvagia";,
"githubId": "vishalsuvagia",
"name": "Vishal Suvagia"
- }
- ,
+ },
{
"apacheId": "weichiu",
"avatarUrl": "https://avatars.githubusercontent.com/u/2691807?v=4";,
@@ -144,9 +140,33 @@
"gitUrl": "https://github.com/kevinw66";,
"githubId": "kevinw66",
"name": "Zhiguo Wu"
+ },
+ {
+ "apacheId": "vjasani",
+ "avatarUrl": "https://avatars.githubusercontent.com/u/34790606?v=4";,
+ "email": "[email protected]",
+ "gitUrl": "https://github.com/virajjasani";,
+ "githubId": "virajjasani",
+ "name": "Viraj Jasani"
}
],
"committer": [
+ {
+ "apacheId": "jialiang",
+ "avatarUrl": "https://avatars.githubusercontent.com/u/18082602?v=4";,
+ "email": "[email protected]",
+ "gitUrl": "https://github.com/Jialiangc";,
+ "githubId": "Jialiangc",
+ "name": "Jialiang Cai"
+ },
+ {
+ "apacheId": "zhangyu",
+ "avatarUrl": "https://avatars.githubusercontent.com/u/16508606?v=4";,
+ "email": "[email protected]",
+ "gitUrl": "https://github.com/zRains";,
+ "githubId": "zRains",
+ "name": "Yu Zhang"
+ },
{
"apacheId": "yaolei",
"avatarUrl": "https://avatars.githubusercontent.com/u/13212217?v=4";,
@@ -156,12 +176,12 @@
"name": "Lei Yao"
},
{
- "apacheId": "vjasani",
- "avatarUrl": "https://avatars.githubusercontent.com/u/34790606?v=4";,
- "email": "[email protected]",
- "gitUrl": "https://github.com/virajjasani";,
- "githubId": "virajjasani",
- "name": "Viraj Jasani"
+ "apacheId": "houyu",
+ "avatarUrl": "https://avatars.githubusercontent.com/u/16263438?v=4 ",
+ "email": "[email protected]",
+ "gitUrl": "https://github.com/timyuer";,
+ "githubId": "timyuer",
+ "name": "Yu Hou"
}
]
}
diff --git a/versioned_docs/version-3.0.0/ambari-dev/bigtop-guide.md
b/versioned_docs/version-3.0.0/ambari-dev/bigtop-guide.md
new file mode 100644
index 0000000..875468a
--- /dev/null
+++ b/versioned_docs/version-3.0.0/ambari-dev/bigtop-guide.md
@@ -0,0 +1,154 @@
+---
+title: Compiling Components for Ambari Bigtop Stack
+---
+
+# Compiling Components for Ambari Bigtop Stack
+
+## Introduction
+
+Apache Bigtop is designed for infrastructure engineers and data scientists
seeking comprehensive packaging, testing, and configuration of leading
open-source big data components. Bigtop supports a wide range of
components/projects, including but not limited to Hadoop, HBase, and Spark.
This guide specifically focuses on how to compile components for the **Ambari
Bigtop Stack**.
+
+## Use Cases for Apache Bigtop
+
+1. **Simplified Package Building**: Bigtop significantly simplifies the
process of compiling RPM or DEB packages for big data components across
different operating systems through pre-configured Docker images, making it
quick and efficient.
+
+2. **Dependency Management**: Bigtop integrates complex dependencies required
during the compilation process, effectively resolving common compilation errors
and ensuring a smooth compilation experience through patches in the code. This
means users no longer need to worry about official packages failing to compile
or setting up complex compilation environments.
+
+3. **Apache Ambari Support**: Bigtop provides support for Apache Ambari,
allowing users to easily package big data software that is compatible with
Ambari and meets installation requirements.
+
+## Getting Started with Bigtop
+
+This guide uses the official Bigtop 3.3.0 as an example, with CentOS 7 as the
compilation operating system. The same operations apply to other systems and
versions.
+
+### Prerequisites
+
+- Linux environment
+- Docker installed on your system
+- Git
+
+### Step-by-Step Guide
+
+#### 1. Create a Development Directory
+
+```bash
+mkdir ~/dev/
+```
+
+#### 2. Clone Bigtop Repository
+
+```bash
+cd ~/dev/
+git clone https://github.com/apache/bigtop.git
+```
+
+#### 3. Switch to Version 3.3.0
+
+```bash
+cd bigtop
+git checkout release-3.3.0
+```
+
+#### 4. Pull the Bigtop CentOS 7 Compilation Environment Image
+
+```bash
+# If you need to compile for other operating systems or architectures (e.g.,
ARM),
+# you can search for the corresponding Bigtop version in the image repository
+# https://hub.docker.com/r/bigtop/slaves/tags
+docker pull bigtop/slaves:3.3.0-centos-7
+```
+
+#### 5. Launch the Container
+
+**Scenario 1**: If you've previously compiled big data components locally and
have a Maven repository cache, it's best to map this directory to the
container's default Maven download directory to avoid downloading packages
again.
+
+For example, if your local Maven repository directory is `/data/repository`:
+
+```bash
+cd ~/dev/bigtop
+docker run -d -it --network host -v `pwd`:/ws -v
/data/repository:/root/.m2/repository --workdir /ws --name bigtopr
bigtop/slaves:3.3.0-centos-7
+```
+
+**Scenario 2**: If you don't have a Maven cache locally or are unfamiliar with
this, you should still map a directory to the Bigtop container to facilitate
repeated compilations using downloaded Maven cache. Otherwise, when the
container is deleted, your Maven cache will be lost, and dependency downloading
is the most time-consuming stage of recompilation.
+
+```bash
+mkdir -p ~/m2/repository
+cd ~/dev/bigtop
+docker run -d -it --network host -v `pwd`:/ws -v
~/m2/repository:/root/.m2/repository --workdir /ws --name bigtopr
bigtop/slaves:3.3.0-centos-7
+```
+
+#### 6. Modify Maven Repository Settings (Optional)
+
+You can configure Maven to use mirrors that are faster for your location. This
step is optional but can significantly improve download speeds.
+
+1. Enter the container:
+```bash
+docker exec -it bigtopr /bin/bash
+```
+
+2. Edit the Maven settings file:
+```bash
+vi /usr/local/maven/conf/settings.xml
+```
+
+3. Add appropriate mirror repositories based on your location. For example:
+
+```xml
+<mirrors>
+ <mirror>
+ <id>central-mirror</id>
+ <mirrorOf>central</mirrorOf>
+ <name>Central Repository Mirror</name>
+ <url>https://repo1.maven.org/maven2/</url>
+ </mirror>
+ <!-- Add other mirrors as needed -->
+</mirrors>
+```
+
+#### 7. Compile Big Data Components
+
+Enter your running container:
+
+```bash
+docker exec -it bigtopr /bin/bash
+```
+
+Compile components:
+
+```bash
+. /etc/profile.d/bigtop.sh
+./gradlew flink-clean flink-pkg -PparentDir=/usr/bigtop -PpkgSuffix
-PbuildThreads=2C repo
+```
+
+**Explanation of compilation parameters**:
+
+- `-PparentDir=/usr/bigtop`: Changes the default installation path of the
package, making Bigtop-built packages conform to Ambari installation
specifications.
+- `-PpkgSuffix`: Makes the output package include the Bigtop version number
(e.g., hadoop_3_3_0), conforming to Ambari Bigtop service specifications.
+- `-PbuildThreads=2C`: Sets the number of threads for compilation (2 times the
number of CPU cores).
+
+## Parallel Compilation for Improved Performance
+
+A pull request for parallel compilation to speed up the build process has been
submitted to the community and is currently under review. Once merged, all Java
components in Bigtop will be able to compile in parallel, expected to be
available in versions after Bigtop 3.3.1.
+
+Performance comparison for parallel compilation (after all dependencies are
downloaded):
+
+| Component | Time Before | Time After |
+|------------|-------------|------------|
+| Alluxio | 21min | 07:43min |
+| Hive | 05:33min | 03:04min |
+| HBase | 06:18min | 02:55min |
+| Zookeeper | 01:25min | 35s |
+| Livy | 03:29min | 03:12min |
+| Phoenix | 11:23min | 05:32min |
+| Zeppelin | 14:15min | 13:19min |
+| Flink | 36:27min | 14:16min |
+| Hadoop | 50min | 16min |
+
+Example of parallel compilation command:
+
+```bash
+docker run -d -it --network host -v `pwd`:/ws -v
/data/repository:/data/repository --workdir /ws --name bigtop
bigtop/slaves:trunk-centos-7 --cpus 16
+source /etc/profile.d/bigtop.sh
+./gradlew alluxio-clean alluxio-pkg -PcompileThreads=2C
+```
+
+This approach shows a 2-3x improvement in compilation speed, with even more
significant effects during initial compilation (e.g., Hadoop initial
compilation time reduced from 3 hours to 1 hour).
diff --git a/versioned_docs/version-3.0.0/ambari-dev/building-from-source.md
b/versioned_docs/version-3.0.0/ambari-dev/building-from-source.md
new file mode 100644
index 0000000..514760b
--- /dev/null
+++ b/versioned_docs/version-3.0.0/ambari-dev/building-from-source.md
@@ -0,0 +1,125 @@
+---
+title: Building from Source
+---
+
+# Building Apache Ambari from Source
+
+This guide explains how to build Apache Ambari 3.0 and its related subprojects
from source code.
+
+## Prerequisites
+
+Before you begin, ensure you have the following requirements installed:
+
+### System Requirements
+- Operating System: Rocky Linux 8 or 9 (recommended)
+- Python 3 Development Tools (`python3-devel`)
+
+### Java Requirements
+- Ambari Main Project: JDK 17
+- Ambari Metrics: JDK 8
+- Ambari Infra: JDK 8
+
+## Building Ambari Main Project
+
+### 1. Clone the Repository
+```bash
+git clone [email protected]:apache/ambari.git
+cd ambari
+```
+
+### 2. Build Options
+
+#### Build Without RPM
+To build Ambari without creating RPM packages:
+```bash
+mvn -B -T 2C clean install package \
+ -Drat.skip=true \
+ -DskipTests \
+ -Dmaven.test.skip=true \
+ -Dfindbugs.skip=true \
+ -Dcheckstyle.skip=true
+```
+
+#### Build With RPM
+To build Ambari and create RPM packages:
+```bash
+mvn -B -T 2C clean install package rpm:rpm \
+ -Drat.skip=true \
+ -DskipTests \
+ -Dmaven.test.skip=true \
+ -Dfindbugs.skip=true \
+ -Dcheckstyle.skip=true
+```
+
+The RPM packages will be generated at:
+- Ambari Agent:
`ambari/ambari-agent/target/rpm/ambari-agent/RPMS/x86_64/ambari-agent-3.0.0.0-SNAPSHOT.x86_64.rpm`
+- Ambari Server:
`ambari/ambari-server/target/rpm/ambari-server/RPMS/x86_64/ambari-server-3.0.0.0-SNAPSHOT.x86_64.rpm`
+
+## Building Ambari Metrics
+
+:::tip Performance Optimization
+To significantly improve build performance, download binary dependencies
locally before building:
+
+1. Create a local directory for dependencies:
+```bash
+mkdir -p /ws/dl/
+```
+
+2. Download the required binary files:
+```bash
+wget -P /ws/dl/
http://repo.bigtop.apache.org.s3.amazonaws.com/bigtop-stack-binary/3.2.0/centos-7/x86_64/hbase-2.4.13-bin.tar.gz
+wget -P /ws/dl/
http://repo.bigtop.apache.org.s3.amazonaws.com/bigtop-stack-binary/3.2.0/centos-7/x86_64/hadoop-3.3.4.tar.gz
+wget -P /ws/dl/
https://dl.grafana.com/oss/release/grafana-11.1.4.linux-amd64.tar.gz
+wget -P /ws/dl/
http://repo.bigtop.apache.org.s3.amazonaws.com/bigtop-stack-binary/3.2.0/centos-7/x86_64/phoenix-hbase-2.4-5.1.2-bin.tar.gz
+```
+
+3. Modify the `pom.xml` in ambari-metrics project to use local files:
+```xml
+<!-- Update these properties to use local files -->
+<properties>
+ <hbase.tar>file:///ws/dl/hbase-2.4.13-bin.tar.gz</hbase.tar>
+ <hadoop.tar>file:///ws/dl/hadoop-3.3.4.tar.gz</hadoop.tar>
+ <grafana.tar>file:///ws/dl/grafana-11.1.4.linux-amd64.tar.gz</grafana.tar>
+ <phoenix.tar>file:///ws/dl/phoenix-hbase-2.4-5.1.2-bin.tar.gz</phoenix.tar>
+</properties>
+```
+
+This optimization will save significant time during repeated builds by
avoiding large downloads.
+:::
+
+### 1. Clone the Repository
+```bash
+git clone [email protected]:apache/ambari-metrics.git
+cd ambari-metrics
+```
+
+### 2. Build Options
+
+#### Build Without RPM
+To build Ambari Metrics without creating RPM packages:
+```bash
+mvn -T 2C clean install -DskipTests
+```
+
+#### Build With RPM
+To build Ambari Metrics and create RPM packages:
+```bash
+mvn -T 2C clean install -DskipTests -Dbuild-rpm
+```
+
+To locate the generated RPM packages:
+```bash
+find ./ -name "*.rpm"
+```
+
+## Building Ambari Infra
+
+### 1. Clone the Repository
+```bash
+git clone [email protected]:apache/ambari-infra.git
+cd ambari-infra
+```
+
+### 2. Build RPM Package
+```bash
+make rpm
diff --git a/versioned_docs/version-3.0.0/ambari-dev/how-to-contribute.md
b/versioned_docs/version-3.0.0/ambari-dev/how-to-contribute.md
index 1e44e8c..216d0bf 100644
--- a/versioned_docs/version-3.0.0/ambari-dev/how-to-contribute.md
+++ b/versioned_docs/version-3.0.0/ambari-dev/how-to-contribute.md
@@ -36,11 +36,21 @@ Repeat these steps for all the branches that needs to be
synced with the remote.
Apache Ambari uses JIRA to track issues including bugs and improvements, and
uses Github pull requests to manage code reviews and code merges. Major design
changes are discussed in JIRA and implementation changes are discussed in pull
requests after a pull request is created.
+:::note Important Changes to JIRA Registration
+* JIRA registration is currently closed to the public
+* To get a JIRA account:
+ 1. Register on [Apache JIRA](https://issues.apache.org/jira)
+ 2. Contact a PMC member to approve your registration
+* Alternatively, you can:
+ 1. Submit your Pull Request first
+ 2. Community members will help create the corresponding JIRA ticket for you
+:::
+
* Find an existing Apache JIRA that the change pertains to
* Do not create a new JIRA if the change is minor and relates to an
existing JIRA; add to the existing discussion and work instead
* Look for existing pull requests that are linked from the JIRA, to
understand if someone is already working on the JIRA
-* If the change is new, then create a new JIRA:
+* If the change is new and you have JIRA access, then create a new JIRA:
* Provide a descriptive Title
* Write a detailed Description. For bug reports, this should ideally
include a short reproduction of the problem. For new features, it may include a
design document.
* Fill the required fields:
@@ -55,6 +65,11 @@ Apache Ambari uses JIRA to track issues including bugs and
improvements, and use
* Affects Version. For Bugs, assign at least one version that is known
to exhibit the problem or need the change
* Do not include a patch file; pull requests are used to propose the
actual change.
+* If you don't have JIRA access:
+ * Submit your Pull Request first
+ * In the PR description, clearly describe the issue or improvement
+ * A community member will create a JIRA ticket and link it to your PR
+
### Pull Request
Apache Ambari uses [Github pull
requests](https://github.com/apache/ambari/pulls) to review and merge changes
to the source code. Before creating a pull request, one must have a fork of
apache/ambari checked out. Follow instructions in step 1 to create a fork if
you haven't already.
diff --git a/versioned_docs/version-3.0.0/ambari-dev/running-tests.md
b/versioned_docs/version-3.0.0/ambari-dev/running-tests.md
new file mode 100644
index 0000000..310988a
--- /dev/null
+++ b/versioned_docs/version-3.0.0/ambari-dev/running-tests.md
@@ -0,0 +1,109 @@
+---
+title: Running Tests
+---
+
+# Running Tests in Apache Ambari
+
+This guide explains how to run different types of tests in Apache Ambari. For
the official test configuration, you can refer to the [Ambari
Jenkinsfile](https://github.com/apache/ambari/blob/trunk/Jenkinsfile).
+
+## Java Tests
+
+### Running All Java Tests
+To run all Java tests for the Ambari Server:
+```bash
+mvn -am test -pl ambari-server \
+ -DskipPythonTests \
+ -Dmaven.test.failure.ignore \
+ -Dmaven.artifact.threads=10 \
+ -Drat.skip \
+ -DskipAdminWebTests=true
+```
+
+### Running Specific Java Tests
+To run a specific Java test class:
+```bash
+mvn -am test -pl ambari-server \
+ -DskipPythonTests \
+ -Dmaven.test.failure.ignore \
+ -Dmaven.artifact.threads=10 \
+ -Drat.skip \
+ -DskipAdminWebTests=true \
+ -Dtest=AmbariServerTest
+```
+
+To run a specific test method within a test class:
+```bash
+mvn -am test -pl ambari-server \
+ -DskipPythonTests \
+ -Dmaven.test.failure.ignore \
+ -Dmaven.artifact.threads=10 \
+ -Drat.skip \
+ -DskipAdminWebTests=true \
+ -Dtest=AmbariServerTest#testMethodName
+```
+
+### Test Parameters Explained
+- `-am`: Also build dependencies
+- `-pl ambari-server`: Only build the ambari-server module
+- `-DskipPythonTests`: Skip Python tests
+- `-Dmaven.test.failure.ignore`: Continue the build even if tests fail
+- `-Dmaven.artifact.threads=10`: Use 10 threads for parallel artifact
resolution
+- `-Drat.skip`: Skip Apache RAT (Release Audit Tool) checks
+- `-DskipAdminWebTests`: Skip admin web interface tests
+- `-Dtest`: Specify which test class or method to run
+
+## Python Tests
+
+### Running All Python Tests
+To run all Python tests:
+```bash
+mvn test -pl ambari-server \
+ -DskipJavaTests \
+ -Dpython.test.mask="*_test.py" \
+ -Dpython.test.skip.pattern="agent_perf.py"
+```
+
+### Running Specific Python Tests
+To run a specific Python test file:
+```bash
+mvn test -pl ambari-server \
+ -DskipJavaTests \
+ -Dpython.test.mask="test_file_name.py"
+```
+
+### Python Test Parameters Explained
+- `-DskipJavaTests`: Skip Java tests
+- `-Dpython.test.mask`: Pattern to match test files to run
+- `-Dpython.test.skip.pattern`: Pattern to match test files to skip
+
+## Integration Tests
+
+### Running Integration Tests
+To run integration tests:
+```bash
+mvn verify -pl ambari-server \
+ -P integration-tests \
+ -DskipPythonTests \
+ -DskipJavaTests
+```
+
+### Integration Test Parameters
+- `-P integration-tests`: Activate the integration-tests profile
+- `-DskipPythonTests`: Skip Python tests
+- `-DskipJavaTests`: Skip Java tests
+
+## Test Reports
+
+Test reports can be found in the following locations after test execution:
+
+### Java Test Reports
+- Unit Tests: `ambari-server/target/surefire-reports/`
+- Integration Tests: `ambari-server/target/failsafe-reports/`
+
+### Python Test Reports
+- Test Results: `ambari-server/target/python-test-results/`
+- Coverage Reports: `ambari-server/target/python-coverage/`
+
+:::tip
+When debugging test failures, check these report directories for detailed test
execution logs and stack traces.
+:::
diff --git a/versioned_docs/version-3.0.0/quick-start/installation-guide.md
b/versioned_docs/version-3.0.0/quick-start/installation-guide.md
index db52d1d..485a50e 100644
--- a/versioned_docs/version-3.0.0/quick-start/installation-guide.md
+++ b/versioned_docs/version-3.0.0/quick-start/installation-guide.md
@@ -2,6 +2,42 @@
This guide covers the installation and setup of Apache Ambari on bare metal,
KVM, Docker, or Vagrant environments.
+## Important: Firewall Configuration
+
+:::tip Before You Begin
+Firewall settings can significantly impact cluster functionality by blocking
necessary communication ports between components. Please review the following
guidelines carefully.
+:::
+
+### Development & Testing Environments
+For development or testing environments, consider disabling the firewall:
+```bash
+systemctl stop firewalld
+systemctl disable firewalld
+```
+
+### Production Environments
+
+#### For System Administrators
+Configure the firewall to allow required Hadoop ecosystem ports based on your
specific component deployment:
+
+| Component | Ports | Purpose |
+|-----------|-------|---------|
+| Ambari Server | 8080, 8440, 8441 | Web UI, Agent communication |
+| Core Hadoop | 8020, 9000, 50070, 50075 | HDFS NameNode, DataNode HTTP |
+| YARN | 8032, 8088, 19888 | ResourceManager, UI, JobHistory |
+| Hive | 9083, 10000 | Metastore, HiveServer2 |
+
+:::note NodeManager Ports
+YARN NodeManagers allocate containers on dynamically assigned ports (default
range: 32768-65535).
+This range can be restricted via `yarn.nodemanager.resource.ports` in
`yarn-site.xml`
+:::
+
+:::info For General Users
+If you're not familiar with advanced network configuration, we recommend:
+- Disabling the firewall during initial setup and cluster testing
+- Consulting with a network security specialist for production deployments
+:::
+
## Prerequisites
Ensure you have a working environment (bare metal, KVM, Docker, or [Vagrant
setup](./environment-setup/vagrant-environment-setup.md)) before proceeding.
@@ -204,16 +240,10 @@ Default credentials:
## Troubleshooting
-1. If you encounter firewall issues, disable it (for development environments
only):
-```bash
-systemctl stop firewalld
-systemctl disable firewalld
-```
-
-2. Ensure proper hostname resolution by configuring `/etc/hosts` on all nodes.
+1. Ensure proper hostname resolution by configuring `/etc/hosts` on all nodes.
-3. For MySQL 8 connection issues, verify the JDBC URL includes the correct SSL
parameters in `ambari.properties`.
+2. For MySQL 8 connection issues, verify the JDBC URL includes the correct SSL
parameters in `ambari.properties`.
-4. Check service logs:
+3. Check service logs:
- Ambari Server: `/var/log/ambari-server/ambari-server.log`
- Ambari Agent: `/var/log/ambari-agent/ambari-agent.log`
diff --git a/versioned_docs/version-3.0.0/release-notes.md
b/versioned_docs/version-3.0.0/release-notes.md
index 25f84e9..f6e491a 100644
--- a/versioned_docs/version-3.0.0/release-notes.md
+++ b/versioned_docs/version-3.0.0/release-notes.md
@@ -6,17 +6,92 @@ sidebar_position: 2
## Overview
+:::tip Major Release
Apache Ambari 3.0.0 represents a significant milestone in the project's
development, bringing major improvements to cluster management capabilities,
user experience, and platform support. This release focuses on modernizing the
technology stack, enhancing security features, and improving overall stability
and performance.
+:::
+
+## Acknowledgment and Important Notes
+
+### A Message to Our Community
+
+:::info Important Transition
+We want to acknowledge that Ambari 3.0 represents a significant milestone as
the first major release since the HDP project became closed source. Due to this
transition, we have made some important decisions:
+
+- The 2.7.x series has reached End-of-Life (EOL) and will no longer be
maintained, as we no longer have access to the HDP packaging source code.
+- Version 3.0 marks the beginning of our new mainline branch, which will
receive primary maintenance and development focus.
+:::
+
+### From Our Hearts to the Community
+
+:::note A Journey Together
+The path to this release has been long and challenging. Our community has
weathered significant changes and obstacles:
+
+<div class="alert alert--secondary">
+
+- 🔄 Community restructuring and transitions
+- 👥 A significantly reduced contributor base
+- 🛠 Critical infrastructure challenges, especially around RPM distribution
+- 🔨 Profound organizational changes
+
+</div>
+:::
+
+:::tip Thank You
+To every user who has stood by Ambari through these times: your patience and
understanding mean more to us than words can express. We know this release has
been a long time coming, and we are truly sorry for keeping you waiting.
+:::
+
+:::info With Gratitude
+Our hearts are filled with gratitude for:
+
+<div class="alert alert--success">
+
+- 🌟 The pioneers who laid Ambari's foundation
+- ⭐️ The steadfast contributors who've stayed with us through the storms
+- 💫 Every community member whose belief in open source has never wavered
+
+</div>
+
+This release exists because of your unwavering support and dedication to the
open source spirit. Your commitment has been our guiding light through these
challenging times.
+:::
+
+### Technical Changes
+
+#### Apache Bigtop Integration
+:::tip New Integration
+We are pleased to announce that Ambari 3.0 now uses Apache Bigtop for
component packaging:
+
+<div class="alert alert--success">
+
+- 📦 [Apache Bigtop](https://github.com/apache/bigtop/) is now our default
packaging system
+- 🔧 Bigtop Stack serves as the default project stack
+- 🚀 This integration provides a more sustainable and community-driven approach
to package management
+
+</div>
+:::
## Release Highlights
+### Major Improvements
+
+- **Apache Bigtop Integration**:
+ - Ambari 3.0.0 now uses Apache Bigtop as the default packaging system for
component management.
+ - Bigtop Stack serves as the default project stack, providing a more
sustainable and community-driven approach to package management.
+
+:::tip Bigtop Component Compilation
+For developers interested in compiling components for the Ambari Bigtop Stack,
we have prepared a **[detailed guide](ambari-dev/bigtop-guide.md)** that walks
through the process step by step.
+:::
+
### Platform Support
-- Added support for Rocky Linux 8 and 9
([AMBARI-26133](https://issues.apache.org/jira/browse/AMBARI-26133))
-- Added support for openEuler-22.03
([AMBARI-26126](https://issues.apache.org/jira/browse/AMBARI-26126))
-- Added Java 17 support
([AMBARI-26142](https://issues.apache.org/jira/browse/AMBARI-26142),
[AMBARI-26186](https://issues.apache.org/jira/browse/AMBARI-26186))
-- Added MySQL 8 support for Hive
([AMBARI-26130](https://issues.apache.org/jira/browse/AMBARI-26130))
-- Enhanced Docker environment with bigtop/puppet:trunk-rockylinux-8 image
+<div class="alert alert--info">
+
+- ✅ Added support for Rocky Linux 8 and 9
([AMBARI-26133](https://issues.apache.org/jira/browse/AMBARI-26133))
+- ✅ Added support for openEuler-22.03
([AMBARI-26126](https://issues.apache.org/jira/browse/AMBARI-26126))
+- ☕️ Added Java 17 support
([AMBARI-26142](https://issues.apache.org/jira/browse/AMBARI-26142),
[AMBARI-26186](https://issues.apache.org/jira/browse/AMBARI-26186))
+- 🗄 Added MySQL 8 support for Hive
([AMBARI-26130](https://issues.apache.org/jira/browse/AMBARI-26130))
+- 📈 Enhanced Docker environment with bigtop/puppet:trunk-rockylinux-8 image
+
+</div>
### New Services and Components
@@ -236,53 +311,73 @@ The PGP signatures can be verified using the public keys
of the Apache Ambari Re
## Acknowledgments
-Apache Ambari 3.0.0 is the result of significant contributions from the Apache
Ambari community. We would like to thank all the contributors who have helped
make this release possible:
-
-- jialiang
-- zrain
-- Peng Lu
-- Mohammad Arshad
-- yaruyng
-- tongxiaojun
-- Sandeep Kumar
-- coldless177
-- Vishal Suvagia
-- zhenye zhang
-- yaolei
-- xjmu
-- Himanshu Maurya
-- timyuer
-- Ananya Singh
-- rzuo
-- Viraj Jasani
-- William Horn
-- vanshuhassija
-- Yu Hou
-- basapuram-kumar
-- Prabhjyot Singh
-- Brahma Reddy Battula
-- Basapuram Kumar
-- Shreeya Sand
-- Bhavik Patel
-- lupeng
-- piaolingzxh
-- Murali Krishna
-- LiJie20190102
-- HARSHITH GANDHE
-- userhimanshuverma
-- Arnout Engelen
-- wangda
-- Dmytro Sen
-- Rich Bowen
-- Shubham Sharma
-- Weijian Wen
-- Will Guo
-- guluo
-- Peng Lee
-- liqinwyyx
-- and many others
-
-Special thanks to our Release Manager(s) for their dedication and hard work in
preparing this release.
+:::tip Our Amazing Contributors 🌟
+Apache Ambari 3.0.0 represents the collective effort of our incredible
community. We are deeply honored to recognize the individuals who have shaped
this milestone release:
+
+<div class="alert alert--success">
+
+### Core Contributors 💫
+These dedicated individuals have made substantial contributions to this
release:
+
+- **jialiang** 🏆
+- **zrain** 🏆
+- **Peng Lu** 🏆
+- **Mohammad Arshad** 🏆
+- **Sandeep Kumar** 🏆
+- **coldless177** 🏆
+- **Vishal Suvagia** 🏆
+- **zhenye zhang** 🏆
+- **yaolei** 🏆
+- **xjmu** 🏆
+- **Himanshu Maurya** 🏆
+- **timyuer** 🏆
+- **yaruyng** 🏆
+- **tongxiaojun** 🏆
+### Community Champions 🌠
+Their valuable contributions have helped drive the project forward:
+
+- **Ananya Singh**
+- **rzuo**
+- **Viraj Jasani**
+- **William Horn**
+- **vanshuhassija**
+- **Yu Hou**
+- **basapuram-kumar**
+- **Prabhjyot Singh**
+- **Brahma Reddy Battula**
+- **Basapuram Kumar**
+- **Shreeya Sand**
+- **Bhavik Patel**
+
+### Technical Innovators ⚡
+Their technical expertise has been instrumental:
+
+- **lupeng**
+- **piaolingzxh**
+- **Murali Krishna**
+- **LiJie20190102**
+- **HARSHITH GANDHE**
+- **userhimanshuverma**
+- **Arnout Engelen**
+- **wangda**
+- **Dmytro Sen**
+- **Rich Bowen**
+- **Shubham Sharma**
+- **Weijian Wen**
+- **Will Guo**
+- **guluo**
+- **Peng Lee**
+- **liqinwyyx**
+
+</div>
+
+:::note Special Recognition 🎖
+A special note of appreciation goes to our Release Manager(s), whose tireless
coordination and attention to detail have been crucial in bringing this release
to fruition.
+:::
+
+:::tip Join Our Community! 🤝
+Every contribution, no matter how small, helps make Ambari better. We welcome
new contributors to join our vibrant community!
+:::
## Community
diff --git a/versioned_sidebars/version-3.0.0-sidebars.json
b/versioned_sidebars/version-3.0.0-sidebars.json
index 1551e3e..6b26dc6 100644
--- a/versioned_sidebars/version-3.0.0-sidebars.json
+++ b/versioned_sidebars/version-3.0.0-sidebars.json
@@ -123,17 +123,22 @@
{
"type": "category",
"label": "Ambari Development",
+ "collapsible": true,
+ "collapsed": false,
"link": {
"type": "doc",
"id": "ambari-dev/index"
},
"items": [
+ "ambari-dev/building-from-source",
+ "ambari-dev/running-tests",
"ambari-dev/development-process-for-new-major-features",
"ambari-dev/ambari-code-layout",
"ambari-dev/apache-ambari-jira",
"ambari-dev/coding-guidelines-for-ambari",
"ambari-dev/how-to-commit",
"ambari-dev/how-to-contribute",
+ "ambari-dev/bigtop-guide",
"ambari-dev/code-review-guidelines",
"ambari-dev/releasing-ambari",
"ambari-dev/admin-view-ambari-admin-development",
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
