Re: [PR] TEZ-4682: [Cloud] Tez AM docker image [tez]

[via GitHub]

Aggarwal-Raghav commented on PR #456:
URL: https://github.com/apache/tez/pull/456#issuecomment-4109078988

   > 1. I have used official hadoop docker image 3.4.2-lean
   > 2. namenode and datanode will be separate service otherwise we need to use 
a custom entrypoint to start in a single container as happening in 
minimal-hadoop docker image. I think it is better to have separate. I have 
added a wait also to ensure namenode is not in safemode before datanode and 
tez-am is up.
   > 3. Handled the review comments regarding docs and file naming conventions.
   > 4. Added a sample program to run in tez-docker am and here it get tricky 
⚠️. It is throwing same error as mentioned in TEZ-4686 with standalone program.
   >    Steps:
   > 
   > ```
   > 1. cd tez-dist/src/docker/tez-am/
   > 2. docker compose up -d --build
   > ```
   > 
   > <img alt="Screenshot 2026-03-20 at 12 26 15 AM" width="1231" height="278" 
src="https://private-user-images.githubusercontent.com/49618756/566467592-4019dec9-6ee0-48c8-ab44-d42677281ad6.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NzQyNTczNTgsIm5iZiI6MTc3NDI1NzA1OCwicGF0aCI6Ii80OTYxODc1Ni81NjY0Njc1OTItNDAxOWRlYzktNmVlMC00OGM4LWFiNDQtZDQyNjc3MjgxYWQ2LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNjAzMjMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjYwMzIzVDA5MTA1OFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTU3OGRkZGMwMjg4ZTIyYTQ1YTBjZDYyNjhjYjhmMGNmNGFkMDM4MzE2NDA1YzZhNDViYjcxMjVhNDljNjZkZGImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.Q462h733j2EscyfvQZoXWAJSYlitRHGX9igipfYCtQk";>
   > **Everything should be running at this point**
   > 
   > ```
   > 3. docker exec -it tez-am bash
   > 4. echo "Hello world Hello" > /tmp/input.txt
   > 5. java -cp ./*:./lib/*:tez-examples-1.0.0-SNAPSHOT.jar 
org.apache.tez.examples.ExternalAmWordCount /tmp/input.txt /tmp/output
   > ```
   > 
   > **With TEZ-4686 cherrypick:**
   > 
   > ```
   > ❯ docker exec -it tez-am bash
   > bash-5.1$ echo "Hello world Hello" > /tmp/input.txt
   > java -cp ./*:./lib/*:tez-examples-1.0.0-SNAPSHOT.jar 
org.apache.tez.examples.ExternalAmWordCount /tmp/input.txt /tmp/output
   > log4j:WARN No appenders could be found for logger 
(org.apache.hadoop.util.Shell).
   > log4j:WARN Please initialize the log4j system properly.
   > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for 
more info.
   > bash-5.1$ ls /tmp/output
   > attempt_1773945902335_0000_1_00_000000_0_10003     
attempt_1773945902335_0000_1_00_000000_0_10003_0  part-v001-o000-r-00000  
_SUCCESS
   > bash-5.1$ cat /tmp/output/part-v001-o000-r-00000
   > Hello      2
   > world      1
   > bash-5.1$
   > ```
   > 
   > **Without TEZ-4686 cherrypick:**
   > 
   > ```
   > ❯ docker exec -it tez-am bash
   > bash-5.1$ echo "Hello world Hello" > /tmp/input.txt
   > java -cp ./*:./lib/*:tez-examples-1.0.0-SNAPSHOT.jar 
org.apache.tez.examples.ExternalAmWordCount /tmp/input.txt /tmp/output
   > log4j:WARN No appenders could be found for logger 
(org.apache.hadoop.util.Shell).
   > log4j:WARN Please initialize the log4j system properly.
   > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for 
more info.
   > Exception in thread "main" java.lang.NullPointerException: Cannot invoke 
"org.apache.tez.client.registry.AMRecord.getApplicationId()" because 
"this.amRecord" is null
   >    at 
org.apache.tez.client.registry.zookeeper.ZkFrameworkClient.createApplication(ZkFrameworkClient.java:114)
   >    at 
org.apache.tez.client.TezClient.createApplication(TezClient.java:1103)
   >    at org.apache.tez.client.TezClient.start(TezClient.java:399)
   >    at 
org.apache.tez.examples.ExternalAmWordCount.main(ExternalAmWordCount.java:74)
   > ```
   
   @abstractdog , any input on this?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@tez.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org