Hean-Chhinling commented on code in PR #8227:
URL: https://github.com/apache/hadoop/pull/8227#discussion_r2774062999
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/jsonprovider/NMJsonProvider.java:
##########
@@ -0,0 +1,67 @@
+package org.apache.hadoop.yarn.server.nodemanager.webapp.jsonprovider;
+
+import
org.apache.hadoop.yarn.server.nodemanager.webapp.dao.NMDeviceResourceInfo;
+import
org.apache.hadoop.yarn.server.nodemanager.webapp.dao.gpu.NMGpuResourceInfo;
+import org.apache.hadoop.yarn.server.webapp.dao.ContainerLogsInfoes;
+import org.eclipse.persistence.jaxb.MarshallerProperties;
+import org.eclipse.persistence.jaxb.rs.MOXyJsonProvider;
+
+import javax.ws.rs.Consumes;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.MultivaluedMap;
+import javax.ws.rs.ext.Provider;
+import javax.xml.bind.JAXBException;
+import javax.xml.bind.Marshaller;
+import javax.xml.bind.Unmarshaller;
+import java.lang.annotation.Annotation;
+import java.lang.reflect.Type;
+
+/**
+ * MOXy JSON provider for NodeManager WebService.
+ *
+ * <p>This class configures a MOXy JSON provider for the NodeManager REST API
endpoints.
+ * The endpoints should be able to provide two types of JSON responses:</p>
+ * <ul>
+ * <li>
+ * <b>Wrapped classes</b> – classes whose JSON representation includes a
root wrapper element.
+ * </li>
+ * <li>
+ * <b>Unwrapped classes</b> – classes whose JSON representation omits a
root wrapper element.
+ * </li>
+ * </ul>
+ *
+ * <p>This behaviour can be configured by the
MarshallerProperties.JSON_INCLUDE_ROOT property.
+ *
+ * By default NodeManager REST API endpoints should include the root wrapper
element in the
+ * responses, however there are some exceptions (e.g. ContainerLogsInfoes
class) which
+ * was introduced to provide backward-compatibility with the Jersey 1 response
format.</p>
+ */
+@Provider
+@Produces(MediaType.APPLICATION_JSON)
+@Consumes(MediaType.APPLICATION_JSON)
+public class NMJsonProvider extends MOXyJsonProvider {
+
+ private boolean isRootElementNeeded (Class<?> type) {
+ return !type.equals(ContainerLogsInfoes.class)
+ && !type.equals(NMGpuResourceInfo.class)
+ && !type.equals(NMDeviceResourceInfo.class);
+ }
+
+ @Override
+ protected void preReadFrom(Class<Object> type, Type genericType,
+ Annotation[] annotations, MediaType mediaType,
+ MultivaluedMap<String, String> httpHeaders,
+ Unmarshaller unmarshaller) throws JAXBException {
+ unmarshaller.setProperty(MarshallerProperties.JSON_INCLUDE_ROOT,
isRootElementNeeded(type));
+ }
+
+ @Override
+ protected void preWriteTo(Object object, Class<?> type, Type genericType,
+ Annotation[] annotations, MediaType mediaType,
+ MultivaluedMap<String, Object> httpHeaders,
Marshaller marshaller)
+ throws JAXBException {
+
marshaller.setProperty(MarshallerProperties.JSON_MARSHAL_EMPTY_COLLECTIONS,
false);
+ marshaller.setProperty(MarshallerProperties.JSON_INCLUDE_ROOT,
isRootElementNeeded(type));
Review Comment:
I tested in both cases with/without adding the JSON_REDUCE_ANY_ARRAYS
property.
And surprisingly in both cases the containers log is not available when the
application is in running state.
With JSON_REDUCE_ANY_ARRAYS:
<img width="2560" height="1440" alt="jsonreduce-no-container-log"
src="https://github.com/user-attachments/assets/bdc651c3-0f20-4524-b270-48b491f46285";
/>
Without JSON_REDUCE_ANY_ARRAYS:
<img width="2560" height="1440" alt="without-still-nolog"
src="https://github.com/user-attachments/assets/62d86bee-b8c8-4b7a-85bc-01ad02ccea49";
/>
I did the following:
- Run Mapreduce job
- While Application is running
- Access container log YARN UI2 -> Application -> logs
I will check with the NM endpoints that give this logs probably there is a
mismatch in the structure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
