I have looked into the logs of worker node by going to the logs folder of
spark. I didn't find in logs my print statements from log4j or System.out.
Can you give detailed response regarding that. Much appreciated
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
the logs printed in the map function exist in the worker node, you can
access it directly, or you can browse through webui.
abby37 于2020年12月23日周三 下午1:53写道：
> I want to print some logs in transformation mapPartitions to logs the
> internal working of function.
> I have used following
I want to print some logs in transformation mapPartitions to logs the
internal working of function.
I have used following techniques without any success.
1. System.out.println()
2. System.err.println()
3. Log4j - logger.info
4. Log4j - logger.debug
My code for mapPartitions is similar to this
