Re: Printing Logs in map-partition

2020-12-27 Thread abby37
I have looked into the logs of worker node by going to the logs folder of spark. I didn't find in logs my print statements from log4j or System.out. Can you give detailed response regarding that. Much appreciated -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

Printing Logs in map-partition

2020-12-22 Thread abby37
I want to print some logs in transformation mapPartitions to logs the internal working of function. I have used following techniques without any success. 1. System.out.println() 2. System.err.println() 3. Log4j - logger.info 4. Log4j - logger.debug My code for mapPartitions is similar to this