comphead commented on code in PR #6401:
URL: https://github.com/apache/datafusion-comet/pull/6401#discussion_r4139750538
##########
spark/src/main/scala/org/apache/comet/CometExecIterator.scala:
##########
@@ -508,7 +529,10 @@ object CometExecIterator extends Logging {
try {
val usage = nativeLib.getMemoryUsage()
val jvmArrow = JvmArrowMemory.current()
- memoryUsageMessage(usage, jvmArrow,
plansAtLastMemoryUsageLog).foreach(logInfo(_))
+ memoryUsageMessage(usage, jvmArrow, plansAtLastMemoryUsageLog).foreach {
message =>
+ // A sample the event log records stays in the executor's log at DEBUG
only.
+ if (sendToEventLog(usage, jvmArrow)) logDebug(message) else
logInfo(message)
Review Comment:
Done. Every sample is logged at INFO again, so the event log only adds to
the executor's log. A send that fails outright now only stops the event log
path, after one warning. The executor also reads `spark.eventLog.enabled`
through Spark's typed entry, as the driver does, so a value with surrounding
whitespace can no longer stop the log.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
