I am trying to understand if I should be concerned about this warning:

"WARN  Utils:66 - Truncated the string representation of a plan since it
was too large. This behavior can be adjusted by setting
'spark.debug.maxToStringFields' in SparkEnv.conf"

It occurs while writing a data frame to parquet.

Has any one on this list looked into this warning before and could help
explain what it means?

Thank you for your help,
Henry

Reply via email to