Lihang Shen created SPARK-61419:
-----------------------------------
Summary: from_avro/schema_of_avro can corrupt a shared options map
when the plan is canonicalized concurrently
Key: SPARK-61419
URL: https://issues.apache.org/jira/browse/SPARK-61419
Project: Spark
Issue Type: Bug
Components: Bug
Affects Versions: 4.0.0
Reporter: Lihang Shen
{{Same issue as SPARK-60945. \{{FromAvro.replacement}} and
\{{SchemaOfAvro.replacement}} evaluate the
stateful \{{options}} map (\{{CreateMap}}) directly. Concurrent
canonicalizations of one plan share
that map and race on its map builder, failing with
\{{IllegalArgumentException}} or
\{{DUPLICATED_MAP_KEY}}, or silently dropping options under LAST_WIN. The fix
is the same:
evaluate \{{options.freshCopyIfContainsStatefulExpression()}}.}}
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]