Abacn commented on code in PR #31580:
URL: https://github.com/apache/beam/pull/31580#discussion_r1637100484
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/util/RowJsonUtils.java:
##########
@@ -34,6 +34,36 @@
@Internal
public class RowJsonUtils {
+ //
+ private static int defaultBufferLimit;
+
+ /**
+ * Increase the default jackson-databind stream read constraint.
+ *
+ * <p>StreamReadConstraints was introduced in jackson 2.15 causing string >
20MB (5MB in 2.15.0)
+ * parsing failure. This has caused regressions in its dependencies include
Beam. Here we
+ * overwrite the default buffer size limit to 100 MB, and exposes this
interface for higher limit.
+ * If needed, call this method during pipeline run time, e.g. in DoFn.setup.
+ */
+ public static void increaseDefaultStreamReadConstraints(int newLimit) {
Review Comment:
was considering adding a pipeline option to configure this. However,
overrideDefaultStreamReadConstraints needs to be called as early as in static
context to make it effective for ObjectMapper used in random places, makes it
ineffective in non-static context
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
