rrwright commented on code in PR #744:
URL: https://github.com/apache/pekko-http/pull/744#discussion_r2331434068
##########
docs/src/main/paradox/common/sse-support.md:
##########
@@ -56,6 +56,49 @@ Scala
Java
: @@snip
[EventStreamMarshallingTest.java](/http-tests/src/test/java/org/apache/pekko/http/javadsl/unmarshalling/sse/EventStreamUnmarshallingTest.java)
{ #event-stream-unmarshalling-example }
+## Configuration
+
+Apache Pekko HTTP provides several configuration options for Server-Sent
Events handling:
+
+### Message Size Limits
+
+The SSE client parser has configurable limits to handle various message sizes:
+
+```hocon
+pekko.http.sse {
+ # The maximum size for parsing received server-sent events.
+ # This value must be larger than `max-line-size`. Set to 0 to disable limit
entirely (unlimited).
+ max-event-size = 115713
+
+ # The maximum size for parsing received lines of a server-sent event. Set to
0 to disable limit entirely (unlimited).
+ max-line-size = 115712
+}
+```
+
+### Oversized Message Handling
+
+When SSE messages exceed the configured `max-line-size`, Apache Pekko HTTP
provides four handling strategies:
+
+- **fail-stream** (default): Fails the stream with a clear error message,
maintaining backward compatibility
+- **log-and-skip**: Logs a warning and skips the oversized message, continuing
stream processing
+- **truncate**: Logs a warning and truncates the message to the configured
limit, continuing processing
+- **dead-letter**: Logs a warning and sends the oversized message to the dead
letter queue, continuing processing
+
+```hocon
+pekko.http.sse {
+ # How to handle messages that exceed max-line-size limit
+ # Options:
+ # "fail-stream" - Fail the stream with a clear error message (default)
+ # "log-and-skip" - Log a warning and skip the oversized message
+ # "truncate" - Log a warning and truncate the message to max-line-size
+ # "dead-letter" - Log a warning, send oversized message to dead letters
+ oversized-message-handling = "fail-stream"
+}
+```
+
+For applications that need to handle very large messages (like blockchain data
or detailed JSON payloads), consider
+setting `max-line-size = 0` to disable limits entirely, or use one of the
non-failing handling modes.
+
Notice that if you are looking for a resilient way to permanently subscribe to
an event stream,
-Apache Pekko Connectors provides the
[EventSource](https://pekko.apache.org/docs/pekko-connectors/current/sse.html)
-connector which reconnects automatically with the id of the last seen event.
+Apache Pekko Connectors provides the
[EventSource](https://pekko.apache.org/docs/pekko-connectors/current/sse.html)connector
which reconnects automatically with the id of the
Review Comment:
✅
##########
http-tests/src/test/java/org/apache/pekko/http/javadsl/settings/OversizedSseStrategySimpleTest.java:
##########
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * license agreements; and to You under the Apache License, version 2.0:
+ *
+ * https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * This file is part of the Apache Pekko project, which was derived from Akka.
Review Comment:
👍 I believe this is updated now for all the relevant files.
##########
http/src/main/resources/reference.conf:
##########
@@ -52,11 +52,20 @@ pekko.http {
# server-sent events
sse {
- # The maximum size for parsing server-sent events.
- max-event-size = 8192
+ # The maximum size for parsing received server-sent events.
+ # This value must be larger than `max-line-size`. Set to 0 to disable
limit entirely (unlimited).
+ max-event-size = 115713
Review Comment:
As noted in the PR description and in [other reviewer
comments](https://github.com/apache/pekko-http/pull/744#issuecomment-3260298481),
8k is extremely small for any modern use case. I believe the argument for
keeping these values small is because Pekko messages should be small (and
instead convey a link to fetch larger payloads through an outside channel if
needed), but these limits are not actually limiting Pekko messages, they are
limiting what is read out of the HTTP connection. Small limits combined with
the existing behavior of the SSE connector (swallows stream failure messages
caused by this limit being triggered) cause silent failures and lost data.
Also noted in the PR description, this is another new arbitrary value and
very reasonably argued against, and not core the changes proposed in this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
