xintongsong commented on issue #541:
URL: https://github.com/apache/flink-agents/issues/541#issuecomment-3949780988
Hi @weiqingy, Thanks for working on this.
The event log enhancement is an important feature that directly affects user
experience, and is also relatively complex. I think this deserves a design doc,
based on which we can first discuss and align on the expected behaviors before
moving to the implementation. Would you like to open a discussion thread?
Based on the issue description, I have a few comments:
- I like the idea of truncating long fields when the level is `STANDARD`. In
addition to string fields, I think long lists or deeply nested structures can
also be truncated. Maybe we should define the semantic of `STANDARD` as
"details might be omitted to keep the logs concise". This would give us the
flexibility to gradually optimize the truncating strategy w/o breaking the
semantics.
- This also means the per-string-field length limitation, which exposes
the truncating strategy, might not be ideal. I'd suggest an per-event overall
length limitation.
- For per-class log level, a single config option is probably not enough.
- Imagine you have configured a list of log levels in `config.yaml`, which
is shared by all your jobs. Then for a certain job you want to overwrite the
config option in the job submitting command, setting a certain class to
`VERBOSE` for debugging. In this case, you can only overwrite the entire log
level list, which means either include everything from `config.yaml` in the
command, or lose the log levels configured in `config.yaml`.
- I think we should introduce a config key pattern. E.g.,
`event-log.level.foo.bar`: OFF|STANDARD|VERBOSE`, where `foo.bar` is the event
type. In this way, each class will have its own option that can be overwritten
separately.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
