Github user NicoK commented on the issue:
https://github.com/apache/flink/pull/4529
let me close this PR since this will be done by upgrading netty instead
---
Github user zhijiangW commented on the issue:
https://github.com/apache/flink/pull/4529
Thanks for telling me the plan of it. I will do that if necessary. :)
---
Github user NicoK commented on the issue:
https://github.com/apache/flink/pull/4529
@zhijiangW looks like we're not upgrading the netty version in Flink 1.5 at
this stage due to the timing, i.e. too quite late in the development cycle. We
will do the upgrade after releasing Flink 1.5
Github user NicoK commented on the issue:
https://github.com/apache/flink/pull/4529
With the recent update of the netty version in flink-shaded (see
https://github.com/apache/flink-shaded/commit/1233f1bb0e2b9fafa4260603aa130b7eb9995a7a),
most of this PR is indirectly included.
Github user zhijiangW commented on the issue:
https://github.com/apache/flink/pull/4529
Hey @NicoK , would this PR be covered in FLINK-1.5? We experienced the
netty direct memory out of memory sometimes in production cased by
`extractFrame`, so we expect this improvement. :)
---
Github user NicoK commented on the issue:
https://github.com/apache/flink/pull/4529
I haven't had time to test this yet - I doubt that this is simply solved by
using our memory segments but I need to understand the problem better.
Maybe this also goes away with [flow
Github user KurtYoung commented on the issue:
https://github.com/apache/flink/pull/4529
@zentol So the rationale behind this is when all netty's memory comes from
flink's managed memory, then this is not an issue, right?
---
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/4529
@KurtYoung I think
https://github.com/apache/flink/pull/4529#issuecomment-321851427 addresses that.
---
Github user KurtYoung commented on the issue:
https://github.com/apache/flink/pull/4529
If I remember correctly, the reason we chose to do an extra copy inside
`extractFrame` is that we faced some memory problem when using `slice`. Is this
no longer an issue or have taken care by
Github user NicoK commented on the issue:
https://github.com/apache/flink/pull/4529
Thanks, @greghogan - I put most of them under FLINK-7315 now (at least the
ones which are loosely related).
Regarding the PR itself, I still need to investigate whether the previous
issues
Github user greghogan commented on the issue:
https://github.com/apache/flink/pull/4529
@NicoK what do you think of creating a parent issue for your collection of
network improvements?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user NicoK commented on the issue:
https://github.com/apache/flink/pull/4529
we also need to double-check this against the previous issue of
d92e422ec7089376583a8f57043274d236c340a4 which may be solved by the way I am
using the `LengthFieldBasedFrameDecoder` now compared to
Github user NicoK commented on the issue:
https://github.com/apache/flink/pull/4529
FYI:
- newer netty 4.0.xx versions also use `buffer.slice(index, length)` inside
their `extractFrame()` method (see
https://github.com/netty/netty/commit/891be30a28c6dc5a1edf1cb5a3690644cf4ff66e)
13 matches
Mail list logo
