[
https://issues.apache.org/jira/browse/THRIFT-5322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248227#comment-17248227
]
Yuxuan Wang commented on THRIFT-5322:
-------------------------------------
[~jensg] Thanks for the pointer. I did not know that TConfiguration is in the
spec. This will actually suit the go implementation better than the string
length limit and container length limit currently implemented in Java and
python, which was the first approach I tried in the go fix. I discarded that
approach because of the zero value feature in go will make it either a breaking
change or behave differently from existing Java/python implementations.
But I do not think implementing TConfiguration in go is required to fix this
issue. The current fix [#2292|https://github.com/apache/thrift/pull/2292] fixes
the issue in a way that no additional limit is needed, we just avoid doing the
whole allocation up front. Implementing TConfiguration can be added on top of
that, either by create a new ticket or reuse this ticket.
> Go compact_protocol allocating unreasonable buffer size
> -------------------------------------------------------
>
> Key: THRIFT-5322
> URL: https://issues.apache.org/jira/browse/THRIFT-5322
> Project: Thrift
> Issue Type: Bug
> Components: Go - Library
> Affects Versions: 0.13.0
> Reporter: Juraci Paixão Kröhling
> Assignee: Yuxuan Wang
> Priority: Major
> Attachments: main.go
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> I don't yet know all the pieces to this puzzle, and it's quite possible that
> the problem is on our side, but we use the Thrift Go library in the Jaeger
> Agent and we are seeing a case where the memory consumption for a payload of
> 4k bytes to result in a buffer allocation in the compact_protocol.go with
> unreasonable sizes. I found buffers of 1.4GiB while debugging the issue.
>
> This is the code that we are seeing this memory usage:
> [https://github.com/apache/thrift/blob/b75e88a33d67ae05ef9b5fa001d2a63a2effe377/lib/go/thrift/compact_protocol.go#L556-L577]
>
> Here's more information about this, including a reproducer and initial
> diagnostics:
> [https://github.com/jaegertracing/jaeger/issues/2638#issuecomment-741848201]
>
> As mentioned above, I'm still getting all the pieces together, but perhaps
> you've seen this before or know what might be going on. What I know for sure
> at the moment is that this happens on malformed payloads, but I would expect
> the library to have an upper limit on the buffer size.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
