[ https://issues.apache.org/jira/browse/THRIFT-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17761495#comment-17761495 ]
Jens Geyer commented on THRIFT-5733: ------------------------------------ {quote}things can get {_}really weird{_}.\{quote} because ... > Building code with circular `include`s can result in tons of memory usage and > eventual segfault > ----------------------------------------------------------------------------------------------- > > Key: THRIFT-5733 > URL: https://issues.apache.org/jira/browse/THRIFT-5733 > Project: Thrift > Issue Type: Bug > Components: Compiler (General) > Affects Versions: 0.18.1 > Environment: I'm on Linux, but this also happens to my coworkers on > macOS. > Reporter: Jeremy Fleischman > Priority: Major > Attachments: 2023-09-01_16-30-26_pattern.png > > > If I try to build the following thrift code, it pretty quickly segfaults: > *setup:* > {{$ cat foo.thrift}} > {{include "bar.thrift"}} > {{$ cat bar.thrift}} > {{include "foo.thrift"}} > *build:* > {{$ thrift --allow-64bit-consts --gen py:slots foo.thrift}} > {{[2] 210654 segmentation fault (core dumped) thrift --allow-64bit-consts > --gen py:slots foo.thrift}} > Not very user friendly error message I've ever received ;), but pretty must > just a cosmetic issue (maybe there's a buffer overflow somewhere and some > potential security exploit to worry about if you're compiling untrusted > thrift code, but I personally never do that, so it doesn't stress me out). > However, if you add a 3rd file to the mix, things can get {_}really weird{_}. > *setup:* > {{$ cat foo.thrift}} > {{include "bar.thrift"}} > {{$ cat bar.thrift}} > {{include "large-enum.thrift"}} > {{include "foo.thrift"}} > {{$ cat large-enum.thrift}} > {{enum LargeEnum {}} > {{ FOO0 = 0,}} > {{ FOO1 = 1,}} > {{ ... [FOO2 through FOO1998] ...}} > {{ FOO1999 = 1999, > }} > {{}}} > If I try to build this, it'll suck up all 32 GiB of RAM on my machine and > render my computer completely unusable. If you reduce the number of entries > in {{{}LargeEnum{}}}, you can get the thrift compiler to use a ton of RAM > before it finally segfaults as in the first example. I've attached a > screenshot so you can see how RAM and CPU gets used on my machine while > attempting to build the above code. > I've also put together a simple repro on > [https://github.com/jfly/2023-09-01-thrift-circular-import,] which can > autogenerate the 3 files described above. (Just be careful when running it > that you kill it before it soaks up all of your ram!) > Yesterday, this explosive use of RAM brought our company's build server (with > 128 GiB of RAM!) to its knees. We spent a lot of time flailing around before > we finally tracked it down to one problematic PR that introduced a circular > include. -- This message was sent by Atlassian Jira (v8.20.10#820010)