适当调小chunk.size 的大小可能有用。 Guo Baoxing 郭宝兴 (集团IT) <guobaox...@haier.com> 于2025年5月16日周五 11:05写道:
> 实时任务flink on yarn运行,实时同步MySQL表到StarRocks经常遇到的内存问题:十万级别的数据量使用jm > 1024mb,tm1024mb启动会报连接超时的问题,tm改成2048mb就会同步成功,但是源表十万级别数据量tm用2048mb内存非常浪费,看jobmanager日志,主要原因应该在chunk上,因为源表主键id的最大值为190562215,最小id为 > 1,可能是因为chunk太大导致内存不够,这种应该怎么优化下呢? > The information transmitted is intended solely for the use of the > addressee and may contain confidential and/or privileged material. Any > unauthorized disclosure, reproduction, distribution, dissemination, or > taking of any action in reliance upon, this information by persons or > entities other than the intended recipient is prohibited. If you received > this in error, please contact the sender and delete the email together with > any material attached (if any) completely from any device immediately. > Unless otherwise stated, any views or opinions expressed in this email are > solely those of the author and do not necessarily represent those of Haier > Group. > > 本邮件可能包含敏感信息且仅限于发给指定的收件人,任何未经授权泄露、复制、散布或传播此信息的行为将被禁止。如果您错误收到了此邮件,请及时告知发送者并立即从所有设备中完全删除此邮件及附件。除非另有说明,否则邮件中可能包含的观点或建议仅代表发件者本人,并不代表海尔集团。 > >