目前遇到个报错,这个有人看得懂不。 org.apache.flink.runtime.io.network.netty.exception.LocalTransportException: readAddress(..) failed: Connection timed out (connection to '10.35.213.143/10.35.213.143:2008') at org.apache.flink.runtime.io.network.netty.CreditBasedPartitionRequestClientHandler.exceptionCaught(CreditBasedPartitionRequestClientHandler.java:173) at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at org.apache.flink.shaded.netty4.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at org.apache.flink.shaded.netty4.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at org.apache.flink.shaded.netty4.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at org.apache.flink.shaded.netty4.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.flink.shaded.netty4.io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection timed out
13631283359 <13631283...@163.com> 于2021年6月8日周二 下午3:41写道: > > > > > > > > > > > 1.不知道这边有没有flink数据链路监控 > 2.flink反压可能有以下几种原因: > 数据倾斜 > TaskManager配置内存太小,导致full gc > checkpoint太慢 > 状态太大 > > > > > > > > > 在 2021-06-08 10:36:49,"LakeShen" <shenleifight...@gmail.com> 写道: > >你可以先结合你的任务逻辑,以及 Flink Web UI 反压监控,看看到底是什么地方引起反压。 > >一般 source 反压,是下游的算子一直反压到 source。可以看看那个算子引起 > > > >Best, > >LakeShen > > > > > >yidan zhao <hinobl...@gmail.com> 于2021年6月8日周二 上午10:28写道: > > > >> 该任务有时候正常,偶尔反压。 > >> 最近观察发现,反压时,kafkaSouce节点100%反压到停滞,后续算子什么也收不到,任务整体停滞。 > >> > >> 这类错误遇到过很多次了,目前我生产中flink有个很大问题就是这些稳定性,压力大不是需要时间去追赶,而是压力一大就整体处于停滞状态。 > >>