flink_1.13 rocksdb????

2023-08-29 文章 crazy 
??
   ??rocksdb??
   flink??1.13.5
   flink on yarn 
??block_cache??write_buffer_manager
   block_cache_capacity=((3 - writeBufferRatio) * 
totalMemorySize / 3)
   write_buffer_manager=(2 * totalMemorySize * 
writeBufferRatio / 3)

   ??Managed 
memory??3G(slot)??block_cache_capacity=2.5G,
 write_buffer_manager=1G??
   1. block_cache_capacity + write_buffer_manager 
=3.5G??Managed memory??3G
   2. 
write_buffer_manager??block_cache??write_buffer_manager+ 
block_cache 
   3.pmap -x 
rocksdbblock_cache_capacity??block_cacheblock_cache+write_buffer??


  






crazy
2463829...@qq.com

Flink????tm Connection timed out????????

2023-03-05 文章 crazy 
failover


2023-03-05 11:41:07,847 INFO  
org.apache.flink.runtime.executiongraph.ExecutionGraph   [] - Process 
(287/300) (b3ef27fec49fe3777f830802ef3501e9) switched from RUNNING to FAILED on 
container_e26_1646120234560_82135_01_97 @ xx.xx.xx.xx (dataPort=26882). 
org.apache.flink.runtime.io.network.netty.exception.LocalTransportException: 
readAddress(..) failed: Connection timed out (connection to 
'xxx/10.70.89.25:43923')   at 
org.apache.flink.runtime.io.network.netty.CreditBasedPartitionRequestClientHandler.exceptionCaught(CreditBasedPartitionRequestClientHandler.java:201)
 ~[flink-dist_2.11-1.13.5.jar:1.13.5]   at 
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302)
 ~[flink-dist_2.11-1.13.5.jar:1.13.5] at 
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281)
 ~[flink-dist_2.11-1.13.5.jar:1.13.5] at 
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273)
 ~[flink-dist_2.11-1.13.5.jar:1.13.5]   at 
org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377)
 ~[flink-dist_2.11-1.13.5.jar:1.13.5]at 
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302)
 ~[flink-dist_2.11-1.13.5.jar:1.13.5] at 
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281)
 ~[flink-dist_2.11-1.13.5.jar:1.13.5] at 
org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907)
 ~[flink-dist_2.11-1.13.5.jar:1.13.5] at 
org.apache.flink.shaded.netty4.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728)
 ~[flink-dist_2.11-1.13.5.jar:1.13.5] at 
org.apache.flink.shaded.netty4.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818)
 ~[flink-dist_2.11-1.13.5.jar:1.13.5]at 
org.apache.flink.shaded.netty4.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475)
 ~[flink-dist_2.11-1.13.5.jar:1.13.5]  at 
org.apache.flink.shaded.netty4.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
 ~[flink-dist_2.11-1.13.5.jar:1.13.5]   at 
org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
 ~[flink-dist_2.11-1.13.5.jar:1.13.5] at 
org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
 ~[flink-dist_2.11-1.13.5.jar:1.13.5]at 
java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_131] Caused by: 
org.apache.flink.shaded.netty4.io.netty.channel.unix.Errors$NativeIoException: 
readAddress(..) failed: Connection timed out

A??switched from RUNNING to FAILED on 
container_e26_1646120234560_82135_01_97 @A (dataPort=26882) , 
A







crazy
2463829...@qq.com