toolongframeexception

[Jiacheng Jiang]

大家好：

   我搭建了一个单机standealone 的flink用来测试mysql 
CDC，我发现每天晚上4点和4.30，我的taskmanager和jobmanager日志都有akka的TooLongFrameException，是BlobServerConnection报出来的，那个时候可能有定时任务在操作大量操作mysql数据库，我想问：

  1.  TooLongFrameException可能是mysql操作比较大造成的吗？如果是，有解决方法吗？
  2.  BlobServer是干嘛的？我一直认为blobserver是用来放job jar之类的，应该和cdc job的数据无关吧
  3.  哪些情况可能造成TooLongFrameException？

感谢大家

安装operator

[Jiacheng Jiang]

请问没有helm的情况下能否安装flink-k8s-operator？安装operator是否可以不要clusterrole

flink on k8s的taskmanager为啥不是Replica Sets

[Jiacheng Jiang]

请问大伙：

Flink on k8s里面，为什么taskmanager是单独的pod，而不是做成Replica Sets 利用副本数来控制taskmanager的数量





从 Windows 版邮件发送

??????Re:Re: flink-1.10 checkpoint ?????? NullPointerException

[??????(Jiacheng Jiang)]

hi



--  --
??: "chenkaibit"https://github.com/apache/flink/blob/release-1.10.0/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java#L1421
 kafka?? wordcount?? kafka??checkpoint ??
| Checkpointing Mode | Exactly Once |
| Interval | 5s |
| Timeout | 10m 0s |
| Minimum Pause Between Checkpoints | 0ms |
| Maximum Concurrent Checkpoints | 1 |


 5377 ?? checkpoint  NPE


 https://github.com/yuchuanchen/flink/commit/e5122d9787be1fee9bce141887e0d70c9b0a4f19 NPE ;


?? 2020-04-21 10:21:56??"chenkaibit" https://github.com/apache/flink/blob/aa4eb8f0c9ce74e6b92c3d9be5dc8e8cb536239d/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java#L1349
>>
>>
>>
>>
>>
>>From: chenkaibit 

?????? flink on yarn ????

[??????(Jiacheng Jiang)]

hi Xintong:
  
 directgc??maxDirectMemorysystem.gc()??maxDirectMemorykill??




--  --
??: 
   "user-zh"

flink on yarn ????

[??????(Jiacheng Jiang)]

         ??flink on 
yarnflink??1.7.2??on 
yarntaskmanager.heap.sizecontainer??container??kill??taskmanager.heap.size
 = xmx+maxDirectMemorySizexmx = taskmanager.heap.size - cutoff 
- network??maxDirectMemorySize = cutoff + 
network??networkmaxDirectMemorySize??cutoffcutoff??maxDirectMemorySize??cutoffmaxDirectMemorySize??metaspacecontainerkill??

????????????????????????

[??????(Jiacheng Jiang)]

??ok??FlinkKafkaProducerpreCommit??KafkaProducer??flush??kafka??checkpointjob??preCommit??commit??




--  --
??: ""

????????????????????????

[??????(Jiacheng Jiang)]

??ok??FlinkKafkaProducerpreCommit??KafkaProducer??flush??kafka??checkpointjob??preCommit??commit??




--  --
??: ""

??????????flink????

[??????(Jiacheng Jiang)]

1.10job??--  --
??: ""

liststate

[??????(Jiacheng Jiang)]

    
 ??RocksDBListState#get??list??next??

??????flink??????????????????

[??????(Jiacheng Jiang)]

checkpoint




--  --
??: "??"

??????RichMapFunction??????

[??????(Jiacheng Jiang)]

flink 1.10??slot??tmcluster.evenly-spread-out-slots: true






--  --
??: "xue...@outlook.com"https://go.microsoft.com/fwlink/?LinkId=550986>

??????flink on yarn????????kerberos??kafka

[??????(Jiacheng Jiang)]

krb5.conf??FLINK_ENV_JAVA_OPTS=-Djava.security.krb5.conf=x/krb5.conf




--  --
??: "zjfpla...@hotmail.com"

??????webui??????job

[??????(Jiacheng Jiang)]

hi all
    flink web ui??jobweb 
ui??job??savepointweb??job??

?????? flink 1.10????????

[??????(Jiacheng Jiang)]

direct??nativeunsafe.allocateMemoryDirectByteBuffer??newdirectMaxDirectMemorySize,jvm??DirectByteBuffer




--  --
??: "Xintong Song"http://apache-flink.147419.n8.nabble.com/Flink-tt1869.html
> > <
> http://apache-flink.147419.n8.nabble.com/Flink-tt1869.html#a1872&gt;
> >
> >
>
> > On&nbsp;Tue,&nbsp;Apr&nbsp;28,&nbsp;2020&nbsp;at&nbsp;9:32&nbsp;AM&nbsp;??(Jiacheng&nbsp;Jiang)&nbsp;<
> > 920334...@qq.com&gt;
> > wrote:
> >
> > &gt;&nbsp;hi&amp;nbsp;Xintong
> > &gt;&nbsp;&amp;nbsp;&nbsp;&amp;nbsp;
> >
>
> > &gt;&nbsp;flink1.10??-Xmx,-Xms,-XX:MaxDirectMomerySize,-XX:MaxMetaspaceSizemanaged
>
> > &gt;&nbsp;memorymanaged&nbsp;memory??jvmmetaspace??managed
> > &gt;&nbsp;memoryjvm
> > &gt;
> > &gt;
> > &gt;
> > &gt;
>
> > &gt;&nbsp;--&amp;nbsp;&amp;nbsp;--
>
> > &gt;&nbsp;??:&nbsp;&quot;Xintong&nbsp;Song&quot;<
> tonysong...@gmail.com
> > &amp;gt;;
> > &gt;&nbsp;:&nbsp;2020??4??27??(??)&nbsp;6:56
> > &gt;&nbsp;??:&nbsp;&quot;user-zh&quot;<
> user-zh@flink.apache.org&amp;gt;;
>
> > &gt;&nbsp;:&nbsp;Re:&nbsp;flink&nbsp;1.10
> > &gt;
> > &gt;
> > &gt;
> > &gt;
> >
>
> > &gt;&nbsp;&amp;nbsp;Framework&amp;nbsp;Off-heap&amp;nbsp;??&amp;nbsp;Task&amp;nbsp;Off-Heap&amp;nbsp;??Network
> >
>
> > &gt;&nbsp;&amp;nbsp;buffer&amp;nbsp;pool??
> > &gt;
> > &gt;&nbsp;Thank&amp;nbsp;you~
> > &gt;
> > &gt;&nbsp;Xintong&amp;nbsp;Song
> > &gt;
> > &gt;
> > &gt;
> > &gt;
> >
>
> > &gt;&nbsp;On&amp;nbsp;Mon,&amp;nbsp;Apr&amp;nbsp;27,&amp;nbsp;2020&amp;nbsp;at&amp;nbsp;6:14&amp;nbsp;PM&amp;nbsp;??(Jiacheng&amp;nbsp;Jiang)&amp;nbsp;<
> > &gt;&nbsp;920334...@qq.com&amp;gt;
> > &gt;&nbsp;wrote:
> > &gt;
> >
>
> > &gt;&nbsp;&amp;gt;&amp;nbsp;Thank&amp;amp;nbsp;you&amp;amp;nbsp;Xintong.??
> > &gt;
> >
>
> > &gt;&nbsp;&amp;gt;&amp;nbsp;-XX:MaxDirectMemorySizeFramework&amp;nbsp;+&amp;nbsp;Task&amp;nbsp;Off-Heap&amp;nbsp;+&amp;nbsp;Network&amp;nbsp;Memory
> > &gt;
> >
>
> > &gt;&nbsp;&amp;gt;&amp;nbsp;MaxDirectMemorySize=Framework&amp;amp;nbsp;Off-Heap&amp;amp;nbsp;+&amp;nbsp;Task&amp;nbsp;Off-Heap&amp;nbsp;+&amp;nbsp;Network
> > &gt;
> >
>
> > &gt;&nbsp;&amp;gt;&amp;nbsp;Memory??MaxDirectMemorySize=10??10=1+1+8??10=1+8+1,??2??3??task
>
> > &gt;&nbsp;&amp;gt;&amp;nbsp;off-heapjob
> > &gt;&nbsp;&amp;gt;
> > &gt;&nbsp;&amp;gt;
> > &gt;&nbsp;&amp;gt;
> >
>
> > &gt;&nbsp;&amp;gt;&amp;nbsp;--&amp;amp;nbsp;&amp;amp;nbsp;--
> >
>
> > &gt;&nbsp;&amp;gt;&amp;nbsp;??:&amp;nbsp;&amp;quot;Xintong&amp;nbsp;Song&amp;quot;<
> > tonysong...@gmail.com
> > &gt;&nbsp;&amp;amp;gt;;
>
> > &gt;&nbsp;&amp;gt;&amp;nbsp;:&amp;nbsp;2020??4??27??(??)&amp;nbsp;12:06
>
> > &gt;&nbsp;&amp;gt;&amp;nbsp;??:&amp;nbsp;&amp;quot;user-zh&amp;quot;<
> > user-zh@flink.apache.org&amp;amp;gt;;
> >
>
> > &gt;&nbsp;&amp;gt;&amp;nbsp;:&amp;nbsp;Re:&amp;nbsp;flink&amp;nbsp;1.10
> > &gt;&nbsp;&amp;gt;
> > &gt;&nbsp;&amp;gt;
> > &gt;&nbsp;&amp;gt;
> > &gt;&nbsp;&amp;gt;&amp;nbsp;??
> > &gt;&nbsp;&amp;gt;
> > &gt;&nbsp;&amp;gt;
>

??????flink on kubernetes ????????????????

[??????(Jiacheng Jiang)]

Map??es




--  --
??: "a511955993"

?????? flink 1.10????????

[??????(Jiacheng Jiang)]

hi Xintong??flink??managed 
memoryflink??native memory




--  --
??: "Xintong Song"http://apache-flink.147419.n8.nabble.com/Flink-tt1869.html
> 
>
> On Tue, Apr 28, 2020 at 9:32 AM ??(Jiacheng Jiang) <
> 920334...@qq.com>
> wrote:
>
> > hi&nbsp;Xintong
> > &nbsp; &nbsp;
>
> > flink1.10??-Xmx,-Xms,-XX:MaxDirectMomerySize,-XX:MaxMetaspaceSizemanaged
> > memorymanaged memory??jvmmetaspace??managed
> > memoryjvm
> >
> >
> >
> >
> > --&nbsp;&nbsp;--
> > ??: "Xintong Song"https://ci.apache.org/projects/flink/flink-docs-release-1.10/zh/ops/memory/mem_detail.html
> > &gt;&nbsp;[2]
> > &gt;
> > &gt;&nbsp;
> > 
> https://ci.apache.org/projects/flink/flink-docs-release-1.10/zh/ops/memory/mem_trouble.html#outofmemoryerror-direct-buffer-memory
> > &gt;&nbsp;[3]
> ;
> > <
> https://ci.apache.org/projects/flink/flink-docs-release-1.10/zh/ops/memory/mem_trouble.html#outofmemoryerror-direct-buffer-memory&gt;&nbsp;[3]>
> ;
> > &gt;
> > &gt;&nbsp;
> > 
> https://ci.apache.org/projects/flink/flink-docs-release-1.10/zh/ops/memory/mem_setup.html#%E6%89%98%E7%AE%A1%E5%86%85%E5%AD%98
> > &gt
> ;
> > <
> https://ci.apache.org/projects/flink/flink-docs-release-1.10/zh/ops/memory/mem_setup.html#%E6%89%98%E7%AE%A1%E5%86%85%E5%AD%98&gt>
> ;
> > ;
> > &gt;
> >
>
> > &gt;&nbsp;On&amp;nbsp;Mon,&amp;nbsp;Apr&amp;nbsp;27,&amp;nbsp;2020&amp;nbsp;at&amp;nbsp;10:31&amp;nbsp;AM&amp;nbsp;??(Jiacheng&amp;nbsp;Jiang)&amp;nbsp;<
> > &gt;&nbsp;920334...@qq.com&amp;gt;
> > &gt;&nbsp;wrote:
> > &gt;
> > &gt;&nbsp;&amp;gt;&amp;nbsp;hi
> > &gt;&nbsp;&amp;gt;
> > &gt;&nbsp;&amp;gt;
> >
>
> > &gt;&nbsp;&amp;gt;&amp;nbsp;&amp;amp;nbsp;&amp;nbsp;&amp;amp;nbsp;??standalone
> > &gt;
> >
>
> > &gt;&nbsp;&amp;gt;&amp;nbsp;flinkjob??tm??8gflink1.10??taskmanager.memory.flink.size??10g??job??jobjava.lang.OutOfMemoryError:
> >
>
> > &gt;&nbsp;&amp;gt;&amp;nbsp;Direct&amp;nbsp;buffer&amp;nbsp;memory??1??job??Direct
> > &gt;
> >
>
> > &gt;&nbsp;&amp;gt;&amp;nbsp;Memory??flink1.10standalone&amp;nbsp;clusterslot
> > &gt;
> >
>
> > &gt;&nbsp;&amp;gt;&amp;nbsp;??slotA&amp;nbsp;TaskManager&amp;nbsp;with&amp;nbsp;three&amp;nbsp;slots,&amp;nbsp;for&amp;nbsp;example,&amp;nbsp;will
> > &gt;
> >
>
> > &gt;&nbsp;&amp;gt;&amp;nbsp;dedicate&amp;nbsp;1/3&amp;nbsp;of&amp;nbsp;its&amp;nbsp;managed&amp;nbsp;memory&amp;nbsp;to&amp;nbsp;each&amp;nbsp;slot.&amp;amp;nbsp;??its&amp;nbsp;managed
> >
>
> > &gt;&nbsp;&amp;gt;&amp;nbsp;memory??taskmanager.memory.managed.size??
> > &gt;&nbsp;&amp;gt;
> > &gt;&nbsp;&amp;gt;
>
> > &gt;&nbsp;&amp;gt;&amp;nbsp;best&amp;amp;nbsp;
> > &gt;&nbsp;&amp;gt;&amp;nbsp;Jungle

?????? flink 1.10????????

[??????(Jiacheng Jiang)]

hi Xintong??java??native memoryapi




--  --
??: "Xintong Song"http://apache-flink.147419.n8.nabble.com/Flink-tt1869.html
<http://apache-flink.147419.n8.nabble.com/Flink-tt1869.html#a1872>;

On Tue, Apr 28, 2020 at 9:32 AM ??(Jiacheng Jiang) <920334...@qq.com>
wrote:

> hi&nbsp;Xintong
> &nbsp; &nbsp;
> flink1.10??-Xmx,-Xms,-XX:MaxDirectMomerySize,-XX:MaxMetaspaceSizemanaged
> memorymanaged memory??jvmmetaspace??managed
> memoryjvm
>
>
>
>
> --&nbsp;&nbsp;--
> ??: "Xintong Song"https://ci.apache.org/projects/flink/flink-docs-release-1.10/zh/ops/memory/mem_detail.html
> &gt;&nbsp;[2]
> &gt;
> &gt;&nbsp;
> https://ci.apache.org/projects/flink/flink-docs-release-1.10/zh/ops/memory/mem_trouble.html#outofmemoryerror-direct-buffer-memory
> &gt;&nbsp;[3]
> <https://ci.apache.org/projects/flink/flink-docs-release-1.10/zh/ops/memory/mem_trouble.html#outofmemoryerror-direct-buffer-memory&gt;&nbsp;[3]>;
> &gt;
> &gt;&nbsp;
> https://ci.apache.org/projects/flink/flink-docs-release-1.10/zh/ops/memory/mem_setup.html#%E6%89%98%E7%AE%A1%E5%86%85%E5%AD%98
> &gt
> <https://ci.apache.org/projects/flink/flink-docs-release-1.10/zh/ops/memory/mem_setup.html#%E6%89%98%E7%AE%A1%E5%86%85%E5%AD%98&gt>;
> ;
> &gt;
>
> &gt;&nbsp;On&amp;nbsp;Mon,&amp;nbsp;Apr&amp;nbsp;27,&amp;nbsp;2020&amp;nbsp;at&amp;nbsp;10:31&amp;nbsp;AM&amp;nbsp;??(Jiacheng&amp;nbsp;Jiang)&amp;nbsp;<
> &gt;&nbsp;920334...@qq.com&amp;gt;
> &gt;&nbsp;wrote:
> &gt;
> &gt;&nbsp;&amp;gt;&amp;nbsp;hi
> &gt;&nbsp;&amp;gt;
> &gt;&nbsp;&amp;gt;
>
> &gt;&nbsp;&amp;gt;&amp;nbsp;&amp;amp;nbsp;&amp;nbsp;&amp;amp;nbsp;??standalone
> &gt;
>
> &gt;&nbsp;&amp;gt;&amp;nbsp;flinkjob??tm??8gflink1.10??taskmanager.memory.flink.size??10g??job??jobjava.lang.OutOfMemoryError:
>
> &gt;&nbsp;&amp;gt;&amp;nbsp;Direct&amp;nbsp;buffer&amp;nbsp;memory??1??job??Direct
> &gt;
>
> &gt;&nbsp;&amp;gt;&amp;nbsp;Memory??flink1.10standalone&amp;nbsp;clusterslot
> &gt;
>
> &gt;&nbsp;&amp;gt;&amp;nbsp;??slotA&amp;nbsp;TaskManager&amp;nbsp;with&amp;nbsp;three&amp;nbsp;slots,&amp;nbsp;for&amp;nbsp;example,&amp;nbsp;will
> &gt;
>
> &gt;&nbsp;&amp;gt;&amp;nbsp;dedicate&amp;nbsp;1/3&amp;nbsp;of&amp;nbsp;its&amp;nbsp;managed&amp;nbsp;memory&amp;nbsp;to&amp;nbsp;each&amp;nbsp;slot.&amp;amp;nbsp;??its&amp;nbsp;managed
>
> &gt;&nbsp;&amp;gt;&amp;nbsp;memory??taskmanager.memory.managed.size??
> &gt;&nbsp;&amp;gt;
> &gt;&nbsp;&amp;gt;
> &gt;&nbsp;&amp;gt;&amp;nbsp;best&amp;amp;nbsp;
> &gt;&nbsp;&amp;gt;&amp;nbsp;Jungle

execution.checkpointing.tolerable-failed-checkpoints ????

[??????(Jiacheng Jiang)]

hi 
  
 flink-conf.yaml??execution.checkpointing.tolerable-failed-checkpoints??
 
1000checkpoint??job??
best
jungglge

??????flink????????hdfs parquest,hdfs????ha??????????

[??????(Jiacheng Jiang)]

??fs.defaultFS??



--  --
??: "196371551"<196371...@qq.com>; 
: 2020??4??30??(??) 8:50
??: "user"

?????? ????????

[??????(Jiacheng Jiang)]

java.lang.OutOfMemoryError: unable to create newnative 
thread??XmxMaxDirectMemorySize??




--  --
??: "LakeShen"http://apache-flink.147419.n8.nabble.com/flink-kafka-td2386.html
> >> [2]  
http://apache-flink.147419.n8.nabble.com/Flink-Kafka-td2390.html
> >> On Sun, Apr 26, 2020 at 10:58 yanggang_it_job 

?????? flink 1.10????????

[??????(Jiacheng Jiang)]

hi Xintong
    
flink1.10??-Xmx,-Xms,-XX:MaxDirectMomerySize,-XX:MaxMetaspaceSizemanaged
 memorymanaged memory??jvmmetaspace??managed 
memoryjvm




--  --
??: "Xintong Song"https://ci.apache.org/projects/flink/flink-docs-release-1.10/zh/ops/memory/mem_detail.html
> [2]
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.10/zh/ops/memory/mem_trouble.html#outofmemoryerror-direct-buffer-memory
> [3]
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.10/zh/ops/memory/mem_setup.html#%E6%89%98%E7%AE%A1%E5%86%85%E5%AD%98
>
>
> On Mon, Apr 27, 2020 at 10:31 AM ??(Jiacheng Jiang) <
> 920334...@qq.com>
> wrote:
>
> > hi
> >
> >
> > &nbsp; &nbsp;??standalone
>
> > flinkjob??tm??8gflink1.10??taskmanager.memory.flink.size??10g??job??jobjava.lang.OutOfMemoryError:
> > Direct buffer memory??1??job??Direct
>
> > Memory??flink1.10standalone clusterslot
>
> > ??slotA TaskManager with three slots, for example, will
>
> > dedicate 1/3 of its managed memory to each slot.&nbsp;??its managed
> > memory??taskmanager.memory.managed.size??
> >
> >
> > best&nbsp;
> > Jungle

?????? flink 1.10????????

[??????(Jiacheng Jiang)]

Thank you Xintong.??
-XX:MaxDirectMemorySizeFramework + Task Off-Heap + Network Memory
MaxDirectMemorySize=Framework Off-Heap + Task Off-Heap + Network 
Memory??MaxDirectMemorySize=10??10=1+1+8??10=1+8+1,??2??3??task
 off-heapjob



--  --
??: "Xintong Song"https://ci.apache.org/projects/flink/flink-docs-release-1.10/zh/ops/memory/mem_detail.html
[2]
https://ci.apache.org/projects/flink/flink-docs-release-1.10/zh/ops/memory/mem_trouble.html#outofmemoryerror-direct-buffer-memory
[3]
https://ci.apache.org/projects/flink/flink-docs-release-1.10/zh/ops/memory/mem_setup.html#%E6%89%98%E7%AE%A1%E5%86%85%E5%AD%98

On Mon, Apr 27, 2020 at 10:31 AM ??(Jiacheng Jiang) <920334...@qq.com>
wrote:

> hi
>
>
> &nbsp; &nbsp;??standalone
> flinkjob??tm??8gflink1.10??taskmanager.memory.flink.size??10g??job??jobjava.lang.OutOfMemoryError:
> Direct buffer memory??1??job??Direct
> Memory??flink1.10standalone clusterslot
> ??slotA TaskManager with three slots, for example, will
> dedicate 1/3 of its managed memory to each slot.&nbsp;??its managed
> memory??taskmanager.memory.managed.size??
>
>
> best&nbsp;
> Jungle

flink 1.10????????

[??????(Jiacheng Jiang)]

hi


   ??standalone 
flinkjob??tm??8gflink1.10??taskmanager.memory.flink.size??10g??job??jobjava.lang.OutOfMemoryError:
 Direct buffer memory??1??job??Direct 
Memory??flink1.10standalone 
clusterslot
??slotA TaskManager with three slots, for example, 
will dedicate 1/3 of its managed memory to each slot. ??its managed 
memory??taskmanager.memory.managed.size??


best 
Jungle

?????? flink1.10????win10????Standlone????

[??????(Jiacheng Jiang)]

Caused by: org.apache.flink.configuration.IllegalConfigurationException:
The network memory min (64 mb) and max (1 gb) mismatch, the network memory
has to be resolved and set to a fixed value before task executor starts




??1.10??




--  --
??: ""

??????flink1.10????win10????Standlone????

[??????(Jiacheng Jiang)]

??




--  --
??: ""

??????1.10????????????--??????????????

[??????(Jiacheng Jiang)]

??MailboxProcessorstreamtask??processInputMailboxDefaultAction??MailboxProcessorInputStatus
 status = 
inputProcessor.processInput();inputProcessor??StreamOneInputProcessor??InputStatus
 status = 
input.emitNext(output);input??StreamTaskNetworkInputprocessElement??StreamTaskAbstractInvokable??StreamTask??invoke()??runMailboxLoop()??StreamTask??invoke()



--  --
??: ""<17626017...@163.com>; 
: 2020??4??19??(??) 5:37
??: "user-zh"

?????? keyby??????????

[??????(Jiacheng Jiang)]

keybywatermark??windowAll??keyby??




--  --
??: "tingli ke"

??????Flink?????? ?????? Keyed Watermarks

[??????(Jiacheng Jiang)]

??




--  --
??: "Utopia"https://bigdata.cs.ut.ee/keyed-watermarks-partition-aware-watermark-generation-apache-flink



Best  regards
Utopia