I think you need provide all the parallelism information, such like the operator info 'Id: b0936afefebc629e050a0f423f44e6ba, maxparallsim: 4096'. What is the parallelism, the maxparallism maybe be generated from the parallelism you have set.
Arvid Heise <ar...@apache.org> 于2021年1月22日周五 下午11:03写道： > Hi Lu, > > if you are using data stream API make sure to set manual uids for each > operator. Only then migrating of savepoints to other major versions of > Flink is supported. [1] > > Best, > > Arvid > > [1] > https://ci.apache.org/projects/flink/flink-docs-stable/ops/state/savepoints.html > > On Fri, Jan 22, 2021 at 3:45 PM Matthias Pohl <matth...@ververica.com> > wrote: > >> Hi Lu, >> thanks for reaching out to the community, Lu. Interesting observation. >> There's no change between 1.9.1 and 1.11 that could explain this behavior >> as far as I can tell. Have you had a chance to debug the code? Can you >> provide the code so that we could look into it more closely? >> Another thing: Are you using the TableAPI in your job? There might be >> some problems with setting the maxParallelism in the TableAPI. >> >> Keep in mind that you could use the State Processor API [1] to adjust the >> maxParallelism per Operator in a Savepoint. >> >> Best, >> Matthias >> >> [1] >> https://ci.apache.org/projects/flink/flink-docs-stable/dev/libs/state_processor_api.html#modifying-savepoints >> >> On Fri, Jan 22, 2021 at 12:49 AM Lu Niu <qqib...@gmail.com> wrote: >> >>> Hi, >>> >>> We recently migrated from 1.9.1 to flink 1.11 and notice the new job >>> cannot consume from savepoint taken in 1.9.1. Here is the list of operator >>> id and max parallelism of savepoints taken in both versions. The only code >>> change is version upgrade. >>> >>> savepoint 1.9.1: >>> ``` >>> Id: 8a74550ce6afad759d5f1d6212f43f4a, maxparallsim: 1024 >>> Id: 21753033b264736cab2e32934441d610, maxparallsim: 4096 >>> Id: e03cdfcd66012e06dc52531958e54e8d, maxparallsim: 1024 >>> Id: d003b5c018424b83b771743563711891, maxparallsim: 900 >>> Id: bb0026f9180b3842f4d781c5f7a4a88f, maxparallsim: 4096 >>> Id: b0936afefebc629e050a0f423f44e6ba, maxparallsim: 4096 >>> Id: 5b3f7c70ad9b86408e6af3715f928ad1, maxparallsim: 1024 >>> Id: 278b3965ca58e95e78ab40884c5ddceb, maxparallsim: 900 >>> Id: 6d0402ca998f4658c7632930a69811ac, maxparallsim: 1024 >>> Id: 594970a50fc65ebd163a055fb972541e, maxparallsim: 900 >>> Id: fba56b0a0ee00414d9913103a7c19ff7, maxparallsim: 4096 >>> ``` >>> >>> savepoint 1.11: >>> ``` >>> Id: 8a74550ce6afad759d5f1d6212f43f4a, maxparallsim: 900 >>> Id: 21753033b264736cab2e32934441d610, maxparallsim: 900 >>> Id: e03cdfcd66012e06dc52531958e54e8d, maxparallsim: 900 >>> Id: d1bc8d10e5b8e98e55b2b6c5444f83c7, maxparallsim: 900 >>> Id: d003b5c018424b83b771743563711891, maxparallsim: 900 >>> Id: bb0026f9180b3842f4d781c5f7a4a88f, maxparallsim: 900 >>> Id: b0936afefebc629e050a0f423f44e6ba, maxparallsim: 900 >>> Id: 5b3f7c70ad9b86408e6af3715f928ad1, maxparallsim: 900 >>> Id: 278b3965ca58e95e78ab40884c5ddceb, maxparallsim: 900 >>> Id: 6d0402ca998f4658c7632930a69811ac, maxparallsim: 900 >>> Id: 594970a50fc65ebd163a055fb972541e, maxparallsim: 900 >>> Id: fba56b0a0ee00414d9913103a7c19ff7, maxparallsim: 900 >>> ``` >>> >>> In the code we use env.setMaxParallsim(900). it is strange that >>> savepoint 1.9.1 has different max parallelism for different operators and >>> we don't know where 1024 and 4096 come from. Here I want to ask the >>> community is it possible these are set by flink itself? >>> >>> Best >>> Lu >>> >>
