On Thu, Jul 14, 2022 at 9:59 AM Asaf Mesika <asaf.mes...@gmail.com> wrote:
> >
> > No, as it's mentioned in PIP: this API will terminate the topic so, it
> will
> > not allow any new write and producers on that topic, and then it flags
> the
> > topic as migrated and finally it sends migrated-topic response to
> > producers/caught-up consumers(with 0 backlog) for further redirection.
>
> So, this method will complete once "termination" of topic is complete: wait
> for all producers to disconnect, wait for consumers to finish backlog and
> then all disconnect?
>
>> Producer can be immediately disconnected and allowed to redirect to
green cluster as soon as topic is terminated and marked as part of
termination. (considering the topic doesn't have geo-replication enabled.
For geo-replicated topics , you can read "Replicator and message ordering
handling" section in PIP).
>
> This flag is to persist the state of managed-ledger that it's considered
> > for migration so, if a broker crashes then it knows about the
> > managed-ledger state.
> >
>
So perhaps the flag means "migrationInProgress" or "migrationStarted"?
>
>> It's just a state to mark that broker has considered it for migration
similar to the Terminated state. we don't need extra state for completion
because the broker is going to delete the topic once all subscribers reach
the end of the topic.
>
> On Wed, Jul 13, 2022 at 7:23 PM Rajan Dhabalia <rdhaba...@apache.org>
> wrote:
>
> > On Wed, Jul 13, 2022 at 1:55 AM Asaf Mesika <asaf.mes...@gmail.com>
> wrote:
> >
> > > Few questions
> > >
> > > "CompletableFuture<Position> asyncMigrate();"
> > > Does this method only change the status of the managed ledger?
> > >
> > No, as it's mentioned in PIP: this API will terminate the topic so, it
> will
> > not allow any new write and producers on that topic, and then it flags
> the
> > topic as migrated and finally it sends migrated-topic response to
> > producers/caught-up consumers(with 0 backlog) for further redirection.
> >
> > >
> > > "message ManagedLedgerInfo {
> > >
> > > // Flag to check if topic is terminated and migrated to different
> > > cluster
> > > optional bool migrated = 4;
> > >
> > > }"
> > >
> > > This flag then is only changed to true when it has finished migration:
> > i.e.
> > > no new messages were written, all existing consumers finished reading
> all
> > > messages and disconnected and the topic can now be deleted?
> > >
> > This flag is to persist the state of managed-ledger that it's considered
> > for migration so, if a broker crashes then it knows about the
> > managed-ledger state.
> >
> >
> > > "Broker sends topic migration message to client so, producer/consumer
> at
> > > client side can handle redirection accordingly"
> > >
> > > For producers, the message will be sent the moment the status of the
> > topic
> > > has changed, so all messages from there on will be written to the new
> > > cluster?
> > >
> > Yes.
> >
> >
> > > For consumers, the message will be sent when there are no more messages
> > to
> > > read?
> > >
> > Yes.
> >
> > >
> > >
> > >
> > > On Tue, Jul 12, 2022 at 8:23 PM Rajan Dhabalia <rdhaba...@apache.org>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > We have created PIP-184 which helps users to perform cluster
> migration
> > > with
> > > > Apache Pulsar. Cluster migration or Blue-Green cluster deployment is
> > one
> > > of
> > > > the proven solutions to migrate live traffic from one cluster to
> > another.
> > > > One of the examples is applications running on Kubernetes sometimes
> > > require
> > > > a Kubernetes cluster upgrade which can cause downtime for the entire
> > > > application during a Kubernetes cluster upgrade. Blue-green
> deployment
> > is
> > > > an application release model that gradually transfers user traffic
> > from a
> > > > previous version of an app or microservice to a nearly identical new
> > > > release—both of which are running in production.
> > > >
> > > > The old version can be called the blue environment while the new
> > version
> > > > can be known as the green environment. Once production traffic is
> fully
> > > > transferred from blue to green, blue can standby in case of rollback
> or
> > > be
> > > > pulled from production and updated to become the template upon which
> > the
> > > > next update is made. We need such capability in Apache pulsar to
> > migrate
> > > > live traffic from the blue cluster to the green cluster so,
> eventually,
> > > the
> > > > entire traffic moves from the blue cluster to the green cluster
> without
> > > > causing downtime for the topics.
> > > >
> > > > PIP: https://github.com/apache/pulsar/issues/16551
> > > >
> > > > Thanks,
> > > > Rajan
> > > >
> > >
> >
>
