FYI, I created https://issues.apache.org/jira/browse/SAMZA-1177 to track
this.
We will ensure that we have explicit upgrade documentation for upcoming
releases.
On Thu, Mar 30, 2017 at 2:21 PM, Jagadish Venkatraman <
jagadish1...@gmail.com> wrote:
> Hi Thomas and Maxim,
>
> Thank you for bringin
Hi Thomas and Maxim,
Thank you for bringing up this concern and pointing out the gap in the
documentation. We will ensure that this is documented in a separate upgrade
page going forward.
Best,
Jagadish
On Thu, Mar 30, 2017 at 12:02 PM, Navina Ramesh <
nram...@linkedin.com.invalid> wrote:
> Hi
Hi everyone,
Apologize for re-chiming in late on this issue.
> I'm not sure I agree with the policy (removing migration code and wanting
people to upgrade seem at odds to me), but minimally I think we should not
assume people are upgrading to each new Samza version.
I agree that we should not ass
Is there some sort of document on how to upgrade Samza through various
versions like the page here for Kafka:
https://kafka.apache.org/documentation/#upgrade ?
Having something like this would be ideal.
On Thu, Mar 30, 2017 at 1:51 PM Thomas Becker wrote:
> Thanks for the reply Yi, and I apologiz
Thanks for the reply Yi, and I apologize if I came off a bit snarky.
I'm not sure I agree with the policy (removing migration code and
wanting people to upgrade seem at odds to me), but minimally I think we
should not assume people are upgrading to each new Samza version. We
have done so when featu
Hi, Thomas,
Sorry to hear that you were hit by the removal of migration in Samza 0.11.
The reason we removed it is following a deprecate-removal policy in two
versions. We are not aware that people still using 0.9 after we released
0.11 and were not expecting a direct upgrade from 0.9 to 0.12. Doc
Yes, we were burned by this. The changelog mapping will be regenerated
instead of migrated and the result will completely hose the job
(because the mapping was not generated deterministically in previous
versions of Samza). I don't understand why the migration code was
removed but it was, and to th
Hi colleagues,
If I understand samza source code correctly, without migration code we
would not lose offsets but lose TaskName-to-ChangelogPartition mapping.
State restore for TaskStorage starts from ChangelogSystemStreamPartition
beginning (if we don't use data locality, offsets to start from are
@Jake: Yes. We removed the migration code (for 0.9 to 0.10) in the 0.11
release, I believe.
@XiaoChuan: As per Jagadish's recommendation, if you have changelog backed
stores, you should upgrade from 0.9.1 to 0.10.0 before upgrading to samza
0.12.0.
I checked with LinkedIn's internal release notes
Good observation Jake!
The code for migration was removed in Samza 11. The migration would read
change-log offsets from the checkpoint topic and write them to the
coordinator stream.
If you're using change-logged stores, I'd recommend upgrading from 0.9.1 to
0.10.0 first.
Otherwise, you will loos
Good observation Jake!
The code for migration was removed in Samza 11. The migration would read
change-log offsets from the checkpoint topic and write them to the
coordinator stream.
If you're using change-logged stores, I'd recommend upgrading from 0.9.1 to
0.10.0 first.
Otherwise, you will loos
As I recall, samza 0.10 introduced the coordinator stream and there was
code to do an automatic migration to use that feature. @navina, @yi, do you
know if that migration code is still in samza 12?
If not, then it's probably better to update from 0.9.1 to 0.10.0 and then
to 0.12.0. I don't think t
Hi Xiaochuan,
>> Do I need to upgrade Kafka and/or YARN?
*Yarn version:*
- Samza 0.12 supports Yarn 2.6.1 and 2.7.1.
- If you already have 2.6.0 installed (as you have said), I believe you
will be fine. (but I'm not sure)
*Kafka version: *
- Samza 0.12 upgraded the version of Kafka
13 matches
Mail list logo
