Hi Jay,
Why wouldn't that work, the tombstone value is only looked at by the broker, on
a topic configured for compaction as such is benign on non compacted topics.
This is as much as sending a null value currently
Regards
Mike
Sent using OWA for iPhone
________________________________________
From: Jay Kreps <j...@confluent.io>
Sent: Sunday, December 11, 2016 8:58:53 PM
To: dev@kafka.apache.org
Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
Hey Michael,
I'm not quite sure that works as that would translate ALL null values to
tombstones, even for non-compacted topics that use null as an acceptable
value sent by the producer and expected by the consumer.
-Jay
On Sun, Dec 11, 2016 at 3:26 AM, Michael Pearce <michael.pea...@ig.com>
wrote:
> Hi Ewen,
>
> I think the easiest way to show this is with code.
>
> As you can see we keep the existing behaviour for code/binaries calling
> the pre-existing constructors, whereby if the value is null the tombstone
> is set to true.
>
> Regards
> Mike
>
>
>
> /**
> * Creates a record with a specified timestamp to be sent to a
> specified topic and partition
> *
> * @param topic The topic the record will be appended to
> * @param partition The partition to which the record should be sent
> * @param timestamp The timestamp of the record
> * @param tombstone if the record should be treated as a tombstone if
> the topic is compacted
> * @param key The key that will be included in the record
> * @param value The record contents
> */
> public ProducerRecord(String topic, Integer partition, Boolean
> tombstone, Long timestamp, K key, V value) {
> if (topic == null)
> throw new IllegalArgumentException("Topic cannot be null.");
> if (timestamp != null && timestamp < 0)
> throw new IllegalArgumentException(
> String.format("Invalid timestamp: %d. Timestamp should
> always be non-negative or null.", timestamp));
> if (partition != null && partition < 0)
> throw new IllegalArgumentException(
> String.format("Invalid partition: %d. Partition number
> should always be non-negative or null.", partition));
> if (!tombstone && value == null){
> throw new IllegalArgumentException(
> String.format("Tombstone must be true if null value"));
> }
> this.topic = topic;
> this.partition = partition;
> this.tombstone = tombstone;
> this.key = key;
> this.value = value;
> this.timestamp = timestamp;
> }
>
> public ProducerRecord(String topic, Integer partition, Long timestamp,
> K key, V value) {
> this(topic, partition, value == null, timestamp, key, value);
> }
> ________________________________________
> From: Ewen Cheslack-Postava <e...@confluent.io>
> Sent: Sunday, December 11, 2016 5:45 AM
> To: dev@kafka.apache.org
> Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
>
> It seemed like this was addressed in the migration section, wasn't it? The
> V2 vs V3 requests and the fact that the broker will downgrade the message
> format (losing zero copy) if you issues a V2 request to a broker using V3
> format handles compatibility, does it not? The existing consumers won't see
> the extra metadata in the value, but they will get a null instead and treat
> it as a tombstone. Certainly there is a performance impact, but it seemed
> compatible.
>
> I'm worried about this though:
>
>
> - *NOTE *: With the new version of producer client using ProduceRequest
> V3 (magic byte = 2), a non tombstone (tombstone bit not set) and null
> value
> should not be allowed as the older version of consumer using
> FetchRequest
> V2 will think of this as a tombstone when its actually not.
>
>
> Unless I'm misunderstanding, this ends up breaking binary compatibility for
> the Java API. It sounds like this suggests that passing a null value to the
> existing ProducerRecord constructors would generate an exception since you
> didn't explicitly enable the tombstone (via whatever new constructor is
> provided). But that means you can't swap in a newer client jar without
> recompiling and get the same behavior if your app deletes keys using the
> current approach because your app will start throwing exceptions. Maybe
> this is a tradeoff we're ok with, but we've tried pretty hard to avoid
> breaking compatibility like this so far -- it makes picking up bug fixes in
> the clients harder for users of frameworks that have to pin to earlier
> default versions for compatibility.
>
> But then later the KIP says:
>
>
> - The old constructors for ProducerRecord without this parameter will be
> kept but updated so that their default behaviour if setting null value
> will
> be the tombstone will be set to true to keep existing behaviour.
>
>
> So maybe I am misinterpreting something.
>
> And just a nit re: motivation section, item 6 would be clearer for a union
> schema with null (or optional schemas in other formats), e.g. [null,
> string], in which case losing the schema truly is losing information
> (whereas null is already the only valid value for a pure null schema).
>
> -Ewen
>
>
> On Sat, Dec 10, 2016 at 9:24 PM, Michael Pearce <michael.pea...@ig.com>
> wrote:
>
> > Hi Jay,
> >
> > Good point this detail is missing in the KIP write up. Ive added this
> now.
> >
> > Essentially simply just upgrading the clients we do not expect any client
> > app code change needed.
> >
> > Cheers
> > Mike
> > ________________________________________
> > From: Jay Kreps <j...@confluent.io>
> > Sent: Saturday, December 10, 2016 10:51 PM
> > To: dev@kafka.apache.org
> > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
> >
> > Michael,
> >
> > The compatibility section goes through the migration path, but isn't the
> > bigger compatibility issue with existing apps? There are many (probably
> > thousands) of apps in production that use this feature and send null to
> > mean delete. It seems like this would break compatibility with them, and
> > they would have to be rewritten to right?
> >
> > -Jay
> >
> > On Thu, Dec 8, 2016 at 12:12 AM, Michael Pearce <michael.pea...@ig.com>
> > wrote:
> >
> > > Hi Jun,
> > >
> > > 4) On v3 we honour the tombstone. As such we expect it to be set
> > correctly
> > > as per the KIP.
> > >
> > > 4.1) why would we want to produce an error when its v3? This is the
> exact
> > > purpose to support non-null tombstone’s
> > > 4.2) again here im unclear on the question on the v3, produce request
> we
> > > expect the tombstone flag to be set correctly.
> > >
> > > 4.4) compaction only occurs on compacted topics, the bit makes no
> > > difference and not looked at on un-compacted (time/size based
> eviction).
> > >
> > >
> > > On 06/12/2016, 20:08, "Jun Rao" <j...@confluent.io> wrote:
> > >
> > > Hi, Michael,
> > >
> > > 4. Then, I think I misunderstood this point. Could you document the
> > > following points in the wiki?
> > > 4.1 If producer V3 sets tombstone, but provides a non-null value,
> > does
> > > the
> > > send() get an error or does the producer automatically set the
> value
> > to
> > > null?
> > > 4.2 If producer V3 doesn't set tombstone, but provides a null
> value,
> > > does
> > > the send() get an error or does the producer automatically sets the
> > > tombstone?
> > > 4.3 Does the broker only expect messages that (a) have no tombstone
> > and
> > > non-null value; (b) have tombstone and null value and reject the
> > > messages
> > > with an error code otherwise?
> > > 4.4 Do 4.1, 4.2, 4.3 depend on whether the topic is compacted on
> > not?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Tue, Dec 6, 2016 at 10:35 AM, Michael Pearce <
> > michael.pea...@ig.com
> > > >
> > > wrote:
> > >
> > > > Not at all. This only acts on compacted topics just as what
> occurs
> > > today
> > > >
> > > > Sent using OWA for iPhone
> > > > ________________________________________
> > > > From: Jun Rao <j...@confluent.io>
> > > > Sent: Tuesday, December 6, 2016 6:25:28 PM
> > > > To: dev@kafka.apache.org
> > > > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
> > > >
> > > > Hi, Michael,
> > > >
> > > > 4. Hmm, does that mean the new client library can never send a
> null
> > > message
> > > > even to a regular topic? This seems like a change of the existing
> > > behavior.
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Tue, Dec 6, 2016 at 9:51 AM, Michael Pearce <
> > > michael.pea...@ig.com>
> > > > wrote:
> > > >
> > > > > Hi Jun,
> > > > >
> > > > > Re 4) That's because we expect the tombstone value to be set
> > > correctly if
> > > > > message bit is 2, as such if an older client sends in on old
> > > message the
> > > > > message is upcast and the bit is set correctly. And such no
> > longer
> > > need
> > > > to
> > > > > check the value. Mayuresh can you confirm my thinking and
> > > understanding
> > > > of
> > > > > what we've discussed?
> > > > >
> > > > > The second point I understand what you're getting at now my
> > > apologies.
> > > > Yes
> > > > > this makes sense to save on touching the message, if we're the
> > > only kip
> > > > > going in, in this release.
> > > > >
> > > > > Cheers
> > > > > Mike
> > > > >
> > > > > Sent using OWA for iPhone
> > > > > ________________________________________
> > > > > From: Jun Rao <j...@confluent.io>
> > > > > Sent: Tuesday, December 6, 2016 5:22:13 PM
> > > > > To: dev@kafka.apache.org
> > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
> > > > >
> > > > > Hi, Michael,
> > > > >
> > > > > 4. Is this updated in the wiki? The text "If the magic byte on
> > > message is
> > > > > 2, the broker should use the tombstone bit for log compaction."
> > > doesn't
> > > > > seem to have changed.
> > > > >
> > > > > 2. My point is that if we change the message format just for
> this
> > > KIP, we
> > > > > should consider whether it's worth optimizing the down
> conversion
> > > path
> > > > > (i.e., decide whether a conversion is needed by just looking at
> > the
> > > > > tombstone bit in the wrapper message) since tombstone will be
> > used
> > > > rarely.
> > > > > However, if the message format change here is combined with
> other
> > > KIPs,
> > > > > then this optimization likely won't be needed. The latter
> > probably
> > > makes
> > > > > the code simpler. Jiangjie, Mayuresh, what do you think?
> > > > >
> > > > > Other than those, +1 from me,
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Tue, Dec 6, 2016 at 8:54 AM, Michael Pearce <
> > > michael.pea...@ig.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Jun
> > > > > >
> > > > > > do we have your vote on this now?
> > > > > >
> > > > > > Any other concerns?
> > > > > >
> > > > > > Cheers
> > > > > > Mike
> > > > > >
> > > > > > Sent using OWA for iPhone
> > > > > > ________________________________________
> > > > > > From: Michael Pearce <michael.pea...@ig.com>
> > > > > > Sent: Saturday, December 3, 2016 1:37:45 AM
> > > > > > To: dev@kafka.apache.org
> > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
> > > > > >
> > > > > > Hi Jun,
> > > > > >
> > > > > > Have updated. Thanks again for the feedback.
> > > > > >
> > > > > > Agree yes we should align up when it gets to that, I assume
> > > you’ve
> > > > > flagged
> > > > > > the same in the other KIP?
> > > > > >
> > > > > > I think for now let’s get this KIP approved, then we can
> start
> > > the work
> > > > > to
> > > > > > get an initial PR, then we can discuss how to align the two
> if
> > > needed.
> > > > > >
> > > > > > Cheers,
> > > > > > Mike
> > > > > >
> > > > > > On 02/12/2016, 18:26, "Jun Rao" <j...@confluent.io> wrote:
> > > > > >
> > > > > > Hi, Michael,
> > > > > >
> > > > > > For 2), this is fine. Could you update the KIP wiki to
> make
> > > this
> > > > and
> > > > > > other
> > > > > > points clearer? Other than that, the KIP looks good to
> me.
> > > > > >
> > > > > > An orthogonal thing is that there are other KIPs such as
> > > KIP-98
> > > > that
> > > > > > also
> > > > > > intend to change the message format. If they all get
> > > approved, we
> > > > > > should
> > > > > > think about whether it's better to just bump up the magic
> > > byte once
> > > > > to
> > > > > > incorporate multiple format changes like we did in
> > > KIP-31/KIP-32.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > >
> > > > > > On Fri, Dec 2, 2016 at 3:19 AM, Michael Pearce <
> > > > > michael.pea...@ig.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Jao
> > > > > > >
> > > > > > > Thanks for the response. Sorry for slow reply, both
> with
> > > personal
> > > > > > sickness
> > > > > > > and also battling some critical issues encountered
> since
> > > > upgrading
> > > > > to
> > > > > > > 0.10.1.0
> > > > > > >
> > > > > > > 1) Thans for spotting, Document error where we branched
> > > this KIP
> > > > > from
> > > > > > > KIP-82, will get that removed.
> > > > > > > 2) Intent is to do this just at the record message
> level.
> > > > > > > 3) Thanks for spotting, Will ensure this is corrected.
> > > > > > > 4) As per discussion thread we will support tombstone +
> > > null
> > > > value,
> > > > > > > tombstone + non null value, no tombstone + null value.
> > > > > > > 5) I believe this was in the discussion thread,
> @Mayuresh
> > > is this
> > > > > > > something we’ve overlooked? I thought we would down
> > > convert and
> > > > > > remove the
> > > > > > > value so the old consumer had existing behavior, or is
> > > there
> > > > > > something we
> > > > > > > haven’t thought about?
> > > > > > >
> > > > > > > Cheers
> > > > > > > Mike
> > > > > > >
> > > > > > > On 30/11/2016, 18:12, "Jun Rao" <j...@confluent.io>
> > wrote:
> > > > > > >
> > > > > > > Hi, Michael,
> > > > > > >
> > > > > > > Thanks for the KIP. A few comments below.
> > > > > > >
> > > > > > > 1. The message format change contains
> "HeadersLength
> > > > Headers".
> > > > > > Is that
> > > > > > > intended?
> > > > > > >
> > > > > > > 2. For compressed messageset, is the tombstone bit
> > > only set
> > > > at
> > > > > > the
> > > > > > > shallow
> > > > > > > level? Do we always leave that bit in the wrapper
> > > message
> > > > > unset?
> > > > > > An
> > > > > > > alternative is to set the tombstone bit in the
> > wrapper
> > > if at
> > > > > > least one
> > > > > > > inner message has the tombstone bit set. This makes
> > > things a
> > > > > bit
> > > > > > more
> > > > > > > complicated, but we could potentially exploit that
> > for
> > > > > > optimizing down
> > > > > > > conversion. For example, we only need to convert
> > > messages
> > > > with
> > > > > > magic 2
> > > > > > > to
> > > > > > > magic 1 if the wrapper's tombstone bit is set
> > > (conversion is
> > > > > > always
> > > > > > > needed
> > > > > > > from magic 2 to magic 0). Not sure if the
> > optimization
> > > is
> > > > worth
> > > > > > the
> > > > > > > complexity though.
> > > > > > >
> > > > > > > 3. The referencing of the new version of
> > > > > > ProducerRequest/FetchRequest
> > > > > > > is
> > > > > > > inconsistent (v4 vs v3). Since our convention
> starts
> > at
> > > > version
> > > > > > at 0, I
> > > > > > > think the new version would be 3.
> > > > > > >
> > > > > > > 4. "If the magic byte on message is 2, the broker
> > > should use
> > > > > the
> > > > > > > tombstone
> > > > > > > bit for log compaction." What about null value? My
> > > > > understanding
> > > > > > is
> > > > > > > that
> > > > > > > null value will be treated the same as setting the
> > > tombstone
> > > > > bit.
> > > > > > >
> > > > > > > 5. For the migration path, it would be useful to
> > > describe the
> > > > > > down
> > > > > > > conversion path to consumers (i.e., brokers on
> > message
> > > format
> > > > > > 0.10.2
> > > > > > > and
> > > > > > > consumers on older version).
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Nov 29, 2016 at 3:18 AM, Michael Pearce <
> > > > > > michael.pea...@ig.com
> > > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi All,
> > > > > > > >
> > > > > > > > We have been discussing in the below thread and
> > final
> > > > changes
> > > > > > have
> > > > > > > been
> > > > > > > > made to the KIP wiki based on these discussions.
> > > > > > > >
> > > > > > > > We would now like to put to the vote the
> following
> > > KIP:
> > > > > > > > https://cwiki.apache.org/
> > > confluence/display/KAFKA/KIP-
> > > > 87+-+
> > > > > > > > Add+Compaction+Tombstone+Flag
> > > > > > > >
> > > > > > > > This kip is for having a distinct compaction
> > > attribute
> > > > > > “tombstone”
> > > > > > > flag
> > > > > > > > instead of relying on null value, allowing
> non-null
> > > value
> > > > > > delete
> > > > > > > messages.
> > > > > > > >
> > > > > > > > Many thanks,
> > > > > > > > Michael
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On 22/11/2016, 15:52, "Michael Pearce" <
> > > > > michael.pea...@ig.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > Hi Mayuresh,
> > > > > > > >
> > > > > > > > LGTM. Ive just made one small adjustment
> > > updating the
> > > > > wire
> > > > > > > protocol to
> > > > > > > > show the magic byte bump.
> > > > > > > >
> > > > > > > > Do we think we’re good to put to a vote? Is
> > > there any
> > > > > > other bits
> > > > > > > > needing discussion?
> > > > > > > >
> > > > > > > > Cheers
> > > > > > > > Mike
> > > > > > > >
> > > > > > > > On 21/11/2016, 18:26, "Mayuresh Gharat" <
> > > > > > > gharatmayures...@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > Hi Michael,
> > > > > > > >
> > > > > > > > I have updated the migration section of
> the
> > > KIP.
> > > > Can
> > > > > > you
> > > > > > > please
> > > > > > > > take a look?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Mayuresh
> > > > > > > >
> > > > > > > > On Fri, Nov 18, 2016 at 9:07 AM, Mayuresh
> > > Gharat <
> > > > > > > > gharatmayures...@gmail.com
> > > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Michael,
> > > > > > > > >
> > > > > > > > > That whilst sending tombstone and non
> > null
> > > value,
> > > > > the
> > > > > > > consumer
> > > > > > > > can expect
> > > > > > > > > only to receive the non-null message
> only
> > > in step
> > > > > > (3) is
> > > > > > > this
> > > > > > > > correct?
> > > > > > > > > ---> I do agree with you here.
> > > > > > > > >
> > > > > > > > > Becket, Ismael : can you guys review
> the
> > > > migration
> > > > > > plan
> > > > > > > listed
> > > > > > > > above using
> > > > > > > > > magic byte?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > > Mayuresh
> > > > > > > > >
> > > > > > > > > On Fri, Nov 18, 2016 at 8:58 AM,
> Michael
> > > Pearce <
> > > > > > > > michael.pea...@ig.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > >> Many thanks for this Mayuresh. I don't
> > > have any
> > > > > > > objections.
> > > > > > > > >>
> > > > > > > > >> I assume we should state:
> > > > > > > > >>
> > > > > > > > >> That whilst sending tombstone and non
> > null
> > > > value,
> > > > > > the
> > > > > > > consumer
> > > > > > > > can expect
> > > > > > > > >> only to receive the non-null message
> > only
> > > in
> > > > step
> > > > > > (3) is
> > > > > > > this
> > > > > > > > correct?
> > > > > > > > >>
> > > > > > > > >> Cheers
> > > > > > > > >> Mike
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> Sent using OWA for iPhone
> > > > > > > > >> ______________________________
> > __________
> > > > > > > > >> From: Mayuresh Gharat <
> > > > gharatmayures...@gmail.com
> > > > > >
> > > > > > > > >> Sent: Thursday, November 17, 2016
> > 5:18:41
> > > PM
> > > > > > > > >> To: dev@kafka.apache.org
> > > > > > > > >> Subject: Re: [DISCUSS] KIP-87 - Add
> > > Compaction
> > > > > > Tombstone
> > > > > > > Flag
> > > > > > > > >>
> > > > > > > > >> Hi Ismael,
> > > > > > > > >>
> > > > > > > > >> Thanks for the explanation.
> > > > > > > > >> Specially I like this part where in
> you
> > > > mentioned
> > > > > > we can
> > > > > > > get
> > > > > > > > rid of the
> > > > > > > > >> older null value support for log
> > > compaction
> > > > later
> > > > > > on,
> > > > > > > here :
> > > > > > > > >> We can't change semantics of the
> message
> > > format
> > > > > > without
> > > > > > > having
> > > > > > > > a long
> > > > > > > > >> transition period. And we can't rely
> > > > > > > > >> on people reading documentation or
> > acting
> > > on a
> > > > > > warning for
> > > > > > > > something so
> > > > > > > > >> fundamental. As such, my take is that
> we
> > > need to
> > > > > > bump the
> > > > > > > magic
> > > > > > > > byte. The
> > > > > > > > >> good news is
> > > > > > > > >> that we don't have to support all
> > versions
> > > > > forever.
> > > > > > We
> > > > > > > have
> > > > > > > > said that we
> > > > > > > > >> will support direct upgrades for 2
> > years.
> > > That
> > > > > > means that
> > > > > > > > message format
> > > > > > > > >> version n could, in theory, be
> removed 2
> > > years
> > > > > > after the
> > > > > > > it's
> > > > > > > > introduced.
> > > > > > > > >>
> > > > > > > > >> Just a heads up, I would like to
> mention
> > > that
> > > > even
> > > > > > without
> > > > > > > > bumping magic
> > > > > > > > >> byte, we will *NOT* loose zero copy as
> > in
> > > the
> > > > > > client(x+1)
> > > > > > > in my
> > > > > > > > >> explanation
> > > > > > > > >> above will convert internally a null
> > > value to
> > > > > have a
> > > > > > > tombstone
> > > > > > > > bit set and
> > > > > > > > >> a tombstone bit set to have a null
> value
> > > > > > automatically
> > > > > > > > internally and by
> > > > > > > > >> the time we move to version (x+2), the
> > > clients
> > > > > > would have
> > > > > > > > upgraded.
> > > > > > > > >> Obviously if we support a request from
> > > > > consumer(x),
> > > > > > we
> > > > > > > will
> > > > > > > > loose zero
> > > > > > > > >> copy
> > > > > > > > >> but that is the same case with magic
> > byte.
> > > > > > > > >>
> > > > > > > > >> But if magic byte bump makes life
> easier
> > > for
> > > > > > transition
> > > > > > > for the
> > > > > > > > above
> > > > > > > > >> reasons that you explained, I am OK
> with
> > > it
> > > > since
> > > > > > we are
> > > > > > > going
> > > > > > > > to meet the
> > > > > > > > >> end goal down the road :)
> > > > > > > > >>
> > > > > > > > >> On a side note can we update the doc
> > here
> > > on
> > > > magic
> > > > > > byte
> > > > > > > to say
> > > > > > > > that "*it
> > > > > > > > >> should be bumped whenever the message
> > > format is
> > > > > > changed
> > > > > > > or the
> > > > > > > > >> interpretation of message format
> (usage
> > > of the
> > > > > > reserved
> > > > > > > bits as
> > > > > > > > well) is
> > > > > > > > >> changed*".
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> Hi Michael,
> > > > > > > > >>
> > > > > > > > >> Here is the update plan that we
> > discussed
> > > > offline
> > > > > > > yesterday :
> > > > > > > > >>
> > > > > > > > >> Currently the magic-byte which
> > > corresponds to
> > > > the
> > > > > > > > "message.format.version"
> > > > > > > > >> is set to 1.
> > > > > > > > >>
> > > > > > > > >> 1) On broker it will be set to 1
> > > initially.
> > > > > > > > >>
> > > > > > > > >> 2) When a producer client sends a
> > message
> > > with
> > > > > > magic-byte
> > > > > > > = 2,
> > > > > > > > since the
> > > > > > > > >> broker is on magic-byte = 1, we will
> > down
> > > > convert
> > > > > > it,
> > > > > > > which
> > > > > > > > means if the
> > > > > > > > >> tombstone bit is set, the value will
> be
> > > set to
> > > > > > null. A
> > > > > > > consumer
> > > > > > > > >> understanding magic-byte = 1, will
> still
> > > work
> > > > with
> > > > > > this. A
> > > > > > > > consumer
> > > > > > > > >> working
> > > > > > > > >> with magic-byte =2 will also be able
> to
> > > > understand
> > > > > > this,
> > > > > > > since
> > > > > > > > it
> > > > > > > > >> understands the tombstone.
> > > > > > > > >> Now there is still the question of
> > > supporting a
> > > > > > > non-tombstone
> > > > > > > > and null
> > > > > > > > >> value from producer client with
> > > magic-byte = 2.*
> > > > > (I
> > > > > > am
> > > > > > > not sure
> > > > > > > > if we
> > > > > > > > >> should support this. Ismael/Becket can
> > > comment
> > > > > > here)*
> > > > > > > > >>
> > > > > > > > >> 3) When almost all the clients have
> > > upgraded,
> > > > the
> > > > > > > > message.format.version
> > > > > > > > >> on
> > > > > > > > >> the broker can be changed to 2, where
> in
> > > the
> > > > down
> > > > > > > conversion in
> > > > > > > > the above
> > > > > > > > >> step will not happen. If at this point
> > we
> > > get a
> > > > > > consumer
> > > > > > > > request from a
> > > > > > > > >> older consumer, we might have to down
> > > convert
> > > > > where
> > > > > > in we
> > > > > > > loose
> > > > > > > > zero copy,
> > > > > > > > >> but these cases should be rare.
> > > > > > > > >>
> > > > > > > > >> Becket can you review this plan and
> add
> > > more
> > > > > > details if I
> > > > > > > have
> > > > > > > > >> missed/wronged something, before we
> put
> > > it on
> > > > KIP.
> > > > > > > > >>
> > > > > > > > >> Thanks,
> > > > > > > > >>
> > > > > > > > >> Mayuresh
> > > > > > > > >>
> > > > > > > > >> On Wed, Nov 16, 2016 at 11:07 PM,
> > Michael
> > > > Pearce <
> > > > > > > > michael.pea...@ig.com>
> > > > > > > > >> wrote:
> > > > > > > > >>
> > > > > > > > >> > Thanks guys, for discussing this
> > > offline and
> > > > > > getting
> > > > > > > some
> > > > > > > > consensus.
> > > > > > > > >> >
> > > > > > > > >> > So its clear for myself and others
> > what
> > > is
> > > > > > proposed now
> > > > > > > (i
> > > > > > > > think i
> > > > > > > > >> > understand, but want to make sure)
> > > > > > > > >> >
> > > > > > > > >> > Could i ask either directly update
> the
> > > kip to
> > > > > > detail the
> > > > > > > > migration
> > > > > > > > >> > strategy, or (re-)state your offline
> > > discussed
> > > > > and
> > > > > > > agreed
> > > > > > > > migration
> > > > > > > > >> > strategy based on a magic byte is in
> > > this
> > > > > thread.
> > > > > > > > >> >
> > > > > > > > >> >
> > > > > > > > >> > The main original driver for the KIP
> > > was to
> > > > > > support
> > > > > > > > compaction where
> > > > > > > > >> value
> > > > > > > > >> > isn't null, based off the
> discussions
> > on
> > > > KIP-82
> > > > > > thread.
> > > > > > > > >> >
> > > > > > > > >> > We should be able to support
> > > non-tombstone +
> > > > > null
> > > > > > value
> > > > > > > by the
> > > > > > > > >> completion
> > > > > > > > >> > of the KIP, as we noted when
> > discussing
> > > this
> > > > > kip,
> > > > > > having
> > > > > > > > logic based on
> > > > > > > > >> a
> > > > > > > > >> > null value isn't very clean and also
> > > separates
> > > > > the
> > > > > > > concerns.
> > > > > > > > >> >
> > > > > > > > >> > As discussed already though we can
> > > split this
> > > > > into
> > > > > > > KIP-87a
> > > > > > > > and KIP-87b
> > > > > > > > >> >
> > > > > > > > >> > Where we look to deliver KIP-87a on
> a
> > > > compacted
> > > > > > topic
> > > > > > > (to
> > > > > > > > address the
> > > > > > > > >> > immediate issues)
> > > > > > > > >> > * tombstone + null value
> > > > > > > > >> > * tombstone + non-null value
> > > > > > > > >> > * non-tombstone + non-null value
> > > > > > > > >> >
> > > > > > > > >> > Then we can discuss once KIP-87a is
> > > completed
> > > > > > options
> > > > > > > later
> > > > > > > > and how we
> > > > > > > > >> > support the second part KIP-87b to
> > > deliver:
> > > > > > > > >> > * non-tombstone + null value
> > > > > > > > >> >
> > > > > > > > >> > Cheers
> > > > > > > > >> > Mike
> > > > > > > > >> >
> > > > > > > > >> >
> > > > > > > > >> >
> > > > > > > > >> > ______________________________
> > > __________
> > > > > > > > >> > From: Becket Qin <
> > becket....@gmail.com>
> > > > > > > > >> > Sent: Thursday, November 17, 2016
> 1:43
> > > AM
> > > > > > > > >> > To: dev@kafka.apache.org
> > > > > > > > >> > Subject: Re: [DISCUSS] KIP-87 - Add
> > > Compaction
> > > > > > > Tombstone Flag
> > > > > > > > >> >
> > > > > > > > >> > Renu, Mayuresh and I had an offline
> > > > discussion,
> > > > > > and
> > > > > > > following
> > > > > > > > is a brief
> > > > > > > > >> > summary.
> > > > > > > > >> >
> > > > > > > > >> > 1. We agreed that not bumping up
> magic
> > > value
> > > > may
> > > > > > result
> > > > > > > in
> > > > > > > > losing zero
> > > > > > > > >> copy
> > > > > > > > >> > during migration.
> > > > > > > > >> > 2. Given that bumping up magic value
> > is
> > > almost
> > > > > > free and
> > > > > > > has
> > > > > > > > benefit of
> > > > > > > > >> > avoiding potential performance
> issue.
> > > It is
> > > > > > probably
> > > > > > > worth
> > > > > > > > doing.
> > > > > > > > >> >
> > > > > > > > >> > One issue we still need to think
> about
> > > is
> > > > > whether
> > > > > > we
> > > > > > > want to
> > > > > > > > support a
> > > > > > > > >> > non-tombstone message with null
> value.
> > > > > > > > >> > Currently it is not supported by
> > Kafka.
> > > If we
> > > > > > allow a
> > > > > > > > non-tombstone null
> > > > > > > > >> > value message to exist after KIP-87.
> > The
> > > > problem
> > > > > > is
> > > > > > > that such
> > > > > > > > message
> > > > > > > > >> will
> > > > > > > > >> > not be supported by the consumers
> > prior
> > > to
> > > > > KIP-87.
> > > > > > > Because a
> > > > > > > > null value
> > > > > > > > >> > will always be interpreted to a
> > > tombstone.
> > > > > > > > >> >
> > > > > > > > >> > One option is that we keep the
> current
> > > way,
> > > > i.e.
> > > > > > do not
> > > > > > > > support such
> > > > > > > > >> > message. It would be good to know if
> > > there is
> > > > a
> > > > > > > concrete use
> > > > > > > > case for
> > > > > > > > >> such
> > > > > > > > >> > message. If there is not, we can
> > > probably just
> > > > > not
> > > > > > > support it.
> > > > > > > > >> >
> > > > > > > > >> > Thanks,
> > > > > > > > >> >
> > > > > > > > >> > JIangjie (Becket) Qin
> > > > > > > > >> >
> > > > > > > > >> >
> > > > > > > > >> >
> > > > > > > > >> > On Wed, Nov 16, 2016 at 1:28 PM,
> > > Mayuresh
> > > > > Gharat <
> > > > > > > > >> > gharatmayures...@gmail.com
> > > > > > > > >> > > wrote:
> > > > > > > > >> >
> > > > > > > > >> > > Hi Ismael,
> > > > > > > > >> > >
> > > > > > > > >> > > This is something I can think of
> for
> > > > migration
> > > > > > plan:
> > > > > > > > >> > > So the migration plan can look
> > > something
> > > > like
> > > > > > this,
> > > > > > > with up
> > > > > > > > >> conversion :
> > > > > > > > >> > >
> > > > > > > > >> > > 1) Currently lets say we have
> Broker
> > > at
> > > > > version
> > > > > > x.
> > > > > > > > >> > > 2) Currently we have clients at
> > > version x.
> > > > > > > > >> > > 3) a) We move the version to
> > > Broker(x+1) :
> > > > > > supports
> > > > > > > both
> > > > > > > > tombstone and
> > > > > > > > >> > null
> > > > > > > > >> > > for log compaction.
> > > > > > > > >> > > b) We upgrade the client to
> > > version
> > > > > > client(x+1) :
> > > > > > > if in
> > > > > > > > the
> > > > > > > > >> producer
> > > > > > > > >> > > client(x+1) the value is set to
> > null,
> > > we
> > > > will
> > > > > > > automatically
> > > > > > > > set the
> > > > > > > > >> > > Tombstone bit internally. If the
> > > producer
> > > > > > client(x+1)
> > > > > > > sets
> > > > > > > > the
> > > > > > > > >> tombstone
> > > > > > > > >> > > itself, well and good. For
> producer
> > > > client(x),
> > > > > > the
> > > > > > > broker
> > > > > > > > will up
> > > > > > > > >> convert
> > > > > > > > >> > > to have the tombstone bit.
> > > Broker(x+1) is
> > > > > > supporting
> > > > > > > both.
> > > > > > > > Consumer
> > > > > > > > >> > > client(x+1) will be aware of this
> > and
> > > should
> > > > > be
> > > > > > able
> > > > > > > to
> > > > > > > > handle this.
> > > > > > > > >> For
> > > > > > > > >> > > consumer client(x) we will down
> > > convert the
> > > > > > message
> > > > > > > on the
> > > > > > > > broker
> > > > > > > > >> side.
> > > > > > > > >> > > c) At this point we will have
> to
> > > > specify a
> > > > > > > warning or
> > > > > > > > clearly
> > > > > > > > >> specify
> > > > > > > > >> > > in docs that this behavior is
> about
> > > to be
> > > > > > changed for
> > > > > > > log
> > > > > > > > compaction.
> > > > > > > > >> > > 4) a) In next release of the
> > > Broker(x+2), we
> > > > > > say that
> > > > > > > only
> > > > > > > > Tombstone
> > > > > > > > >> is
> > > > > > > > >> > > used for log compaction on the
> > Broker
> > > side.
> > > > > > > Clients(x+1)
> > > > > > > > still is
> > > > > > > > >> > > supported.
> > > > > > > > >> > > b) We upgrade the client to
> > > version
> > > > > > client(x+2) :
> > > > > > > if
> > > > > > > > value is set
> > > > > > > > >> to
> > > > > > > > >> > > null, tombstone will not be set
> > > > automatically.
> > > > > > The
> > > > > > > client
> > > > > > > > will have to
> > > > > > > > >> > call
> > > > > > > > >> > > setTombstone() to actually set the
> > > > tombstone.
> > > > > > > > >> > >
> > > > > > > > >> > > We should compare this migration
> > plan
> > > with
> > > > the
> > > > > > > migration
> > > > > > > > plan for
> > > > > > > > >> magic
> > > > > > > > >> > > byte bump and do whatever looks
> > good.
> > > > > > > > >> > > I am just worried that if we go
> down
> > > magic
> > > > > byte
> > > > > > route,
> > > > > > > > unless I am
> > > > > > > > >> > missing
> > > > > > > > >> > > something, it sounds like kafka
> will
> > > be
> > > > stuck
> > > > > > with
> > > > > > > > supporting both
> > > > > > > > >> null
> > > > > > > > >> > > value and tombstone bit for log
> > > compaction
> > > > for
> > > > > > life
> > > > > > > long,
> > > > > > > > which does
> > > > > > > > >> not
> > > > > > > > >> > > look like a good end state.
> > > > > > > > >> > >
> > > > > > > > >> > > Thanks,
> > > > > > > > >> > >
> > > > > > > > >> > > Mayuresh
> > > > > > > > >> > >
> > > > > > > > >> > >
> > > > > > > > >> > >
> > > > > > > > >> > >
> > > > > > > > >> > > On Wed, Nov 16, 2016 at 9:32 AM,
> > > Mayuresh
> > > > > > Gharat <
> > > > > > > > >> > > gharatmayures...@gmail.com
> > > > > > > > >> > > > wrote:
> > > > > > > > >> > >
> > > > > > > > >> > > > Hi Ismael,
> > > > > > > > >> > > >
> > > > > > > > >> > > > That's a very good point which I
> > > might
> > > > have
> > > > > > not
> > > > > > > > considered earlier.
> > > > > > > > >> > > >
> > > > > > > > >> > > > Here is a plan that I can think
> > of:
> > > > > > > > >> > > >
> > > > > > > > >> > > > Stage 1) The broker from now on,
> > up
> > > > converts
> > > > > > the
> > > > > > > message
> > > > > > > > to have the
> > > > > > > > >> > > > tombstone marker. The log
> > compaction
> > > > thread
> > > > > > does log
> > > > > > > > compaction
> > > > > > > > >> based
> > > > > > > > >> > on
> > > > > > > > >> > > > both null and tombstone marker.
> > > This is
> > > > our
> > > > > > > transition
> > > > > > > > period.
> > > > > > > > >> > > > Stage 2) The next release we
> only
> > > say that
> > > > > log
> > > > > > > compaction
> > > > > > > > is based
> > > > > > > > >> on
> > > > > > > > >> > > > tombstone marker. (Open source
> > > kafka makes
> > > > > > this as a
> > > > > > > > policy). By
> > > > > > > > >> this
> > > > > > > > >> > > time,
> > > > > > > > >> > > > the organization which is moving
> > to
> > > this
> > > > > > release
> > > > > > > will be
> > > > > > > > sure that
> > > > > > > > >> they
> > > > > > > > >> > > > have gone through the entire
> > > transition
> > > > > > period.
> > > > > > > > >> > > >
> > > > > > > > >> > > > My only goal of doing this is
> that
> > > Kafka
> > > > > > clearly
> > > > > > > > specifies the end
> > > > > > > > >> > state
> > > > > > > > >> > > > about what log compaction means
> > (is
> > > it
> > > > null
> > > > > > value
> > > > > > > or a
> > > > > > > > tombstone
> > > > > > > > >> > marker,
> > > > > > > > >> > > > but not both).
> > > > > > > > >> > > >
> > > > > > > > >> > > > What do you think?
> > > > > > > > >> > > >
> > > > > > > > >> > > > Thanks,
> > > > > > > > >> > > >
> > > > > > > > >> > > > Mayuresh
> > > > > > > > >> > > > .
> > > > > > > > >> > > >
> > > > > > > > >> > > > On Wed, Nov 16, 2016 at 9:17 AM,
> > > Ismael
> > > > > Juma <
> > > > > > > > ism...@juma.me.uk>
> > > > > > > > >> > wrote:
> > > > > > > > >> > > >
> > > > > > > > >> > > >> One comment below.
> > > > > > > > >> > > >>
> > > > > > > > >> > > >> On Wed, Nov 16, 2016 at 5:08
> PM,
> > > Mayuresh
> > > > > > Gharat <
> > > > > > > > >> > > >> gharatmayures...@gmail.com
> > > > > > > > >> > > >> > wrote:
> > > > > > > > >> > > >>
> > > > > > > > >> > > >> > - If we don't bump up the
> > > magic
> > > > byte,
> > > > > > on the
> > > > > > > broker
> > > > > > > > side, the
> > > > > > > > >> > > broker
> > > > > > > > >> > > >> > will always have to look
> at
> > > both
> > > > > > tombstone
> > > > > > > bit and
> > > > > > > > the value
> > > > > > > > >> when
> > > > > > > > >> > > do
> > > > > > > > >> > > >> the
> > > > > > > > >> > > >> > compaction. Assuming we do
> > > not bump
> > > > up
> > > > > > the
> > > > > > > magic
> > > > > > > > byte,
> > > > > > > > >> > > >> > imagine the broker sees a
> > > message
> > > > > which
> > > > > > does
> > > > > > > not
> > > > > > > > have a
> > > > > > > > >> tombstone
> > > > > > > > >> > > bit
> > > > > > > > >> > > >> > set. The broker does not
> > know
> > > when
> > > > the
> > > > > > > message was
> > > > > > > > produced
> > > > > > > > >> (i.e.
> > > > > > > > >> > > >> > whether
> > > > > > > > >> > > >> > the message has been up
> > > converted or
> > > > > > not), it
> > > > > > > has
> > > > > > > > to take a
> > > > > > > > >> > further
> > > > > > > > >> > > >> > look at
> > > > > > > > >> > > >> > the value to see if it is
> > > null or
> > > > not
> > > > > in
> > > > > > > order to
> > > > > > > > determine
> > > > > > > > >> if it
> > > > > > > > >> > > is
> > > > > > > > >> > > >> a
> > > > > > > > >> > > >> > tombstone. The same logic
> > has
> > > to be
> > > > > put
> > > > > > on the
> > > > > > > > consumer as
> > > > > > > > >> well
> > > > > > > > >> > > >> because
> > > > > > > > >> > > >> > the
> > > > > > > > >> > > >> > consumer does not know if
> > the
> > > > message
> > > > > > has
> > > > > > > been up
> > > > > > > > converted or
> > > > > > > > >> > not.
> > > > > > > > >> > > >> > - If we upconvert while
> > > > appending,
> > > > > > this is
> > > > > > > not
> > > > > > > > the case,
> > > > > > > > >> > right?
> > > > > > > > >> > > >>
> > > > > > > > >> > > >>
> > > > > > > > >> > > >> If I understand you correctly,
> > > this is
> > > > not
> > > > > > > sufficient
> > > > > > > > because the
> > > > > > > > >> log
> > > > > > > > >> > > may
> > > > > > > > >> > > >> have messages appended before
> it
> > > was
> > > > > > upgraded to
> > > > > > > include
> > > > > > > > KIP-87.
> > > > > > > > >> > > >>
> > > > > > > > >> > > >> Ismael
> > > > > > > > >> > > >>
> > > > > > > > >> > > >
> > > > > > > > >> > > >
> > > > > > > > >> > > >
> > > > > > > > >> > > > --
> > > > > > > > >> > > > -Regards,
> > > > > > > > >> > > > Mayuresh R. Gharat
> > > > > > > > >> > > > (862) 250-7125
> > > > > > > > >> > > >
> > > > > > > > >> > >
> > > > > > > > >> > >
> > > > > > > > >> > >
> > > > > > > > >> > > --
> > > > > > > > >> > > -Regards,
> > > > > > > > >> > > Mayuresh R. Gharat
> > > > > > > > >> > > (862) 250-7125
> > > > > > > > >> > >
> > > > > > > > >> > The information contained in this
> > email
> > > is
> > > > > > strictly
> > > > > > > > confidential and for
> > > > > > > > >> > the use of the addressee only,
> unless
> > > > otherwise
> > > > > > > indicated. If
> > > > > > > > you are
> > > > > > > > >> not
> > > > > > > > >> > the intended recipient, please do
> not
> > > read,
> > > > > copy,
> > > > > > use or
> > > > > > > > disclose to
> > > > > > > > >> others
> > > > > > > > >> > this message or any attachment.
> Please
> > > also
> > > > > > notify the
> > > > > > > sender
> > > > > > > > by
> > > > > > > > >> replying
> > > > > > > > >> > to this email or by telephone
> (+44(020
> > > 7896
> > > > > 0011)
> > > > > > and
> > > > > > > then
> > > > > > > > delete the
> > > > > > > > >> email
> > > > > > > > >> > and any copies of it. Opinions,
> > > conclusion
> > > > (etc)
> > > > > > that
> > > > > > > do not
> > > > > > > > relate to
> > > > > > > > >> the
> > > > > > > > >> > official business of this company
> > shall
> > > be
> > > > > > understood as
> > > > > > > > neither given
> > > > > > > > >> nor
> > > > > > > > >> > endorsed by it. IG is a trading name
> > of
> > > IG
> > > > > Markets
> > > > > > > Limited (a
> > > > > > > > company
> > > > > > > > >> > registered in England and Wales,
> > company
> > > > number
> > > > > > > 04008957) and
> > > > > > > > IG Index
> > > > > > > > >> > Limited (a company registered in
> > > England and
> > > > > > Wales,
> > > > > > > company
> > > > > > > > number
> > > > > > > > >> > 01190902). Registered address at
> > Cannon
> > > Bridge
> > > > > > House, 25
> > > > > > > > Dowgate Hill,
> > > > > > > > >> > London EC4R 2YA. Both IG Markets
> > Limited
> > > > > (register
> > > > > > > number
> > > > > > > > 195355) and IG
> > > > > > > > >> > Index Limited (register number
> 114059)
> > > are
> > > > > > authorised
> > > > > > > and
> > > > > > > > regulated by
> > > > > > > > >> the
> > > > > > > > >> > Financial Conduct Authority.
> > > > > > > > >> >
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> --
> > > > > > > > >> -Regards,
> > > > > > > > >> Mayuresh R. Gharat
> > > > > > > > >> (862) 250-7125
> > > > > > > > >> The information contained in this
> email
> > is
> > > > > strictly
> > > > > > > > confidential and for
> > > > > > > > >> the use of the addressee only, unless
> > > otherwise
> > > > > > > indicated. If
> > > > > > > > you are not
> > > > > > > > >> the intended recipient, please do not
> > > read,
> > > > copy,
> > > > > > use or
> > > > > > > > disclose to others
> > > > > > > > >> this message or any attachment. Please
> > > also
> > > > notify
> > > > > > the
> > > > > > > sender
> > > > > > > > by replying
> > > > > > > > >> to this email or by telephone (+44(020
> > > 7896
> > > > 0011)
> > > > > > and then
> > > > > > > > delete the email
> > > > > > > > >> and any copies of it. Opinions,
> > > conclusion (etc)
> > > > > > that do
> > > > > > > not
> > > > > > > > relate to the
> > > > > > > > >> official business of this company
> shall
> > be
> > > > > > understood as
> > > > > > > > neither given nor
> > > > > > > > >> endorsed by it. IG is a trading name
> of
> > IG
> > > > Markets
> > > > > > > Limited (a
> > > > > > > > company
> > > > > > > > >> registered in England and Wales,
> company
> > > number
> > > > > > 04008957)
> > > > > > > and
> > > > > > > > IG Index
> > > > > > > > >> Limited (a company registered in
> England
> > > and
> > > > > Wales,
> > > > > > > company
> > > > > > > > number
> > > > > > > > >> 01190902). Registered address at
> Cannon
> > > Bridge
> > > > > > House, 25
> > > > > > > > Dowgate Hill,
> > > > > > > > >> London EC4R 2YA. Both IG Markets
> Limited
> > > > (register
> > > > > > number
> > > > > > > > 195355) and IG
> > > > > > > > >> Index Limited (register number 114059)
> > are
> > > > > > authorised and
> > > > > > > > regulated by the
> > > > > > > > >> Financial Conduct Authority.
> > > > > > > > >>
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > -Regards,
> > > > > > > > > Mayuresh R. Gharat
> > > > > > > > > (862) 250-7125
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > -Regards,
> > > > > > > > Mayuresh R. Gharat
> > > > > > > > (862) 250-7125
> > > > > > > >
> > > > > > > >
> > > > > > > > The information contained in this email is
> > > strictly
> > > > > > confidential
> > > > > > > and
> > > > > > > > for the use of the addressee only, unless
> otherwise
> > > > > indicated.
> > > > > > If
> > > > > > > you are
> > > > > > > > not the intended recipient, please do not read,
> > > copy, use
> > > > or
> > > > > > > disclose to
> > > > > > > > others this message or any attachment. Please
> also
> > > notify
> > > > the
> > > > > > sender
> > > > > > > by
> > > > > > > > replying to this email or by telephone (+44(020
> > 7896
> > > 0011)
> > > > > and
> > > > > > then
> > > > > > > delete
> > > > > > > > the email and any copies of it. Opinions,
> > conclusion
> > > (etc)
> > > > > > that do
> > > > > > > not
> > > > > > > > relate to the official business of this company
> > > shall be
> > > > > > understood
> > > > > > > as
> > > > > > > > neither given nor endorsed by it. IG is a trading
> > > name of
> > > > IG
> > > > > > Markets
> > > > > > > > Limited (a company registered in England and
> Wales,
> > > company
> > > > > > number
> > > > > > > > 04008957) and IG Index Limited (a company
> > registered
> > > in
> > > > > > England and
> > > > > > > Wales,
> > > > > > > > company number 01190902). Registered address at
> > > Cannon
> > > > Bridge
> > > > > > House,
> > > > > > > 25
> > > > > > > > Dowgate Hill, London EC4R 2YA. Both IG Markets
> > > Limited
> > > > > > (register
> > > > > > > number
> > > > > > > > 195355) and IG Index Limited (register number
> > > 114059) are
> > > > > > authorised
> > > > > > > and
> > > > > > > > regulated by the Financial Conduct Authority.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > The information contained in this email is strictly
> > > confidential
> > > > > and
> > > > > > for
> > > > > > > the use of the addressee only, unless otherwise
> > indicated.
> > > If you
> > > > > > are not
> > > > > > > the intended recipient, please do not read, copy, use
> or
> > > disclose
> > > > > to
> > > > > > others
> > > > > > > this message or any attachment. Please also notify the
> > > sender by
> > > > > > replying
> > > > > > > to this email or by telephone (+44(020 7896 0011) and
> > then
> > > delete
> > > > > > the email
> > > > > > > and any copies of it. Opinions, conclusion (etc) that
> do
> > > not
> > > > relate
> > > > > > to the
> > > > > > > official business of this company shall be understood
> as
> > > neither
> > > > > > given nor
> > > > > > > endorsed by it. IG is a trading name of IG Markets
> > Limited
> > > (a
> > > > > company
> > > > > > > registered in England and Wales, company number
> 04008957)
> > > and IG
> > > > > > Index
> > > > > > > Limited (a company registered in England and Wales,
> > company
> > > > number
> > > > > > > 01190902). Registered address at Cannon Bridge House,
> 25
> > > Dowgate
> > > > > > Hill,
> > > > > > > London EC4R 2YA. Both IG Markets Limited (register
> number
> > > 195355)
> > > > > > and IG
> > > > > > > Index Limited (register number 114059) are authorised
> and
> > > > regulated
> > > > > > by the
> > > > > > > Financial Conduct Authority.
> > > > > > >
> > > > > >
> > > > > >
> > > > > > The information contained in this email is strictly
> > confidential
> > > and
> > > > for
> > > > > > the use of the addressee only, unless otherwise indicated. If
> > > you are
> > > > not
> > > > > > the intended recipient, please do not read, copy, use or
> > > disclose to
> > > > > others
> > > > > > this message or any attachment. Please also notify the sender
> > by
> > > > replying
> > > > > > to this email or by telephone (+44(020 7896 0011) and then
> > > delete the
> > > > > email
> > > > > > and any copies of it. Opinions, conclusion (etc) that do not
> > > relate to
> > > > > the
> > > > > > official business of this company shall be understood as
> > neither
> > > given
> > > > > nor
> > > > > > endorsed by it. IG is a trading name of IG Markets Limited (a
> > > company
> > > > > > registered in England and Wales, company number 04008957) and
> > IG
> > > Index
> > > > > > Limited (a company registered in England and Wales, company
> > > number
> > > > > > 01190902). Registered address at Cannon Bridge House, 25
> > Dowgate
> > > Hill,
> > > > > > London EC4R 2YA. Both IG Markets Limited (register number
> > > 195355) and
> > > > IG
> > > > > > Index Limited (register number 114059) are authorised and
> > > regulated by
> > > > > the
> > > > > > Financial Conduct Authority.
> > > > > >
> > > > > The information contained in this email is strictly
> confidential
> > > and for
> > > > > the use of the addressee only, unless otherwise indicated. If
> you
> > > are not
> > > > > the intended recipient, please do not read, copy, use or
> disclose
> > > to
> > > > others
> > > > > this message or any attachment. Please also notify the sender
> by
> > > replying
> > > > > to this email or by telephone (+44(020 7896 0011) and then
> delete
> > > the
> > > > email
> > > > > and any copies of it. Opinions, conclusion (etc) that do not
> > > relate to
> > > > the
> > > > > official business of this company shall be understood as
> neither
> > > given
> > > > nor
> > > > > endorsed by it. IG is a trading name of IG Markets Limited (a
> > > company
> > > > > registered in England and Wales, company number 04008957) and
> IG
> > > Index
> > > > > Limited (a company registered in England and Wales, company
> > number
> > > > > 01190902). Registered address at Cannon Bridge House, 25
> Dowgate
> > > Hill,
> > > > > London EC4R 2YA. Both IG Markets Limited (register number
> 195355)
> > > and IG
> > > > > Index Limited (register number 114059) are authorised and
> > > regulated by
> > > > the
> > > > > Financial Conduct Authority.
> > > > >
> > > > >
> > > > The information contained in this email is strictly confidential
> > and
> > > for
> > > > the use of the addressee only, unless otherwise indicated. If you
> > > are not
> > > > the intended recipient, please do not read, copy, use or disclose
> > to
> > > others
> > > > this message or any attachment. Please also notify the sender by
> > > replying
> > > > to this email or by telephone (+44(020 7896 0011) and then delete
> > > the email
> > > > and any copies of it. Opinions, conclusion (etc) that do not
> relate
> > > to the
> > > > official business of this company shall be understood as neither
> > > given nor
> > > > endorsed by it. IG is a trading name of IG Markets Limited (a
> > company
> > > > registered in England and Wales, company number 04008957) and IG
> > > Index
> > > > Limited (a company registered in England and Wales, company
> number
> > > > 01190902). Registered address at Cannon Bridge House, 25 Dowgate
> > > Hill,
> > > > London EC4R 2YA. Both IG Markets Limited (register number 195355)
> > > and IG
> > > > Index Limited (register number 114059) are authorised and
> regulated
> > > by the
> > > > Financial Conduct Authority.
> > > >
> > > >
> > >
> > >
> > > The information contained in this email is strictly confidential and
> for
> > > the use of the addressee only, unless otherwise indicated. If you are
> not
> > > the intended recipient, please do not read, copy, use or disclose to
> > others
> > > this message or any attachment. Please also notify the sender by
> replying
> > > to this email or by telephone (+44(020 7896 0011) and then delete the
> > email
> > > and any copies of it. Opinions, conclusion (etc) that do not relate to
> > the
> > > official business of this company shall be understood as neither given
> > nor
> > > endorsed by it. IG is a trading name of IG Markets Limited (a company
> > > registered in England and Wales, company number 04008957) and IG Index
> > > Limited (a company registered in England and Wales, company number
> > > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill,
> > > London EC4R 2YA. Both IG Markets Limited (register number 195355) and
> IG
> > > Index Limited (register number 114059) are authorised and regulated by
> > the
> > > Financial Conduct Authority.
> > >
> > The information contained in this email is strictly confidential and for
> > the use of the addressee only, unless otherwise indicated. If you are not
> > the intended recipient, please do not read, copy, use or disclose to
> others
> > this message or any attachment. Please also notify the sender by replying
> > to this email or by telephone (+44(020 7896 0011) and then delete the
> email
> > and any copies of it. Opinions, conclusion (etc) that do not relate to
> the
> > official business of this company shall be understood as neither given
> nor
> > endorsed by it. IG is a trading name of IG Markets Limited (a company
> > registered in England and Wales, company number 04008957) and IG Index
> > Limited (a company registered in England and Wales, company number
> > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill,
> > London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG
> > Index Limited (register number 114059) are authorised and regulated by
> the
> > Financial Conduct Authority.
> >
> The information contained in this email is strictly confidential and for
> the use of the addressee only, unless otherwise indicated. If you are not
> the intended recipient, please do not read, copy, use or disclose to others
> this message or any attachment. Please also notify the sender by replying
> to this email or by telephone (+44(020 7896 0011) and then delete the email
> and any copies of it. Opinions, conclusion (etc) that do not relate to the
> official business of this company shall be understood as neither given nor
> endorsed by it. IG is a trading name of IG Markets Limited (a company
> registered in England and Wales, company number 04008957) and IG Index
> Limited (a company registered in England and Wales, company number
> 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill,
> London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG
> Index Limited (register number 114059) are authorised and regulated by the
> Financial Conduct Authority.
>
The information contained in this email is strictly confidential and for the
use of the addressee only, unless otherwise indicated. If you are not the
intended recipient, please do not read, copy, use or disclose to others this
message or any attachment. Please also notify the sender by replying to this
email or by telephone (+44(020 7896 0011) and then delete the email and any
copies of it. Opinions, conclusion (etc) that do not relate to the official
business of this company shall be understood as neither given nor endorsed by
it. IG is a trading name of IG Markets Limited (a company registered in England
and Wales, company number 04008957) and IG Index Limited (a company registered
in England and Wales, company number 01190902). Registered address at Cannon
Bridge House, 25 Dowgate Hill, London EC4R 2YA. Both IG Markets Limited
(register number 195355) and IG Index Limited (register number 114059) are
authorised and regulated by the Financial Conduct Authority.
