Hi Dongjin,
One of the complications is that old versions of the API will not expect a
new error code. However, since we expect this to be a fatal error anyway
for old clients, it may still be more useful to return the correct error
code. For example, the Kafka clients use the following code to convert the
error code:
public static Errors forCode(short code) {
Errors error = codeToError.get(code);
if (error != null) {
return error;
} else {
log.warn("Unexpected error code: {}.", code);
return UNKNOWN_SERVER_ERROR;
}
}
If we return an unsupported error code, it will be converted to an UNKNOWN
error, but at least we will get the message in the log with the correct
code. That seems preferable to returning a misleading error code. So I
wonder if we can use the new UNSUPPORTED_COMPRESSION_TYPE error even for
older versions.
Also, one question just to check my understanding. I think we would only
use this error code when we /know/ that zstd was in use and the client
doesn't support it? This is true if either 1) the message needs
down-conversion and we encounter a zstd compressed message, or 2) if the
topic is explicitly configured to use zstd. However, if the compression
type is set to "producer," then the fetched data may or may not be
compressed with zstd. In this case, we return the data to the client and
expect it to fail parsing. Is that correct?
Thanks,
Jason
On Tue, Aug 21, 2018 at 9:08 AM, Dongjin Lee <dong...@apache.org> wrote:
> Ismael, Jason and all,
>
> I rewrote the backward compatibility strategy & its alternatives like
> following, based on Ismael & Jason's comments. Since it is not updated to
> the wiki yet, don't hesitate to give me a message if you have any opinion
> on it.
>
> ```
> *Backward Compatibility*
>
> We need to establish some backward-compatibility strategy for the case an
> old client subscribes a topic using ZStandard implicitly (i.e.,
> 'compression.type' configuration of given topic is 'producer' and the
> producer compressed the records with ZStandard). We have the following
> options for this situation:
>
> *A. Support ZStandard to the old clients which can understand v0, v1
> messages only.*
>
> This strategy necessarily requires the down-conversion of v2 message
> compressed with Zstandard into v0 or v1 messages, which means a
> considerable performance degradation. So we rejected this strategy.
>
> *B. Bump the API version and support only v2-available clients*
>
> With this approach, we can message the old clients that they are old and
> should be upgraded. However, there are still several options for the Error
> code.
>
> *B.1. INVALID_REQUEST (42)*
>
> This option gives the client so little information; the user can be
> confused about why the client worked correctly in the past suddenly
> encounters a problem. So we rejected this strategy.
>
> *B.2. CORRUPT_MESSAGE (2)*
>
> This option gives inaccurate information; the user can be surprised and
> misunderstand that the log files are broken in some way. So we rejected
> this strategy.
>
> *B.3 UNSUPPORTED_FOR_MESSAGE_FORMAT (43)*
>
> The advantage of this approach is we don't need to define a new error code;
> we can reuse it and that's all.
>
> The disadvantage of this approach is that it is also a little bit vague;
> This error code is defined as a work for KIP-98[^1] and now returned in the
> transaction error.
>
> *B.4. UNSUPPORTED_COMPRESSION_TYPE (new)*
>
> The advantage of this approach is that it is clear and provides an exact
> description. The disadvantage is we need to add a new error code.
> ```
>
> *It seems like what we need to choose is now so clear:
> UNSUPPORTED_FOR_MESSAGE_FORMAT (B.3) or UNSUPPORTED_COMPRESSION_TYPE
> (B.4).*
> The first one doesn't need a new error message but the latter is more
> explicit. Which one do you prefer? Since all of you have much more
> experience and knowledge than me, I will follow your decision. The wiki
> page will be updated following the decision also.
>
> Best,
> Dongjin
>
> [^1]: https://issues.apache.org/jira/browse/KAFKA-4990
>
> On Sun, Aug 19, 2018 at 4:58 AM Ismael Juma <isma...@gmail.com> wrote:
>
> > Sounds reasonable to me.
> >
> > Ismael
> >
> > On Sat, 18 Aug 2018, 12:20 Jason Gustafson, <ja...@confluent.io> wrote:
> >
> > > Hey Ismael,
> > >
> > > Your summary looks good to me. I think it might also be a good idea to
> > add
> > > a new UNSUPPORTED_COMPRESSION_TYPE error code to go along with the
> > version
> > > bumps. We won't be able to use it for old api versions since the
> clients
> > > will not understand it, but we can use it going forward so that we're
> not
> > > stuck in a similar situation with a new message format and a new codec
> to
> > > support. Another option is to use UNSUPPORTED_FOR_MESSAGE_FORMAT, but
> it
> > is
> > > not as explicit.
> > >
> > > -Jason
> > >
> > > On Fri, Aug 17, 2018 at 5:19 PM, Ismael Juma <ism...@juma.me.uk>
> wrote:
> > >
> > > > Hi Dongjin and Jason,
> > > >
> > > > I would agree. My summary:
> > > >
> > > > 1. Support zstd with message format 2 only.
> > > > 2. Bump produce and fetch request versions.
> > > > 3. Provide broker errors whenever possible based on the request
> version
> > > and
> > > > rely on clients for the cases where the broker can't validate
> > efficiently
> > > > (example message format 2 consumer that supports the latest fetch
> > version
> > > > but doesn't support zstd).
> > > >
> > > > If there's general agreement on this, I suggest we update the KIP to
> > > state
> > > > the proposal and to move the rejected options to its own section. And
> > > then
> > > > start a vote!
> > > >
> > > > Ismael
> > > >
> > > > On Fri, Aug 17, 2018 at 4:00 PM Jason Gustafson <ja...@confluent.io>
> > > > wrote:
> > > >
> > > > > Hi Dongjin,
> > > > >
> > > > > Yes, that's a good summary. For clients which support v2, the
> client
> > > can
> > > > > parse the message format and hopefully raise a useful error message
> > > > > indicating the unsupported compression type. For older clients, our
> > > > options
> > > > > are probably (1) to down-convert to the old format using no
> > compression
> > > > > type, or (2) to return an error code. I'm leaning toward the latter
> > as
> > > > the
> > > > > simpler solution, but the challenge is finding a good error code.
> Two
> > > > > possibilities might be INVALID_REQUEST or CORRUPT_MESSAGE. The
> > downside
> > > > is
> > > > > that old clients probably won't get a helpful message. However, at
> > > least
> > > > > the behavior will be consistent in the sense that all clients will
> > fail
> > > > if
> > > > > they do not support zstandard.
> > > > >
> > > > > What do you think?
> > > > >
> > > > > Thanks,
> > > > > Jason
> > > > >
> > > > > On Fri, Aug 17, 2018 at 8:08 AM, Dongjin Lee <dong...@apache.org>
> > > wrote:
> > > > >
> > > > > > Thanks Jason, I reviewed the down-converting logic following your
> > > > > > explanation.[^1] You mean the following routines, right?
> > > > > >
> > > > > > -
> > > > > > https://github.com/apache/kafka/blob/trunk/core/src/
> > > > > > main/scala/kafka/server/KafkaApis.scala#L534
> > > > > > -
> > > > > > https://github.com/apache/kafka/blob/trunk/clients/src/
> > > > > > main/java/org/apache/kafka/common/record/
> LazyDownConversionRecords.
> > > > > > java#L165
> > > > > > -
> > > > > > https://github.com/apache/kafka/blob/trunk/clients/src/
> > > > > > main/java/org/apache/kafka/common/record/RecordsUtil.java#L40
> > > > > >
> > > > > > It seems like your stance is like following:
> > > > > >
> > > > > > 1. In principle, Kafka does not change the compression codec when
> > > > > > down-converting, since it requires inspecting the fetched data,
> > which
> > > > is
> > > > > > expensive.
> > > > > > 2. However, there are some cases the fetched data is inspected
> > > anyway.
> > > > In
> > > > > > this case, we can provide compression conversion from Zstandard
> to
> > > > > > classical ones[^2].
> > > > > >
> > > > > > And from what I understand, the cases where the client without
> > > > ZStandard
> > > > > > support receives ZStandard compressed records can be organized
> into
> > > two
> > > > > > cases:
> > > > > >
> > > > > > a. The 'compression.type' configuration of given topic is
> > 'producer'
> > > > and
> > > > > > the producer compressed the records with ZStandard. (that is,
> using
> > > > > > ZStandard implicitly.)
> > > > > > b. The 'compression.type' configuration of given topic is
> 'zstd';
> > > that
> > > > > is,
> > > > > > using ZStandard explicitly.
> > > > > >
> > > > > > As you stated, we don't have to handle the case b specially. So,
> It
> > > > seems
> > > > > > like we can narrow the focus of the problem by joining case 1 and
> > > case
> > > > b
> > > > > > like the following:
> > > > > >
> > > > > > > Given the topic with 'producer' as its 'compression.type'
> > > > > configuration,
> > > > > > ZStandard compressed records and old client without ZStandard, is
> > > there
> > > > > any
> > > > > > case we need to inspect the records and can change the
> compression
> > > > type?
> > > > > If
> > > > > > so, can we provide compression type converting?
> > > > > >
> > > > > > Do I understand correctly?
> > > > > >
> > > > > > Best,
> > > > > > Dongjin
> > > > > >
> > > > > > [^1]: I'm sorry, I found that I was a little bit misunderstanding
> > how
> > > > API
> > > > > > version works, after reviewing the downconvert logic & the
> protocol
> > > > > > documentation <https://kafka.apache.org/protocol>.
> > > > > > [^2]: None, Gzip, Snappy, Lz4.
> > > > > >
> > > > > > On Tue, Aug 14, 2018 at 2:16 AM Jason Gustafson <
> > ja...@confluent.io>
> > > > > > wrote:
> > > > > >
> > > > > > > >
> > > > > > > > But in my opinion, since the client will fail with the API
> > > version,
> > > > > so
> > > > > > we
> > > > > > > > don't need to down-convert the messages anyway. Isn't it?
> So, I
> > > > think
> > > > > > we
> > > > > > > > don't care about this case. (I'm sorry, I am not familiar
> with
> > > > > > > down-convert
> > > > > > > > logic.)
> > > > > > >
> > > > > > >
> > > > > > > Currently the broker down-converts automatically when it
> receives
> > > an
> > > > > old
> > > > > > > version of the fetch request (a version which is known to
> predate
> > > the
> > > > > > > message format in use). Typically when down-converting the
> > message
> > > > > > format,
> > > > > > > we use the same compression type, but there is not much point
> in
> > > > doing
> > > > > so
> > > > > > > when we know the client doesn't support it. So if zstandard is
> in
> > > > use,
> > > > > > and
> > > > > > > we have to down-convert anyway, then we can choose to use a
> > > different
> > > > > > > compression type or no compression type.
> > > > > > >
> > > > > > > From my perspective, there is no significant downside to
> bumping
> > > the
> > > > > > > protocol version and it has several potential benefits. Version
> > > bumps
> > > > > are
> > > > > > > cheap. The main question mark in my mind is about
> > down-conversion.
> > > > > > Figuring
> > > > > > > out whether down-conversion is needed is hard generally without
> > > > > > inspecting
> > > > > > > the fetched data, which is expensive. I think we agree in
> > principle
> > > > > that
> > > > > > we
> > > > > > > do not want to have to pay this cost generally and prefer the
> > > clients
> > > > > to
> > > > > > > fail when they see an unhandled compression type. The point I
> was
> > > > > making
> > > > > > is
> > > > > > > that there are some cases where we are either inspecting the
> data
> > > > > anyway
> > > > > > > (because we have to down-convert the message format), or we
> have
> > an
> > > > > easy
> > > > > > > way to tell whether zstandard is in use (the topic has it
> > > configured
> > > > > > > explicitly). In the latter case, we don't have to handle it
> > > > specially.
> > > > > > But
> > > > > > > we do have to decide how we will handle down-conversion to
> older
> > > > > formats.
> > > > > > >
> > > > > > > -Jason
> > > > > > >
> > > > > > > On Sun, Aug 12, 2018 at 5:15 PM, Dongjin Lee <
> dong...@apache.org
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Colin and Jason,
> > > > > > > >
> > > > > > > > Thanks for your opinions. In summarizing, the Pros and Cons
> of
> > > > > bumping
> > > > > > > > fetch API version are:
> > > > > > > >
> > > > > > > > Cons:
> > > > > > > >
> > > > > > > > - The Broker can't know whether a given message batch is
> > > compressed
> > > > > > with
> > > > > > > > zstd or not.
> > > > > > > > - Need some additional logic for the topic explicitly
> > configured
> > > to
> > > > > use
> > > > > > > > zstd.
> > > > > > > >
> > > > > > > > Pros:
> > > > > > > >
> > > > > > > > - The broker doesn't need to conduct expensive
> down-conversion.
> > > > > > > > - Can message the users to update their client.
> > > > > > > >
> > > > > > > > So, opinions for the backward-compatibility policy by far:
> > > > > > > >
> > > > > > > > - A: bump the API version - +2 (Colin, Jason)
> > > > > > > > - B: leave unchanged - +1 (Viktor)
> > > > > > > >
> > > > > > > > Here are my additional comments:
> > > > > > > >
> > > > > > > > @Colin
> > > > > > > >
> > > > > > > > I greatly appreciate your response. In the case of the
> > dictionary
> > > > > > > support,
> > > > > > > > of course, this issue should be addressed later so we don't
> > need
> > > it
> > > > > in
> > > > > > > the
> > > > > > > > first version. You are right - it is not late to try it after
> > > some
> > > > > > > > benchmarks. What I mean is, we should keep in mind on that
> > > > potential
> > > > > > > > feature.
> > > > > > > >
> > > > > > > > @Jason
> > > > > > > >
> > > > > > > > You wrote,
> > > > > > > >
> > > > > > > > > Similarly, if we have to down-convert anyway because the
> > client
> > > > > does
> > > > > > > not
> > > > > > > > understand the message format, then we could also use a
> > different
> > > > > > > > compression type.
> > > > > > > >
> > > > > > > > But in my opinion, since the client will fail with the API
> > > version,
> > > > > so
> > > > > > we
> > > > > > > > don't need to down-convert the messages anyway. Isn't it?
> So, I
> > > > think
> > > > > > we
> > > > > > > > don't care about this case. (I'm sorry, I am not familiar
> with
> > > > > > > down-convert
> > > > > > > > logic.)
> > > > > > > >
> > > > > > > > Please give more opinions. Thanks!
> > > > > > > >
> > > > > > > > - Dongjin
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Aug 8, 2018 at 6:41 AM Jason Gustafson <
> > > ja...@confluent.io
> > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hey Colin,
> > > > > > > > >
> > > > > > > > > The problem for the fetch API is that the broker does not
> > > > generally
> > > > > > > know
> > > > > > > > if
> > > > > > > > > a batch was compressed with zstd unless it parses it. I
> think
> > > the
> > > > > > goal
> > > > > > > > here
> > > > > > > > > is to avoid the expensive down-conversion that is needed to
> > > > ensure
> > > > > > > > > compatibility because it is only necessary if zstd is
> > actually
> > > in
> > > > > > use.
> > > > > > > > But
> > > > > > > > > as long as old clients can parse the message format, they
> > > should
> > > > > get
> > > > > > a
> > > > > > > > > reasonable error if they see an unsupported compression
> type
> > in
> > > > the
> > > > > > > > > attributes. Basically the onus is on users to ensure that
> > their
> > > > > > > consumers
> > > > > > > > > have been updated prior to using zstd. It seems like a
> > > reasonable
> > > > > > > > tradeoff
> > > > > > > > > to me. There are a couple cases that might be worth
> thinking
> > > > > through:
> > > > > > > > >
> > > > > > > > > 1. If a topic is explicitly configured to use zstd, then we
> > > don't
> > > > > > need
> > > > > > > to
> > > > > > > > > check the fetched data for the compression type to know if
> we
> > > > need
> > > > > > > > > down-conversion. If we did bump the Fetch API version, then
> > we
> > > > > could
> > > > > > > > handle
> > > > > > > > > this case by either down-converting using a different
> > > compression
> > > > > > type
> > > > > > > or
> > > > > > > > > returning an error.
> > > > > > > > > 2. Similarly, if we have to down-convert anyway because the
> > > > client
> > > > > > does
> > > > > > > > not
> > > > > > > > > understand the message format, then we could also use a
> > > different
> > > > > > > > > compression type.
> > > > > > > > >
> > > > > > > > > For the produce API, I think it's reasonable to bump the
> api
> > > > > version.
> > > > > > > > This
> > > > > > > > > can be used by clients to check whether a broker supports
> > zstd.
> > > > For
> > > > > > > > > example, we might support a list of preferred compression
> > types
> > > > in
> > > > > > the
> > > > > > > > > producer and we could use the broker to detect which
> version
> > to
> > > > > use.
> > > > > > > > >
> > > > > > > > > -Jason
> > > > > > > > >
> > > > > > > > > On Tue, Aug 7, 2018 at 1:32 PM, Colin McCabe <
> > > cmcc...@apache.org
> > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Thanks for bumping this, Dongjin. ZStd is a good
> > compression
> > > > > codec
> > > > > > > > and I
> > > > > > > > > > hope we can get this support in soon!
> > > > > > > > > >
> > > > > > > > > > I would say we can just bump the API version to indicate
> > that
> > > > > ZStd
> > > > > > > > > support
> > > > > > > > > > is expected in new clients. We probably need some way of
> > > > > > indicating
> > > > > > > to
> > > > > > > > > the
> > > > > > > > > > older clients that they can't consume the partitions, as
> > > well.
> > > > > > > Perhaps
> > > > > > > > > we
> > > > > > > > > > can use the UNSUPPORTED_FOR_MESSAGE_FORMAT error?
> > > > > > > > > >
> > > > > > > > > > The license thing seems straightforward -- it's just a
> > matter
> > > > of
> > > > > > > adding
> > > > > > > > > > the text to the right files as per ASF guidelines.
> > > > > > > > > >
> > > > > > > > > > With regard to the dictionary support, do we really need
> > that
> > > > in
> > > > > > the
> > > > > > > > > first
> > > > > > > > > > version? Hopefully message batches are big enough that
> > this
> > > > > isn't
> > > > > > > > > needed.
> > > > > > > > > > Some benchmarks might help here.
> > > > > > > > > >
> > > > > > > > > > best,
> > > > > > > > > > Colin
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Tue, Aug 7, 2018, at 08:02, Dongjin Lee wrote:
> > > > > > > > > > > As Kafka 2.0.0 was released, let's reboot this issue,
> > > KIP-110
> > > > > > > > > > > <https://cwiki.apache.org/
> confluence/display/KAFKA/KIP-
> > > > > > > > > > 110%3A+Add+Codec+for+ZStandard+Compression>
> > > > > > > > > > > .
> > > > > > > > > > >
> > > > > > > > > > > For newcomers, Here is some summary of the history:
> > KIP-110
> > > > was
> > > > > > > > > > originally
> > > > > > > > > > > worked for the issue KAFKA-4514 but, it lacked
> benchmark
> > > > > results
> > > > > > to
> > > > > > > > get
> > > > > > > > > > the
> > > > > > > > > > > agreement of the community. Later, Ivan Babrou and some
> > > other
> > > > > > users
> > > > > > > > who
> > > > > > > > > > > adopted the patch provided their excellent performance
> > > report
> > > > > > which
> > > > > > > > is
> > > > > > > > > > now
> > > > > > > > > > > included in the KIP, but it postponed again because of
> > the
> > > > > > > community
> > > > > > > > > was
> > > > > > > > > > > busy for 2.0.0 release. It is why I now reboot this
> > issue.
> > > > > > > > > > >
> > > > > > > > > > > The following is the current status of the feature: You
> > can
> > > > > check
> > > > > > > the
> > > > > > > > > > > current draft implementation here
> > > > > > > > > > > <https://github.com/apache/kafka/pull/2267>. It is
> based
> > > on
> > > > > zstd
> > > > > > > > 1.3.5
> > > > > > > > > > and
> > > > > > > > > > > periodically rebased onto the latest trunk[^1].
> > > > > > > > > > >
> > > > > > > > > > > The issues that should be addressed is like following:
> > > > > > > > > > >
> > > > > > > > > > > *1. Backward Compatibility*
> > > > > > > > > > >
> > > > > > > > > > > To support old consumers, we need to take a strategy to
> > > > handle
> > > > > > the
> > > > > > > > old
> > > > > > > > > > > consumers. Current candidates are:
> > > > > > > > > > >
> > > > > > > > > > > - Bump API version
> > > > > > > > > > > - Leave unchanged: let the old clients fail.
> > > > > > > > > > > - Improve the error messages:
> > > > > > > > > > >
> > > > > > > > > > > *2. Dictionary Support*
> > > > > > > > > > >
> > > > > > > > > > > To support zstd's dictionary feature in the future (if
> > > > needed),
> > > > > > we
> > > > > > > > need
> > > > > > > > > > to
> > > > > > > > > > > sketch how it should be and leave some room for it. As
> of
> > > > now,
> > > > > > > there
> > > > > > > > > has
> > > > > > > > > > > been no discussion on this topic yet.
> > > > > > > > > > >
> > > > > > > > > > > *3. License*
> > > > > > > > > > >
> > > > > > > > > > > To use this feature, we need to add license of zstd and
> > > > > zstd-jni
> > > > > > to
> > > > > > > > the
> > > > > > > > > > > project. (Thanks to Viktor Somogyi for raising this
> > issue!)
> > > > It
> > > > > > > seems
> > > > > > > > > like
> > > > > > > > > > > what Apache Spark did would be a good example but there
> > has
> > > > > been
> > > > > > no
> > > > > > > > > > > discussion yet.
> > > > > > > > > > >
> > > > > > > > > > > You can find the details of the above issues in the KIP
> > > > > document.
> > > > > > > > > Please
> > > > > > > > > > > have a look when you are free, and give me feedback.
> All
> > > > kinds
> > > > > of
> > > > > > > > > > > participating are welcome.
> > > > > > > > > > >
> > > > > > > > > > > Best,
> > > > > > > > > > > Dongjin
> > > > > > > > > > >
> > > > > > > > > > > [^1]: At the time of writing, commit 6b4fb8152.
> > > > > > > > > > >
> > > > > > > > > > > On Sat, Jul 14, 2018 at 10:45 PM Dongjin Lee <
> > > > > dong...@apache.org
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Sorry for the late reply.
> > > > > > > > > > > >
> > > > > > > > > > > > In short, I could not submit the updated KIP by the
> > > feature
> > > > > > > freeze
> > > > > > > > > > > > deadline of 2.0.0. For this reason, it will not be
> > > included
> > > > > in
> > > > > > > the
> > > > > > > > > > 2.0.0
> > > > > > > > > > > > release and all discussion for this issue were
> > postponed
> > > > > after
> > > > > > > the
> > > > > > > > > > release
> > > > > > > > > > > > of 2.0.0.
> > > > > > > > > > > >
> > > > > > > > > > > > I have been updating the PR following recent updates.
> > > Just
> > > > > > now, I
> > > > > > > > > > rebased
> > > > > > > > > > > > it against the latest trunk and updated the zstd
> > version
> > > > into
> > > > > > > > 1.3.5.
> > > > > > > > > > If you
> > > > > > > > > > > > need some request, don't hesitate to notify me. (But
> > not
> > > > this
> > > > > > > > thread
> > > > > > > > > -
> > > > > > > > > > just
> > > > > > > > > > > > send me the message directly.)
> > > > > > > > > > > >
> > > > > > > > > > > > Best,
> > > > > > > > > > > > Dongjin
> > > > > > > > > > > >
> > > > > > > > > > > > On Tue, Jul 10, 2018 at 11:57 PM Bobby Evans <
> > > > > bo...@apache.org
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > >> I there any update on this. The performance
> > > improvements
> > > > > are
> > > > > > > > quite
> > > > > > > > > > > >> impressive and I really would like to stop forking
> > kafka
> > > > > just
> > > > > > to
> > > > > > > > get
> > > > > > > > > > this
> > > > > > > > > > > >> in.
> > > > > > > > > > > >>
> > > > > > > > > > > >> Thanks,
> > > > > > > > > > > >>
> > > > > > > > > > > >> Bobby
> > > > > > > > > > > >>
> > > > > > > > > > > >> On Wed, Jun 13, 2018 at 8:56 PM Dongjin Lee <
> > > > > > dong...@apache.org
> > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > > >>
> > > > > > > > > > > >> > Ismael,
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > Oh, I forgot all of you are on working frenzy for
> > 2.0!
> > > > No
> > > > > > > > problem,
> > > > > > > > > > take
> > > > > > > > > > > >> > your time. I am also working at another issue now.
> > > Thank
> > > > > you
> > > > > > > for
> > > > > > > > > > > >> letting me
> > > > > > > > > > > >> > know.
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > Best,
> > > > > > > > > > > >> > Dongjin
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > On Wed, Jun 13, 2018, 11:44 PM Ismael Juma <
> > > > > > isma...@gmail.com
> > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > > Sorry for the delay Dongjin. Everyone is busy
> > > > finalising
> > > > > > > > 2.0.0.
> > > > > > > > > > This
> > > > > > > > > > > >> KIP
> > > > > > > > > > > >> > > seems like a great candidate for 2.1.0 and
> > hopefully
> > > > > there
> > > > > > > > will
> > > > > > > > > be
> > > > > > > > > > > >> more
> > > > > > > > > > > >> > of
> > > > > > > > > > > >> > > a discussion next week. :)
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > Ismael
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > On Wed, 13 Jun 2018, 05:17 Dongjin Lee, <
> > > > > > dong...@apache.org
> > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > > Hello. I just updated my draft implementation:
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > > 1. Rebased to latest trunk (commit 5145d6b)
> > > > > > > > > > > >> > > > 2. Apply ZStd 1.3.4
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > > You can check out the implementation from here
> > > > > > > > > > > >> > > > <https://github.com/apache/kafka/pull/2267>.
> If
> > > you
> > > > > > > > > experience
> > > > > > > > > > any
> > > > > > > > > > > >> > > problem
> > > > > > > > > > > >> > > > running it, don't hesitate to give me a
> mention.
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > > Best,
> > > > > > > > > > > >> > > > Dongjin
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > > On Tue, Jun 12, 2018 at 6:50 PM Dongjin Lee <
> > > > > > > > > dong...@apache.org
> > > > > > > > > > >
> > > > > > > > > > > >> > wrote:
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > > > Here is the short conclusion about the
> license
> > > > > > problem:
> > > > > > > > *We
> > > > > > > > > > can
> > > > > > > > > > > >> use
> > > > > > > > > > > >> > > zstd
> > > > > > > > > > > >> > > > > and zstd-jni without any problem, but we
> need
> > to
> > > > > > include
> > > > > > > > > their
> > > > > > > > > > > >> > license,
> > > > > > > > > > > >> > > > > e.g., BSD license.*
> > > > > > > > > > > >> > > > >
> > > > > > > > > > > >> > > > > Both of BSD 2 Clause License & 3 Clause
> > License
> > > > > > requires
> > > > > > > > to
> > > > > > > > > > > >> include
> > > > > > > > > > > >> > the
> > > > > > > > > > > >> > > > > license used, and BSD 3 Clause License
> > requires
> > > > that
> > > > > > the
> > > > > > > > > name
> > > > > > > > > > of
> > > > > > > > > > > >> the
> > > > > > > > > > > >> > > > > contributor can't be used to endorse or
> > promote
> > > > the
> > > > > > > > product.
> > > > > > > > > > > >> That's
> > > > > > > > > > > >> > it
> > > > > > > > > > > >> > > > > <
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> >
> > > > > > > > > > > >> http://www.mikestratton.net/2011/12/is-bsd-license-
> > > > > > > > > > compatible-with-apache-2-0-license/
> > > > > > > > > > > >> > > > >
> > > > > > > > > > > >> > > > > - They are not listed in the list of
> > prohibited
> > > > > > licenses
> > > > > > > > > > > >> > > > > <https://www.apache.org/legal/
> > > > > > resolved.html#category-x>
> > > > > > > > > also.
> > > > > > > > > > > >> > > > >
> > > > > > > > > > > >> > > > > Here is how Spark did for it
> > > > > > > > > > > >> > > > > <https://issues.apache.org/
> > > > jira/browse/SPARK-19112
> > > > > >:
> > > > > > > > > > > >> > > > >
> > > > > > > > > > > >> > > > > - They made a directory dedicated to the
> > > > dependency
> > > > > > > > license
> > > > > > > > > > files
> > > > > > > > > > > >> > > > > <
> > > > > https://github.com/apache/spark/tree/master/licenses
> > > > > > >
> > > > > > > > and
> > > > > > > > > > added
> > > > > > > > > > > >> > > > licenses
> > > > > > > > > > > >> > > > > for Zstd
> > > > > > > > > > > >> > > > > <
> > > > > > > > > > > >> >
> > https://github.com/apache/spark/blob/master/licenses/
> > > > > > > > > > LICENSE-zstd.txt
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > > &
> > > > > > > > > > > >> > > > > Zstd-jni
> > > > > > > > > > > >> > > > > <
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> >
> > > > > > > > > > > >> https://github.com/apache/
> spark/blob/master/licenses/
> > > > > > > > > > LICENSE-zstd-jni.txt
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > > > > .
> > > > > > > > > > > >> > > > > - Added a link to the original license files
> > in
> > > > > > LICENSE.
> > > > > > > > > > > >> > > > > <
> > > https://github.com/apache/spark/pull/18805/files
> > > > >
> > > > > > > > > > > >> > > > >
> > > > > > > > > > > >> > > > > If needed, I can make a similar update.
> > > > > > > > > > > >> > > > >
> > > > > > > > > > > >> > > > > Thanks for pointing out this problem,
> Viktor!
> > > Nice
> > > > > > > catch!
> > > > > > > > > > > >> > > > >
> > > > > > > > > > > >> > > > > Best,
> > > > > > > > > > > >> > > > > Dongjin
> > > > > > > > > > > >> > > > >
> > > > > > > > > > > >> > > > >
> > > > > > > > > > > >> > > > >
> > > > > > > > > > > >> > > > > On Mon, Jun 11, 2018 at 11:50 PM Dongjin
> Lee <
> > > > > > > > > > dong...@apache.org>
> > > > > > > > > > > >> > > wrote:
> > > > > > > > > > > >> > > > >
> > > > > > > > > > > >> > > > >> I greatly appreciate your comprehensive
> > > > reasoning.
> > > > > > so:
> > > > > > > +1
> > > > > > > > > > for b
> > > > > > > > > > > >> > until
> > > > > > > > > > > >> > > > now.
> > > > > > > > > > > >> > > > >>
> > > > > > > > > > > >> > > > >> For the license issues, I will have a check
> > on
> > > > how
> > > > > > the
> > > > > > > > over
> > > > > > > > > > > >> projects
> > > > > > > > > > > >> > > are
> > > > > > > > > > > >> > > > >> doing and share the results.
> > > > > > > > > > > >> > > > >>
> > > > > > > > > > > >> > > > >> Best,
> > > > > > > > > > > >> > > > >> Dongjin
> > > > > > > > > > > >> > > > >>
> > > > > > > > > > > >> > > > >> On Mon, Jun 11, 2018 at 10:08 PM Viktor
> > > Somogyi <
> > > > > > > > > > > >> > > > viktorsomo...@gmail.com>
> > > > > > > > > > > >> > > > >> wrote:
> > > > > > > > > > > >> > > > >>
> > > > > > > > > > > >> > > > >>> Hi Dongjin,
> > > > > > > > > > > >> > > > >>>
> > > > > > > > > > > >> > > > >>> A couple of comments:
> > > > > > > > > > > >> > > > >>> I would vote for option b. in the
> "backward
> > > > > > > > compatibility"
> > > > > > > > > > > >> section.
> > > > > > > > > > > >> > > My
> > > > > > > > > > > >> > > > >>> reasoning for this is that users upgrading
> > to
> > > a
> > > > > zstd
> > > > > > > > > > compatible
> > > > > > > > > > > >> > > version
> > > > > > > > > > > >> > > > >>> won't start to use it automatically, so
> > manual
> > > > > > > > > > reconfiguration
> > > > > > > > > > > >> is
> > > > > > > > > > > >> > > > >>> required.
> > > > > > > > > > > >> > > > >>> Therefore an upgrade won't mess up the
> > > cluster.
> > > > If
> > > > > > not
> > > > > > > > all
> > > > > > > > > > the
> > > > > > > > > > > >> > > clients
> > > > > > > > > > > >> > > > >>> are
> > > > > > > > > > > >> > > > >>> upgraded but just some of them and they'd
> > > start
> > > > to
> > > > > > use
> > > > > > > > > zstd
> > > > > > > > > > > >> then it
> > > > > > > > > > > >> > > > would
> > > > > > > > > > > >> > > > >>> cause errors in the cluster. I'd like to
> > > presume
> > > > > > > though
> > > > > > > > > that
> > > > > > > > > > > >> this
> > > > > > > > > > > >> > is
> > > > > > > > > > > >> > > a
> > > > > > > > > > > >> > > > >>> very
> > > > > > > > > > > >> > > > >>> obvious failure case and nobody should be
> > > > > surprised
> > > > > > if
> > > > > > > > it
> > > > > > > > > > didn't
> > > > > > > > > > > >> > > work.
> > > > > > > > > > > >> > > > >>> I wouldn't choose a. as I think we should
> > bump
> > > > the
> > > > > > > fetch
> > > > > > > > > and
> > > > > > > > > > > >> > produce
> > > > > > > > > > > >> > > > >>> requests if it's a change in the message
> > > format.
> > > > > > > > Moreover
> > > > > > > > > if
> > > > > > > > > > > >> some
> > > > > > > > > > > >> > of
> > > > > > > > > > > >> > > > the
> > > > > > > > > > > >> > > > >>> producers and the brokers are upgraded but
> > > some
> > > > of
> > > > > > the
> > > > > > > > > > consumers
> > > > > > > > > > > >> > are
> > > > > > > > > > > >> > > > not,
> > > > > > > > > > > >> > > > >>> then we wouldn't prevent the error when
> the
> > > old
> > > > > > > consumer
> > > > > > > > > > tries
> > > > > > > > > > > >> to
> > > > > > > > > > > >> > > > consume
> > > > > > > > > > > >> > > > >>> the zstd compressed messages.
> > > > > > > > > > > >> > > > >>> I wouldn't choose c. either as I think
> > binding
> > > > the
> > > > > > > > > > compression
> > > > > > > > > > > >> type
> > > > > > > > > > > >> > > to
> > > > > > > > > > > >> > > > an
> > > > > > > > > > > >> > > > >>> API is not so obvious from the developer's
> > > > > > > perspective.
> > > > > > > > > > > >> > > > >>>
> > > > > > > > > > > >> > > > >>> I would also prefer to use the existing
> > > binding,
> > > > > > > however
> > > > > > > > > we
> > > > > > > > > > must
> > > > > > > > > > > >> > > > respect
> > > > > > > > > > > >> > > > >>> the licenses:
> > > > > > > > > > > >> > > > >>> "The code for these JNI bindings is
> licenced
> > > > under
> > > > > > > > > 2-clause
> > > > > > > > > > BSD
> > > > > > > > > > > >> > > > license.
> > > > > > > > > > > >> > > > >>> The native Zstd library is licensed under
> > > > 3-clause
> > > > > > BSD
> > > > > > > > > > license
> > > > > > > > > > > >> and
> > > > > > > > > > > >> > > > GPL2"
> > > > > > > > > > > >> > > > >>> Based on the FAQ page
> > > > > > > > > > > >> > > > >>> https://www.apache.org/legal/
> > > > > > resolved.html#category-a
> > > > > > > > > > > >> > > > >>> we may use 2- and 3-clause BSD licenses
> but
> > > the
> > > > > > Apache
> > > > > > > > > > license
> > > > > > > > > > > >> is
> > > > > > > > > > > >> > not
> > > > > > > > > > > >> > > > >>> compatible with GPL2. I'm hoping that the
> > > > > "3-clause
> > > > > > > BSD
> > > > > > > > > > license
> > > > > > > > > > > >> and
> > > > > > > > > > > >> > > > GPL2"
> > > > > > > > > > > >> > > > >>> is really not an AND but an OR in this
> case,
> > > but
> > > > > I'm
> > > > > > > no
> > > > > > > > > > lawyer,
> > > > > > > > > > > >> > just
> > > > > > > > > > > >> > > > >>> wanted
> > > > > > > > > > > >> > > > >>> to make the point that we should watch out
> > for
> > > > > > > licenses.
> > > > > > > > > :)
> > > > > > > > > > > >> > > > >>>
> > > > > > > > > > > >> > > > >>> Regards,
> > > > > > > > > > > >> > > > >>> Viktor
> > > > > > > > > > > >> > > > >>>
> > > > > > > > > > > >> > > > >>>
> > > > > > > > > > > >> > > > >>> On Sun, Jun 10, 2018 at 3:02 AM Ivan
> Babrou
> > <
> > > > > > > > > > ibob...@gmail.com>
> > > > > > > > > > > >> > > wrote:
> > > > > > > > > > > >> > > > >>>
> > > > > > > > > > > >> > > > >>> > Hello,
> > > > > > > > > > > >> > > > >>> >
> > > > > > > > > > > >> > > > >>> > This is Ivan and I still very much
> support
> > > the
> > > > > > fact
> > > > > > > > that
> > > > > > > > > > zstd
> > > > > > > > > > > >> > > > >>> compression
> > > > > > > > > > > >> > > > >>> > should be included out of the box.
> > > > > > > > > > > >> > > > >>> >
> > > > > > > > > > > >> > > > >>> > Please think about the environment, you
> > can
> > > > save
> > > > > > > > quite a
> > > > > > > > > > lot
> > > > > > > > > > > >> of
> > > > > > > > > > > >> > > > >>> hardware
> > > > > > > > > > > >> > > > >>> > with it.
> > > > > > > > > > > >> > > > >>> >
> > > > > > > > > > > >> > > > >>> > Thank you.
> > > > > > > > > > > >> > > > >>> >
> > > > > > > > > > > >> > > > >>> > On Sat, Jun 9, 2018 at 14:14 Dongjin
> Lee <
> > > > > > > > > > dong...@apache.org>
> > > > > > > > > > > >> > > wrote:
> > > > > > > > > > > >> > > > >>> >
> > > > > > > > > > > >> > > > >>> > > Since there are no responses for a
> > week, I
> > > > > > decided
> > > > > > > > to
> > > > > > > > > > > >> > reinitiate
> > > > > > > > > > > >> > > > the
> > > > > > > > > > > >> > > > >>> > > discussion thread.
> > > > > > > > > > > >> > > > >>> > >
> > > > > > > > > > > >> > > > >>> > >
> > > > > > > > > > > >> > > > >>> > >
> > > > > > > > > > > >> > > > >>> >
> > > > > > > > > > > >> > > > >>>
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> >
> > > > > > > > > > > >>
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > > > > > > 110%3A+Add+Codec+for+ZStandard+Compression
> > > > > > > > > > > >> > > > >>> > >
> > > > > > > > > > > >> > > > >>> > > This KIP is about to introduce
> ZStandard
> > > > > > > Compression
> > > > > > > > > > into
> > > > > > > > > > > >> > Apache
> > > > > > > > > > > >> > > > >>> Kafka.
> > > > > > > > > > > >> > > > >>> > > The reason why it is posted again has
> a
> > > > story:
> > > > > > It
> > > > > > > > was
> > > > > > > > > > > >> > originally
> > > > > > > > > > > >> > > > >>> posted
> > > > > > > > > > > >> > > > >>> > to
> > > > > > > > > > > >> > > > >>> > > the dev mailing list more than one
> year
> > > ago
> > > > > but
> > > > > > > > since
> > > > > > > > > it
> > > > > > > > > > > >> has no
> > > > > > > > > > > >> > > > >>> > performance
> > > > > > > > > > > >> > > > >>> > > report included, it was postponed
> later.
> > > But
> > > > > > Some
> > > > > > > > > people
> > > > > > > > > > > >> > > (including
> > > > > > > > > > > >> > > > >>> Ivan)
> > > > > > > > > > > >> > > > >>> > > reported excellent performance report
> > with
> > > > the
> > > > > > > draft
> > > > > > > > > PR,
> > > > > > > > > > > >> this
> > > > > > > > > > > >> > > work
> > > > > > > > > > > >> > > > >>> is now
> > > > > > > > > > > >> > > > >>> > > reactivated.
> > > > > > > > > > > >> > > > >>> > >
> > > > > > > > > > > >> > > > >>> > > The updated KIP document includes some
> > > > > expected
> > > > > > > > > > problems and
> > > > > > > > > > > >> > > their
> > > > > > > > > > > >> > > > >>> > > candidate alternatives. Please have a
> > look
> > > > > when
> > > > > > > you
> > > > > > > > > are
> > > > > > > > > > > >> free,
> > > > > > > > > > > >> > and
> > > > > > > > > > > >> > > > >>> give
> > > > > > > > > > > >> > > > >>> > me a
> > > > > > > > > > > >> > > > >>> > > feedback. All kinds of participating
> are
> > > > > > welcome.
> > > > > > > > > > > >> > > > >>> > >
> > > > > > > > > > > >> > > > >>> > > Best,
> > > > > > > > > > > >> > > > >>> > > Dongjin
> > > > > > > > > > > >> > > > >>> > >
> > > > > > > > > > > >> > > > >>> > > --
> > > > > > > > > > > >> > > > >>> > > *Dongjin Lee*
> > > > > > > > > > > >> > > > >>> > >
> > > > > > > > > > > >> > > > >>> > > *A hitchhiker in the mathematical
> > world.*
> > > > > > > > > > > >> > > > >>> > >
> > > > > > > > > > > >> > > > >>> > > *github: <http://goog_969573159/
> > >github
> > > > > > > > > > .com/dongjinleekr
> > > > > > > > > > > >> > > > >>> > > <http://github.com/dongjinleekr
> > >linkedin:
> > > > > > > > > > > >> > > > >>> > kr.linkedin.com/in/dongjinleekr
> > > > > > > > > > > >> > > > >>> > > <http://kr.linkedin.com/in/
> dongjinleekr
> > > > > > > >slideshare:
> > > > > > > > > > > >> > > > >>> > www.slideshare.net/dongjinleekr
> > > > > > > > > > > >> > > > >>> > > <http://www.slideshare.net/
> dongjinleekr
> > >*
> > > > > > > > > > > >> > > > >>> > >
> > > > > > > > > > > >> > > > >>> >
> > > > > > > > > > > >> > > > >>>
> > > > > > > > > > > >> > > > >> --
> > > > > > > > > > > >> > > > >> *Dongjin Lee*
> > > > > > > > > > > >> > > > >>
> > > > > > > > > > > >> > > > >> *A hitchhiker in the mathematical world.*
> > > > > > > > > > > >> > > > >>
> > > > > > > > > > > >> > > > >> *github: <http://goog_969573159/>github
> > > > > > > > .com/dongjinleekr
> > > > > > > > > > > >> > > > >> <http://github.com/dongjinleekr>linkedin:
> > > > > > > > > > > >> > > > kr.linkedin.com/in/dongjinleekr
> > > > > > > > > > > >> > > > >> <http://kr.linkedin.com/in/dongjinleekr
> > > > > >slideshare:
> > > > > > > > > > > >> > > > www.slideshare.net/dongjinleekr
> > > > > > > > > > > >> > > > >> <http://www.slideshare.net/dongjinleekr>*
> > > > > > > > > > > >> > > > >>
> > > > > > > > > > > >> > > > >
> > > > > > > > > > > >> > > > >
> > > > > > > > > > > >> > > > > --
> > > > > > > > > > > >> > > > > *Dongjin Lee*
> > > > > > > > > > > >> > > > >
> > > > > > > > > > > >> > > > > *A hitchhiker in the mathematical world.*
> > > > > > > > > > > >> > > > >
> > > > > > > > > > > >> > > > > *github: <http://goog_969573159/>
> > > > > > > github.com/dongjinleekr
> > > > > > > > > > > >> > > > > <http://github.com/dongjinleekr>linkedin:
> > > > > > > > > > > >> > > > kr.linkedin.com/in/dongjinleekr
> > > > > > > > > > > >> > > > > <http://kr.linkedin.com/in/dongjinleekr
> > > > >slideshare:
> > > > > > > > > > > >> > > > www.slideshare.net/dongjinleekr
> > > > > > > > > > > >> > > > > <http://www.slideshare.net/dongjinleekr>*
> > > > > > > > > > > >> > > > >
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > > --
> > > > > > > > > > > >> > > > *Dongjin Lee*
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > > *A hitchhiker in the mathematical world.*
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > > *github: <http://goog_969573159/>github
> > > > > > .com/dongjinleekr
> > > > > > > > > > > >> > > > <http://github.com/dongjinleekr>linkedin:
> > > > > > > > > > > >> > > kr.linkedin.com/in/dongjinleekr
> > > > > > > > > > > >> > > > <http://kr.linkedin.com/in/dongjinleekr
> > > >slideshare:
> > > > > > > > > > > >> > > > www.slideshare.net/dongjinleekr
> > > > > > > > > > > >> > > > <http://www.slideshare.net/dongjinleekr>*
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> >
> > > > > > > > > > > >>
> > > > > > > > > > > >> --
> > > > > > > > > > > >> *Dongjin Lee*
> > > > > > > > > > > >>
> > > > > > > > > > > >> *A hitchhiker in the mathematical world.*
> > > > > > > > > > > >>
> > > > > > > > > > > >> *github: <http://goog_969573159/>
> > > github.com/dongjinleekr
> > > > > > > > > > > >> <http://github.com/dongjinleekr>linkedin:
> > > > > kr.linkedin.com/in/
> > > > > > > > > > dongjinleekr
> > > > > > > > > > > >> <http://kr.linkedin.com/in/dongjinleekr>slideshare:
> > > > > > > > > > www.slideshare.net/dongjinleekr
> > > > > > > > > > > >> <http://www.slideshare.net/dongjinleekr>*
> > > > > > > > > > > >>
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > *Dongjin Lee*
> > > > > > > > > > >
> > > > > > > > > > > *A hitchhiker in the mathematical world.*
> > > > > > > > > > >
> > > > > > > > > > > *github: <http://goog_969573159/>
> > github.com/dongjinleekr
> > > > > > > > > > > <http://github.com/dongjinleekr>linkedin:
> > > > kr.linkedin.com/in/
> > > > > > > > > > dongjinleekr
> > > > > > > > > > > <http://kr.linkedin.com/in/dongjinleekr>slideshare:
> > > > > > > > > > > www.slideshare.net/dongjinleekr
> > > > > > > > > > > <http://www.slideshare.net/dongjinleekr>*
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > *Dongjin Lee*
> > > > > > > > >
> > > > > > > > > *A hitchhiker in the mathematical world.*
> > > > > > > > >
> > > > > > > > > *github: <http://goog_969573159/>github.com/dongjinleekr
> > > > > > > > > <http://github.com/dongjinleekr>linkedin:
> > kr.linkedin.com/in/
> > > > > > > > dongjinleekr
> > > > > > > > > <http://kr.linkedin.com/in/dongjinleekr>slideshare:
> > > > > > > www.slideshare.net/
> > > > > > > > dongjinleekr
> > > > > > > > > <http://www.slideshare.net/dongjinleekr>*
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > *Dongjin Lee*
> > > > > > >
> > > > > > > *A hitchhiker in the mathematical world.*
> > > > > > >
> > > > > > > *github: <http://goog_969573159/>github.com/dongjinleekr
> > > > > > > <http://github.com/dongjinleekr>linkedin: kr.linkedin.com/in/
> > > > > > dongjinleekr
> > > > > > > <http://kr.linkedin.com/in/dongjinleekr>slideshare:
> > > > > www.slideshare.net/
> > > > > > dongjinleekr
> > > > > > > <http://www.slideshare.net/dongjinleekr>*
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> > --
> > *Dongjin Lee*
> >
> > *A hitchhiker in the mathematical world.*
> >
> > *github: <http://goog_969573159/>github.com/dongjinleekr
> > <http://github.com/dongjinleekr>linkedin: kr.linkedin.com/in/
> dongjinleekr
> > <http://kr.linkedin.com/in/dongjinleekr>slideshare: www.slideshare.net/
> dongjinleekr
> > <http://www.slideshare.net/dongjinleekr>*
> >
>
