[jira] [Assigned] (KAFKA-8958) Fix Kafka Streams JavaDocs with regard to used Serdes
[
https://issues.apache.org/jira/browse/KAFKA-8958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
bibin sebastian reassigned KAFKA-8958:
--
Assignee: (was: bibin sebastian)
> Fix Kafka Streams JavaDocs with regard to used Serdes
> -
>
> Key: KAFKA-8958
> URL: https://issues.apache.org/jira/browse/KAFKA-8958
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Reporter: Matthias J. Sax
>Priority: Minor
> Labels: beginner, newbie
>
> In older released, Kafka Streams applied operator specific overwrites of
> Serdes as in-place overwrites. In newer releases, Kafka Streams tries to
> re-use Serdes more "aggressively" by pushing serde information downstream if
> the key and/or value did not change.
> However, we never updated the JavaDocs accordingly. For example
> `KStream#through(String topic)` JavaDocs say:
> {code:java}
> Materialize this stream to a topic and creates a new {@code KStream} from the
> topic using default serializers, deserializers, and producer's {@link
> DefaultPartitioner}.
> {code}
> The JavaDocs don't put into account that Serdes might have been set further
> upstream, and the defaults from the config would not be used.
> `KStream#through()` is just one example. We should address this through all
> JavaDocs over all operators (ie, KStream, KGroupedStream,
> TimeWindowedKStream, SessionWindowedKStream, KTable, and KGroupedTable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-8977) Remove MockStreamsMetrics Since it is not a Mock
[
https://issues.apache.org/jira/browse/KAFKA-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16973497#comment-16973497
]
bibin sebastian commented on KAFKA-8977:
[~cadonna] not sure if you got a chance to look at my PR.
> Remove MockStreamsMetrics Since it is not a Mock
>
>
> Key: KAFKA-8977
> URL: https://issues.apache.org/jira/browse/KAFKA-8977
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: bibin sebastian
>Priority: Minor
> Labels: newbie
>
> The class {{MockStreamsMetrics}} is used throughout unit tests as a mock but
> it is not really a mock since it only hides two parameters of the
> {{StreamsMetricsImpl}} constructor. Either a real mock or the real
> {{StreamsMetricsImpl}} should be used in the tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-8977) Remove MockStreamsMetrics Since it is not a Mock
[
https://issues.apache.org/jira/browse/KAFKA-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16970811#comment-16970811
]
bibin sebastian commented on KAFKA-8977:
[~cadonna] ^^
> Remove MockStreamsMetrics Since it is not a Mock
>
>
> Key: KAFKA-8977
> URL: https://issues.apache.org/jira/browse/KAFKA-8977
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: bibin sebastian
>Priority: Minor
> Labels: newbie
>
> The class {{MockStreamsMetrics}} is used throughout unit tests as a mock but
> it is not really a mock since it only hides two parameters of the
> {{StreamsMetricsImpl}} constructor. Either a real mock or the real
> {{StreamsMetricsImpl}} should be used in the tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-8977) Remove MockStreamsMetrics Since it is not a Mock
[
https://issues.apache.org/jira/browse/KAFKA-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967683#comment-16967683
]
bibin sebastian commented on KAFKA-8977:
[~cadonna] do you think that my PR above is up for review?
> Remove MockStreamsMetrics Since it is not a Mock
>
>
> Key: KAFKA-8977
> URL: https://issues.apache.org/jira/browse/KAFKA-8977
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: bibin sebastian
>Priority: Minor
> Labels: newbie
>
> The class {{MockStreamsMetrics}} is used throughout unit tests as a mock but
> it is not really a mock since it only hides two parameters of the
> {{StreamsMetricsImpl}} constructor. Either a real mock or the real
> {{StreamsMetricsImpl}} should be used in the tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-8977) Remove MockStreamsMetrics Since it is not a Mock
[
https://issues.apache.org/jira/browse/KAFKA-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965595#comment-16965595
]
bibin sebastian commented on KAFKA-8977:
Ok, I get your point.
Yes, I would like to remove {{MockStreamsMetrics}} with {{StreamsMetricsImpl
for now as the first step.}}
> Remove MockStreamsMetrics Since it is not a Mock
>
>
> Key: KAFKA-8977
> URL: https://issues.apache.org/jira/browse/KAFKA-8977
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: bibin sebastian
>Priority: Minor
> Labels: newbie
>
> The class {{MockStreamsMetrics}} is used throughout unit tests as a mock but
> it is not really a mock since it only hides two parameters of the
> {{StreamsMetricsImpl}} constructor. Either a real mock or the real
> {{StreamsMetricsImpl}} should be used in the tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-8977) Remove MockStreamsMetrics Since it is not a Mock
[
https://issues.apache.org/jira/browse/KAFKA-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964669#comment-16964669
]
bibin sebastian commented on KAFKA-8977:
I am assuming you are suggesting to create a generic mock of
`StreamsMetricsImpl` using the PowerMock/EasyMock and externalise for all
tests. Technically I think that is possible.
(Actually I think that is the only way to create mock for `StreamsMetricsImpl`.
Creating the mock by implementing the interface `StreamsMetrics` won't really
work because in most of the places `StreamsMetricsImpl` is directly referred
instead of using the interface references. Also this class have multiple static
methods. So I have a feeling that this class is used like a util class.)
So going by your suggestion, even if I create a generic mock, I am still not
convinced on the value of creating a mock in these scenarios where we have
indirect access to `StreamsMetricsImpl`.
Let me try to explain my thought process here.
Generally I use a mock instance to mock any external (direct) dependencies of
the class/method in test and I verify the usage of mock as part of assertion in
the end. In our case since StreamsMetricsImpl is not being directly referred
(most of the time) in classes/methods in test, even if we create a mock we
won't be realistically be able to verify the usage (methods invocations) on the
mock (using verify(mock) method). This is because we wouldn't want our tests to
fail assertion if the indirect usage of StreamsMetricsImpl changes at a later
point of time, so verifying the mock usage in the assertion wont really make
sense. So my point is what is the real value we gain if we can't verify the
usage of mock?
So I was preferring to leave its usage as it is in its full form rather than
using a mock. What do think?
[~cadonna]
> Remove MockStreamsMetrics Since it is not a Mock
>
>
> Key: KAFKA-8977
> URL: https://issues.apache.org/jira/browse/KAFKA-8977
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: bibin sebastian
>Priority: Minor
> Labels: newbie
>
> The class {{MockStreamsMetrics}} is used throughout unit tests as a mock but
> it is not really a mock since it only hides two parameters of the
> {{StreamsMetricsImpl}} constructor. Either a real mock or the real
> {{StreamsMetricsImpl}} should be used in the tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-8977) Remove MockStreamsMetrics Since it is not a Mock
[
https://issues.apache.org/jira/browse/KAFKA-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961278#comment-16961278
]
bibin sebastian commented on KAFKA-8977:
[~cadonna] Having going through all the usages of existing MockStreamsMetrics,
what I generally see is that MockStreamsMetrics is used during initialisation
steps of the test (For example to create ThreadCache and session store). The
tests are not always directly accessing/invoking the methods of StreamsMetrics.
Almost all the times access is indirect (As mentioned in the previos above
observation as well). So I am not really seeing any great the value in creating
a mock on StreamsMetricsImpl via PowerMock/EasyMock and replacing
MockStreamsMetrics.
Actually at first, I started with creating the mock using PowerMock/EasyMock.
First of all it forced me to look at the initialisation steps and mock most of
the methods for the initialisation steps. On top of these, since these tests
are indirectly using StreamsMetricsImpl, I have to create separate expectation
on mocks just to make the test pass.
Had it been StreamsMetricsImpl being directly referred in the test methods, it
would have been easy to mock the relevant methods but that is not the case in
almost all the cases.
All these findings makes me feel that replacing MockStreamsMetrics with
StreamsMetricsImpl is the easiest and cleanest option at the moment. In some
simple cases, I have used niceMock().
Updated the PR above. Can you please take a look?
> Remove MockStreamsMetrics Since it is not a Mock
>
>
> Key: KAFKA-8977
> URL: https://issues.apache.org/jira/browse/KAFKA-8977
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: bibin sebastian
>Priority: Minor
> Labels: newbie
>
> The class {{MockStreamsMetrics}} is used throughout unit tests as a mock but
> it is not really a mock since it only hides two parameters of the
> {{StreamsMetricsImpl}} constructor. Either a real mock or the real
> {{StreamsMetricsImpl}} should be used in the tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-8977) Remove MockStreamsMetrics Since it is not a Mock
[
https://issues.apache.org/jira/browse/KAFKA-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951855#comment-16951855
]
bibin sebastian commented on KAFKA-8977:
Got it. Glancing through the test cases/classes where MockStreamsMetrics is
used, what I see is that in lot of the test cases, class/object that is tested,
already have references to many other classes. streamsMetrics instance is
passed down to these reference classes and they in turn call the methods on
streamsMetrics.
Here is an example. ThreadCache accepts StreamsMetrics as a constructor
parameter (this.metrics) and this is passed down to NamesCache instance as
follows in getOrCreateCache() method.
{code:java}
public ThreadCache(final LogContext logContext, final long maxCacheSizeBytes,
final StreamsMetricsImpl metrics) {
this.maxCacheSizeBytes = maxCacheSizeBytes;
this.metrics = metrics;
this.log = logContext.logger(getClass());
}
private synchronized NamedCache getOrCreateCache(final String name) {
NamedCache cache = caches.get(name);
if (cache == null) {
cache = new NamedCache(name, this.metrics);
caches.put(name, cache);
}
return cache;
}
{code}
Now if you look at the constructor of NamedCache (shown below), streamsMetrics
is again passed down to NamedCacheMetrics.
{code:java}
NamedCache(final String name, final StreamsMetricsImpl streamsMetrics) {
this.name = name;
this.streamsMetrics = streamsMetrics;
storeName = ThreadCache.underlyingStoreNamefromCacheName(name);
taskName = ThreadCache.taskIDfromCacheName(name);
hitRatioSensor = NamedCacheMetrics.hitRatioSensor(
streamsMetrics,
Thread.currentThread().getName(),
taskName,
storeName
);
}
{code}
While replacing MockStreamsMetrics with an EasyMock/PowerMock instances, I am
also trying to mock the external dependencies (wherever appropriate) to reduce
the scope of the mocking StreamsMetricsImpl. So in this just above mentioned
case I will also mock NamedCacheMetrics.hitRatioSensor() as this is an external
logic to ThreadCache. IMO this will leave tests in a better shape as a general
practice it is good to mock all the external dependencies in a unit test.
There are many test files where we have this above mentioned situation. I will
share the PR once I am done with the changes.
> Remove MockStreamsMetrics Since it is not a Mock
>
>
> Key: KAFKA-8977
> URL: https://issues.apache.org/jira/browse/KAFKA-8977
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: bibin sebastian
>Priority: Minor
> Labels: newbie
>
> The class {{MockStreamsMetrics}} is used throughout unit tests as a mock but
> it is not really a mock since it only hides two parameters of the
> {{StreamsMetricsImpl}} constructor. Either a real mock or the real
> {{StreamsMetricsImpl}} should be used in the tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-8977) Remove MockStreamsMetrics Since it is not a Mock
[
https://issues.apache.org/jira/browse/KAFKA-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951857#comment-16951857
]
bibin sebastian commented on KAFKA-8977:
[~cadonna]^^
> Remove MockStreamsMetrics Since it is not a Mock
>
>
> Key: KAFKA-8977
> URL: https://issues.apache.org/jira/browse/KAFKA-8977
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: bibin sebastian
>Priority: Minor
> Labels: newbie
>
> The class {{MockStreamsMetrics}} is used throughout unit tests as a mock but
> it is not really a mock since it only hides two parameters of the
> {{StreamsMetricsImpl}} constructor. Either a real mock or the real
> {{StreamsMetricsImpl}} should be used in the tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-8977) Remove MockStreamsMetrics Since it is not a Mock
[
https://issues.apache.org/jira/browse/KAFKA-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16945258#comment-16945258
]
bibin sebastian commented on KAFKA-8977:
[~cadonna] shall I remove MockStreamsMetrics and instead use StreamsMetricsImpl
in the testcases?
I dont see a real need to create a mock for StreamsMetricsImpl now.
> Remove MockStreamsMetrics Since it is not a Mock
>
>
> Key: KAFKA-8977
> URL: https://issues.apache.org/jira/browse/KAFKA-8977
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: bibin sebastian
>Priority: Minor
> Labels: newbie
>
> The class {{MockStreamsMetrics}} is used throughout unit tests as a mock but
> it is not really a mock since it only hides two parameters of the
> {{StreamsMetricsImpl}} constructor. Either a real mock or the real
> {{StreamsMetricsImpl}} should be used in the tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (KAFKA-8977) Remove MockStreamsMetrics Since it is not a Mock
[
https://issues.apache.org/jira/browse/KAFKA-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
bibin sebastian reassigned KAFKA-8977:
--
Assignee: bibin sebastian
> Remove MockStreamsMetrics Since it is not a Mock
>
>
> Key: KAFKA-8977
> URL: https://issues.apache.org/jira/browse/KAFKA-8977
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Reporter: Bruno Cadonna
>Assignee: bibin sebastian
>Priority: Minor
> Labels: newbie
>
> The class {{MockStreamsMetrics}} is used throughout unit tests as a mock but
> it is not really a mock since it only hides two parameters of the
> {{StreamsMetricsImpl}} constructor. Either a real mock or the real
> {{StreamsMetricsImpl}} should be used in the tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-8958) Fix Kafka Streams JavaDocs with regard to used Serdes
[
https://issues.apache.org/jira/browse/KAFKA-8958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16945238#comment-16945238
]
bibin sebastian commented on KAFKA-8958:
Going through the docs, I see these below places where it says it will use
default serializers and deserializers. Do you think we need to update the docs
in all these places?
*Kstream*
KStream#groupBy(KeyValueMapper selector)
{code:java}
Group the records of this KStream on a new key that is selected using the
provided KeyValueMapper and default serializers and deserializers.
{code}
KStream#groupByKey()
{code:java}
Group the records by their current key into a KGroupedStream while preserving
the original values and default serializers and deserializers.
{code}
KStream#join(KStream otherStream, ValueJoiner joiner, JoinWindows windows)
{code:java}
Join records of this stream with another KStream's records using windowed inner
equi join with default serializers and deserializers.
{code}
KStream#join(KTable table, ValueJoiner
joiner)
{code:java}
Join records of this stream with KTable's records using non-windowed inner equi
join with default serializers and deserializers.
{code}
KStream#join(KTable table, ValueJoiner
joiner, Joined joined)
{code:java}
Join records of this stream with KTable's records using non-windowed inner equi
join with default serializers and deserializers.
{code}
KStream#leftJoin(KStream otherStream, ValueJoiner joiner, JoinWindows windows)
{code:java}
Join records of this stream with another KStream's records using windowed left
equi join with default serializers and deserializers.
{code}
KStream#leftJoin(KTable table, ValueJoiner joiner)
{code:java}
Join records of this stream with KTable's records using non-windowed left equi
join with default serializers and deserializers.
{code}
KStream#leftJoin(KTable table, ValueJoiner joiner, Joined joined)
{code:java}
Join records of this stream with KTable's records using non-windowed left equi
join with default serializers and deserializers.
{code}
KStream#outerJoin(KStream otherStream, ValueJoiner joiner, JoinWindows windows)
{code:java}
Join records of this stream with another KStream's records using windowed outer
equi join with default serializers and deserializers.
{code}
KStream#through(String topic)
{code:java}
Materialize this stream to a topic and creates a new KStream from the topic
using default serializers, deserializers, and producer's DefaultPartitioner.
{code}
KStream#to(String topic)
{code:java}
Materialize this stream to a topic using default serializers specified in the
config and producer's DefaultPartitioner.
{code}
KStream#to(TopicNameExtractor topicExtractor)
{code:java}
Dynamically materialize this stream to topics using default serializers
specified in the config and producer's DefaultPartitioner.
{code}
*KTable*
KTable#filter(Predicate predicate)
{code:java}
Create a new KTable that consists of all records of this KTable which satisfy
the given predicate, with default serializers, deserializers, and state store.
{code}
KTable#filterNot(Predicate predicate)
{code:java}
Create a new KTable that consists all records of this KTable which do not
satisfy the given predicate, with default serializers, deserializers, and state
store.
{code}
KTable#groupBy(KeyValueMapper> selector)
{code:java}
Re-groups the records of this KTable using the provided KeyValueMapper and
default serializers and deserializers.
{code}
KTable#join(KTable other, ValueJoiner
joiner)
{code:java}
Join records of this KTable with another KTable's records using non-windowed
inner equi join, with default serializers, deserializers, and state store.
{code}
KTable#leftJoin(KTable other, ValueJoiner joiner)
{code:java}
Join records of this KTable (left input) with another KTable's (right input)
records using non-windowed left equi join, with default serializers,
deserializers, and state store.
{code}
KTable#mapValues(ValueMapper mapper)
{code:java}
Create a new KTable by transforming the value of each record in this KTable
into a new value (with possibly a new type) in the new KTable, with default
serializers, deserializers, and state store.
{code}
KTable#mapValues(ValueMapperWithKey mapper)
{code:java}
Create a new KTable by transforming the value of each record in this KTable
into a new value (with possibly a new type) in the new KTable, with default
serializers, deserializers, and state store.
{code}
KTable#outerJoin(KTable other, ValueJoiner joiner)
{code:java}
Join records of this KTable (left input) with another KTable's (right input)
records using non-windowed outer equi join, with default serializers,
deserializers, and state store.
{code}
KTable#transformValues(ValueTransformerWithKeySupplier transformerSupplier, String... stateStoreNames)
{code:java}
Create a new KTable by transforming the value of each record in this KTable
into a new value (with possibly a
[jira] [Commented] (KAFKA-8958) Fix Kafka Streams JavaDocs with regard to used Serdes
[
https://issues.apache.org/jira/browse/KAFKA-8958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942600#comment-16942600
]
bibin sebastian commented on KAFKA-8958:
[~mjsax] i will work on this.
> Fix Kafka Streams JavaDocs with regard to used Serdes
> -
>
> Key: KAFKA-8958
> URL: https://issues.apache.org/jira/browse/KAFKA-8958
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Reporter: Matthias J. Sax
>Assignee: bibin sebastian
>Priority: Minor
> Labels: beginner, newbie
>
> In older released, Kafka Streams applied operator specific overwrites of
> Serdes as in-place overwrites. In newer releases, Kafka Streams tries to
> re-use Serdes more "aggressively" by pushing serde information downstream if
> the key and/or value did not change.
> However, we never updated the JavaDocs accordingly. For example
> `KStream#through(String topic)` JavaDocs say:
> {code:java}
> Materialize this stream to a topic and creates a new {@code KStream} from the
> topic using default serializers, deserializers, and producer's {@link
> DefaultPartitioner}.
> {code}
> The JavaDocs don't put into account that Serdes might have been set further
> upstream, and the defaults from the config would not be used.
> `KStream#through()` is just one example. We should address this through all
> JavaDocs over all operators (ie, KStream, KGroupedStream,
> TimeWindowedKStream, SessionWindowedKStream, KTable, and KGroupedTable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (KAFKA-8958) Fix Kafka Streams JavaDocs with regard to used Serdes
[
https://issues.apache.org/jira/browse/KAFKA-8958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
bibin sebastian reassigned KAFKA-8958:
--
Assignee: bibin sebastian
> Fix Kafka Streams JavaDocs with regard to used Serdes
> -
>
> Key: KAFKA-8958
> URL: https://issues.apache.org/jira/browse/KAFKA-8958
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Reporter: Matthias J. Sax
>Assignee: bibin sebastian
>Priority: Minor
> Labels: beginner, newbie
>
> In older released, Kafka Streams applied operator specific overwrites of
> Serdes as in-place overwrites. In newer releases, Kafka Streams tries to
> re-use Serdes more "aggressively" by pushing serde information downstream if
> the key and/or value did not change.
> However, we never updated the JavaDocs accordingly. For example
> `KStream#through(String topic)` JavaDocs say:
> {code:java}
> Materialize this stream to a topic and creates a new {@code KStream} from the
> topic using default serializers, deserializers, and producer's {@link
> DefaultPartitioner}.
> {code}
> The JavaDocs don't put into account that Serdes might have been set further
> upstream, and the defaults from the config would not be used.
> `KStream#through()` is just one example. We should address this through all
> JavaDocs over all operators (ie, KStream, KGroupedStream,
> TimeWindowedKStream, SessionWindowedKStream, KTable, and KGroupedTable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-8504) Update Docs to explain how to use suppress() in more details
[
https://issues.apache.org/jira/browse/KAFKA-8504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938852#comment-16938852
]
bibin sebastian commented on KAFKA-8504:
[~vvcephei] Is this something I can pick up?
> Update Docs to explain how to use suppress() in more details
>
>
> Key: KAFKA-8504
> URL: https://issues.apache.org/jira/browse/KAFKA-8504
> Project: Kafka
> Issue Type: Improvement
> Components: documentation, streams
>Affects Versions: 2.2.1
>Reporter: Simone Esposito
>Priority: Minor
> Labels: newbie
>
> Hi, I'm playing a bit with KafkaStream and the new suppress feature. I
> noticed that when using a {{TimeWindows}} without explicitly setting the
> grace {{suppress}} will not emit any message if used with
> {{Suppressed.untilWindowCloses.}}
> I look a bit into the code and from what I understood with this configuration
> {{suppress}} should use the {{grace}} setting of the {{TimeWindows}}. But
> since using {{TimeWindows.of(Duration)}} default the grace to {{-1}} and when
> getting the grace using the method {{TimeWindows.gracePeriodMs()}} in case of
> grace equals to -1 the return value is set to {{maintainMs() - size()}} I
> think that the end of window is not properly calculated.
> Of course is possible to avoid this problem forcing the {{grace}} to 0 when
> creating the TimeWindows but I think that this should be the default
> behaviour at least when it comes to the suppress feature.
> I hope I have not misunderstood the code in my analysis, thank you :)
> Simone
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-8666) Improve Documentation on usage of Materialized config object
[
https://issues.apache.org/jira/browse/KAFKA-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938720#comment-16938720
]
bibin sebastian commented on KAFKA-8666:
[~mjsax][~zafirov] I feel there is still some scope for improvement - to reduce
confusion here.
When an API can support a construct like this
{code:java}
Materialized.as("MyStoreName").with(keySerde, valueSerde)
{code}
general perception is that "MyStoreName" is set in the first object created is
never lost in the chain.
This perception could also be because method name `with` - usually when I see a
method name `with` in the builder pattern context, I tend to think that we are
updating an existing object. I think that is what happened with KAFKA-8646 as
well.
I know, if we use an IDE it will show it as a warning because the static method
"with(keySerde, valueSerde)" is called on an object instance, but it is quite
possible that developer never notices as that.
Looking at attributes of "Materialized" class, looks like fields storeName,
keySerde and valueSerde are all optional.
So, I think doing these steps will remove the confusion and satisfy our needs.
1 - deprecate static method `Materialized.as("MyStoreName")` and replace it
with a non static `Materialized.withStoreName(storeName)`.
2 - provide static method `Materialized.default()` to return a Materialized
object instance with storeName as null.
3 - deprecate static method `with(final Serde keySerde, final Serde
valueSerde)` - we wouldn't need this anymore.
Or creating a builder class for "Materialized" class might be a better approach.
> Improve Documentation on usage of Materialized config object
>
>
> Key: KAFKA-8666
> URL: https://issues.apache.org/jira/browse/KAFKA-8666
> Project: Kafka
> Issue Type: Improvement
> Components: documentation, streams
>Reporter: Bill Bejeck
>Priority: Major
> Labels: newbie
>
> When using the Materialized object if the user wants to name the statestore
> with
> {code:java}
> Materialized.as("MyStoreName"){code}
> then subsequently provide the key and value serde the calls to do so must
> take the form of
> {code:java}
> Materialized.as("MyStoreName").withKeySerde(keySerde).withValueSerde(valueSerde)
> {code}
> If users do the following
> {code:java}
> Materialized.as("MyStoreName").with(keySerde, valueSerde)
> {code}
> the Materialized instance created by the "as(storeName)" call is replaced by
> a new Materialized instance resulting from the "with(...)" call and any
> configuration on the first Materialized instance is lost.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-7131) Update release script to generate announcement email text
[ https://issues.apache.org/jira/browse/KAFKA-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624496#comment-16624496 ] bibin sebastian commented on KAFKA-7131: [~mjsax], [~ewencp] can you please take a look? > Update release script to generate announcement email text > - > > Key: KAFKA-7131 > URL: https://issues.apache.org/jira/browse/KAFKA-7131 > Project: Kafka > Issue Type: Improvement >Reporter: Matthias J. Sax >Assignee: bibin sebastian >Priority: Minor > Labels: newbie > > When a release is finalized, we send out an email to announce the release. > Atm, we have a template in the wiki > ([https://cwiki.apache.org/confluence/display/KAFKA/Release+Process|https://cwiki.apache.org/confluence/display/KAFKA/Release+Process]). > However, the template needs some manual changes to fill in the release > number, number of contributors, etc. > Some parts could be automated – the corresponding commands are document in > the wiki already. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (KAFKA-7131) Update release script to generate announcement email text
[ https://issues.apache.org/jira/browse/KAFKA-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16617098#comment-16617098 ] bibin sebastian edited comment on KAFKA-7131 at 9/17/18 5:20 AM: - [~mjsax] [~ewencp] can you please the PR? was (Author: bibin84): [@ewencp|https://github.com/ewencp] [@mjsax|https://github.com/mjsax] can you please the PR? > Update release script to generate announcement email text > - > > Key: KAFKA-7131 > URL: https://issues.apache.org/jira/browse/KAFKA-7131 > Project: Kafka > Issue Type: Improvement >Reporter: Matthias J. Sax >Assignee: bibin sebastian >Priority: Minor > Labels: newbie > > When a release is finalized, we send out an email to announce the release. > Atm, we have a template in the wiki > ([https://cwiki.apache.org/confluence/display/KAFKA/Release+Process|https://cwiki.apache.org/confluence/display/KAFKA/Release+Process]). > However, the template needs some manual changes to fill in the release > number, number of contributors, etc. > Some parts could be automated – the corresponding commands are document in > the wiki already. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7131) Update release script to generate announcement email text
[ https://issues.apache.org/jira/browse/KAFKA-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16617098#comment-16617098 ] bibin sebastian commented on KAFKA-7131: [@ewencp|https://github.com/ewencp] [@mjsax|https://github.com/mjsax] can you please the PR? > Update release script to generate announcement email text > - > > Key: KAFKA-7131 > URL: https://issues.apache.org/jira/browse/KAFKA-7131 > Project: Kafka > Issue Type: Improvement >Reporter: Matthias J. Sax >Assignee: bibin sebastian >Priority: Minor > Labels: newbie > > When a release is finalized, we send out an email to announce the release. > Atm, we have a template in the wiki > ([https://cwiki.apache.org/confluence/display/KAFKA/Release+Process|https://cwiki.apache.org/confluence/display/KAFKA/Release+Process]). > However, the template needs some manual changes to fill in the release > number, number of contributors, etc. > Some parts could be automated – the corresponding commands are document in > the wiki already. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7131) Update release script to generate announcement email text
[ https://issues.apache.org/jira/browse/KAFKA-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592668#comment-16592668 ] bibin sebastian commented on KAFKA-7131: [~mjsax] ^^ apologies for the delay in updating this. can you please take a look at this PR? > Update release script to generate announcement email text > - > > Key: KAFKA-7131 > URL: https://issues.apache.org/jira/browse/KAFKA-7131 > Project: Kafka > Issue Type: Improvement >Reporter: Matthias J. Sax >Assignee: bibin sebastian >Priority: Minor > Labels: newbie > > When a release is finalized, we send out an email to announce the release. > Atm, we have a template in the wiki > ([https://cwiki.apache.org/confluence/display/KAFKA/Release+Process|https://cwiki.apache.org/confluence/display/KAFKA/Release+Process]). > However, the template needs some manual changes to fill in the release > number, number of contributors, etc. > Some parts could be automated – the corresponding commands are document in > the wiki already. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7131) Update release script to generate announcement email text
[ https://issues.apache.org/jira/browse/KAFKA-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562228#comment-16562228 ] bibin sebastian commented on KAFKA-7131: [~mjsax] ^^ > Update release script to generate announcement email text > - > > Key: KAFKA-7131 > URL: https://issues.apache.org/jira/browse/KAFKA-7131 > Project: Kafka > Issue Type: Improvement >Reporter: Matthias J. Sax >Assignee: bibin sebastian >Priority: Minor > Labels: newbie > > When a release is finalized, we send out an email to announce the release. > Atm, we have a template in the wiki > ([https://cwiki.apache.org/confluence/display/KAFKA/Release+Process|https://cwiki.apache.org/confluence/display/KAFKA/Release+Process]). > However, the template needs some manual changes to fill in the release > number, number of contributors, etc. > Some parts could be automated – the corresponding commands are document in > the wiki already. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7131) Update release script to generate announcement email text
[ https://issues.apache.org/jira/browse/KAFKA-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560822#comment-16560822 ] bibin sebastian commented on KAFKA-7131: [~mjsax], i have a question on the scope of the work. Is the ask is to automate all what is suggested below (as mentioned in the release.py) ? * Grab the source archive and make sure it compiles: http://home.apache.org/~%(apache_id)s/kafka-%(rc_tag)s/kafka-%(release_version)s-src.tgz * Grab one of the binary distros and run the quickstarts against them: http://home.apache.org/~%(apache_id)s/kafka-%(rc_tag)s/kafka_2.11-%(release_version)s.tgz * Extract and verify one of the site docs jars: http://home.apache.org/~%(apache_id)s/kafka-%(rc_tag)s/kafka_2.11-%(release_version)s-site-docs.tgz * Build a sample against jars in the staging repo: (TODO: Can we get a temporary URL before "closing" the staged artifacts?) * Validate GPG signatures on at least one file: wget http://home.apache.org/~%(apache_id)s/kafka-%(rc_tag)s/kafka-%(release_version)s-src.tgz && wget http://home.apache.org/~%(apache_id)s/kafka-%(rc_tag)s/kafka-%(release_version)s-src.tgz.asc && wget http://home.apache.org/~%(apache_id)s/kafka-%(rc_tag)s/kafka-%(release_version)s-src.tgz.md5 && wget http://home.apache.org/~%(apache_id)s/kafka-%(rc_tag)s/kafka-%(release_version)s-src.tgz.sha1 && wget http://home.apache.org/~%(apache_id)s/kafka-%(rc_tag)s/kafka-%(release_version)s-src.tgz.sha512 && gpg --verify kafka-%(release_version)s-src.tgz.asc kafka-%(release_version)s-src.tgz && gpg --print-md md5 kafka-%(release_version)s-src.tgz | diff - kafka-%(release_version)s-src.tgz.md5 && gpg --print-md sha1 kafka-%(release_version)s-src.tgz | diff - kafka-%(release_version)s-src.tgz.sha1 && gpg --print-md sha512 kafka-%(release_version)s-src.tgz | diff - kafka-%(release_version)s-src.tgz.sha512 && rm kafka-%(release_version)s-src.tgz* && echo "OK" || echo "Failed" * Validate the javadocs look ok. They are at [http://home.apache.org/~%(apache_id)s/kafka-%(rc_tag)s/javadoc/] > Update release script to generate announcement email text > - > > Key: KAFKA-7131 > URL: https://issues.apache.org/jira/browse/KAFKA-7131 > Project: Kafka > Issue Type: Improvement >Reporter: Matthias J. Sax >Assignee: bibin sebastian >Priority: Minor > Labels: newbie > > When a release is finalized, we send out an email to announce the release. > Atm, we have a template in the wiki > ([https://cwiki.apache.org/confluence/display/KAFKA/Release+Process|https://cwiki.apache.org/confluence/display/KAFKA/Release+Process]). > However, the template needs some manual changes to fill in the release > number, number of contributors, etc. > Some parts could be automated – the corresponding commands are document in > the wiki already. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7131) Update release script to generate announcement email text
[ https://issues.apache.org/jira/browse/KAFKA-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16555168#comment-16555168 ] bibin sebastian commented on KAFKA-7131: can i pick this up? i can work on this with some guidance/help on the required changes to the script > Update release script to generate announcement email text > - > > Key: KAFKA-7131 > URL: https://issues.apache.org/jira/browse/KAFKA-7131 > Project: Kafka > Issue Type: Improvement >Reporter: Matthias J. Sax >Priority: Minor > Labels: newbie > > When a release is finalized, we send out an email to announce the release. > Atm, we have a template in the wiki > ([https://cwiki.apache.org/confluence/display/KAFKA/Release+Process|https://cwiki.apache.org/confluence/display/KAFKA/Release+Process]). > However, the template needs some manual changes to fill in the release > number, number of contributors, etc. > Some parts could be automated – the corresponding commands are document in > the wiki already. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
