Hi All,
I modified wiki page to replace Serdes with Serializer / Deserializer.
I will start the vote.
Jukka
pe 20. syysk. 2019 klo 4.21 John Roesler (j...@confluent.io) kirjoitti:
> Hey, all,
>
> For what it's worth, I agree with Matthias. While it's true that
> you're likely to have Serdes on
Hey, all,
For what it's worth, I agree with Matthias. While it's true that
you're likely to have Serdes on hand, if for whatever reason, you
don't, it's a pain to make one.
Plus, with just requiring Serializer and Deserializer, each component
is only asking for the minimum thing it needs, rather
Personally, I think it would be cleaner to just pass `Deserializer` into
`InputTopic` and `Serializer` into `OutputTopic`. Keeping the interface
simple is a good argument and we should not have overloads -- but it
seems that picking `Serdes` instead of the more straight forward
`(De)Serializer` is
Hi,
In many cases you have already serde for the stream definition. So I see
natural to use it also for the tests.
So you can write:
public void setup() {
testDriver = new TopologyTestDriver(TestStream.getTopology(),
TestStream.getConfig());
inputTopic = testDriver.createInputTopic(Test
Maybe one follow up question:
Why do we pass `Serdes` into `createInputTopic` and createOutputTopic`
-- seems `Serializer` (for input) and `Deserialized` (for output) should
be sufficient?
-Matthias
On 9/12/19 4:59 PM, Matthias J. Sax wrote:
> Thanks for updating the KIP.
>
> I agree with John
Thanks for updating the KIP.
I agree with John that we (ie, you :)) can start a vote.
-Matthias
On 9/11/19 9:23 AM, John Roesler wrote:
> Thanks for the update, Jukka!
>
> I'd be in favor of the current proposal. Not sure how the others feel.
> If people generally feel positive, it might be ti
Thanks for the update, Jukka!
I'd be in favor of the current proposal. Not sure how the others feel.
If people generally feel positive, it might be time to start a vote.
Thanks,
-John
On Sat, Sep 7, 2019 at 12:40 AM Jukka Karvanen
wrote:
>
> Hi,
>
> Sorry; I need to rollback right away the one
Hi,
Sorry; I need to rollback right away the one method removal what I was
proposing.
One constructor with Long restored to TestRecord, which is needed by
TestInputTopic.
Regards,
Jukka
la 7. syysk. 2019 klo 8.06 Jukka Karvanen (jukka.karva...@jukinimi.com)
kirjoitti:
> Hi,
>
> Let's get back
Hi,
Let's get back to this after summer break.
Thanks Antony to offering help, it might be needed.
I modified the KIP based on the feedback to be a mixture of variations 4
and 5.
In TestInputTopic I removed deprecation from one variation with long
timestamp and removed totally one version withou
Hi Jukka! I just came across your work - it looks great! I was taking a
stab at improving the existing API, but yours already looks great and just
about complete! Are you planning on continuing your work and submitting a
PR? If you want some help, I'd be happy to jump in.
Regards,
Antony.
On Thu,
Hi Jukka,
I also think 3, 4, and 5 are all good options.
My personal preference is 4, but I also wouldn't mind going with 5 if that
is what others want to do.
Thanks,
Bill
On Tue, Jul 30, 2019 at 9:31 AM John Roesler wrote:
> Hey Jukka,
>
> Sorry for the delay.
>
> For what it's worth, I thin
Hey Jukka,
Sorry for the delay.
For what it's worth, I think 3, 4, and 5 are all good options. I guess my
own preference is 5.
It seems like the migration pain is a one-time concern vs. having more
maintainable code for years thereafter.
Thanks,
-John
On Tue, Jul 2, 2019 at 4:03 AM Jukka Kar
Hi Matthias,
Generally I think using Instant and Duration make the test more readable
and that's why I modified KIP based on your suggestion.
Now a lot of code use time with long or Long and that make the change more
complicated.
What I tried to say about the migration is the lines without timest
Thanks Jukka!
The idea to use `Instant/Duration` was a proposal. If we think it's not
a good one, we could still stay with `long`. Because `ProducerRecord`
and `ConsumerRecord` are both based on `long,` it might make sense to
keep `long`?
> The result of converting millis to Instant directly gene
Hi,
>>(4) Should we switch from `long` for timestamps to `Instant` and
`Duration` ?
>This version startTimestamp is Instant and autoAdvance Duration in
Initialization and with manual configured collection pipe methods.
>Now timestamp of TestRecord is still Long and similarly single record
pipeInpu
Hi Jukka
> These topic objects are only interfacing TopologyTestDriver, not affecting
> the internal functionality of it. In my plan the internal data structures
> are using those Producer/ConsumerRecords as earlier. That way I don't see
> how those could be affected.
I mistakenly thought the KIP
Hi All,
Hi Matthias,
>(1) It's a little confusing that you list all method (existing, proposed
>to deprecate, and new one) of `TopologyTestDriver` in the KIP. Maybe
>only list the ones you propose to deprecate and the new ones you want to
>add?
Ok. Unmodified methods removed.
>(2) `TopologyTestD
Hi Bill,
These topic objects are only interfacing TopologyTestDriver, not affecting
the internal functionality of it. In my plan the internal data structures
are using those Producer/ConsumerRecords as earlier. That way I don't see
how those could be affected.
Jukka
On Fri, 21 Jun 2019, 20:57
Hi,
TestRecord is already included in wiki page. It has mainly constructors and
methods to access data fields.
Jukka
On Fri, 21 Jun 2019, 19:05 Guozhang Wang, wrote:
> 1) Got it, could you list this class along with all its functions in the
> proposed public APIs as well?
>
> 2) Ack, thanks!
>
Thanks for the KIP. The idea to add InputTopic and OutputTopic
abstractions is really neat!
Couple of minor comment:
(1) It's a little confusing that you list all method (existing, proposed
to deprecate, and new one) of `TopologyTestDriver` in the KIP. Maybe
only list the ones you propose to dep
Jukka,
Thanks for the KIP. I like the changes overall.
One thing I wanted to confirm, and this may be me being paranoid, but will
the changes for input/output topic affect how the TopologyTestDriver works
with internal topics when there are sub-topologies created?
On Fri, Jun 21, 2019 at 12:05 PM
1) Got it, could you list this class along with all its functions in the
proposed public APIs as well?
2) Ack, thanks!
On Thu, Jun 20, 2019 at 11:27 PM Jukka Karvanen
wrote:
> Hi Guozhang,
>
> 1) This TestRecord is new class in my proposal. So it is a simplified
> version of ProducerRecord and
Hi Guozhang,
1) This TestRecord is new class in my proposal. So it is a simplified
version of ProducerRecord and ConsumerRecord containing only the fields
needed to test record content.
2)
public final TestInputTopic createInputTopic(final String
topicName, final Serde keySerde, final Serde val
Hello Jukka,
Thanks for writing the KIP, I have a couple of quick questions:
1) Is "TestRecord" an existing class that you propose to piggy-back on?
Right now we have a scala TestRecord case class but I doubt that was your
proposal, or are you proposing to add a new Java class?
2) Would the new
Woah, I wasn't aware of that Hamcrest test style. Awesome!
Thanks for the updates. I look forward to hearing what others think.
-John
On Mon, Jun 17, 2019 at 4:12 AM Jukka Karvanen
wrote:
>
> Wiki page updated:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-470%3A+TopologyTestDriver+te
Wiki page updated:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-470%3A+TopologyTestDriver+test+input+and+output+usability+improvements
ClientRecord removed and replaced with TestRecord in method calls.
TestRecordFactory removed (time tracking functionality to be included to
TestInputTopi
Sounds good. Thanks as always for considering my feedback!
-John
On Fri, Jun 14, 2019 at 12:12 PM Jukka Karvanen
wrote:
>
> Ok, I will modify KIP Public Interface in a wiki based on the feedback.
>
> TestRecordFactory / ConsumerRecordFactory was used by TestInputTopic with
> the version I had wit
Ok, I will modify KIP Public Interface in a wiki based on the feedback.
TestRecordFactory / ConsumerRecordFactory was used by TestInputTopic with
the version I had with KIP456, but maybe I can merge That functionality to
InputTopic or TestRecordFactory can kept non public maybe moving it to
int
Thanks, Jukka,
Ok, I buy this reasoning.
Just to echo what I think I read, you would just drop ClientRecord
from the proposal, and TestRecord would stand on its own, with the
same methods and properties you proposed, and the "input topic" would
accept TestRecord, and the "output topic" would prod
Hi,
I am not a fan of swapping only ProducerRecord and ConsumerRecord.
As a test writer point of view I do not want to care about the difference
of those and
that way I like to have object type which can be used to pipe records in
and compare outputs.
That way avoid unnecessary conversions between
Hey, all, maybe we can jump-start this discussion.
I think this approach would be very ergonomic for testing, and would
help reduce boilerplate in tests.
The think I like most about it is that it mirrors the mental model
that people already have from Kafka Streams, in which you write to an
"input
Hi All,
I would like to start the discussion on KIP-470: TopologyTestDriver test
input and output usability improvements:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-470%3A+TopologyTestDriver+test+input+and+output+usability+improvements
This KIP is inspired by the Discussion in KIP-456
32 matches
Mail list logo
