Re: [VOTE] KIP-349 Priorities for Source Topics
@Nick: A KIP is only accepted if it got 3 binding votes, ie, votes from committers. If you close the vote before that, the KIP would not be accepted. Note that committers need to pay attention to a lot of KIPs and it can take a while until people can look into it. Thanks for your understanding. @Jan: Can you give a little bit more context on your concerns? It's unclear why you mean atm. @Adam: anyone can vote :) -Matthias On 8/19/18 9:58 AM, Adam Bellemare wrote: > While I am not sure if I can or can’t vote, my question re: Jan’s comment is, > “should we be implementing it as Samza does?” > > I am not familiar with the drawbacks of the current approach vs how samza > does it. > >> On Aug 18, 2018, at 5:06 PM, n...@afshartous.com wrote: >> >> >> I only saw one vote on KIP-349, just checking to see if anyone else would >> like to vote before closing this out. >> -- >> Nick >> >> >>> On Aug 13, 2018, at 9:19 PM, n...@afshartous.com wrote: >>> >>> >>> Hi All, >>> >>> Calling for a vote on KIP-349 >>> >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-349%3A+Priorities+for+Source+Topics >>> >>> -- >>> Nick >>> >>> >>> >> >> >> >> signature.asc Description: OpenPGP digital signature
Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times
bq. // or just Duration windowSize();
+1 to the above choice.
The duration is obvious from the return type. For getter methods, we don't
use get as prefix (as least for new code).
Cheers
On Sun, Aug 19, 2018 at 8:03 AM Nikolay Izhikov wrote:
> Hello, John.
>
> Thank you very much for your feedback!
> I've addressed all your comments.
> Please, see my answers and let my know is anything in KIP [1] needs to be
> improved.
>
> > The correct choice is actually "Instant", not> "LocalDateTime"
>
> I've changed the methods proposed in KIP [1] to use Instant.
>
> > I noticed some recent APIs are> missing (see KIP-328)
> > those APIs were just added and have never been released... you can just
> replace them.
>
> I've added new methods to KIP [1].
> Not released methods marked for remove.
>
> > any existing method that's already deprecated, don't bother
> transitioning it to Duration.
>
> Fixed.
>
> > IllegalArgumentException... we should plan to mention this in the
> javadoc for those methods.
>
> Got it.
>
> > In Stores, windowSize and segmentInterval should also be durations.
>
> Fixed.
>
> > StreamsMetrics, recordLatency ... this one is better left alone.
>
> OK. I removed this method from KIP [1].
>
> Two more questions question about implementation:
>
> 1. We have serveral methods without parameters.
> In java we can't have two methods with parameters with the same name.
> It wouldn't compile.
> So we have to rename new methods. Please, see suggested names and share
> your thoughts:
>
> Windows {
> long size() -> Duration windowSize();
> }
>
> Window {
> long start() -> Instant startTime();
> long end() -> Instant endTime();
> }
>
> SessionWindows {
> long inactivityGap() -> Duration inactivityGapDuration();
> }
>
> TimeWindowedDeserializer {
> Long getWindowSize() -> Duration getWindowSizeDuration(); // or just
> Duration windowSize();
> }
>
> SessionBytesStoreSupplier {
> long retentionPeriod() -> Duration retentionPeriodDuration();
> }
>
> WindowBytesStoreSupplier {
> long windowSize() -> Duration windowSizeDuration();
> long retentionPeriod() -> Duration retentionPeriodDuration();
> }
>
> 2. Do we want to use Duration and Instant inside API implementations?
>
> IGNITE-7277: "Durations potentially worsen memory pressure and gc
> performance, so internally, we will still use longMs as the representation."
> IGNITE-7222: Duration used to store retention.
>
> [1]
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-358%3A+Migrate+Streams+API+to+Duration+instead+of+long+ms+times
> [2]
> https://github.com/apache/kafka/commit/b3771ba22acad7870e38ff7f58820c5b50946787#diff-47289575d3e3e2449f27b3a7b6788e1aR64
>
> В Пт, 17/08/2018 в 14:46 -0500, John Roesler пишет:
> > Hi Nikolay,
> >
> > Thanks for this very nice KIP!
> >
> > To answer your questions:
> > 1. Correct, we should not delete existing methods that have been
> released,
> > but ...
> >
> > 2. Yes, we should deprecate the 'long' variants so that we can drop them
> > later on. Personally, I like to mention which version deprecated the
> method
> > so everyone can see later on how long it's been deprecated, but this may
> be
> > controversial, so let's let other weigh in.
> >
> > 3. I think you're asking whether it's appropriate to drop the "Ms"
> suffix,
> > and I think yes. So "long inactivityGapMs" would become "Duration
> > inactivityGap".
> > In the places where the parameter's name is just "duration", I think we
> can
> > pick something more descriptive (I realize it was already "durationMs";
> > this is just a good time to improve it).
> > Also, you're correct that we shouldn't use a Duration to represent a
> moment
> > in time, like "startTime". The correct choice is actually "Instant", not
> > "LocalDateTime", though.
> >
> https://stackoverflow.com/questions/32437550/whats-the-difference-between-instant-and-localdatetime
> > explains why.
> >
> > I also had a few notes on the KIP itself:
> > 4. You might want to pull trunk again. I noticed some recent APIs are
> > missing (see KIP-328).
> >
> > 5. Speaking of KIP-328: those APIs were just added and have never been
> > released, so there's no need to deprecate the methods, you can just
> replace
> > them.
> >
> > 6. For any existing method that's already deprecated, don't bother
> > transitioning it to Duration. I think the examples I noticed were
> > deprecated in KIP-328, so you'll see what I'm talking about when you pull
> > trunk again.
> >
> > 7. Any method taking a Duration argument may throw an
> > IllegalArgumentException (we choose to convert ArithmeticException to
> > IllegalArgumentException, as I mentioned in the Jira ticket). We don't
> need
> > a "throws" declaration, but we should plan to mention this in the javadoc
> > for those methods.
> >
> > 8. In Stores, windowSize and segmentInterval should also be durations.
> >
> > 9. In StreamsMetrics, recordLatency could be just a Duration, but I
> > actually think this one is better left alone.
Re: Add to contributor list
You need an apache id for the project (github account is separate), you can create one on apache website. Guozhang On Sun, Aug 19, 2018 at 2:02 PM, Murali Mani wrote: > Yes, it is correct. muralimani is the user id of Github. The mail id for > github - murali.m...@gmail.com > > On Sun, Aug 19, 2018 at 7:03 PM Guozhang Wang wrote: > > > Hello, > > > > I cannot find the id "muralimani" for either JIRA or wiki system. Could > you > > double check if it is created? > > > > > > Guozhang > > > > > > On Sun, Aug 19, 2018 at 12:46 AM, Murali Mani > > wrote: > > > > > Hi, > > > > > > Could you please add me to the contributor list? My jira id is > muralimani > > > > > > -- > > > regards > > > Murali Krishnan Mani > > > Ph: +44 (0)7432128519 > > > > > > > > > > > -- > > -- Guozhang > > > > > -- > regards > Murali Krishnan Mani > Ph: +44 (0)7432128519 > -- -- Guozhang
Re: Add to contributor list
Yes, it is correct. muralimani is the user id of Github. The mail id for github - murali.m...@gmail.com On Sun, Aug 19, 2018 at 7:03 PM Guozhang Wang wrote: > Hello, > > I cannot find the id "muralimani" for either JIRA or wiki system. Could you > double check if it is created? > > > Guozhang > > > On Sun, Aug 19, 2018 at 12:46 AM, Murali Mani > wrote: > > > Hi, > > > > Could you please add me to the contributor list? My jira id is muralimani > > > > -- > > regards > > Murali Krishnan Mani > > Ph: +44 (0)7432128519 > > > > > > -- > -- Guozhang > -- regards Murali Krishnan Mani Ph: +44 (0)7432128519
[jira] [Created] (KAFKA-7313) StopReplicaRequest should not remove partition from ReplicaManager.allPartitions if partition is offline
Dong Lin created KAFKA-7313: --- Summary: StopReplicaRequest should not remove partition from ReplicaManager.allPartitions if partition is offline Key: KAFKA-7313 URL: https://issues.apache.org/jira/browse/KAFKA-7313 Project: Kafka Issue Type: Improvement Reporter: Dong Lin -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Add to contributor list
Hello, I cannot find the id "muralimani" for either JIRA or wiki system. Could you double check if it is created? Guozhang On Sun, Aug 19, 2018 at 12:46 AM, Murali Mani wrote: > Hi, > > Could you please add me to the contributor list? My jira id is muralimani > > -- > regards > Murali Krishnan Mani > Ph: +44 (0)7432128519 > -- -- Guozhang
[jira] [Created] (KAFKA-7312) Transient failure in kafka.api.AdminClientIntegrationTest.testMinimumRequestTimeouts
Guozhang Wang created KAFKA-7312:
Summary: Transient failure in
kafka.api.AdminClientIntegrationTest.testMinimumRequestTimeouts
Key: KAFKA-7312
URL: https://issues.apache.org/jira/browse/KAFKA-7312
Project: Kafka
Issue Type: Improvement
Components: unit tests
Reporter: Guozhang Wang
{code}
Error Message
org.junit.runners.model.TestTimedOutException: test timed out after 12
milliseconds
Stacktrace
org.junit.runners.model.TestTimedOutException: test timed out after 12
milliseconds
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at
org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:92)
at
org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:262)
at
kafka.utils.TestUtils$.assertFutureExceptionTypeEquals(TestUtils.scala:1345)
at
kafka.api.AdminClientIntegrationTest.testMinimumRequestTimeouts(AdminClientIntegrationTest.scala:1080)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: [VOTE] KIP-349 Priorities for Source Topics
While I am not sure if I can or can’t vote, my question re: Jan’s comment is, “should we be implementing it as Samza does?” I am not familiar with the drawbacks of the current approach vs how samza does it. > On Aug 18, 2018, at 5:06 PM, n...@afshartous.com wrote: > > > I only saw one vote on KIP-349, just checking to see if anyone else would > like to vote before closing this out. > -- > Nick > > >> On Aug 13, 2018, at 9:19 PM, n...@afshartous.com wrote: >> >> >> Hi All, >> >> Calling for a vote on KIP-349 >> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-349%3A+Priorities+for+Source+Topics >> >> -- >> Nick >> >> >> > > > >
Re: [DISCUSS] KIP-358: Migrate Streams API to Duration instead of long ms times
Hello, John.
Thank you very much for your feedback!
I've addressed all your comments.
Please, see my answers and let my know is anything in KIP [1] needs to be
improved.
> The correct choice is actually "Instant", not> "LocalDateTime"
I've changed the methods proposed in KIP [1] to use Instant.
> I noticed some recent APIs are> missing (see KIP-328)
> those APIs were just added and have never been released... you can just
> replace them.
I've added new methods to KIP [1].
Not released methods marked for remove.
> any existing method that's already deprecated, don't bother transitioning it
> to Duration.
Fixed.
> IllegalArgumentException... we should plan to mention this in the javadoc for
> those methods.
Got it.
> In Stores, windowSize and segmentInterval should also be durations.
Fixed.
> StreamsMetrics, recordLatency ... this one is better left alone.
OK. I removed this method from KIP [1].
Two more questions question about implementation:
1. We have serveral methods without parameters.
In java we can't have two methods with parameters with the same name.
It wouldn't compile.
So we have to rename new methods. Please, see suggested names and share your
thoughts:
Windows {
long size() -> Duration windowSize();
}
Window {
long start() -> Instant startTime();
long end() -> Instant endTime();
}
SessionWindows {
long inactivityGap() -> Duration inactivityGapDuration();
}
TimeWindowedDeserializer {
Long getWindowSize() -> Duration getWindowSizeDuration(); // or just
Duration windowSize();
}
SessionBytesStoreSupplier {
long retentionPeriod() -> Duration retentionPeriodDuration();
}
WindowBytesStoreSupplier {
long windowSize() -> Duration windowSizeDuration();
long retentionPeriod() -> Duration retentionPeriodDuration();
}
2. Do we want to use Duration and Instant inside API implementations?
IGNITE-7277: "Durations potentially worsen memory pressure and gc performance,
so internally, we will still use longMs as the representation."
IGNITE-7222: Duration used to store retention.
[1]
https://cwiki.apache.org/confluence/display/KAFKA/KIP-358%3A+Migrate+Streams+API+to+Duration+instead+of+long+ms+times
[2]
https://github.com/apache/kafka/commit/b3771ba22acad7870e38ff7f58820c5b50946787#diff-47289575d3e3e2449f27b3a7b6788e1aR64
В Пт, 17/08/2018 в 14:46 -0500, John Roesler пишет:
> Hi Nikolay,
>
> Thanks for this very nice KIP!
>
> To answer your questions:
> 1. Correct, we should not delete existing methods that have been released,
> but ...
>
> 2. Yes, we should deprecate the 'long' variants so that we can drop them
> later on. Personally, I like to mention which version deprecated the method
> so everyone can see later on how long it's been deprecated, but this may be
> controversial, so let's let other weigh in.
>
> 3. I think you're asking whether it's appropriate to drop the "Ms" suffix,
> and I think yes. So "long inactivityGapMs" would become "Duration
> inactivityGap".
> In the places where the parameter's name is just "duration", I think we can
> pick something more descriptive (I realize it was already "durationMs";
> this is just a good time to improve it).
> Also, you're correct that we shouldn't use a Duration to represent a moment
> in time, like "startTime". The correct choice is actually "Instant", not
> "LocalDateTime", though.
> https://stackoverflow.com/questions/32437550/whats-the-difference-between-instant-and-localdatetime
> explains why.
>
> I also had a few notes on the KIP itself:
> 4. You might want to pull trunk again. I noticed some recent APIs are
> missing (see KIP-328).
>
> 5. Speaking of KIP-328: those APIs were just added and have never been
> released, so there's no need to deprecate the methods, you can just replace
> them.
>
> 6. For any existing method that's already deprecated, don't bother
> transitioning it to Duration. I think the examples I noticed were
> deprecated in KIP-328, so you'll see what I'm talking about when you pull
> trunk again.
>
> 7. Any method taking a Duration argument may throw an
> IllegalArgumentException (we choose to convert ArithmeticException to
> IllegalArgumentException, as I mentioned in the Jira ticket). We don't need
> a "throws" declaration, but we should plan to mention this in the javadoc
> for those methods.
>
> 8. In Stores, windowSize and segmentInterval should also be durations.
>
> 9. In StreamsMetrics, recordLatency could be just a Duration, but I
> actually think this one is better left alone. IMO, it's more effort for
> little gain to require users to construct a Duration before they call the
> method, since they vary likely call System.currentTimeNanos before and
> after the code in question.
>
> These are quite a few notes, but they're all minor. Overall the KIP looks
> really good to me. Thanks for picking this up!
> -John
>
> On Thu, Aug 16, 2018 at 9:55 AM Nikolay Izhikov wrote:
>
> > Hello, Kafka developers.
> >
> > I would like to start a
Add to contributor list
Hi, Could you please add me to the contributor list? My jira id is muralimani -- regards Murali Krishnan Mani Ph: +44 (0)7432128519
