GitHub user helowken opened a pull request:
https://github.com/apache/kafka/pull/4292
Is there something wrong? (for kafka 1.0)
In kafka stream source code: InternalTopologyBuilder.java
I think it should be: **globalGroups.add(node);**
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/kafka 1.0
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/4292.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4292
----
commit dc907d9b7b9b0ed719c228237640d5049bb8e483
Author: Apurva Mehta <[email protected]>
Date: 2017-10-05T05:27:03Z
KAFKA-6003; Accept appends on replicas unconditionally when local producer
state doesn't exist
Without this patch, if the replica's log was somehow truncated before
the leader's, it is possible for the replica fetcher thread to
continuously throw an OutOfOrderSequenceException because the
incoming sequence would be non-zero and there is no local state.
This patch changes the behavior so that the replica state is updated to
the leader's state if there was no local state for the producer at the
time of the append.
Author: Apurva Mehta <[email protected]>
Reviewers: Ismael Juma <[email protected]>, Jason Gustafson
<[email protected]>
Closes #4004 from apurvam/KAFKA-6003-handle-unknown-producer-on-replica
(cherry picked from commit 6ea4fffdd287a0c6a02c1b6dc1006b1a7b614405)
Signed-off-by: Jason Gustafson <[email protected]>
commit 6c01d68c994d966da1cf74e7127473bda2ea3a46
Author: Rajini Sivaram <[email protected]>
Date: 2017-10-05T16:25:34Z
KAFKA-6012; Close request metrics only after closing request handlers
Author: Rajini Sivaram <[email protected]>
Reviewers: Ismael Juma <[email protected]>
Closes #4024 from rajinisivaram/KAFKA-6012-error-metric
(cherry picked from commit e40b3a2e74133de6d60599beefb65407ca4cc7dd)
Signed-off-by: Rajini Sivaram <[email protected]>
commit 5e2767a26b9e4662df301df9fd57843c36e2cdc7
Author: Joel Hamill <[email protected]>
Date: 2017-10-05T16:30:38Z
Rename streams tutorial and quickstart
Changed these topic titles:
- Write your own Streams Applications -> Tutorial: Write a Streams
Application
- Play with a Streams Application -> Run the Streams Demo Application
Author: Joel Hamill <[email protected]>
Author: Joel Hamill <[email protected]>
Reviewers: Michael G. Noll <[email protected]>, Guozhang Wang
<[email protected]>
Closes #4017 from joel-hamill/joel-hamill/streams-titles
commit 1d026269e1ab0af130b78f1efadaabbc4f5a8552
Author: Randall Hauch <[email protected]>
Date: 2017-10-05T18:23:11Z
KAFKA-5903: Added Connect metrics to the worker and distributed herder
(KIP-196)
Added metrics to the Connect worker and rebalancing metrics to the
distributed herder.
This is built on top of #3987, and I can rebase this PR once that is merged.
Author: Randall Hauch <[email protected]>
Reviewers: Ewen Cheslack-Postava <[email protected]>
Closes #4011 from rhauch/kafka-5903
(cherry picked from commit a47bfbcae050659d32f777ed2f4b26dda5fbdbbd)
Signed-off-by: Ewen Cheslack-Postava <[email protected]>
commit b2bb2c6e8f6d430bd84690f25c2c1a29bc1f3864
Author: Damian Guy <[email protected]>
Date: 2017-10-05T19:55:55Z
KAFKA-5989: resume consumption of tasks that have state stores but no
changelogging
Stores where logging is disabled where never consumed as the partitions
were paused, but never resumed.
Author: Damian Guy <[email protected]>
Reviewers: Guozhang Wang <[email protected]>
Closes #4025 from dguy/1.0
commit 42841c9f36c86585b3b7ed7de29baee2b4f8893c
Author: Derrick Or <[email protected]>
Date: 2017-10-05T20:31:18Z
Update docs to reflect kafka trademark status
Updated a couple places in docs with the 'registered' trademark symbol.
Author: Derrick Or <[email protected]>
Reviewers: Jun Rao <[email protected]>
Closes #4028 from derrickdoo/kafka-trademark-status
(cherry picked from commit f29203d022f110761415c9a2ae2c149b4bc5dd21)
Signed-off-by: Jun Rao <[email protected]>
commit b113b4bd3e3802e8e6974e6754e96bd0f1de32e2
Author: Guozhang Wang <[email protected]>
Date: 2017-10-06T00:02:53Z
KAFKA-5576: RocksDB upgrade to 5.8, plus one bug fix on Bytes.wrap
Author: Guozhang Wang <[email protected]>
Reviewers: Ismael Juma <[email protected]>, Bill Bejeck
<[email protected]>, Matthias J. Sax <[email protected]>, Damian Guy
<[email protected]>
Closes #3819 from guozhangwang/KMinor-rocksDB-573
(cherry picked from commit 196bcfca0c56420793f85514d1602bde564b0651)
Signed-off-by: Guozhang Wang <[email protected]>
commit 32ee1a87986916619acebea3b88609cf0a1c7dc4
Author: Konstantine Karantasis <[email protected]>
Date: 2017-10-06T18:42:04Z
KAFKA-5953: Register all jdbc drivers available in plugin and class paths
Author: Konstantine Karantasis <[email protected]>
Reviewers: Randall Hauch <[email protected]>, Ewen Cheslack-Postava
<[email protected]>
Closes #4030 from
kkonstantine/KAFKA-5953-Connect-classloader-isolation-may-be-broken-for-JDBC-drivers
(cherry picked from commit 974d6fec93b779cd2f9036d0ea89cf1b0917de7b)
Signed-off-by: Ewen Cheslack-Postava <[email protected]>
commit ebeee3c8dea748186d00c9c88185499c46ce3949
Author: Manikumar Reddy <[email protected]>
Date: 2017-10-06T19:51:30Z
KAFKA-5547; Return TOPIC_AUTHORIZATION_FAILED error if no describe access
for topics
Author: Manikumar Reddy <[email protected]>
Reviewers: Rajini Sivaram <[email protected]>, Ismael Juma
<[email protected]>, Jason Gustafson <[email protected]>
Closes #3924 from omkreddy/KAFKA-5547-TOPIC-AUTHRO
(cherry picked from commit 10cd98cc894b88c5d1e24fc54c66361ad9914df2)
Signed-off-by: Jason Gustafson <[email protected]>
commit aa1e4c235b882ae98a457cf9176f687951473cee
Author: Apurva Mehta <[email protected]>
Date: 2017-10-06T22:01:22Z
KAFKA-6015; Fix NPE in RecordAccumulator after ProducerId reset
It is possible for batches with sequence numbers to be in the `deque` while
at the same time the in flight batches in the `TransactionManager` are removed
due to a producerId reset.
In this case, when the batches in the `deque` are drained, we will get a
`NullPointerException` in the background thread due to this line:
```java
if (first.hasSequence() && first.baseSequence() !=
transactionManager.nextBatchBySequence(first.topicPartition).baseSequence())
```
Particularly, `transactionManager.nextBatchBySequence` will return null,
because there no inflight batches being tracked.
In this patch, we simply allow the batches in the `deque` to be drained if
there are no in flight batches being tracked in the TransactionManager. If they
succeed, well and good. If the responses come back with an error, the batces
will be ultimately failed in the producer with an `OutOfOrderSequenceException`
when the response comes back.
Author: Apurva Mehta <[email protected]>
Reviewers: Jason Gustafson <[email protected]>
Closes #4022 from apurvam/KAFKA-6015-npe-in-record-accumulator
(cherry picked from commit 105ab47ed90c8a0e83c159c97a8f2294c5582657)
Signed-off-by: Jason Gustafson <[email protected]>
commit 12c77b4b08aa0eb39320915a2796444420a05392
Author: Damian Guy <[email protected]>
Date: 2017-10-06T22:38:23Z
KAFKA-5651; Follow-up: add with method to Materialized
Add a `with(Serde keySerde, Serde valSerde)` to `Materialized` for cases
where people don't care about the state store name.
Author: Damian Guy <[email protected]>
Reviewers: Guozhang Wang <[email protected]>, Ismael Juma
<[email protected]>, Matthias J. Sax <[email protected]>
Closes #4009 from dguy/materialized
(cherry picked from commit 23a014052d39521a3af471b3f95809c2164820f7)
Signed-off-by: Guozhang Wang <[email protected]>
commit 963733cd0166ea47a26aaf32d876fdf3a8e086c5
Author: Guozhang Wang <[email protected]>
Date: 2017-10-06T22:42:18Z
MINOR: log4j improvements on assigned tasks and store changelog reader
Author: Guozhang Wang <[email protected]>
Reviewers: Matthias J. Sax <[email protected]>, Xavier Léauté
<[email protected]>, Damian Guy <[email protected]>, Bill Bejeck
<[email protected]>
Closes #4031 from guozhangwang/KMinor-assigned-task-log4j
(cherry picked from commit 2427a44768f9734179c957b645df0476e7cb6d05)
Signed-off-by: Guozhang Wang <[email protected]>
commit d3a603cb90fc4a0ac248569694f1f15c6e0dfbf1
Author: Ismael Juma <[email protected]>
Date: 2017-10-06T23:45:45Z
KAFKA-5829; Only delete producer snapshots before the recovery point
Author: Ismael Juma <[email protected]>
Reviewers: Jason Gustafson <[email protected]>
Closes #4023 from
ijuma/kafka-5829-avoid-reading-older-segments-on-hard-shutdown
(cherry picked from commit 91517e8fbd7767ba6d7f43b517f5a26b6f870585)
Signed-off-by: Jason Gustafson <[email protected]>
commit f899c735d35f264833d41d802f0ed4010b32905d
Author: Matthias J. Sax <[email protected]>
Date: 2017-10-07T00:48:34Z
KAFKA-5362; Follow up to Streams EOS system test
- improve tests to get rid of calls to `sleep` in Python
- fixed some flaky test conditions
- improve debugging
Author: Matthias J. Sax <[email protected]>
Reviewers: Damian Guy <[email protected]>, Bill Bejeck
<[email protected]>, Guozhang Wang <[email protected]>
Closes #3542 from mjsax/failing-eos-system-tests
(cherry picked from commit 51063441d3ed4dec8a96794f085028b4b8feb20c)
Signed-off-by: Guozhang Wang <[email protected]>
commit 5298731469379abcf197b26b57f89f94785fbc64
Author: Guozhang Wang <[email protected]>
Date: 2017-10-07T00:57:32Z
MINOR: KIP-161 upgrade docs change
Author: Guozhang Wang <[email protected]>
Reviewers: Damian Guy <[email protected]>
Closes #4036 from guozhangwang/KMinor-kip-161-docs
(cherry picked from commit e2e8d4a57a83693b946c3e26f9be2e41a7341ab4)
Signed-off-by: Guozhang Wang <[email protected]>
commit ae4f647dd2e2ea47220ed06b61f9a58ab317c637
Author: Xin Li <[email protected]>
Date: 2017-10-09T21:15:44Z
KAFKA-6030; Fix Integer overflow in cleanable ratio computation
Author: Xin Li <[email protected]>
Reviewers: Ismael Juma <[email protected]>, Jason Gustafson
<[email protected]>
Closes #4043 from lisa2lisa/fix
(cherry picked from commit bb27215ceac9caaac446b5fe43d3f3284624fcb4)
Signed-off-by: Jason Gustafson <[email protected]>
commit 03da215f627599240b1435469d5ac14e18f0285b
Author: bartdevylder <[email protected]>
Date: 2017-10-10T00:49:52Z
KAFKA-6026; Fix for indefinite wait in KafkaFutureImpl
Author: bartdevylder <[email protected]>
Author: Bart De Vylder <[email protected]>
Reviewers: Colin P. Mccabe <[email protected]>, Ismael Juma
<[email protected]>, Jason Gustafson <[email protected]>
Closes #4044 from bartdevylder/KAFKA-6026
commit dadae0a35e4e77c195c5655bc066ccf6e155f4ed
Author: Dong Lin <[email protected]>
Date: 2017-10-10T15:29:17Z
KAFKA-6027; Access to log should throw KafkaStorageException after the log
has been marked offline
Author: Dong Lin <[email protected]>
Reviewers: Ismael Juma <[email protected]>, Jun Rao <[email protected]>
Closes #4045 from lindong28/KAFKA-6027
(cherry picked from commit 29c46ddb91fac8c0fb3f35c1511e2afdcf67fd2e)
Signed-off-by: Jun Rao <[email protected]>
commit 1db2fc4fd72f3d3b4b737cac5dfd9c48b518747d
Author: Matthias J. Sax <[email protected]>
Date: 2017-10-10T17:03:02Z
MINOR: fix EOS test race condition
Author: Matthias J. Sax <[email protected]>
Reviewers: Damian Guy <[email protected]>, Bill Bejeck
<[email protected]>, Guozhang Wang <[email protected]>
Closes #4048 from mjsax/fix-eos-test-race-condition
(cherry picked from commit 64930cd71361853cc46ed02232213cc7ba749e77)
Signed-off-by: Guozhang Wang <[email protected]>
commit 8a175b0ee0839a1395c326a0adb1c0c2217b769e
Author: Apurva Mehta <[email protected]>
Date: 2017-10-10T17:31:52Z
KAFKA-6016; Make the reassign partitions system test use the idempotent
producer
With these changes, we are ensuring that the partitions being reassigned
are from non-zero offsets. We also ensure that every message in the log has
producerId and sequence number.
This means that it successfully reproduces
https://issues.apache.org/jira/browse/KAFKA-6003.
Author: Apurva Mehta <[email protected]>
Reviewers: Jason Gustafson <[email protected]>
Closes #4029 from
apurvam/KAFKA-6016-add-idempotent-producer-to-reassign-partitions
(cherry picked from commit 90b5ce3f04626ee24aebd1d06588489180a4bb05)
Signed-off-by: Jason Gustafson <[email protected]>
commit e409f847f549f4fe10c552110366c646c1e44138
Author: Jason Gustafson <[email protected]>
Date: 2017-10-09T20:03:20Z
MINOR: Factor out some common group/transactional fields in request objects
Author: Jason Gustafson <[email protected]>
Reviewers: tedyu <[email protected]>, Ismael Juma <[email protected]>
Closes #4047 from hachikuji/factor-out-some-common-fields
commit e671c1cd36caf3b767fb9cc4af58c25918292afe
Author: Jason Gustafson <[email protected]>
Date: 2017-10-10T22:01:10Z
MINOR: Avoid some unnecessary collection copies in KafkaApis
Author: Jason Gustafson <[email protected]>
Reviewers: Manikumar Reddy <[email protected]>, Ismael Juma
<[email protected]>
Closes #4035 from hachikuji/KAFKA-5547-followup and squashes the following
commits:
f6b04ce1a [Jason Gustafson] Add a couple missed common fields
d3473b14d [Jason Gustafson] Fix compilation errors and a few warnings
58a0ae695 [Jason Gustafson] MINOR: Avoid some unnecessary collection copies
in KafkaApis
(cherry picked from commit 1027ff3c769906b7b80582217cf5b4703fd6864d)
Signed-off-by: Jason Gustafson <[email protected]>
commit 85f6f8cbb353a7c6ec8606b088b1f35dfcf1f6ea
Author: Bill Bejeck <[email protected]>
Date: 2017-10-10T22:07:53Z
KAFKA-6025: small fix for streams tutorial
Author: Bill Bejeck <[email protected]>
Reviewers: Guozhang Wang <[email protected]>
Closes #4053 from bbejeck/KAFKA_6025_fix_streams_tutorial
(cherry picked from commit dac990aab31ab13ed08a6818e1516f4dd25331dd)
Signed-off-by: Guozhang Wang <[email protected]>
commit 6c1c5efe7f83afc733608122c270642ccdc97399
Author: Matthias J. Sax <[email protected]>
Date: 2017-10-10T22:13:19Z
MINOR: KIP-182 follow up; add deprecation annotations to test classes
Author: Matthias J. Sax <[email protected]>
Reviewers: Bill Bejeck <[email protected]>, Guozhang Wang
<[email protected]>
Closes #4051 from mjsax/minor-kip-182-follow-up
(cherry picked from commit a5a9a901ef78373cda4b743323ef9f6e6da0dbf6)
Signed-off-by: Guozhang Wang <[email protected]>
commit 2cf059f33d8dc64e9ee14769c60e7d6ff559ae55
Author: Guozhang Wang <[email protected]>
Date: 2017-10-11T21:20:41Z
HOTFIX: Updates on release.py before 1.0.0
Author: Guozhang Wang <[email protected]>
Reviewers: Ismael Juma <[email protected]>, Ewen Cheslack-Postava
<[email protected]>
Closes #4054 from guozhangwang/KMinor-pre-1.0-release
(cherry picked from commit 54ed3435bf0660295b0f0fee0df4c9761873225c)
Signed-off-by: Guozhang Wang <[email protected]>
commit 531dfd7268a5642f97e2bf79453b43ba307b9c0f
Author: Ewen Cheslack-Postava <[email protected]>
Date: 2017-10-11T21:32:40Z
MINOR: Push JMX metric name mangling into the JmxReporter (KIP-190 follow
up)
Author: Ewen Cheslack-Postava <[email protected]>
Reviewers: Rajini Sivaram <[email protected]>, Ismael Juma
<[email protected]>
Closes #3980 from ewencp/dont-mangle-names
(cherry picked from commit 7d6ca52a2751908c7fc6b752d70dfaaaaa9bbe8c)
Signed-off-by: Rajini Sivaram <[email protected]>
commit 334f8a48baba60a973880f73c6d9e98fb0785d52
Author: Manjula K <[email protected]>
Date: 2017-10-11T22:16:12Z
MINOR: Redesign of Streams page to include video & logos
guozhangwang Please review.
Author: Manjula K <[email protected]>
Reviewers: Guozhang Wang <[email protected]>
Closes #4059 from manjuapu/redesign-streams-page
(cherry picked from commit f6e724b127891bc950499fd7a6876e500ba78de3)
Signed-off-by: Guozhang Wang <[email protected]>
commit d5614b529274d7781356305fc5bb14bf80683727
Author: Apurva Mehta <[email protected]>
Date: 2017-10-12T09:05:14Z
KAFKA-6053; Fix NoSuchMethodError when creating ProducerRecords with older
client versions
Author: Apurva Mehta <[email protected]>
Reviewers: Ismael Juma <[email protected]>
Closes #4057 from
apurvam/KAFKA-6053-fix-no-such-method-error-in-producer-record
commit a04d18ffb37b32d0a2ffe395ae2a8b279dcf89d1
Author: Vahid Hashemian <[email protected]>
Date: 2017-10-12T09:09:16Z
KAFKA-6055; Fix typo in KAFKA_JVM_PERFORMANCE_OPTS for Windows
Author: Vahid Hashemian <[email protected]>
Reviewers: Ismael Juma <[email protected]>
Closes #4062 from vahidhashemian/KAFKA-6055
(cherry picked from commit bfa82fc0b63a5ad0121da6e2190a587eb672a1d5)
Signed-off-by: Ismael Juma <[email protected]>
commit c49c830690a09ab3925416df5b6429d96b034002
Author: Mickael Maison <[email protected]>
Date: 2017-10-12T09:12:46Z
MINOR: Fix log message in AbstractFetcherThread
Use Scala string templates instead of format
Author: Mickael Maison <[email protected]>
Reviewers: Ismael Juma <[email protected]>
Closes #4058 from mimaison/minor_AFT_logging
----
---
