[jira] [Created] (KAFKA-8145) Broker fails with FATAL Shutdown on Windows, log or index renaming fail

[prehistoricpenguin (JIRA)]

prehistoricpenguin created KAFKA-8145:
-

 Summary: Broker fails with FATAL Shutdown on Windows, log or index 
renaming fail
 Key: KAFKA-8145
 URL: https://issues.apache.org/jira/browse/KAFKA-8145
 Project: Kafka
  Issue Type: Bug
  Components: core, log
Affects Versions: 2.1.1, 2.0.1, 1.1.1, 1.1.0
Reporter: prehistoricpenguin


When consumer offset log cleaning or Kafka log rolling triggered on Windows(The 
default rolling time is 168 hours, eg 1 week). Kafka broker will shut down due 
to file renaming fail, on Windows is't invalid to rename a file when it's 
opened, so this issue is Windows specific.

A topical error log during log rolling:
{quote}at 
kafka.log.Log$$anonfun$deleteSegments$1$$anonfun$apply$mcI$sp$1.apply(Log.scala:1170)
at 
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at kafka.log.Log$$anonfun$deleteSegments$1.apply$mcI$sp(Log.scala:1170)
at kafka.log.Log$$anonfun$deleteSegments$1.apply(Log.scala:1161)
at kafka.log.Log$$anonfun$deleteSegments$1.apply(Log.scala:1161)
at kafka.log.Log.maybeHandleIOException(Log.scala:1678)
at kafka.log.Log.deleteSegments(Log.scala:1161)
at kafka.log.Log.deleteOldSegments(Log.scala:1156)
at kafka.log.Log.deleteRetentionMsBreachedSegments(Log.scala:1228)
at kafka.log.Log.deleteOldSegments(Log.scala:1222)
at kafka.log.LogManager$$anonfun$cleanupLogs$3.apply(LogManager.scala:854)
at kafka.log.LogManager$$anonfun$cleanupLogs$3.apply(LogManager.scala:852)
at 
scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733)
at scala.collection.immutable.List.foreach(List.scala:392)
at 
scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732)
at kafka.log.LogManager.cleanupLogs(LogManager.scala:852)
at kafka.log.LogManager$$anonfun$startup$1.apply$mcV$sp(LogManager.scala:385)
at kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:110)
at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:62)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown
 Source)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
 Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Suppressed: java.nio.file.FileSystemException: 
Z:\tmp\kafka-logs\test-0\.log -> 
Z:\tmp\kafka-logs\test-0\.log.deleted: The process cannot 
access the file because it is being used by another process.
at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
at sun.nio.fs.WindowsFileCopy.move(Unknown Source)
at sun.nio.fs.WindowsFileSystemProvider.move(Unknown Source)
at java.nio.file.Files.move(Unknown Source)
at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:694)
... 32 more{quote}
 
And the log from consumer offset log cleaner:
{quote}[2019-02-26 02:16:05,151] ERROR Failed to clean up log for 
__consumer_offsets-29 in dir C:\Program Files (x86)\...\tmp\kafka-logs due to 
IOException (kafka.server.LogDirFailureChannel)
java.nio.file.FileSystemException: C:\Program Files 
(x86)\...\tmp\kafka-logs\__consumer_offsets-29\.log.cleaned 
-> C:\Program Files 
(x86)...\tmp\kafka-logs\__consumer_offsets-29\.log.swap: 
The process cannot access the file because it is being used by another process.
at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:387)
at sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287)
at java.nio.file.Files.move(Files.java:1395)
at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:697)
at org.apache.kafka.common.record.FileRecords.renameTo(FileRecords.java:212)
at kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:415)
at kafka.log.Log.replaceSegments(Log.scala:1644)
at kafka.log.Cleaner.cleanSegments(LogCleaner.scala:535)
at kafka.log.Cleaner$$anonfun$doClean$4.apply(LogCleaner.scala:462)
at kafka.log.Cleaner$$anonfun$doClean$4.apply(LogCleaner.scala:461)
at scala.collection.immutable.List.foreach(List.scala:392)
at kafka.log.Cleaner.doClean(LogCleaner.scala:461)
at kafka.log.Cleaner.clean(LogCleaner.scala:438)
at kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:305)
at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:291)
at kafka.utils.ShutdownableT

[jira] [Created] (KAFKA-8146) WARNING: An illegal reflective access operation has occurred

[Abhi (JIRA)]

Abhi created KAFKA-8146:
---

 Summary: WARNING: An illegal reflective access operation has 
occurred
 Key: KAFKA-8146
 URL: https://issues.apache.org/jira/browse/KAFKA-8146
 Project: Kafka
  Issue Type: Bug
  Components: clients, core
Affects Versions: 2.1.1
 Environment: Java 11
Kafka v2.1.1
Reporter: Abhi


Hi,
I am running Kafka v2.1.1 and see below warnings at the startup of server and 
clients. What is the cause of these warnings and how they can be avoided or 
fixed?


*Client side:*
WARNING: Illegal reflective access by 
org.apache.kafka.common.network.SaslChannelBuilder 
(file:/local/kafka/kafka_installation/kafka_2.12-2.1.1/libs/kafka-clients-2.1.1.jar)
 to method sun.security.krb5.Config.getInstance()
WARNING: Please consider reporting this to the maintainers of 
org.apache.kafka.common.network.SaslChannelBuilder
WARNING: Use --illegal-access=warn to enable warnings of further illegal 
reflective access operations
WARNING: All illegal access operations will be denied in a future release


*Server side:*
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by 
org.apache.zookeeper.server.util.KerberosUtil 
(file:/local/kafka/kafka_installation/kafka_2.12-2.1.1/libs/zookeeper-3.4.13.jar)
 to method sun.security.krb5.Config.getInstance()
WARNING: Please consider reporting this to the maintainers of 
org.apache.zookeeper.server.util.KerberosUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal 
reflective access operations
WARNING: All illegal access operations will be denied in a future release





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Jenkins build is back to normal : kafka-trunk-jdk8 #3486

[Apache Jenkins Server]

See 

Jenkins build is back to normal : kafka-2.2-jdk8 #70

[Apache Jenkins Server]

See 

Re: Question on performance data for Kafka vs NATS

[Adam Bellemare]

One more thing to note:

You are looking at regular, base NATS. On its own, it is not a direct 1-1
comparison to Kafka because it lacks things like data retention, clustering
and replication. Instead, you would want to compare it to NATS-Streaming, (
https://github.com/nats-io/nats-streaming-server ). You can find a number
of more recent articles and comparisons by a simple web search.

With that being said, this is likely not the best venue for an in-depth
discussion on tradeoffs between the two (especially since I see you're
spanning two very large mailing lists).

Adam




On Fri, Mar 22, 2019 at 1:34 AM Hans Jespersen  wrote:

> Thats a 4.5 year old benchmark and it was run with a single broker node
> and only 1 producer and 1 consumer all running on a single MacBookPro.
> Definitely not the target production environment for Kafka.
>
> -hans
>
> > On Mar 21, 2019, at 11:43 AM, M. Manna  wrote:
> >
> > HI All,
> >
> > https://nats.io/about/
> >
> > this shows a general comparison of sender/receiver throughputs for NATS
> and
> > other messaging system including our favourite Kafka.
> >
> > It appears that Kafka, despite taking the 2nd place, has a very low
> > throughput. My question is, where does Kafka win over NATS? is it the
> > unique partitioning and delivery semantics? Or, is it something else.
> >
> > From what I can see, NATS has traditional pub/sub and queuing. But it
> > doesn't look like there is any proper retention system built for this.
> >
> > Has anyone come across this already?
> >
> > Thanks,
>

Jenkins build is back to normal : kafka-2.1-jdk8 #152

[Apache Jenkins Server]

See 

[jira] [Created] (KAFKA-8147) Add changelog topic configuration to KTable suppress

[Maarten (JIRA)]

Maarten created KAFKA-8147:
--

 Summary: Add changelog topic configuration to KTable suppress
 Key: KAFKA-8147
 URL: https://issues.apache.org/jira/browse/KAFKA-8147
 Project: Kafka
  Issue Type: Improvement
  Components: streams
Affects Versions: 2.1.1
Reporter: Maarten


The streams DSL does not provide a way to configure the changelog topic created 
by KTable.suppress.

>From the perspective of an external user this could be implemented similar to 
>the configuration of aggregate + materialized, i.e., 
{code:java}
changelogTopicConfigs = // Configs
materialized = Materialized.as(..).withLoggingEnabled(changelogTopicConfigs)
..
KGroupedStream.aggregate(..,materialized)
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KAFKA-8148) Kafka Group deletion

[Atul T Renapurkar (JIRA)]

Atul T Renapurkar created KAFKA-8148:


 Summary: Kafka Group deletion
 Key: KAFKA-8148
 URL: https://issues.apache.org/jira/browse/KAFKA-8148
 Project: Kafka
  Issue Type: Bug
  Components: consumer
Affects Versions: 2.0.0
Reporter: Atul T Renapurkar


When I was trying to delete group with below command 

 

./kafka-consumer-groups.sh   --bootstrap-server zookeeper1:2181 zookeeper2:2181 
zookeeper3:2181 --delete --group groupName

 

I am facing 

 

Error: Deletion of some consumer groups failed:

* Group 'groupName' could not be deleted due to: COORDINATOR_NOT_AVAILABLE
Please feel free If you required any additional information

Thanks and Regards,
Atul



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Add to contributors list

[Ivan Yurchenko]

Hello,

Could somebody add me to the Kafka contributor list so I can assign myself
tickets in JIRA? My ID/username is ivanyu.
Thank you.

Best,
Ivan

Add to contributors list

[Ivan Yurchenko]

Hello,

Could somebody add me to the Kafka contributor list so I can assign myself
tickets in JIRA? My ID/username is ivanyu.
Thank you.

Best,
Ivan

[jira] [Created] (KAFKA-8149) ERROR Disk error while writing to recovery point file

[wade wu (JIRA)]

wade wu created KAFKA-8149:
--

 Summary: ERROR Disk error while writing to recovery point file
 Key: KAFKA-8149
 URL: https://issues.apache.org/jira/browse/KAFKA-8149
 Project: Kafka
  Issue Type: Bug
  Components: core
Affects Versions: 1.1.1
 Environment: Windows
Reporter: wade wu


[2019-03-17 02:55:14,458] ERROR Disk error while writing to recovery point file 
in directory I:\data\Kafka\kafka-datalogs (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: 
H:\data\Kafka\kafka-datalogs\AzPubSubCompactTestNew1-0\01170892.snapshot
 at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
 at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
 at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
 at 
sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
 at 
sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(AbstractFileSystemProvider.java:108)
 at java.nio.file.Files.deleteIfExists(Files.java:1165)
 at 
kafka.log.ProducerStateManager$$anonfun$kafka$log$ProducerStateManager$$deleteSnapshotFiles$2.apply(ProducerStateManager.scala:458)
 at 
kafka.log.ProducerStateManager$$anonfun$kafka$log$ProducerStateManager$$deleteSnapshotFiles$2.apply(ProducerStateManager.scala:457)
 at 
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
 at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:35)
 at 
kafka.log.ProducerStateManager$.kafka$log$ProducerStateManager$$deleteSnapshotFiles(ProducerStateManager.scala:457)
 at 
kafka.log.ProducerStateManager$.deleteSnapshotsBefore(ProducerStateManager.scala:454)
 at 
kafka.log.ProducerStateManager.deleteSnapshotsBefore(ProducerStateManager.scala:763)
 at kafka.log.Log.deleteSnapshotsAfterRecoveryPointCheckpoint(Log.scala:1461)
 at 
kafka.log.LogManager$$anonfun$kafka$log$LogManager$$checkpointLogRecoveryOffsetsInDir$1$$anonfun$apply$29$$anonfun$apply$31.apply(LogManager.scala:577)
 at 
kafka.log.LogManager$$anonfun$kafka$log$LogManager$$checkpointLogRecoveryOffsetsInDir$1$$anonfun$apply$29$$anonfun$apply$31.apply(LogManager.scala:577)
 at scala.collection.immutable.List.foreach(List.scala:392)
 at 
kafka.log.LogManager$$anonfun$kafka$log$LogManager$$checkpointLogRecoveryOffsetsInDir$1$$anonfun$apply$29.apply(LogManager.scala:577)
 at 
kafka.log.LogManager$$anonfun$kafka$log$LogManager$$checkpointLogRecoveryOffsetsInDir$1$$anonfun$apply$29.apply(LogManager.scala:573)
 at scala.Option.foreach(Option.scala:257)
 at 
kafka.log.LogManager$$anonfun$kafka$log$LogManager$$checkpointLogRecoveryOffsetsInDir$1.apply(LogManager.scala:573)
 at 
kafka.log.LogManager$$anonfun$kafka$log$LogManager$$checkpointLogRecoveryOffsetsInDir$1.apply(LogManager.scala:572)
 at scala.Option.foreach(Option.scala:257)
 at 
kafka.log.LogManager.kafka$log$LogManager$$checkpointLogRecoveryOffsetsInDir(LogManager.scala:572)
 at 
kafka.log.LogManager$$anonfun$checkpointLogRecoveryOffsets$1.apply(LogManager.scala:556)
 at 
kafka.log.LogManager$$anonfun$checkpointLogRecoveryOffsets$1.apply(LogManager.scala:556)
 at 
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
 at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
 at kafka.log.LogManager.checkpointLogRecoveryOffsets(LogManager.scala:556)
 at kafka.log.LogManager.truncateTo(LogManager.scala:520)
 at 
kafka.cluster.Partition$$anonfun$truncateTo$1.apply$mcV$sp(Partition.scala:665)
 at kafka.cluster.Partition$$anonfun$truncateTo$1.apply(Partition.scala:665)
 at kafka.cluster.Partition$$anonfun$truncateTo$1.apply(Partition.scala:665)
 at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:250)
 at kafka.utils.CoreUtils$.inReadLock(CoreUtils.scala:256)
 at kafka.cluster.Partition.truncateTo(Partition.scala:664)
 at 
kafka.server.ReplicaFetcherThread$$anonfun$maybeTruncate$1.apply(ReplicaFetcherThread.scala:320)
 at 
kafka.server.ReplicaFetcherThread$$anonfun$maybeTruncate$1.apply(ReplicaFetcherThread.scala:301)
 at scala.collection.immutable.Map$Map2.foreach(Map.scala:137)
 at 
kafka.server.ReplicaFetcherThread.maybeTruncate(ReplicaFetcherThread.scala:301)
 at 
kafka.server.AbstractFetcherThread$$anonfun$maybeTruncate$1.apply$mcV$sp(AbstractFetcherThread.scala:133)
 at 
kafka.server.AbstractFetcherThread$$anonfun$maybeTruncate$1.apply(AbstractFetcherThread.scala:130)
 at 
kafka.server.AbstractFetcherThread$$anonfun$maybeTruncate$1.apply(AbstractFetcherThread.scala:130)
 at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:250)
 at 
kafka.server.AbstractFetcherThread.maybeTruncate(AbstractFetcherThread.scala:130)
 at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:100)
 at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [jira] [Created] (KAFKA-8149) ERROR Disk error while writing to recovery point file

[M. Manna]

This is probably a duplicate of Kafka-6188 and Kafka-1194

Please double check and verify.

Thanks,

On Fri, 22 Mar 2019 at 17:36, wade wu (JIRA)  wrote:

> wade wu created KAFKA-8149:
> --
>
>  Summary: ERROR Disk error while writing to recovery point file
>  Key: KAFKA-8149
>  URL: https://issues.apache.org/jira/browse/KAFKA-8149
>  Project: Kafka
>   Issue Type: Bug
>   Components: core
> Affects Versions: 1.1.1
>  Environment: Windows
> Reporter: wade wu
>
>
> [2019-03-17 02:55:14,458] ERROR Disk error while writing to recovery point
> file in directory I:\data\Kafka\kafka-datalogs
> (kafka.server.LogDirFailureChannel)
> java.nio.file.AccessDeniedException:
> H:\data\Kafka\kafka-datalogs\AzPubSubCompactTestNew1-0\01170892.snapshot
>  at
> sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
>  at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
>  at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
>  at
> sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
>  at
> sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(AbstractFileSystemProvider.java:108)
>  at java.nio.file.Files.deleteIfExists(Files.java:1165)
>  at
> kafka.log.ProducerStateManager$$anonfun$kafka$log$ProducerStateManager$$deleteSnapshotFiles$2.apply(ProducerStateManager.scala:458)
>  at
> kafka.log.ProducerStateManager$$anonfun$kafka$log$ProducerStateManager$$deleteSnapshotFiles$2.apply(ProducerStateManager.scala:457)
>  at
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
>  at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:35)
>  at
> kafka.log.ProducerStateManager$.kafka$log$ProducerStateManager$$deleteSnapshotFiles(ProducerStateManager.scala:457)
>  at
> kafka.log.ProducerStateManager$.deleteSnapshotsBefore(ProducerStateManager.scala:454)
>  at
> kafka.log.ProducerStateManager.deleteSnapshotsBefore(ProducerStateManager.scala:763)
>  at
> kafka.log.Log.deleteSnapshotsAfterRecoveryPointCheckpoint(Log.scala:1461)
>  at
> kafka.log.LogManager$$anonfun$kafka$log$LogManager$$checkpointLogRecoveryOffsetsInDir$1$$anonfun$apply$29$$anonfun$apply$31.apply(LogManager.scala:577)
>  at
> kafka.log.LogManager$$anonfun$kafka$log$LogManager$$checkpointLogRecoveryOffsetsInDir$1$$anonfun$apply$29$$anonfun$apply$31.apply(LogManager.scala:577)
>  at scala.collection.immutable.List.foreach(List.scala:392)
>  at
> kafka.log.LogManager$$anonfun$kafka$log$LogManager$$checkpointLogRecoveryOffsetsInDir$1$$anonfun$apply$29.apply(LogManager.scala:577)
>  at
> kafka.log.LogManager$$anonfun$kafka$log$LogManager$$checkpointLogRecoveryOffsetsInDir$1$$anonfun$apply$29.apply(LogManager.scala:573)
>  at scala.Option.foreach(Option.scala:257)
>  at
> kafka.log.LogManager$$anonfun$kafka$log$LogManager$$checkpointLogRecoveryOffsetsInDir$1.apply(LogManager.scala:573)
>  at
> kafka.log.LogManager$$anonfun$kafka$log$LogManager$$checkpointLogRecoveryOffsetsInDir$1.apply(LogManager.scala:572)
>  at scala.Option.foreach(Option.scala:257)
>  at
> kafka.log.LogManager.kafka$log$LogManager$$checkpointLogRecoveryOffsetsInDir(LogManager.scala:572)
>  at
> kafka.log.LogManager$$anonfun$checkpointLogRecoveryOffsets$1.apply(LogManager.scala:556)
>  at
> kafka.log.LogManager$$anonfun$checkpointLogRecoveryOffsets$1.apply(LogManager.scala:556)
>  at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
>  at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
>  at kafka.log.LogManager.checkpointLogRecoveryOffsets(LogManager.scala:556)
>  at kafka.log.LogManager.truncateTo(LogManager.scala:520)
>  at
> kafka.cluster.Partition$$anonfun$truncateTo$1.apply$mcV$sp(Partition.scala:665)
>  at
> kafka.cluster.Partition$$anonfun$truncateTo$1.apply(Partition.scala:665)
>  at
> kafka.cluster.Partition$$anonfun$truncateTo$1.apply(Partition.scala:665)
>  at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:250)
>  at kafka.utils.CoreUtils$.inReadLock(CoreUtils.scala:256)
>  at kafka.cluster.Partition.truncateTo(Partition.scala:664)
>  at
> kafka.server.ReplicaFetcherThread$$anonfun$maybeTruncate$1.apply(ReplicaFetcherThread.scala:320)
>  at
> kafka.server.ReplicaFetcherThread$$anonfun$maybeTruncate$1.apply(ReplicaFetcherThread.scala:301)
>  at scala.collection.immutable.Map$Map2.foreach(Map.scala:137)
>  at
> kafka.server.ReplicaFetcherThread.maybeTruncate(ReplicaFetcherThread.scala:301)
>  at
> kafka.server.AbstractFetcherThread$$anonfun$maybeTruncate$1.apply$mcV$sp(AbstractFetcherThread.scala:133)
>  at
> kafka.server.AbstractFetcherThread$$anonfun$maybeTruncate$1.apply(AbstractFetcherThread.scala:130)
>  at
> kafka.server.AbstractFetcherThread$$anonfun$maybeTruncate$1.apply(AbstractFetcherThread.scala:130)
>  at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:250)

Re: Add to contributors list

[Matthias J. Sax]

Done.

On 3/22/19 8:30 AM, Ivan Yurchenko wrote:
> Hello,
> 
> Could somebody add me to the Kafka contributor list so I can assign myself
> tickets in JIRA? My ID/username is ivanyu.
> Thank you.
> 
> Best,
> Ivan
> 



signature.asc
Description: OpenPGP digital signature

Re: [VOTE] 2.2.0 RC2

[Jason Gustafson]

+1

- Ran the basic quickstart
- Verified the release notes

Just one thing I noticed is that we don't have the usual upgrade
instructions. We can address this separately.

Thanks,
Jason



On Thu, Mar 21, 2019 at 3:41 PM Ewen Cheslack-Postava 
wrote:

> +1
>
> -Ewen
>
> On Thu, Mar 21, 2019 at 10:33 AM Harsha  wrote:
>
> > +1 (non-bidning)
> >  - Download artifacts, setup 3 node cluster
> > - Ran producer/consumer clients
> >
> > Thanks,
> > Harsha
> >
> > On Thu, Mar 21, 2019, at 5:54 AM, Andrew Schofield wrote:
> > > +1 (non-binding)
> > >
> > > - Downloaded the artifacts
> > > - Ran Kafka Connect connectors
> > >
> > > Thanks,
> > > Andrew Schofield
> > > IBM Event Streams
> > >
> > > On 19/03/2019, 19:13, "Manikumar"  wrote:
> > >
> > > +1 (non-binding)
> > >
> > > - Verified the artifacts, build from src, ran tests
> > > - Verified the quickstart, ran producer/consumer performance tests.
> > >
> > > Thanks for running release!.
> > >
> > > Thanks,
> > > Manikumar
> > >
> > > On Wed, Mar 20, 2019 at 12:19 AM David Arthur <
> > davidart...@apache.org>
> > > wrote:
> > >
> > > > +1
> > > >
> > > > Validated signatures, and ran through quick-start.
> > > >
> > > > Thanks!
> > > >
> > > > On Mon, Mar 18, 2019 at 4:00 AM Jakub Scholz 
> > > wrote:
> > > >
> > > > > +1 (non-binding). I used the staged binaries and run some of my
> > > tests
> > > > > against them. All seems to look good to me.
> > > > >
> > > > > On Sat, Mar 9, 2019 at 11:56 PM Matthias J. Sax
> > > 
> > > > > wrote:
> > > > >
> > > > > > Hello Kafka users, developers and client-developers,
> > > > > >
> > > > > > This is the third candidate for release of Apache Kafka
> 2.2.0.
> > > > > >
> > > > > >  - Added SSL support for custom principal name
> > > > > >  - Allow SASL connections to periodically re-authenticate
> > > > > >  - Command line tool bin/kafka-topics.sh adds AdminClient
> > > support
> > > > > >  - Improved consumer group management
> > > > > >- default group.id is `null` instead of empty string
> > > > > >  - API improvement
> > > > > >- Producer: introduce close(Duration)
> > > > > >- AdminClient: introduce close(Duration)
> > > > > >- Kafka Streams: new flatTransform() operator in Streams
> > > DSL
> > > > > >- KafkaStreams (and other classed) now implement
> > > AutoClosable to
> > > > > > support try-with-resource
> > > > > >- New Serdes and default method implementations
> > > > > >  - Kafka Streams exposed internal client.id via
> ThreadMetadata
> > > > > >  - Metric improvements:  All `-min`, `-avg` and `-max`
> > > metrics will now
> > > > > > output `NaN` as default value
> > > > > > Release notes for the 2.2.0 release:
> > > > > >
> > >
> >
> https://eur02.safelinks.protection.outlook.com/?url=https:%2F%2Fhome.apache.org%2F~mjsax%2Fkafka-2.2.0-rc2%2FRELEASE_NOTES.html&data=02%7C01%7C%7Cbc5822a806a749b0638208d6ac9ef756%7C84df9e7fe9f640afb435%7C1%7C0%7C636886196079314852&sdata=zBUbQlQiAuGZzs33TUPUqsuC8IpPavg2lT3yPFO%2F3nA%3D&reserved=0
> > > > > >
> > > > > > *** Please download, test, and vote by Thursday, March 14,
> > > 9am PST.
> > > > > >
> > > > > > Kafka's KEYS file containing PGP keys we use to sign the
> > > release:
> > > > > >
> > >
> >
> https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fkafka.apache.org%2FKEYS&data=02%7C01%7C%7Cbc5822a806a749b0638208d6ac9ef756%7C84df9e7fe9f640afb435%7C1%7C0%7C636886196079314852&sdata=g1Gg%2BoIRgpKUum5%2Bmi2plT1qIfH9d2aZkdK9jw7DLxM%3D&reserved=0
> > > > > >
> > > > > > * Release artifacts to be voted upon (source and binary):
> > > > > >
> > >
> >
> https://eur02.safelinks.protection.outlook.com/?url=https:%2F%2Fhome.apache.org%2F~mjsax%2Fkafka-2.2.0-rc2%2F&data=02%7C01%7C%7Cbc5822a806a749b0638208d6ac9ef756%7C84df9e7fe9f640afb435%7C1%7C0%7C636886196079324862&sdata=dUZrMCGvR4ki8XS%2B9dEDQ5Bavv4A4xq86CtcXQ6tnFs%3D&reserved=0
> > > > > >
> > > > > > * Maven artifacts to be voted upon:
> > > > > >
> > >
> >
> https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fkafka%2F&data=02%7C01%7C%7Cbc5822a806a749b0638208d6ac9ef756%7C84df9e7fe9f640afb435%7C1%7C0%7C636886196079324862&sdata=sCoRIXmcRQd473bRqwFgQaSm2XI%2BBqHw%2FbiddQd4hnE%3D&reserved=0
> > > > > >
> > > > > > * Javadoc:
> > > > > >
> > >
> >
> https://eur02.safelinks.protection.outlook.com/?url=https:%2F%2Fhome.apache.org%2F~mjsax%2Fkafka-2.2.0-rc2%2Fjavadoc%2F&data=02%7C01%7C%7Cbc5822a806a749b0638208d6ac9ef756%7C84df9e7fe9f640afb435%7C1%7C0%7C636886196079324862&sdata=iK4WEFuaK0lCySWROi7BbBv%2Bpg8h%2B9umbVNA7I1rqxc%3D&reserved=0
> > > > > >
> > > > > > * Tag to be voted upon (off 2.2 branch) is the 2.2.0 tag:
> > >

Re: [VOTE] 2.2.0 RC2

[Matthias J. Sax]

+1 (non-binding)

Closing this vote. Thanks a lot for verifying and testing the RC!


-Matthias

On 3/22/19 12:03 PM, Jason Gustafson wrote:
> +1
> 
> - Ran the basic quickstart
> - Verified the release notes
> 
> Just one thing I noticed is that we don't have the usual upgrade
> instructions. We can address this separately.
> 
> Thanks,
> Jason
> 
> 
> 
> On Thu, Mar 21, 2019 at 3:41 PM Ewen Cheslack-Postava 
> wrote:
> 
>> +1
>>
>> -Ewen
>>
>> On Thu, Mar 21, 2019 at 10:33 AM Harsha  wrote:
>>
>>> +1 (non-bidning)
>>>  - Download artifacts, setup 3 node cluster
>>> - Ran producer/consumer clients
>>>
>>> Thanks,
>>> Harsha
>>>
>>> On Thu, Mar 21, 2019, at 5:54 AM, Andrew Schofield wrote:
 +1 (non-binding)

 - Downloaded the artifacts
 - Ran Kafka Connect connectors

 Thanks,
 Andrew Schofield
 IBM Event Streams

 On 19/03/2019, 19:13, "Manikumar"  wrote:

 +1 (non-binding)

 - Verified the artifacts, build from src, ran tests
 - Verified the quickstart, ran producer/consumer performance tests.

 Thanks for running release!.

 Thanks,
 Manikumar

 On Wed, Mar 20, 2019 at 12:19 AM David Arthur <
>>> davidart...@apache.org>
 wrote:

 > +1
 >
 > Validated signatures, and ran through quick-start.
 >
 > Thanks!
 >
 > On Mon, Mar 18, 2019 at 4:00 AM Jakub Scholz 
 wrote:
 >
 > > +1 (non-binding). I used the staged binaries and run some of my
 tests
 > > against them. All seems to look good to me.
 > >
 > > On Sat, Mar 9, 2019 at 11:56 PM Matthias J. Sax
 
 > > wrote:
 > >
 > > > Hello Kafka users, developers and client-developers,
 > > >
 > > > This is the third candidate for release of Apache Kafka
>> 2.2.0.
 > > >
 > > >  - Added SSL support for custom principal name
 > > >  - Allow SASL connections to periodically re-authenticate
 > > >  - Command line tool bin/kafka-topics.sh adds AdminClient
 support
 > > >  - Improved consumer group management
 > > >- default group.id is `null` instead of empty string
 > > >  - API improvement
 > > >- Producer: introduce close(Duration)
 > > >- AdminClient: introduce close(Duration)
 > > >- Kafka Streams: new flatTransform() operator in Streams
 DSL
 > > >- KafkaStreams (and other classed) now implement
 AutoClosable to
 > > > support try-with-resource
 > > >- New Serdes and default method implementations
 > > >  - Kafka Streams exposed internal client.id via
>> ThreadMetadata
 > > >  - Metric improvements:  All `-min`, `-avg` and `-max`
 metrics will now
 > > > output `NaN` as default value
 > > > Release notes for the 2.2.0 release:
 > > >

>>>
>> https://eur02.safelinks.protection.outlook.com/?url=https:%2F%2Fhome.apache.org%2F~mjsax%2Fkafka-2.2.0-rc2%2FRELEASE_NOTES.html&data=02%7C01%7C%7Cbc5822a806a749b0638208d6ac9ef756%7C84df9e7fe9f640afb435%7C1%7C0%7C636886196079314852&sdata=zBUbQlQiAuGZzs33TUPUqsuC8IpPavg2lT3yPFO%2F3nA%3D&reserved=0
 > > >
 > > > *** Please download, test, and vote by Thursday, March 14,
 9am PST.
 > > >
 > > > Kafka's KEYS file containing PGP keys we use to sign the
 release:
 > > >

>>>
>> https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fkafka.apache.org%2FKEYS&data=02%7C01%7C%7Cbc5822a806a749b0638208d6ac9ef756%7C84df9e7fe9f640afb435%7C1%7C0%7C636886196079314852&sdata=g1Gg%2BoIRgpKUum5%2Bmi2plT1qIfH9d2aZkdK9jw7DLxM%3D&reserved=0
 > > >
 > > > * Release artifacts to be voted upon (source and binary):
 > > >

>>>
>> https://eur02.safelinks.protection.outlook.com/?url=https:%2F%2Fhome.apache.org%2F~mjsax%2Fkafka-2.2.0-rc2%2F&data=02%7C01%7C%7Cbc5822a806a749b0638208d6ac9ef756%7C84df9e7fe9f640afb435%7C1%7C0%7C636886196079324862&sdata=dUZrMCGvR4ki8XS%2B9dEDQ5Bavv4A4xq86CtcXQ6tnFs%3D&reserved=0
 > > >
 > > > * Maven artifacts to be voted upon:
 > > >

>>>
>> https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fkafka%2F&data=02%7C01%7C%7Cbc5822a806a749b0638208d6ac9ef756%7C84df9e7fe9f640afb435%7C1%7C0%7C636886196079324862&sdata=sCoRIXmcRQd473bRqwFgQaSm2XI%2BBqHw%2FbiddQd4hnE%3D&reserved=0
 > > >
 > > > * Javadoc:
 > > >

>>>
>> https://eur02.safelinks.protection.outlook.com/?url=https:%2F%2Fhome.apache.org%2F~mjsax%2Fkafka-2.2.0-rc2%2Fjavadoc%2F&data=02%7C01%7C%7Cbc5822a806a749b0638208d6ac9ef756%7C84df9e7fe9f640afb435%7C1%7C0%7C636886196079324862&sdata=iK4WEFuaK0lCySWROi7BbBv%2Bpg8h%2B9umbVNA7I1rqxc%3D&reserve

[RESULT ][VOTE] Release Kafka version 2.2.0

[Matthias J. Sax]

This vote passes with 7 +12 votes (3 bindings) and no 0 or -1 votes.

+1 votes
PMC Members:
* Gwen
* Rajini
* Jason

Committers:
* Manikumar
* Bill
* Harsha
* Ewen
* Matthias

Community:
* Jakob
* David
* Satish
* Andrew

0 votes
* No votes

-1 votes
* No votes

Vote thread:
http://mail-archives.apache.org/mod_mbox/kafka-dev/201903.mbox/%3C8b4e39f0-e906-ecf5-0af8-858fb8989271%40confluent.io%3E

I'll continue with the release process and the release announcement will
follow in the next few days.


-Matthias


On 3/22/19 1:11 PM, Matthias J. Sax wrote:
> +1 (non-binding)
> 
> Closing this vote. Thanks a lot for verifying and testing the RC!
> 
> 
> -Matthias
> 
> On 3/22/19 12:03 PM, Jason Gustafson wrote:
>> +1
>>
>> - Ran the basic quickstart
>> - Verified the release notes
>>
>> Just one thing I noticed is that we don't have the usual upgrade
>> instructions. We can address this separately.
>>
>> Thanks,
>> Jason
>>
>>
>>
>> On Thu, Mar 21, 2019 at 3:41 PM Ewen Cheslack-Postava 
>> wrote:
>>
>>> +1
>>>
>>> -Ewen
>>>
>>> On Thu, Mar 21, 2019 at 10:33 AM Harsha  wrote:
>>>
 +1 (non-bidning)
  - Download artifacts, setup 3 node cluster
 - Ran producer/consumer clients

 Thanks,
 Harsha

 On Thu, Mar 21, 2019, at 5:54 AM, Andrew Schofield wrote:
> +1 (non-binding)
>
> - Downloaded the artifacts
> - Ran Kafka Connect connectors
>
> Thanks,
> Andrew Schofield
> IBM Event Streams
>
> On 19/03/2019, 19:13, "Manikumar"  wrote:
>
> +1 (non-binding)
>
> - Verified the artifacts, build from src, ran tests
> - Verified the quickstart, ran producer/consumer performance tests.
>
> Thanks for running release!.
>
> Thanks,
> Manikumar
>
> On Wed, Mar 20, 2019 at 12:19 AM David Arthur <
 davidart...@apache.org>
> wrote:
>
> > +1
> >
> > Validated signatures, and ran through quick-start.
> >
> > Thanks!
> >
> > On Mon, Mar 18, 2019 at 4:00 AM Jakub Scholz 
> wrote:
> >
> > > +1 (non-binding). I used the staged binaries and run some of my
> tests
> > > against them. All seems to look good to me.
> > >
> > > On Sat, Mar 9, 2019 at 11:56 PM Matthias J. Sax
> 
> > > wrote:
> > >
> > > > Hello Kafka users, developers and client-developers,
> > > >
> > > > This is the third candidate for release of Apache Kafka
>>> 2.2.0.
> > > >
> > > >  - Added SSL support for custom principal name
> > > >  - Allow SASL connections to periodically re-authenticate
> > > >  - Command line tool bin/kafka-topics.sh adds AdminClient
> support
> > > >  - Improved consumer group management
> > > >- default group.id is `null` instead of empty string
> > > >  - API improvement
> > > >- Producer: introduce close(Duration)
> > > >- AdminClient: introduce close(Duration)
> > > >- Kafka Streams: new flatTransform() operator in Streams
> DSL
> > > >- KafkaStreams (and other classed) now implement
> AutoClosable to
> > > > support try-with-resource
> > > >- New Serdes and default method implementations
> > > >  - Kafka Streams exposed internal client.id via
>>> ThreadMetadata
> > > >  - Metric improvements:  All `-min`, `-avg` and `-max`
> metrics will now
> > > > output `NaN` as default value
> > > > Release notes for the 2.2.0 release:
> > > >
>

>>> https://eur02.safelinks.protection.outlook.com/?url=https:%2F%2Fhome.apache.org%2F~mjsax%2Fkafka-2.2.0-rc2%2FRELEASE_NOTES.html&data=02%7C01%7C%7Cbc5822a806a749b0638208d6ac9ef756%7C84df9e7fe9f640afb435%7C1%7C0%7C636886196079314852&sdata=zBUbQlQiAuGZzs33TUPUqsuC8IpPavg2lT3yPFO%2F3nA%3D&reserved=0
> > > >
> > > > *** Please download, test, and vote by Thursday, March 14,
> 9am PST.
> > > >
> > > > Kafka's KEYS file containing PGP keys we use to sign the
> release:
> > > >
>

>>> https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fkafka.apache.org%2FKEYS&data=02%7C01%7C%7Cbc5822a806a749b0638208d6ac9ef756%7C84df9e7fe9f640afb435%7C1%7C0%7C636886196079314852&sdata=g1Gg%2BoIRgpKUum5%2Bmi2plT1qIfH9d2aZkdK9jw7DLxM%3D&reserved=0
> > > >
> > > > * Release artifacts to be voted upon (source and binary):
> > > >
>

>>> https://eur02.safelinks.protection.outlook.com/?url=https:%2F%2Fhome.apache.org%2F~mjsax%2Fkafka-2.2.0-rc2%2F&data=02%7C01%7C%7Cbc5822a806a749b0638208d6ac9ef756%7C84df9e7fe9f640afb435%7C1%7C0%7C636886196079324862&sdata=dUZrMCGvR4ki8XS%2B9dEDQ5Bavv4A4xq86CtcXQ6tnFs%3D&reserved=0
> > > >
> > > > * Maven artifacts to be voted upon:
> > > >
>

>>> htt

Re: [RESULT ][VOTE] Release Kafka version 2.2.0

[Matthias J. Sax]

Updating a template is hard...

This vote passes with 12 +1 votes (3 bindings) and no 0 or -1 votes.


On 3/22/19 1:14 PM, Matthias J. Sax wrote:
> This vote passes with 7 +12 votes (3 bindings) and no 0 or -1 votes.
> 
> +1 votes
> PMC Members:
> * Gwen
> * Rajini
> * Jason
> 
> Committers:
> * Manikumar
> * Bill
> * Harsha
> * Ewen
> * Matthias
> 
> Community:
> * Jakob
> * David
> * Satish
> * Andrew
> 
> 0 votes
> * No votes
> 
> -1 votes
> * No votes
> 
> Vote thread:
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201903.mbox/%3C8b4e39f0-e906-ecf5-0af8-858fb8989271%40confluent.io%3E
> 
> I'll continue with the release process and the release announcement will
> follow in the next few days.
> 
> 
> -Matthias
> 
> 
> On 3/22/19 1:11 PM, Matthias J. Sax wrote:
>> +1 (non-binding)
>>
>> Closing this vote. Thanks a lot for verifying and testing the RC!
>>
>>
>> -Matthias
>>
>> On 3/22/19 12:03 PM, Jason Gustafson wrote:
>>> +1
>>>
>>> - Ran the basic quickstart
>>> - Verified the release notes
>>>
>>> Just one thing I noticed is that we don't have the usual upgrade
>>> instructions. We can address this separately.
>>>
>>> Thanks,
>>> Jason
>>>
>>>
>>>
>>> On Thu, Mar 21, 2019 at 3:41 PM Ewen Cheslack-Postava 
>>> wrote:
>>>
 +1

 -Ewen

 On Thu, Mar 21, 2019 at 10:33 AM Harsha  wrote:

> +1 (non-bidning)
>  - Download artifacts, setup 3 node cluster
> - Ran producer/consumer clients
>
> Thanks,
> Harsha
>
> On Thu, Mar 21, 2019, at 5:54 AM, Andrew Schofield wrote:
>> +1 (non-binding)
>>
>> - Downloaded the artifacts
>> - Ran Kafka Connect connectors
>>
>> Thanks,
>> Andrew Schofield
>> IBM Event Streams
>>
>> On 19/03/2019, 19:13, "Manikumar"  wrote:
>>
>> +1 (non-binding)
>>
>> - Verified the artifacts, build from src, ran tests
>> - Verified the quickstart, ran producer/consumer performance tests.
>>
>> Thanks for running release!.
>>
>> Thanks,
>> Manikumar
>>
>> On Wed, Mar 20, 2019 at 12:19 AM David Arthur <
> davidart...@apache.org>
>> wrote:
>>
>> > +1
>> >
>> > Validated signatures, and ran through quick-start.
>> >
>> > Thanks!
>> >
>> > On Mon, Mar 18, 2019 at 4:00 AM Jakub Scholz 
>> wrote:
>> >
>> > > +1 (non-binding). I used the staged binaries and run some of my
>> tests
>> > > against them. All seems to look good to me.
>> > >
>> > > On Sat, Mar 9, 2019 at 11:56 PM Matthias J. Sax
>> 
>> > > wrote:
>> > >
>> > > > Hello Kafka users, developers and client-developers,
>> > > >
>> > > > This is the third candidate for release of Apache Kafka
 2.2.0.
>> > > >
>> > > >  - Added SSL support for custom principal name
>> > > >  - Allow SASL connections to periodically re-authenticate
>> > > >  - Command line tool bin/kafka-topics.sh adds AdminClient
>> support
>> > > >  - Improved consumer group management
>> > > >- default group.id is `null` instead of empty string
>> > > >  - API improvement
>> > > >- Producer: introduce close(Duration)
>> > > >- AdminClient: introduce close(Duration)
>> > > >- Kafka Streams: new flatTransform() operator in Streams
>> DSL
>> > > >- KafkaStreams (and other classed) now implement
>> AutoClosable to
>> > > > support try-with-resource
>> > > >- New Serdes and default method implementations
>> > > >  - Kafka Streams exposed internal client.id via
 ThreadMetadata
>> > > >  - Metric improvements:  All `-min`, `-avg` and `-max`
>> metrics will now
>> > > > output `NaN` as default value
>> > > > Release notes for the 2.2.0 release:
>> > > >
>>
>
 https://eur02.safelinks.protection.outlook.com/?url=https:%2F%2Fhome.apache.org%2F~mjsax%2Fkafka-2.2.0-rc2%2FRELEASE_NOTES.html&data=02%7C01%7C%7Cbc5822a806a749b0638208d6ac9ef756%7C84df9e7fe9f640afb435%7C1%7C0%7C636886196079314852&sdata=zBUbQlQiAuGZzs33TUPUqsuC8IpPavg2lT3yPFO%2F3nA%3D&reserved=0
>> > > >
>> > > > *** Please download, test, and vote by Thursday, March 14,
>> 9am PST.
>> > > >
>> > > > Kafka's KEYS file containing PGP keys we use to sign the
>> release:
>> > > >
>>
>
 https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fkafka.apache.org%2FKEYS&data=02%7C01%7C%7Cbc5822a806a749b0638208d6ac9ef756%7C84df9e7fe9f640afb435%7C1%7C0%7C636886196079314852&sdata=g1Gg%2BoIRgpKUum5%2Bmi2plT1qIfH9d2aZkdK9jw7DLxM%3D&reserved=0
>> > > >
>> > > > * Release artifacts to be voted upon (source and binary):
>> > > >
>>
>
 https://eur02.safelinks.protection.outlook.com/?u

[jira] [Created] (KAFKA-8150) Generated protocol class toString functions get NPEs when array fields are null

[Colin P. McCabe (JIRA)]

Colin P. McCabe created KAFKA-8150:
--

 Summary: Generated protocol class toString functions get NPEs when 
array fields are null
 Key: KAFKA-8150
 URL: https://issues.apache.org/jira/browse/KAFKA-8150
 Project: Kafka
  Issue Type: Bug
Reporter: Colin P. McCabe
Assignee: Colin P. McCabe


Generated protocol class toString functions get NPEs when array fields are null.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)