KStream: Error Reading and Writing Avro records

I’m trying to write avro record that I read from a topic into another topic, intentions it to augment it with transformation after I get this routing working. I have used the KStream with avro code from one of the example with some modifications to connect to Schema Registry for retrieving the

Re: Comparing Pulsar and Kafka: unified queuing and streaming

Hi Khurrum, Is ready now. https://github.com/Landoop/stream-reactor Regards Andrew From: Khurrum Nasim Sent: Thursday, 7 December, 08:36 Subject: Re: Comparing Pulsar and Kafka: unified queuing and streaming To: dev@kafka.apache.org Cc: us...@kafka.apache.org Andrew, Thank you! Is there any

[GitHub] kafka pull request #4304: KAFKA-6323: document that punctuation is called im...

GitHub user fredfp opened a pull request: https://github.com/apache/kafka/pull/4304 KAFKA-6323: document that punctuation is called immediately. If KAFKA-6323 is not a bug, then it needs better documentation. Alternative to https://github.com/apache/kafka/pull/4301

Jenkins build is back to normal : kafka-trunk-jdk9 #246

See

[jira] [Created] (KAFKA-6329) Load trust store as a resource

Allen Wang created KAFKA-6329: - Summary: Load trust store as a resource Key: KAFKA-6329 URL: https://issues.apache.org/jira/browse/KAFKA-6329 Project: Kafka Issue Type: Improvement

[GitHub] kafka pull request #4242: KAFKA-4857: Replace StreamsKafkaClient with AdminC...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4242 ---

Re: [DISCUSS] KIP-227: Introduce Incremental FetchRequests to Increase Partition Scalability

On Wed, Dec 6, 2017, at 17:07, Jun Rao wrote: > Hi, Collin, > > Thanks for the KIP. A few comments below. > > 20. Not sure that I fully understand session ID and session epoch. Is > session ID tied to a socket connection? Hi Jun, No, the session ID is not tied to a socket connection. > That

[jira] [Created] (KAFKA-6328) Exclude node groups belonging to global stores in InternalTopologyBuilder#makeNodeGroups

Guozhang Wang created KAFKA-6328: Summary: Exclude node groups belonging to global stores in InternalTopologyBuilder#makeNodeGroups Key: KAFKA-6328 URL: https://issues.apache.org/jira/browse/KAFKA-6328

Re: [DISCUSS] KIP-227: Introduce Incremental FetchRequests to Increase Partition Scalability

On Thu, Dec 7, 2017, at 08:57, Jason Gustafson wrote: > Hey Colin, > > A full fetch request will certainly avoid any ambiguity here. But now > > we're back to sending full fetch requests whenever there are network > > issues, which is worse than the current proposal. And has the > > congestion

Re: [DISCUSS] KIP-227: Introduce Incremental FetchRequests to Increase Partition Scalability

On Wed, Dec 6, 2017, at 11:23, Becket Qin wrote: > Hi Colin, > > >A full fetch request will certainly avoid any ambiguity here. But now > >we're back to sending full fetch requests whenever there are network > >issues, which is worse than the current proposal. And has the > >congestion collapse

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

Thank you Bill, I think this is reasonable. Do you have any suggestion for handling oldValues in cases like builder.table().filter(RichPredicate).join() where we process a Change with old and new value and dont have a record context for old. my suggestion would be that instead of SOURCE ->

[GitHub] kafka pull request #4303: KAFKA-6319: Quote strings stored in JSON configs

GitHub user rajinisivaram opened a pull request: https://github.com/apache/kafka/pull/4303 KAFKA-6319: Quote strings stored in JSON configs This is required for ACLs where SSL principals contain special characters (e.g. comma) that are escaped using backslash. The strings need to

[jira] [Created] (KAFKA-6327) IllegalArgumentException in RocksDB when RocksDBException being generated

Anthony May created KAFKA-6327: -- Summary: IllegalArgumentException in RocksDB when RocksDBException being generated Key: KAFKA-6327 URL: https://issues.apache.org/jira/browse/KAFKA-6327 Project: Kafka

Re: [DISCUSS] KIP-228 Negative record timestamp support

Hi Matthias, Indeed for consumers it will be not obvious what −1 means: actual timestamp or no timestamp. Nevertheless, it's just −1 millisecond, so I thought it will be not a big deal to leave it (not clean, but acceptable). I agree that it will much cleaner to have a different type of topics

Re: [DISCUSS] KIP-232: Detect outdated metadata by adding ControllerMetadataEpoch field

Hey Jun, Thanks much for the comments. Great point particularly regarding (3). I haven't thought about this before. It seems that there are two possible ways where the version number can be used. One solution is for client to check the version number at the time it receives MetadataResponse. And

Re: Pulsar and Kafka - Segment Centric vs Partition Centric

Thanks, Brett. I am also a Kafka user following a Kafka mailing list and asking help from Kafka community. Isn't this the place for kafka users to ask questions and seek help? Also I am not an infrastructure guy. That's why I was looking to hear thoughts from Kafka community. I was hoping kafka

Re: [DISCUSS] KIP-227: Introduce Incremental FetchRequests to Increase Partition Scalability

Hey Colin, A full fetch request will certainly avoid any ambiguity here. But now > we're back to sending full fetch requests whenever there are network > issues, which is worse than the current proposal. And has the > congestion collapse problem I talked about earlier when the network is >

RE: [DISCUSS]KIP-235 DNS alias and secured connections

Yes, it would address it I think. Jonathan Skrzypek Middleware Engineering Messaging Engineering Goldman Sachs International Christchurch Court - 10-15 Newgate Street London EC1A 7HD Tel: +442070512977 -Original Message- From: Stephane Maarek [mailto:steph...@simplemachines.com.au]

[jira] [Created] (KAFKA-6326) when broker is unavailable(such as broker's machine is down), controller will wait 30 sec timeout

HongLiang created KAFKA-6326: Summary: when broker is unavailable(such as broker's machine is down), controller will wait 30 sec timeout Key: KAFKA-6326 URL: https://issues.apache.org/jira/browse/KAFKA-6326

[GitHub] kafka pull request #4302: when broker is unavailable, fast revover

GitHub user leonhong opened a pull request: https://github.com/apache/kafka/pull/4302 when broker is unavailable, fast revover - when broker is unavailable(such as broker's machine is down), controller will wait 30 sec timeout by dedault. it seems to be that the timeout waiting is

[jira] [Created] (KAFKA-6325) Producer.flush() doesn't throw exception on timeout

Erik Scheuter created KAFKA-6325: Summary: Producer.flush() doesn't throw exception on timeout Key: KAFKA-6325 URL: https://issues.apache.org/jira/browse/KAFKA-6325 Project: Kafka Issue

Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore

Yes, Matthias. The object will be used togerther with function table and function stream. I didn’t see how this will affect other part but if you do, please explain more on how this will affect generated topic name. Thank you Panuwat On Thu, 7 Dec 2560 at 00:01 Matthias Margush

Re: [DISCUSS] KIP-213 Support non-key joining in KTable

On 05.12.2017 00:42, Matthias J. Sax wrote: Jan, The KTableValueGetter thing is a valid point. I think we would need a backwards mapper (or merge both into one and sacrifices lambdas?). Another alternative would be, to drop the optimization and materialize the KTable.operator() result... (not

Jenkins build is back to normal : kafka-trunk-jdk9 #244

See

[jira] [Created] (KAFKA-6324) Change LogSegment.delete to deleteIfExists and harden log recovery

Ismael Juma created KAFKA-6324: -- Summary: Change LogSegment.delete to deleteIfExists and harden log recovery Key: KAFKA-6324 URL: https://issues.apache.org/jira/browse/KAFKA-6324 Project: Kafka

Jenkins build is back to normal : kafka-trunk-jdk7 #3025

See

[jira] [Resolved] (KAFKA-6313) Kafka Core should have explicit SLF4J API dependency

[ https://issues.apache.org/jira/browse/KAFKA-6313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-6313. Resolution: Fixed > Kafka Core should have explicit SLF4J API dependency >

[GitHub] kafka pull request #4296: KAFKA-6313: Add SLF4J as direct dependency to Kafk...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4296 ---

[GitHub] kafka pull request #4301: KAFKA-6323: punctuate with WALL_CLOCK_TIME trigger...

GitHub user fredfp opened a pull request: https://github.com/apache/kafka/pull/4301 KAFKA-6323: punctuate with WALL_CLOCK_TIME triggered immediately This is the only way I found to fix the issue without altering the API. @mihbor @mjsax the contribution is my