[jira] [Created] (KAFKA-5450) Scripts to startup Connect in system tests have too short a timeout

2017-06-14 Thread Randall Hauch (JIRA)
Randall Hauch created KAFKA-5450: Summary: Scripts to startup Connect in system tests have too short a timeout Key: KAFKA-5450 URL: https://issues.apache.org/jira/browse/KAFKA-5450 Project: Kafka

[jira] [Commented] (KAFKA-5412) Using connect-console-sink/source.properties raises an exception related to "file" property not found

2017-06-12 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16046700#comment-16046700 ] Randall Hauch commented on KAFKA-5412: -- [~ppatierno], the committer that reviews and merges the PR

[jira] [Updated] (KAFKA-5412) Using connect-console-sink/source.properties raises an exception related to "file" property not found

2017-06-09 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-5412: - Description: With the latest 0.11.1.0-SNAPSHOT it happens that the Kafka Connect example using

[jira] [Updated] (KAFKA-5412) Using connect-console-sink/source.properties raises an exception related to "file" property not found

2017-06-09 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-5412: - Description: Hi, with the latest 0.11.1.0-SNAPSHOT it happens that the Kafka Connect example

[jira] [Commented] (KAFKA-5412) Using connect-console-sink/source.properties raises an exception related to "file" property not found

2017-06-09 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16044608#comment-16044608 ] Randall Hauch commented on KAFKA-5412: -- [~ppatierno], you might want to use "Submit Patch&qu

[jira] [Updated] (KAFKA-5412) Using connect-console-sink/source.properties raises an exception related to "file" property not found

2017-06-09 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-5412: - Affects Version/s: (was: 0.11.0.0) 0.10.2.1 > Using connect-cons

[jira] [Commented] (KAFKA-5412) Using connect-console-sink/source.properties raises an exception related to "file" property not found

2017-06-09 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16044606#comment-16044606 ] Randall Hauch commented on KAFKA-5412: -- I've initially targeted this to 0.11.1.0, since I think

[jira] [Updated] (KAFKA-5412) Using connect-console-sink/source.properties raises an exception related to "file" property not found

2017-06-09 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-5412: - Fix Version/s: 0.11.1.0 > Using connect-console-sink/source.properties raises an except

[jira] [Commented] (KAFKA-5412) Using connect-console-sink/source.properties raises an exception related to "file" property not found

2017-06-09 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043986#comment-16043986 ] Randall Hauch commented on KAFKA-5412: -- Please fix both the FileStreamSink and FileStreamSource. BTW

[jira] [Commented] (KAFKA-5412) Using connect-console-sink/source.properties raises an exception related to "file" property not found

2017-06-09 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043984#comment-16043984 ] Randall Hauch commented on KAFKA-5412: -- [~ppatierno], thanks! Go ahead and submit a pull request

[jira] [Updated] (KAFKA-5412) Using connect-console-sink/source.properties raises an exception related to "file" property not found

2017-06-08 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-5412: - Affects Version/s: (was: 0.11.1.0) 0.11.0.0 > Using connect-cons

[jira] [Commented] (KAFKA-5412) Using connect-console-sink/source.properties raises an exception related to "file" property not found

2017-06-08 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043018#comment-16043018 ] Randall Hauch commented on KAFKA-5412: -- I believe the proper correction would be to add a null

[jira] [Assigned] (KAFKA-5164) SetSchemaMetadata does not replace the schemas in structs correctly

2017-06-01 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch reassigned KAFKA-5164: Assignee: Randall Hauch > SetSchemaMetadata does not replace the schemas in stru

[jira] [Commented] (KAFKA-5352) Support automatic restart of failed tasks

2017-05-31 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16031472#comment-16031472 ] Randall Hauch commented on KAFKA-5352: -- Potentially related to KAFKA-3819, and it's possible a single

[jira] [Commented] (KAFKA-5295) Allow Kafka Connect source connectors to specify topic-specific settings for new topics

2017-05-30 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029625#comment-16029625 ] Randall Hauch commented on KAFKA-5295: -- Created [KIP-158|https://cwiki.apache.org/confluence/display

Re: [DISCUSS] KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics

2017-05-23 Thread Randall Hauch
the topic name will be used, and no other rules will be > applied. Is that correct? (I wasn't sure if it might cascade > together multiple matching rules...) > > Looks great, > > Mathieu > > > On Mon, May 22, 2017 at 1:43 PM, Randall Hauch <rha...@gmail.com> wrote:

[jira] [Comment Edited] (KAFKA-3821) Allow Kafka Connect source tasks to produce offset without writing to topics

2017-05-22 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16020279#comment-16020279 ] Randall Hauch edited comment on KAFKA-3821 at 5/22/17 10:12 PM: [~ewencp

[jira] [Commented] (KAFKA-3821) Allow Kafka Connect source tasks to produce offset without writing to topics

2017-05-22 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16020279#comment-16020279 ] Randall Hauch commented on KAFKA-3821: -- [~ewencp], the more I think about this issue, the more I

[jira] [Comment Edited] (KAFKA-5275) Review and potentially tweak AdminClient API for the initial release (KIP-117)

2017-05-22 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017701#comment-16017701 ] Randall Hauch edited comment on KAFKA-5275 at 5/22/17 8:55 PM: --- [~cmccabe

Re: Kafka Connect: To much restarting with a SourceConnector with dynamic set of tasks

2017-05-22 Thread Randall Hauch
You're not doing anything wrong, but I suspect you're requesting task reconfiguration more frequently than was originally envisioned, which means that the current implementation is not as optimal for your case. I'm not sure how much effort is required to implement this new behavior. The logic for

[jira] [Comment Edited] (KAFKA-3821) Allow Kafka Connect source tasks to produce offset without writing to topics

2017-05-22 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16020100#comment-16020100 ] Randall Hauch edited comment on KAFKA-3821 at 5/22/17 8:13 PM: --- The problem

[jira] [Commented] (KAFKA-3821) Allow Kafka Connect source tasks to produce offset without writing to topics

2017-05-22 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16020100#comment-16020100 ] Randall Hauch commented on KAFKA-3821: -- The problem with the connector directly using

[jira] [Comment Edited] (KAFKA-5275) Review and potentially tweak AdminClient API for the initial release (KIP-117)

2017-05-22 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16020058#comment-16020058 ] Randall Hauch edited comment on KAFKA-5275 at 5/22/17 7:48 PM: --- [~cmccabe

[jira] [Comment Edited] (KAFKA-5275) Review and potentially tweak AdminClient API for the initial release (KIP-117)

2017-05-22 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16020058#comment-16020058 ] Randall Hauch edited comment on KAFKA-5275 at 5/22/17 7:48 PM: --- [~cmccabe

[DISCUSS] KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics

2017-05-22 Thread Randall Hauch
Hi, all. We recently added the ability for Kafka Connect to create *internal* topics using the new AdminClient, but it still would be great if Kafka Connect could do this for new topics that result from source connector records. I've outlined an approach to do this in "KIP-158 Kafka Connect

[jira] [Commented] (KAFKA-5275) Review and potentially tweak AdminClient API for the initial release (KIP-117)

2017-05-22 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16020058#comment-16020058 ] Randall Hauch commented on KAFKA-5275: -- [~cmccabe] wrote: {quote} Hmm. Methods like these, can't

[jira] [Commented] (KAFKA-5275) Review and potentially tweak AdminClient API for the initial release (KIP-117)

2017-05-19 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017701#comment-16017701 ] Randall Hauch commented on KAFKA-5275: -- [~colinmccabe] wrote: {quote} We could add an option

[jira] [Created] (KAFKA-5295) Allow Kafka Connect source connectors to specify topic-specific settings for new topics

2017-05-19 Thread Randall Hauch (JIRA)
Randall Hauch created KAFKA-5295: Summary: Allow Kafka Connect source connectors to specify topic-specific settings for new topics Key: KAFKA-5295 URL: https://issues.apache.org/jira/browse/KAFKA-5295

[jira] [Commented] (KAFKA-5275) Review and potentially tweak AdminClient API for the initial release (KIP-117)

2017-05-18 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16016513#comment-16016513 ] Randall Hauch commented on KAFKA-5275: -- Also, a common use case for creating topics is going

[jira] [Commented] (KAFKA-5275) Review and potentially tweak AdminClient API for the initial release (KIP-117)

2017-05-18 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16016495#comment-16016495 ] Randall Hauch commented on KAFKA-5275: -- It'd be great to simplify the creation of the {{NewTopic

Re: [DISCUSS] KIP-154 Add Kafka Connect configuration properties for creating internal topics

2017-05-08 Thread Randall Hauch
wrote: > So, when Kafka broker is less than 3, and the user has not set the > replication configuration it will throw an error to the user, to correct > the configuration according to his setup? Is this the approach you are > suggesting here? > > > > On Mon, May 8, 2017

Re: [DISCUSS] KIP-154 Add Kafka Connect configuration properties for creating internal topics

2017-05-08 Thread Randall Hauch
introduced config > > properties. > > > > This is by no means a blocking comment. > > > > Thanks, > > Konstantine > > > > On Fri, May 5, 2017 at 7:18 PM, Randall Hauch <rha...@gmail.com> wrote: > > > > > Thanks, Gwen. > > &g

[VOTE] KIP-154 Add Kafka Connect configuration properties for creating internal topics

2017-05-08 Thread Randall Hauch
Hi, everyone. Given the simple and non-controversial nature of the KIP, I would like to start the voting process for KIP-154: Add Kafka Connect configuration properties for creating internal topics:

Re: [DISCUSS] KIP-154 Add Kafka Connect configuration properties for creating internal topics

2017-05-05 Thread Randall Hauch
on I recommend leaving them out of the example > config files - we already have issues with users playing with configs > without understanding what they are doing and not liking the results. > >> On Fri, May 5, 2017 at 3:42 PM, Randall Hauch <rha...@gmail.com> wrote: >&g

[jira] [Comment Edited] (KAFKA-4667) Connect should create internal topics

2017-05-05 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999061#comment-15999061 ] Randall Hauch edited comment on KAFKA-4667 at 5/5/17 10:45 PM: --- Per [~ewencp

[DISCUSS] KIP-154 Add Kafka Connect configuration properties for creating internal topics

2017-05-05 Thread Randall Hauch
Hi, all. I've been working on KAFKA-4667 to change the distributed worker of Kafka Connect to look for the topics used to store connector and task configurations, offsets, and status, and if those tasks do not exist to create them using the new AdminClient. To make this as useful as possible and

[jira] [Commented] (KAFKA-4667) Connect should create internal topics

2017-05-05 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999061#comment-15999061 ] Randall Hauch commented on KAFKA-4667: -- Per [~ewencp], I created [KIP-153|https://cwiki.apache.org

Please give me permission to create/edit Kafka Improvement Proposals

2017-05-05 Thread Randall Hauch
See subject. I'm a contributor to Kafka Connect. Thanks! Randall

[jira] [Work started] (KAFKA-4667) Connect should create internal topics

2017-05-05 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4667 started by Randall Hauch. > Connect should create internal top

[jira] [Assigned] (KAFKA-4667) Connect should create internal topics

2017-05-05 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch reassigned KAFKA-4667: Assignee: Randall Hauch > Connect should create internal top

Re: Kafka Connect and Partitions

2017-05-02 Thread Randall Hauch
chema, Object value) > > Many thanks once again for your guidance. I think this puzzle is now > solved :) > Best wishes, > David > > -Original Message- > From: Randall Hauch [mailto:rha...@gmail.com] > Sent: 28 April 2017 16:08 > To: dev@kafka.apache.org

[jira] [Comment Edited] (KAFKA-4667) Connect should create internal topics

2017-05-01 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991697#comment-15991697 ] Randall Hauch edited comment on KAFKA-4667 at 5/1/17 10:47 PM

[jira] [Commented] (KAFKA-4667) Connect should create internal topics

2017-05-01 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991697#comment-15991697 ] Randall Hauch commented on KAFKA-4667: -- The AdminClient pull request is being added as part of [PR

Re: [DISCUSS] KIP-146: Classloading Isolation in Connect

2017-05-01 Thread Randall Hauch
Very nice work, Konstantine. Conflicting dependencies of connectors is indeed a big issue that makes it hard to manage installed connectors. I do like Gwen's idea about removing the 'module.isolation.enabled' property. However, I would have anticipated always using classpath isolation for *only*

Re: Kafka Connect and Partitions

2017-04-28 Thread Randall Hauch
The source connector creates SourceRecord object and can set a number of fields, including the message's key and value, the Kafka topic name and, if desired, the Kafka topic partition number. If the connector does se the the topic partition to a non-null value, then that's the partition to which

[jira] [Comment Edited] (KAFKA-3821) Allow Kafka Connect source tasks to produce offset without writing to topics

2017-04-18 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973506#comment-15973506 ] Randall Hauch edited comment on KAFKA-3821 at 4/18/17 9:17 PM: --- [~criccomini

[jira] [Commented] (KAFKA-3821) Allow Kafka Connect source tasks to produce offset without writing to topics

2017-04-18 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973506#comment-15973506 ] Randall Hauch commented on KAFKA-3821: -- [~criccomini] logged KAFKA-5084, but that is really another

[jira] [Commented] (KAFKA-5084) Allow Kafka connect source tasks to commit offsets without messages being sent

2017-04-18 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973500#comment-15973500 ] Randall Hauch commented on KAFKA-5084: -- Looking back, it seems that this may actually be a duplicate

[jira] [Commented] (KAFKA-5084) Allow Kafka connect source tasks to commit offsets without messages being sent

2017-04-18 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973498#comment-15973498 ] Randall Hauch commented on KAFKA-5084: -- A bit more background may be in order. The Debezium MySQL

[jira] [Updated] (KAFKA-3832) Kafka Connect's JSON Converter never outputs a null value

2017-03-03 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-3832: - Description: Kafka Connect's JSON Converter will never output a null value when {{enableSchemas

[jira] [Commented] (KAFKA-2376) Add Copycat metrics

2017-02-03 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852164#comment-15852164 ] Randall Hauch commented on KAFKA-2376: -- I agree with [~criccomini]. It's be great to be able

[jira] [Commented] (KAFKA-4353) Add semantic types to Kafka Connect

2016-11-08 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15647999#comment-15647999 ] Randall Hauch commented on KAFKA-4353: -- Logical types and semantic types are not the same thing

[jira] [Created] (KAFKA-4353) Add semantic types to Kafka Connect

2016-10-27 Thread Randall Hauch (JIRA)
Randall Hauch created KAFKA-4353: Summary: Add semantic types to Kafka Connect Key: KAFKA-4353 URL: https://issues.apache.org/jira/browse/KAFKA-4353 Project: Kafka Issue Type: Improvement

[jira] [Commented] (KAFKA-4183) Logical converters in JsonConverter don't properly handle null values

2016-09-16 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15497694#comment-15497694 ] Randall Hauch commented on KAFKA-4183: -- [~shikhar], that'd be great. Note that I created two PRs: one

[jira] [Commented] (KAFKA-4183) Logical converters in JsonConverter don't properly handle null values

2016-09-16 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15497626#comment-15497626 ] Randall Hauch commented on KAFKA-4183: -- [~hachikuji], thanks. I've created a [pull request|https

[jira] [Commented] (KAFKA-4183) Logical converters in JsonConverter don't properly handle null values

2016-09-16 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15497530#comment-15497530 ] Randall Hauch commented on KAFKA-4183: -- [~hachikuji], will there be a 0.10.0.2? If so, any chance

[jira] [Updated] (KAFKA-4183) Logical converters in JsonConverter don't properly handle null values

2016-09-16 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-4183: - Status: Patch Available (was: Open) Added a [pull request|https://github.com/apache/kafka/pull

[jira] [Commented] (KAFKA-4183) Logical converters in JsonConverter don't properly handle null values

2016-09-16 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15497073#comment-15497073 ] Randall Hauch commented on KAFKA-4183: -- I should be able to provide a pull request to fix this. I

[jira] [Created] (KAFKA-4183) Logical converters in JsonConverter don't properly handle null values

2016-09-16 Thread Randall Hauch (JIRA)
Randall Hauch created KAFKA-4183: Summary: Logical converters in JsonConverter don't properly handle null values Key: KAFKA-4183 URL: https://issues.apache.org/jira/browse/KAFKA-4183 Project: Kafka

[jira] [Commented] (KAFKA-3487) Support per-connector/per-task classloaders in Connect

2016-06-19 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338920#comment-15338920 ] Randall Hauch commented on KAFKA-3487: -- [~ewencp] wrote: {quote} There's lots of other stuff you can

[jira] [Commented] (KAFKA-3487) Support per-connector/per-task classloaders in Connect

2016-06-17 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336254#comment-15336254 ] Randall Hauch commented on KAFKA-3487: -- I hope this can move toward an approach that can eventually

[jira] [Created] (KAFKA-3832) Kafka Connect's JSON Converter never outputs a null value

2016-06-13 Thread Randall Hauch (JIRA)
Randall Hauch created KAFKA-3832: Summary: Kafka Connect's JSON Converter never outputs a null value Key: KAFKA-3832 URL: https://issues.apache.org/jira/browse/KAFKA-3832 Project: Kafka

[jira] [Commented] (KAFKA-3821) Allow Kafka Connect source tasks to produce offset without writing to topics

2016-06-09 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323726#comment-15323726 ] Randall Hauch commented on KAFKA-3821: -- [~ewencp], I'm not sure a {{data}} field is that much

[jira] [Comment Edited] (KAFKA-3816) Provide more context in Kafka Connect log messages

2016-06-09 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323684#comment-15323684 ] Randall Hauch edited comment on KAFKA-3816 at 6/10/16 1:11 AM: --- {quote} One

[jira] [Comment Edited] (KAFKA-3816) Provide more context in Kafka Connect log messages

2016-06-09 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323684#comment-15323684 ] Randall Hauch edited comment on KAFKA-3816 at 6/10/16 1:07 AM: --- {quote} One

[jira] [Commented] (KAFKA-3816) Provide more context in Kafka Connect log messages

2016-06-09 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323684#comment-15323684 ] Randall Hauch commented on KAFKA-3816: -- {quote} One thing we might want to do in conjunction

[jira] [Commented] (KAFKA-3821) Allow Kafka Connect source tasks to produce offset without writing to topics

2016-06-09 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323429#comment-15323429 ] Randall Hauch commented on KAFKA-3821: -- Source tasks produce records by returning them from

[jira] [Created] (KAFKA-3821) Allow Kafka Connect source tasks to produce offset without writing to topics

2016-06-09 Thread Randall Hauch (JIRA)
Randall Hauch created KAFKA-3821: Summary: Allow Kafka Connect source tasks to produce offset without writing to topics Key: KAFKA-3821 URL: https://issues.apache.org/jira/browse/KAFKA-3821 Project

[jira] [Resolved] (KAFKA-3814) Connectors should be able to know their name

2016-06-09 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch resolved KAFKA-3814. -- Resolution: Invalid Apologies. There is a {{name}} property. Not sure what I was thinking

[jira] [Commented] (KAFKA-3816) Provide more context in Kafka Connect log messages

2016-06-09 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323179#comment-15323179 ] Randall Hauch commented on KAFKA-3816: -- One example of a set of 3 parameters would work

[jira] [Created] (KAFKA-3815) Support command line arguments in Kafka Connect distribute worker

2016-06-09 Thread Randall Hauch (JIRA)
Randall Hauch created KAFKA-3815: Summary: Support command line arguments in Kafka Connect distribute worker Key: KAFKA-3815 URL: https://issues.apache.org/jira/browse/KAFKA-3815 Project: Kafka

[jira] [Created] (KAFKA-3814) Connectors should be able to know their name

2016-06-09 Thread Randall Hauch (JIRA)
Randall Hauch created KAFKA-3814: Summary: Connectors should be able to know their name Key: KAFKA-3814 URL: https://issues.apache.org/jira/browse/KAFKA-3814 Project: Kafka Issue Type

[jira] [Commented] (KAFKA-3813) Let SourceConnector implementations access the offset reader

2016-06-09 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323112#comment-15323112 ] Randall Hauch commented on KAFKA-3813: -- The current API defines {{ConnectorContext}} interface

[jira] [Created] (KAFKA-3813) Let SourceConnector implementations access the offset reader

2016-06-09 Thread Randall Hauch (JIRA)
Randall Hauch created KAFKA-3813: Summary: Let SourceConnector implementations access the offset reader Key: KAFKA-3813 URL: https://issues.apache.org/jira/browse/KAFKA-3813 Project: Kafka

[jira] [Comment Edited] (KAFKA-3803) JsonConverter deserialized Struct containing bytes field does not return true for equals()

2016-06-07 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319515#comment-15319515 ] Randall Hauch edited comment on KAFKA-3803 at 6/7/16 10:25 PM: --- The {{equals

[jira] [Commented] (KAFKA-3803) JsonConverter deserialized Struct containing bytes field does not return true for equals()

2016-06-07 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319529#comment-15319529 ] Randall Hauch commented on KAFKA-3803: -- BTW, I had no idea until I started looking into this and saw

[jira] [Commented] (KAFKA-3803) JsonConverter deserialized Struct containing bytes field does not return true for equals()

2016-06-07 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319515#comment-15319515 ] Randall Hauch commented on KAFKA-3803: -- The {{equals(...)}} method

[jira] [Resolved] (KAFKA-3804) Kafka Connect's Struct objects rehydrated with JSON converter are not equal

2016-06-07 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch resolved KAFKA-3804. -- Resolution: Duplicate > Kafka Connect's Struct objects rehydrated with JSON conver

[jira] [Created] (KAFKA-3804) Kafka Connect's Struct objects rehydrated with JSON converter are not equal

2016-06-07 Thread Randall Hauch (JIRA)
Randall Hauch created KAFKA-3804: Summary: Kafka Connect's Struct objects rehydrated with JSON converter are not equal Key: KAFKA-3804 URL: https://issues.apache.org/jira/browse/KAFKA-3804 Project

Re: Kafka Connect connector monitoring tasks via offset storage reader

2016-05-24 Thread Randall Hauch
://github.com/apache/kafka/blob/trunk/connect/api/src/main/java/org/apache/kafka/connect/connector/Connector.java Thanks,  Liquan  On Tue, May 24, 2016 at 1:31 PM, Randall Hauch <rha...@gmail.com> wrote:  > I have a need for one of my SourceConnector implementations to configure a  > bu

Kafka Connect connector monitoring tasks via offset storage reader

2016-05-24 Thread Randall Hauch
compatible, and non-intrusive. Is there any interest in this? If so, I can create an issue and work on a pull request. Best regards, Randall Hauch

Additional Kafka Connect schema logical types?

2016-03-04 Thread Randall Hauch
cleaner, though it does increase the verbosity of the Schema object. Thoughts? Randall Hauch [1]  https://github.com/debezium/debezium/blob/74c5adcc8d30afaa221bbdbecad3bb6f6febbaa5/debezium-core/src/main/java/io/debezium/data/Bits.java [2]  https://github.com/debezium/debezium/blob

Re: KIP-28 does not allow Processor to specify partition of output message

2015-10-14 Thread Randall Hauch
Created https://issues.apache.org/jira/browse/KAFKA-2649 and attached a PR with the proposed change. Thanks! On October 14, 2015 at 3:12:34 AM, Guozhang Wang (wangg...@gmail.com) wrote: Thanks! On Tue, Oct 13, 2015 at 9:34 PM, Randall Hauch <rha...@gmail.com> wrote: Ok, cool. I agree w

[jira] [Updated] (KAFKA-2649) Add support for custom partitioner in sink nodes

2015-10-14 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-2649: - Status: Patch Available (was: Open) > Add support for custom partitioner in sink no

Re: KIP-28 does not allow Processor to specify partition of output message

2015-10-14 Thread Randall Hauch
up. Thus, a single class representing all partitioning logic is not a bad thing at all. (The code organization wise, all logic does not necessarily in the single class, of course.) On Wed, Oct 14, 2015 at 8:47 AM, Randall Hauch <rha...@gmail.com> wrote: > Crea

[jira] [Created] (KAFKA-2649) Add support for custom partitioner in sink nodes

2015-10-14 Thread Randall Hauch (JIRA)
Randall Hauch created KAFKA-2649: Summary: Add support for custom partitioner in sink nodes Key: KAFKA-2649 URL: https://issues.apache.org/jira/browse/KAFKA-2649 Project: Kafka Issue Type

[jira] [Updated] (KAFKA-2649) Add support for custom partitioner in sink nodes

2015-10-14 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-2649: - Description: The only way for Processor implementations to control partitioning of forwarded

Re: KIP-28 does not allow Processor to specify partition of output message

2015-10-13 Thread Randall Hauch
Perhaps we should add a new Partitioner interface in Kafka Streams? > > Guozhang > >> On Tue, Oct 13, 2015 at 6:38 PM, Randall Hauch <rha...@gmail.com> wrote: >> This overrides the partitioning logic for all topics, right? That means I >> have to explicitly ca

KIP-28 does not allow Processor to specify partition of output message

2015-10-13 Thread Randall Hauch
a JIRA and create a PR to add it to the API, but without knowing all of the history I’m wondering if leaving it out of the API was intentional. Thoughts? Best regards, Randall Hauch

Re: KIP-28 does not allow Processor to specify partition of output message

2015-10-13 Thread Randall Hauch
in org.apache.kafka.clients.producer.Partitioner Let me know if it works for you. Guozhang On Tue, Oct 13, 2015 at 10:59 AM, Randall Hauch <rha...@gmail.com> wrote: > The new streams API added with KIP-28 is great. I’ve been using it on a > prototype for a few weeks, and I’m looking forward to it be

[jira] [Commented] (KAFKA-2593) KeyValueStores should not require use of the context's default serializers and deserializers

2015-10-07 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947520#comment-14947520 ] Randall Hauch commented on KAFKA-2593: -- ~guozhang, I've replaced the static factory methods in each

[jira] [Updated] (KAFKA-2600) Make KStream interfaces compatible with Java 8 java.util.function

2015-10-02 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-2600: - Description: As suggested by [~rhauch], if we make the interface method names as the same

[jira] [Updated] (KAFKA-2600) Make KStream interfaces compatible with Java 8 java.util.function

2015-10-01 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-2600: - Reviewer: Guozhang Wang Fix Version/s: 0.9.0.0 Status: Patch Available

[jira] [Assigned] (KAFKA-2600) Make KStream interfaces compatible with Java 8 java.util.function

2015-09-30 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch reassigned KAFKA-2600: Assignee: Randall Hauch > Make KStream interfaces compatible with Jav

[jira] [Updated] (KAFKA-2597) Add Eclipse directories to .gitignore

2015-09-29 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-2597: - Fix Version/s: 0.9.0.0 Status: Patch Available (was: Open) This contribution is ready

[jira] [Updated] (KAFKA-2593) KeyValueStores should not require use of the context's default serializers and deserializers

2015-09-29 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-2593: - Status: Patch Available (was: Open) This contribution is ready for review. > KeyValueSto

[jira] [Updated] (KAFKA-2594) Add a key-value store that is a fixed-capacity in-memory LRU cache

2015-09-29 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-2594: - Status: Patch Available (was: Open) This contribution is ready for review. > Add a key-va

[jira] [Created] (KAFKA-2593) KeyValueStores should not require use of the context's default serializers and deserializers

2015-09-28 Thread Randall Hauch (JIRA)
Randall Hauch created KAFKA-2593: Summary: KeyValueStores should not require use of the context's default serializers and deserializers Key: KAFKA-2593 URL: https://issues.apache.org/jira/browse/KAFKA-2593

[jira] [Created] (KAFKA-2594) Add a key-value store that is a fixed-capacity in-memory LRU cache

2015-09-28 Thread Randall Hauch (JIRA)
Randall Hauch created KAFKA-2594: Summary: Add a key-value store that is a fixed-capacity in-memory LRU cache Key: KAFKA-2594 URL: https://issues.apache.org/jira/browse/KAFKA-2594 Project: Kafka

[jira] [Created] (KAFKA-2597) Add Eclipse directories to .gitignore

2015-09-28 Thread Randall Hauch (JIRA)
Randall Hauch created KAFKA-2597: Summary: Add Eclipse directories to .gitignore Key: KAFKA-2597 URL: https://issues.apache.org/jira/browse/KAFKA-2597 Project: Kafka Issue Type: Improvement

<    2   3   4   5   6   7   8   >