1. Change send to use java.util.concurrent.Future in send():
Future<RecordPosition> send(ProducerRecord record, Callback callback)
The cancel method will always return false and not do anything. Callback
will then be changed to
interface Callback {
public void onCompletion(RecordPosition position, Exception exception);
}
so that the exception is available there too and will be null if no error
occurred. I'm not planning on changing the name callback because I haven't
thought of a better one.
+1
2. We will change the way serialization works to proposal 1A in the
previous discussion. That is the Partitioner and Serializer interfaces will
disappear. ProducerRecord will change to:
class ProducerRecord {
public byte[] key() {...}
public byte[] value() {...}
public Integer partition() {...} // can be null
}
In order to allow correctly choosing a partition the Producer interface
will include a new method:
List<PartitionInfo> partitionsForTopic(String topic);
PartitionInfo will be changed to include the actual Node objects not just
the Node ids.
Mostly agree with this but wanted to confirm my understanding of
partitionsForTopic.
So, is that something a user has to remember to invoke in order to detect
newly
added partitions to a topic? Will each invocation of partitionsForTopic
lead to a
TopicMetadataRequest RPC?
3. I will make the producer implement java.io.Closable but not throw any
exceptions as there doesn't really seem to be any disadvantage to this and
the interface may remind people to call close.
+1
Non-changes
1. I don't plan to change the build system. The SBT=>gradle change is
basically orthoganol and we should debate it in the context of its ticket.
+1
2. I'm going to stick with my oddball kafka.* rather than
org.apache.kafka.* package name and non-getter methods unless everyone
complains.
-1. I prefer org.apache.kafka for consistency with other top level Apache
projects
3. I'm not going to introduce a zookeeper dependency in the client as I
don't see any advantage.
+1. I see the point about non-java clients not having an easy way to talk
to zookeeper,
but don't really think the connections to zookeeper is the problem.
4. There were a number of reasonable suggestions on the Callback execution
model. I'm going to leave it as is, though. Because we are moving to use
Java Future we can't include functionality like ListenableFuture. I think
the simplestic callback model where callbacks are executed in the i/o
thread should be good enough for most cases and other cases can use their
own threading.
+1
On Sat, Feb 1, 2014 at 9:06 AM, Joel Koshy <jjkosh...@gmail.com> wrote:
> .
> >
> > > In order to allow correctly choosing a partition the Producer interface
> > > will include a new method:
> > > List<PartitionInfo> partitionsForTopic(String topic);
> > > PartitionInfo will be changed to include the actual Node objects not
> just
> > > the Node ids.
> >
> > Why are the node id's alone insufficient? You can still do
> > round-robin/connection-limiting, etc. with just the node id right?
>
>
> Actually no - we do need the full node info.
>
>
>
>
> --
> Sent from Gmail Mobile
>
