[
https://issues.apache.org/jira/browse/SPARK-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573563#comment-15573563
]
Cody Koeninger commented on SPARK-17812:
----------------------------------------
So a short term question
- with your proposed interface, what, as a user, do you expect to happen when
you specify startingOffsets for some but not all partitions?
A couple of medium term questions:
- Yes, auto.offset.reset is a mess. Have you read
https://issues.apache.org/jira/browse/KAFKA-3370
- What are you going to do when that ticket is resolved? It should allow users
to answer the questions you raised in very specific ways, that your interface
does not.
And a really leading long term question:
- Is the purpose of your interface to do what you think users should be able to
do, or what they need to be able to do?
> More granular control of starting offsets (assign)
> --------------------------------------------------
>
> Key: SPARK-17812
> URL: https://issues.apache.org/jira/browse/SPARK-17812
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Reporter: Michael Armbrust
>
> Right now you can only run a Streaming Query starting from either the
> earliest or latests offsets available at the moment the query is started.
> Sometimes this is a lot of data. It would be nice to be able to do the
> following:
> - seek to user specified offsets for manually specified topicpartitions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
