[
https://issues.apache.org/jira/browse/SPARK-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560820#comment-15560820
]
Ofir Manor commented on SPARK-17812:
------------------------------------
I'll have to think about it - not sure if it ever makes sense to provide custom
offsets, skip some partitions, and expect a user-configurable default offsets
for the missing ones.
I would actually suggest that if a user provide custom offsets, they would be
used as is. For example, if I provide offsets only for topic 1 partition
1,4,10, then the consumer group would subscribe to only these three partitions.
In other words, custom assignment should be respected as is.
> More granular control of starting offsets
> -----------------------------------------
>
> Key: SPARK-17812
> URL: https://issues.apache.org/jira/browse/SPARK-17812
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Reporter: Michael Armbrust
>
> Right now you can only run a Streaming Query starting from either the
> earliest or latests offsets available at the moment the query is started.
> Sometimes this is a lot of data. It would be nice to be able to do the
> following:
> - seek back {{X}} offsets in the stream from the moment the query starts
> - seek to user specified offsets
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
