[jira] [Commented] (SPARK-8833) Kafka Direct API support offset in zookeeper

2015-07-09 Thread Tathagata Das (JIRA)

[ 
https://issues.apache.org/jira/browse/SPARK-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14621579#comment-14621579
 ] 

Tathagata Das commented on SPARK-8833:
--

[~c...@koeninger.org] I wouldnt call it rejected. I am waiting for getting 
enough usecase to float in before I take a final call. And the fact that 
another person wanted to contribute this is a proof that this feature is 
useful. 

 Kafka Direct API support offset in zookeeper
 

 Key: SPARK-8833
 URL: https://issues.apache.org/jira/browse/SPARK-8833
 Project: Spark
  Issue Type: Improvement
  Components: Streaming
Affects Versions: 1.4.0
Reporter: guowei
Priority: Minor

 Kafka Direct API only support consume the topic from latest or earliest.
 but user usually need to consume message from last offset  when restart 
 stream app .



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Commented] (SPARK-8833) Kafka Direct API support offset in zookeeper

2015-07-06 Thread Apache Spark (JIRA)

[ 
https://issues.apache.org/jira/browse/SPARK-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614677#comment-14614677
 ] 

Apache Spark commented on SPARK-8833:
-

User 'guowei2' has created a pull request for this issue:
https://github.com/apache/spark/pull/7235

 Kafka Direct API support offset in zookeeper
 

 Key: SPARK-8833
 URL: https://issues.apache.org/jira/browse/SPARK-8833
 Project: Spark
  Issue Type: Bug
  Components: Streaming
Affects Versions: 1.4.0
Reporter: guowei

 Kafka Direct API only support consume the topic from latest or earliest.
 but user usually need to consume message from last offset  when restart 
 stream app .



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Commented] (SPARK-8833) Kafka Direct API support offset in zookeeper

2015-07-06 Thread Sean Owen (JIRA)

[ 
https://issues.apache.org/jira/browse/SPARK-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14614670#comment-14614670
 ] 

Sean Owen commented on SPARK-8833:
--

No, you actually pass the offsets you want to begin consuming at. Are you 
looking at {{createDirectStream}}? it's {{fromOffsets}}.

 Kafka Direct API support offset in zookeeper
 

 Key: SPARK-8833
 URL: https://issues.apache.org/jira/browse/SPARK-8833
 Project: Spark
  Issue Type: Bug
  Components: Streaming
Affects Versions: 1.4.0
Reporter: guowei

 Kafka Direct API only support consume the topic from latest or earliest.
 but user usually need to consume message from last offset  when restart 
 stream app .



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Commented] (SPARK-8833) Kafka Direct API support offset in zookeeper

2015-07-06 Thread Cody Koeninger (JIRA)

[ 
https://issues.apache.org/jira/browse/SPARK-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615056#comment-14615056
 ] 

Cody Koeninger commented on SPARK-8833:
---

this basic idea has been discussed before and rejected:

https://issues.apache.org/jira/browse/SPARK-6249

 Kafka Direct API support offset in zookeeper
 

 Key: SPARK-8833
 URL: https://issues.apache.org/jira/browse/SPARK-8833
 Project: Spark
  Issue Type: Improvement
  Components: Streaming
Affects Versions: 1.4.0
Reporter: guowei
Priority: Minor

 Kafka Direct API only support consume the topic from latest or earliest.
 but user usually need to consume message from last offset  when restart 
 stream app .



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org