[
https://issues.apache.org/jira/browse/SPARK-6569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14394457#comment-14394457
]
Cody Koeninger commented on SPARK-6569:
---------------------------------------
Keep in mind that this message is in KafkaRDD, which is designed to be used on
its own outside of a streaming job. If someone asks for a 0 length offset
range, I'd want the system to do something other than silently do nothing.
Platon's point about monitoring makes sense, and we do monitor our jobs. I see
that as orthogonal to logging - a graphite monitor might tell me a job is
running abnormally, Spark logs might tell me that's because a producer is
misconfigured and a particular topic is only getting written for one partition.
I wouldn't expect to have to run my system at debug log level to get that.
Honestly, given the incredible volume of Spark logs at info level, worrying
about one more (potentially useful) log at info level doesn't make sense to me.
> Kafka directInputStream logs what appear to be incorrect warnings
> -----------------------------------------------------------------
>
> Key: SPARK-6569
> URL: https://issues.apache.org/jira/browse/SPARK-6569
> Project: Spark
> Issue Type: Bug
> Components: Streaming
> Affects Versions: 1.3.0
> Environment: Spark 1.3.0
> Reporter: Platon Potapov
> Priority: Minor
>
> During what appears to be normal operation of streaming from a Kafka topic,
> the following log records are observed, logged periodically:
> {code}
> [Stage 391:==========================================> (3 + 0) /
> 4]
> 2015-03-27 12:49:54 WARN KafkaRDD: Beginning offset ${part.fromOffset} is the
> same as ending offset skipping raw 0
> 2015-03-27 12:49:54 WARN KafkaRDD: Beginning offset ${part.fromOffset} is the
> same as ending offset skipping raw 0
> 2015-03-27 12:49:54 WARN KafkaRDD: Beginning offset ${part.fromOffset} is the
> same as ending offset skipping raw 0
> {code}
> * the part.fromOffset placeholder is not correctly substituted to a value
> * is the condition really mandates a warning being logged?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
