[
https://issues.apache.org/jira/browse/SPARK-12134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15043445#comment-15043445
]
Brian London edited comment on SPARK-12134 at 12/5/15 6:55 PM:
---------------------------------------------------------------
That's an interesting idea (although the {{return}} is unnecessary). I was
thinking to convert it into a warning. I believe the region may be used in
other ways later. {{createStream}} currently issues a warning if you specify
both a region and a URL that the region extracted from the URL will overwrite
the one specified -- something that should be fixed on its own. That logic
will likely need to get changed as well. I'm not sure if it lives in Spark or
in the Aws library. It's likely to be a problem if the later.
was (Author: brianlondon):
That's an interesting idea (although the {{return}} is unnecessary). I was
thinking to convert it into a warning. I believe the region may be used in
other ways later. {createStream} currently issues a warning if you specify
both a region and a URL that the region extracted from the URL will overwrite
the one specified -- something that should be fixed on its own. That logic
will likely need to get changed as well. I'm not sure if it lives in Spark or
in the Aws library. It's likely to be a problem if the later.
> Spark Streaming Kinesis: Allow a mechanism to bypass validateRegion
> -------------------------------------------------------------------
>
> Key: SPARK-12134
> URL: https://issues.apache.org/jira/browse/SPARK-12134
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 1.5.2
> Reporter: Brian London
>
> The {{KinesisUtils.createStream}} family of methods internally call a
> function {{validateRegion}} that requires the supplied region to be one of
> the strings on the list of AWS regions. Doing so, prevents the use of
> createStream to connect to a non-AWS kinesis stream such as one run through
> [Kinesalite|https://github.com/mhart/kinesalite], which can be useful for
> testing. The {{validateRegion}} method does no additional transformation so
> it's only effective function is to kill the createStream call if a user
> requests a stream with a non-standard region string (intentionally or
> otherwise).
> There should be a mechanism to bypass this check, if not remove it entirely.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
