[
https://issues.apache.org/jira/browse/STREAMS-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171570#comment-14171570
]
ASF GitHub Bot commented on STREAMS-168:
----------------------------------------
GitHub user steveblackmon opened a pull request:
https://github.com/apache/incubator-streams/pull/104
Streams 168
There's quite a bit going on in this PR, so here's a summary:
To date, modules in streams-contrib have implemented interfaces from
streams-core. Code re-use has been intra-contrib-module, or occasionally
cross-contrib-module, with a few helper classes in streams-pojo playing a
supporting role.
This PR introduces a new top-level sub-module, streams-components, where
I've placed streams-http. streams-http contains a provider and a processor,
both of which can be employed in a stream to collect json documents from remote
REST endpoints. The config properties of streams-http were chosen to align
with camel's http support module, anticipating compatibility will be desirable
at some point in the future.
In addition, the PR contains new processors (in streams-provider-twitter
and streams-processor-peoplepattern) which employ streams-http to collect data
from specific data supplier endpoints, with a high degree of code reuse from
streams-http. Essentially they just override a few methods but do not need to
introduce their configuration or alter any aspect of the REST call other than
parameters and headers. I've built code that uses the provider as well.
The JSON responses from the REST calls in these reference processors are
then appended to specific extensions fields (activity.extensions.X,
activity.actor.extensions.X, activity.object.extensions.X, etc.. for arbitrary
X, sited on the activity or below (on any sub-node whose field extends
ActivityObject.) Utility classes supporting extensions are under
streams-pojo-extensions. Modules that were importing these classes from
streams-pojo would need to update to import streams-pojo-extensions as well.
Philosophically, future modules containing providers, processors, and
persisters which are both useful as standalone components in streams, and which
are designed to simplify/harden channel/API/database-specific components that
extend them, would be placed under streams-components. This gives us a
dedicated sub-partitioned module for valuable utility features (outside of
core), and will drastically curtail tendancies for one contrib module to
explicitly require another.
Lastly, I've built streams-example modules utilizing each of these
processors, which are pushed to w2ogroup/streams-examples under the STREAMS-168
branch.
I look forward to your feedback on this proposed reorg.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/incubator-streams STREAMS-168
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-streams/pull/104.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #104
----
commit 1464819fd7ea147830a4211ea2ac6af4aa715022
Author: sblackmon <[email protected]>
Date: 2014-09-14T15:24:47Z
just pons
commit 34232ad87b50253913fba0dabb6fa9af591f388d
Author: sblackmon <[email protected]>
Date: 2014-09-15T18:50:32Z
added streams-components module
added streams-processor-http module
activated streams-pojo-extensions module
tweaks to pojo, require id and remove non-sensical defaults
commit b8ccf9f6869c64f1f6e25f668942ea0de9fec2eb
Author: sblackmon <[email protected]>
Date: 2014-09-15T23:38:09Z
added HttpConfigurator
commit a576460911464cf260e37886ce840f4e57dd5f30
Author: sblackmon <[email protected]>
Date: 2014-09-16T19:24:53Z
class need not be abstract
rootDocument is what should exit processor
commit d62061dec8628484e66d3494e02f1f65efafda41
Author: Steve Blackmon <[email protected]>
Date: 2014-10-12T22:49:01Z
introduced a SimpleHTTPGetProvider
reorganized configuration
added access_token support
added http basic auth support
commit 9a575322231e4a8a69c56f06da743ffe3211ccb4
Author: Steve Blackmon <[email protected]>
Date: 2014-10-13T01:20:52Z
committing PeoplePattern processor
commit ad5f90cc115c40e0ef480a85957bbb4c98157cf6
Author: Steve Blackmon <[email protected]>
Date: 2014-10-13T01:21:22Z
removing validation, as jackson does not contain a default implementation
until 2.5 release
commit 0c8e67ce26e131282aaaa03815ad6daefd684346
Author: Steve Blackmon <[email protected]>
Date: 2014-10-13T04:13:29Z
a few package name changes
working twitterurlapiprocess which uses streams-http (w/o authentication)
working people pattern processors which use streams-http (w/ authentication)
commit d2ffe62baf88618c7c8a6b9a1e5a79d21ed9e3fd
Author: sblackmon <[email protected]>
Date: 2014-10-13T22:07:19Z
added license header and javadoc comments
----
> Processor for making authenticated REST calls, appending results to datum
> -------------------------------------------------------------------------
>
> Key: STREAMS-168
> URL: https://issues.apache.org/jira/browse/STREAMS-168
> Project: Streams
> Issue Type: New Feature
> Reporter: Steve Blackmon
>
> Think it would be nice to have some sort of REST utility processor or at
> least a processor that can serve as an example for others looking to call an
> endpoint and attach the result.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
