[
https://issues.apache.org/jira/browse/BEAM-1040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886299#comment-15886299
]
ASF GitHub Bot commented on BEAM-1040:
--------------------------------------
GitHub user diptikul opened a pull request:
https://github.com/apache/beam/pull/2115
BEAM-1040 Add HadoopInputFormatIO
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title is formatted like:
`[BEAM-<Jira issue #>] Description of pull request`
- [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
- [ ] Replace `<Jira issue #>` in the title with the actual Jira issue
number, if there is one.
- [ ] If this contribution is large, please file an Apache
[Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.txt).
---
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/diptikul/incubator-beam hif-io-clean
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/2115.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2115
----
commit 45cf382e14fceb32b2738b548c851472659f84d9
Author: Radhika S Kulkarni <[email protected]>
Date: 2016-12-22T13:49:38Z
First Hadoop InputFormat IO project code check in
commit 9c1a54d8d686ac396a1f500c9b69abe4d44e18a5
Author: Radhika S Kulkarni <[email protected]>
Date: 2016-12-23T08:15:07Z
Merge rtest from original
Take latest from original. remote-tracking branch 'upstream/master'
commit 5f927add640a9888ca26662450ceb1ecfa92c08c
Author: Sheetal Tarodekar <[email protected]>
Date: 2016-12-23T08:55:12Z
erg end() method
remote-tracking branch 'upstream/master'
commit 6c7ec6a37286ae55e16f267f126342694eb02b0e
Author: Dipti Kulkarni <[email protected]>
Date: 2016-12-23T09:43:48Z
Merge remote-tracking branch 'upstream/master'
taking latest-23/12
commit e49f22fed8fe0a8798fa1fe729c33df105e54cb5
Author: Radhika S Kulkarni <[email protected]>
Date: 2016-12-23T10:20:45Z
Updated Pom files
commit 9e4fdba32171b86b2f67f0c73780a06f6ca1b187
Author: Dipti Kulkarni <[email protected]>
Date: 2016-12-23T10:24:04Z
Merge branch 'master' of https://github.com/diptikul/incubator-beam
commit 6357bdf3c151c8c7575abad0f0e2f4bc4a424d7a
Author: Sheetal Tarodekar <[email protected]>
Date: 2016-12-23T14:41:23Z
Added code and tests to read from DummyInputFormat
commit a2163e823d7de3a024913acbb6a4035b8d8b6dd3
Author: Sheetal Tarodekar <[email protected]>
Date: 2016-12-23T14:48:38Z
Resolved conflictsMerge branch 'master' of
https://github.com/diptikul/incubator-beam
commit fb1f2de7e479920ac53de25ba18dd673046c69da
Author: Dipti Kulkarni <[email protected]>
Date: 2016-12-23T14:50:06Z
Merge branch 'master' of https://github.com/diptikul/incubator-beam
commit 571fcb40838a61148d0d49669ec18df35779b784
Author: Radhika S Kulkarni <[email protected]>
Date: 2016-12-26T05:01:37Z
Added Hadoop Input Format module in io pom file and minor changes in pom
file.
commit a785180fc3b95f66a9b30a03c184fb7d637777ad
Author: Dipti Kulkarni <[email protected]>
Date: 2016-12-26T05:30:25Z
Merge branch 'master' of https://github.com/diptikul/incubator-beam
commit b29276ae953e9fff7ed6eb0c241d21e34122c5d2
Author: Radhika S Kulkarni <[email protected]>
Date: 2016-12-26T09:49:39Z
Moved MyCassandraRow to test folder
commit 70f05266328d62ba3a9127d54bc9f3769effbde4
Author: Radhika S Kulkarni <[email protected]>
Date: 2016-12-26T10:51:29Z
Removed CassandraRowCoder and Custom coder unit test, added MyCassandraRow
in test, used Avro coder for MyCassandraRow
commit f129ead54b1b7047f4c5ff592bc4f2e7ecc7bc43
Author: Dipti Kulkarni <[email protected]>
Date: 2016-12-26T11:56:58Z
Merge branch 'master' of https://github.com/diptikul/incubator-beam
commit 276c3c217df6c062b8dd4b5deda0e3425c775513
Author: Sheetal Tarodekar <[email protected]>
Date: 2016-12-27T06:02:14Z
Added test cases for validate method, getFractionsConsumed method
commit 020576283b10c7630bb38df8836387404a57b76e
Author: Dipti Kulkarni <[email protected]>
Date: 2016-12-27T07:03:24Z
Merge branch 'master' of https://github.com/diptikul/incubator-beam
commit c01544ffb61d6c129a932eef6d957e57c698b858
Author: Sheetal Tarodekar <[email protected]>
Date: 2016-12-28T10:46:54Z
Added junits for splitIntoBundles(),computeSplits and added autovalue
builder.
commit 6e6819eec404ec131ec1f2ef98d985a2d51e4c9e
Author: Dipti Kulkarni <[email protected]>
Date: 2016-12-28T10:48:28Z
Merge branch 'master' of https://github.com/diptikul/incubator-beam
commit 8ebbd2b124b5f40e67be19ef9caba8f1f3b3c6bc
Author: Dipti Kulkarni <[email protected]>
Date: 2016-12-28T10:56:50Z
Made tests compile with read()
commit e0b291958e804e7c328be87ce860e63ea9253b20
Author: Dipti Kulkarni <[email protected]>
Date: 2016-12-28T11:23:24Z
New package for dummy input formats, and add reference of
DisplayDataMatchers for unit test
commit a1b092a67fd0456a835fb3c4115c0b14a3f2d094
Author: Dipti Kulkarni <[email protected]>
Date: 2016-12-28T11:28:23Z
Added Apache License header to all classes
commit 049473e3350fe12a2b4c217cc1cfae1737872135
Author: Radhika S Kulkarni <[email protected]>
Date: 2016-12-30T07:44:52Z
Merge remote-tracking branch 'upstream/master'
commit 46b13e4a3851b262cb1a0f51bf791cacee2b6d8f
Author: Radhika S Kulkarni <[email protected]>
Date: 2016-12-30T07:55:01Z
Added TypeDescriptors
commit b04eb728120fe42442184eecae16ac987cfc4226
Author: Sheetal Tarodekar <[email protected]>
Date: 2016-12-30T09:55:21Z
Added junits for testing immutability of HadoopInputFormatIO and junit for
checking key/value classes set by user in configuration.
commit 59b0f272a5432b2738416850161f0e61e5739d6f
Author: Dipti Kulkarni <[email protected]>
Date: 2017-01-03T06:29:21Z
InputFormat changes, Exception thrown using annotation , testPipeline used
for creation of pipeline
commit 8f2a57071cfe83a8586328b3dd4c30d5c1596948
Author: Dipti Kulkarni <[email protected]>
Date: 2017-01-03T07:26:06Z
Added few comments, TestPipeline Usage
commit 995bfa9b32ee8e5ded8225ef5fa1e15e3b4cb2df
Author: Sheetal Tarodekar <[email protected]>
Date: 2017-01-03T14:00:09Z
Added cassandra inegration test and modified junits as per test review
comments.
commit 6d5fc47d9a2fb74d5eec634fba4d63a6f87789f7
Author: Sheetal Tarodekar <[email protected]>
Date: 2017-01-04T13:06:13Z
Added logging into HadoopInputFormatIO and modifications in Junits and
cassandra integration test.
commit a5db53f986e4a8e95713f9e7d8a1e1d0a22eab59
Author: Sheetal Tarodekar <[email protected]>
Date: 2017-01-05T12:56:49Z
Modification in code according to code review comments and google code
formatting.
commit 8d6392b058b08cdc7bc9b21623e35d57e996a29a
Author: Sheetal Tarodekar <[email protected]>
Date: 2017-01-06T12:02:26Z
Modifications as per code review comments.
----
> Hadoop InputFormat - IO Transform for reads
> -------------------------------------------
>
> Key: BEAM-1040
> URL: https://issues.apache.org/jira/browse/BEAM-1040
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-extensions
> Reporter: Stephen Sisk
> Assignee: Dipti Kulkarni
>
> We should build a IO read transform that will read data from services
> supporting the Hadoop InputFormat interface [1]
> This will make it easy to connect to a variety of databases while still
> providing some aspects of scalability since the InputFormat interface
> provides for parallel reading.
> [1]
> https://hadoop.apache.org/docs/r2.7.3/api/org/apache/hadoop/mapred/InputFormat.html
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
