Hi Daniel, I know JB has a started a RestIO, it is not yet complete and needs some love but probably some opportunity for converge and collaboration here
Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book 2018-04-11 15:42 GMT+02:00 Daniel Salerno <[email protected]>: > Good morning, > > In my Big Data Google project we need to read batch data from the VTEX > platform every 15 minutes and record json's return to our cloud storage > datalake. > It makes the data available through HTTP GET requests for its API: > (https://documenter.getpostman.com/view/487146/vtex-oms-api/6tjSKqi) > > I have not found a good way to do this reading using standard Apache Beam > components, my idea is to develop an HttpIO component using Source Sink to > read the request and write a JSON file to Cloud Storage. After that another > pipeline will process the file and write to BigQuery. > > Can you please help me with this matter? Is that the best way to go? Do you > have any idea how to develop this new component? > > Thank you. > > Hugs, > > Daniel Salerno de Arruda
