Hi Ankur, Thanks for the reply! Below is more details of the usecase:
1. Call an API which would provide a dictionary as response. 2. Transform dictionary to add / remove few keys. 3. Send transformed dictionary as JSON to an API which prints this JSON as output. Please let me know in case of any clarifications. Thanks, Anjana ________________________________ From: Ankur Goenka [[email protected]] Sent: Saturday, June 01, 2019 6:47 PM To: [email protected] Subject: Re: How to build a beam python pipeline which does GET/POST request to API's Hi Anjana, You can write your API logic in a ParDo and subsequently pass the elements to other ParDos to transform and eventually make an API call to to another endpoint. However, this might not be a good fit for Beam as the input is not well defined and hence scaling and "once processing" of elements will not be possible as their is no well defined input. It will be better to elaborate a bit more on the usecase for better suggestions. Thanks, Ankur On Sat, Jun 1, 2019 at 5:50 PM Anjana Pydi <[email protected]<mailto:[email protected]>> wrote: Hi, I have a requirement to create an apache beam python pipeline to read a JSON from an API endpoint, transform it (add/remove few fields)and send the transformed JSON to another API endpoint. Can anyone please provide some suggestions on how to do it. Thanks, Anjana ----------------------------------------------------------------------------------------------------------------------- The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you are not the intended recipient, please notify us immediately by responding to this email and then delete it from your system. Bahwan Cybertek is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. ----------------------------------------------------------------------------------------------------------------------- The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you are not the intended recipient, please notify us immediately by responding to this email and then delete it from your system. Bahwan Cybertek is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
