Where can I find information on the size of the datasets supported by AWS
Glue? I didn't see it on the documentation
Also, if I want to process TBs of data for eg 1TB what should be the ideal
EMR cluster configuration?
Could you please guide me on this?
Thanks,
Sid.
On Thu, 23 Jun 2022, 23:44 Gourav Sengupta,
wrote:
> Please use EMR, Glue is not made for heavy processing jobs.
>
> On Thu, Jun 23, 2022 at 6:36 AM Sid wrote:
>
>> Hi Team,
>>
>> Could anyone help me in the below problem:
>>
>>
>> https://stackoverflow.com/questions/72724999/how-to-calculate-number-of-g-1-workers-in-aws-glue-for-processing-1tb-data
>>
>> Thanks,
>> Sid
>>
>
