Where can I find information on the size of the datasets supported by AWS Glue? I didn't see it on the documentation
Also, if I want to process TBs of data for eg 1TB what should be the ideal EMR cluster configuration? Could you please guide me on this? Thanks, Sid. On Thu, 23 Jun 2022, 23:44 Gourav Sengupta, <gourav.sengu...@gmail.com> wrote: > Please use EMR, Glue is not made for heavy processing jobs. > > On Thu, Jun 23, 2022 at 6:36 AM Sid <flinkbyhe...@gmail.com> wrote: > >> Hi Team, >> >> Could anyone help me in the below problem: >> >> >> https://stackoverflow.com/questions/72724999/how-to-calculate-number-of-g-1-workers-in-aws-glue-for-processing-1tb-data >> >> Thanks, >> Sid >> >
