Where can I find information on the size of the datasets supported by AWS
Glue? I didn't see it on the documentation
Also, if I want to process TBs of data for eg 1TB what should be the ideal
EMR cluster configuration?
Could you please guide me on this?
Thanks,
Sid.
On Thu, 23 Jun 2022, 23:44
Please use EMR, Glue is not made for heavy processing jobs.
On Thu, Jun 23, 2022 at 6:36 AM Sid wrote:
> Hi Team,
>
> Could anyone help me in the below problem:
>
>
> https://stackoverflow.com/questions/72724999/how-to-calculate-number-of-g-1-workers-in-aws-glue-for-processing-1tb-data
>
>
Hi.
> So the above issue occurs at build and test a maven project with Spark 3.3.0
> and Java 17, rather than test spark-3.3 source code?
Yes, that’s correct — it’s my project’s unit test that fails, not a Spark
source code unit test.
Sorry for the confusion - and thanks for the info about
So the above issue occurs at build and test a maven project with Spark 3.3.0
and Java 17, rather than test spark-3.3 source code?
If yes, you may need to add the following Java Options to `argLine` of
`maven-surefire-plugin` for Java 17:
```
--add-opens=java.base/java.lang=ALL-UNNAMED
Hi.I am running on macOS 12.4, using an ‘Adoptium’ JDK from https://adoptium.net/download. The version details are:$ java -versionopenjdk version "17.0.3" 2022-04-19OpenJDK Runtime Environment Temurin-17.0.3+7 (build 17.0.3+7)OpenJDK 64-Bit Server VM Temurin-17.0.3+7 (build 17.0.3+7, mixed mode,
