We want to migrate our data (approximately 20M rows) from parquet to postgres, when we are using dataframe writer's jdbc method the execution time is very large, we have tried the same with batch insert it was much effective. Is it intentionally implemented in that way?
- DataFrameWriter.jdbc is very slow Aram Mkrtchyan
- Re: DataFrameWriter.jdbc is very slow Michael Armbrust
- Re: DataFrameWriter.jdbc is very slow Aram Mkrtchyan