Thank you Yew
On Wed, May 24, 2023, 11:19 PM Wing Yew Poon
wrote:
> Gaurav,
>
> Is your data partitioned by date? If so, you can compact subsets of
> partitions at a time. To do this using the Spark procedure, you pass a
> where clause:
>
> spark.sql("CALL catalog_name.system.rewrite_data_files(
Gaurav,
Is your data partitioned by date? If so, you can compact subsets of
partitions at a time. To do this using the Spark procedure, you pass a
where clause:
spark.sql("CALL catalog_name.system.rewrite_data_files(table => '...',
where => '...')")
If you use the RewriteDataFilesSparkAction, yo
On Wed, May 24, 2023, 10:41 AM Gaurav Agarwal
wrote:
> I have one more query we are trying to compact files currently it is
> taking time as have never compacted till now this is the first time we are
> trying to perform compaction after 5 months of continuously loading data
> We change the forma
