Re: speed up full table scan using psql

ps. On Wed, May 31, 2023 at 9:16 PM Adrian Klaver wrote: > On 5/31/23 13:57, Lian Jiang wrote: > > The command is: psql $db_url -c "copy (select row_to_json(x_tmp_uniq) > > from public.mytable x_tmp_uniq) to stdout" > > What is taking the stdout and what it is it do

Re: speed up full table scan using psql

The command is: psql $db_url -c "copy (select row_to_json(x_tmp_uniq) from public.mytable x_tmp_uniq) to stdout" postgres version: 14.7 Does this mean COPY and java CopyManager may not help since my psql command already uses copy? Regarding pg_dump, it does not support json format which means ext

Re: speed up full table scan using psql

t; snowflake table. Hope this makes sense. On Tue, May 30, 2023 at 10:17 PM David Rowley wrote: > On Wed, 31 May 2023 at 16:26, Lian Jiang wrote: > > I am using psql to periodically dump the postgres tables into json files > which are imported into snowflake. For large tables (e.g. 7

speed up full table scan using psql

hi, I am using psql to periodically dump the postgres tables into json files which are imported into snowflake. For large tables (e.g. 70M rows), it takes hours for psql to complete. Using spark to read the postgres table seems not to work as the postgres read only replication is the bottleneck so

Re: event trigger should provide more details

Tue, May 30, 2023 at 2:42 PM Erik Wienhold wrote: > > On 30/05/2023 22:23 CEST Lian Jiang wrote: > > > > I plan to create an event trigger to detect schema change (e.g. > add/remove > > a column, change column type), and write it into a separate table (e.g. > > EVENTS

event trigger should provide more details

hi, I plan to create an event trigger to detect schema change (e.g. add/remove a column, change column type), and write it into a separate table (e.g. EVENTS). Then a process periodically reads this table to send schema change notification. However, the event trigger