Re: Backfill bgworker Extension?

2017-12-17 Thread Jeremy Finzel
On Sat, Dec 16, 2017 at 8:31 AM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 12/15/17 23:50, Jeremy Finzel wrote: > > The common ground is some column in some table needs to be bulk updated. > > I may not be explaining well, but in our environment we have done > > hundreds of t

Re: Backfill bgworker Extension?

2017-12-16 Thread Peter Eisentraut
On 12/15/17 23:50, Jeremy Finzel wrote: > The common ground is some column in some table needs to be bulk updated. > I may not be explaining well, but in our environment we have done > hundreds of these using a generic framework to build a backfill. So I’m > not sure what you are questioning about

Re: Backfill bgworker Extension?

2017-12-15 Thread Jeremy Finzel
On Tue, Dec 12, 2017 at 2:26 PM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 12/12/17 13:03, Jeremy Finzel wrote: > > To be clear, what I mean is batch updating a large set of data in small > > pieces so as to avoid things like lock contention and replication lags. > > Sometime

Re: Backfill bgworker Extension?

2017-12-12 Thread Peter Eisentraut
On 12/12/17 13:03, Jeremy Finzel wrote: > To be clear, what I mean is batch updating a large set of data in small > pieces so as to avoid things like lock contention and replication lags.  > Sometimes these have a driving table that has the source data to update > in a destination table based on a

Backfill bgworker Extension?

2017-12-12 Thread Jeremy Finzel
One of our challenges we have is that our engineers have written frameworks to backfill data in several different DSLs, and every time they adopt a new language, they maybe need to write another one. To be clear, what I mean is batch updating a large set of data in small pieces so as to avoid thin