Re: [GENERAL] how to ensure a client waits for a previous transaction to finish?

2009-12-09 Thread Dan Kortschak
Thanks for that clarification Merlin, The server/client is on a workstation that is essentially private (I share some time with students, but they don't have pg access). The locks are across sessions. There are three perl scripts that connect to a pg db, one loads the database and creates some com

Re: [GENERAL] how to ensure a client waits for a previous transaction to finish?

2009-12-09 Thread Dan Kortschak
Thanks Tom, That was my initial view and I'm still not convinced that I'm wrong - I can see no way that the server can tell that the query came from a process without a terminal (though I may be missing something here). Unfortunately I'm working at the boundary of my knowledge for both torque and

Re: [GENERAL] how to ensure a client waits for a previous transaction to finish?

2009-12-08 Thread Merlin Moncure
On Tue, Dec 8, 2009 at 10:13 PM, Dan Kortschak wrote: > I've spoken to people on the torque user mailing list and tried merlin's > suggestion below (which looked like it should work - but unfortunately > did not prevent the problem). > > >From working through things with the torque list, it seems

Re: [GENERAL] how to ensure a client waits for a previous transaction to finish?

2009-12-08 Thread Tom Lane
Dan Kortschak writes: > From working through things with the torque list, it seems to be the > case that postgresql is behaving differently because it is not attached > to a terminal (this has caused problems for others on that list with > sqlite and mysql). That seems unlikely. My best guess at

Re: [GENERAL] how to ensure a client waits for a previous transaction to finish?

2009-12-08 Thread Dan Kortschak
I've spoken to people on the torque user mailing list and tried merlin's suggestion below (which looked like it should work - but unfortunately did not prevent the problem). >From working through things with the torque list, it seems to be the case that postgresql is behaving differently because i

Re: [GENERAL] how to ensure a client waits for a previous transaction to finish?

2009-12-07 Thread Merlin Moncure
On Mon, Dec 7, 2009 at 5:32 PM, Dan Kortschak wrote: > Thanks to everyone who has answered this. The short answer is that > torque is not behaving the way I expected and not the way I have ever > seen it behave in the past. The I/O binding of these jobs may have > something to do with this, but I

Re: [GENERAL] how to ensure a client waits for a previous transaction to finish?

2009-12-07 Thread Dan Kortschak
Thanks to everyone who has answered this. The short answer is that torque is not behaving the way I expected and not the way I have ever seen it behave in the past. The I/O binding of these jobs may have something to do with this, but I will look into it further. cheers On Mon, 2009-12-07 at 13:2

Re: [GENERAL] how to ensure a client waits for a previous transaction to finish?

2009-12-07 Thread Vick Khera
On Mon, Dec 7, 2009 at 4:21 PM, Dan Kortschak wrote: > The automated runs call each script as part of a PBS torque > script (essentially a sh script calls each in turn). > Sounds more like it calls them in parallel, else your populate script would have finished before your query script ran assumi

Re: [GENERAL] how to ensure a client waits for a previous transaction to finish?

2009-12-07 Thread John R Pierce
Dan Kortschak wrote: Yes, they are separate perl files (I'm thinking that perhaps this wasn't the best way to do it now, but for the moment I'm going to have to stick with it). In the case of the manual testing it's jus a matter of command line calls. The automated runs call each script as part

Re: [GENERAL] how to ensure a client waits for a previous transaction to finish?

2009-12-07 Thread Vick Khera
On Mon, Dec 7, 2009 at 4:00 PM, Dan Kortschak wrote: > During manual testing everything works, but automating it results in the > query scripts starting before the population transactions having > completed, so they give erroneous result. > Update your process to have the query scripts wait until

Re: [GENERAL] how to ensure a client waits for a previous transaction to finish?

2009-12-07 Thread Dan Kortschak
Thanks for that, that should help me sort it out. I haven't used the autocommit option in pgdbi. I'll have a look to see if DBI::do has an option to wait for command completion. cheers On Mon, 2009-12-07 at 16:12 -0500, Tom Lane wrote: > It's not. What you want is to COMMIT and make sure you've

Re: [GENERAL] how to ensure a client waits for a previous transaction to finish?

2009-12-07 Thread Dan Kortschak
Yes, they are separate perl files (I'm thinking that perhaps this wasn't the best way to do it now, but for the moment I'm going to have to stick with it). In the case of the manual testing it's jus a matter of command line calls. The automated runs call each script as part of a PBS torque script

Re: [GENERAL] how to ensure a client waits for a previous transaction to finish?

2009-12-07 Thread John R Pierce
Dan Kortschak wrote: Hi, this is a bit of a noob question. I am using PGSql to perform some large analyses, with the clients being a sequentially run set of perl scripts (one to set up and populate tables and then down stream scripts to query the database for the results). when you say 'a

Re: [GENERAL] how to ensure a client waits for a previous transaction to finish?

2009-12-07 Thread Tom Lane
Dan Kortschak writes: > How can I ensure the query scripts wait until the population transaction > processes have finished? I've looked at 'SET TRANSACTION ISOLATION LEVEL > SERIALIZABLE' (to us in the query scripts I thought), but I'm not sure > that is what I want. It's not. What you want is t

[GENERAL] how to ensure a client waits for a previous transaction to finish?

2009-12-07 Thread Dan Kortschak
Hi, this is a bit of a noob question. I am using PGSql to perform some large analyses, with the clients being a sequentially run set of perl scripts (one to set up and populate tables and then down stream scripts to query the database for the results). During manual testing everything works, but