On Fri, Mar 16, 2012 at 8:34 PM, Greg Stark <st...@mit.edu> wrote: > On Fri, Mar 16, 2012 at 11:29 PM, Jeff Davis <pg...@j-davis.com> wrote: > > There is a lot of difference between those two. In particular, it looks > > like the problem you are seeing is coming from the background writer, > > which is not running during initdb. > > The difference that comes to mind is that the postmaster forks. If the > library opens any connections prior to forking and then uses them > after forking that would work at first but it would get confused > quickly once more than one backend tries to use the same connection. > The data being sent would all be mixed together and they would see > responses to requests other processes sent. >
The opened connections thing was the first thing I thought of.. the HDFS documentation claims the C interface is thread-safe.. However, since I noticed that different processes (having different process IDs) are calling the mdwrite, then it might be a possibility. I tried reconnecting to HDFS on every mdwrite request, but it didn't work out. > You need to ensure that any network connections are opened up *after* > the new processes are forked. > > There are other things going on that could cause problems. initdb > probably doesn't deal with many errors so it might not be casing any > longjmps that happen when Postgres handles errors. I suspect it > doesn't create any temporary files, etc. > > There's also a long history of threads not playing well with Postgres. > I think that's overblown and I believe they should work fine. Most of > it was caused by a bug in an old version of libc. But you do have to > ensure that the other threads don't call any Postgres functions > because the Postgres code is very much not thread-aware. > > -- > greg > I'm novice in PG, and if I got this one running then I'll have achieved what I wanted to do without further digging in PG. So I was thinking if there was a configuration option (or something similar) that will eliminate (or reduce) the role the background writer process.. I believe it can be one of the WAL options but I'm not sure.. Any suggestions?
