Re: [PATCHES] [HACKERS] large object regression tests
Sorry, please disregard. Was means for a Japanese FAQ update. --- Bruce Momjian wrote: > > Patch applied. Thanks. > > --- > > > Jeremy Drake wrote: > > On Sun, 24 Sep 2006, Jeremy Drake wrote: > > > > > On Thu, 21 Sep 2006, Tom Lane wrote: > > > > > > > I think we could do without the Moby Dick extract too ... > > > > > > I am open to suggestions. I saw one suggestion that I use an image of an > > > elephant, but I suspect that was tongue-in-cheek. I am not very fond of > > > the idea of generating repetitious data, as I think it would be more > > > difficult to determine whether or not the loseek/tell functions put me in > > > the right place in the middle of the file. > > > > I just had the idea that I could use one of the existing data files which > > are used for testing COPY instead of the Moby Dick extract. They are > > already there, a few of them are pretty good sized, they have data in the > > file which is not just simple repetition so it would be pretty obvious if > > the seek function broke, and they are very unlikely to change. I am > > considering changing the test I put together to use tenk.data as the input > > file tomorrow and send in what I have again, since I also am doing a test > > of \lo_import (which also requires a patch to psql I sent in earlier to > > fix the output of the \lo_* commands to respect the output settings). > > > > -- > > When does summertime come to Minnesota, you ask? > > Well, last year, I think it was a Tuesday. > > > > ---(end of broadcast)--- > > TIP 3: Have you checked our extensive FAQ? > > > >http://www.postgresql.org/docs/faq > > -- > Bruce Momjian [EMAIL PROTECTED] > EnterpriseDBhttp://www.enterprisedb.com > > + If your life is a hard drive, Christ can be your backup. + > > ---(end of broadcast)--- > TIP 1: if posting/reading through Usenet, please send an appropriate >subscribe-nomail command to [EMAIL PROTECTED] so that your >message can get through to the mailing list cleanly -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] [HACKERS] large object regression tests
Patch applied. Thanks. --- Jeremy Drake wrote: > On Sun, 24 Sep 2006, Jeremy Drake wrote: > > > On Thu, 21 Sep 2006, Tom Lane wrote: > > > > > I think we could do without the Moby Dick extract too ... > > > > I am open to suggestions. I saw one suggestion that I use an image of an > > elephant, but I suspect that was tongue-in-cheek. I am not very fond of > > the idea of generating repetitious data, as I think it would be more > > difficult to determine whether or not the loseek/tell functions put me in > > the right place in the middle of the file. > > I just had the idea that I could use one of the existing data files which > are used for testing COPY instead of the Moby Dick extract. They are > already there, a few of them are pretty good sized, they have data in the > file which is not just simple repetition so it would be pretty obvious if > the seek function broke, and they are very unlikely to change. I am > considering changing the test I put together to use tenk.data as the input > file tomorrow and send in what I have again, since I also am doing a test > of \lo_import (which also requires a patch to psql I sent in earlier to > fix the output of the \lo_* commands to respect the output settings). > > -- > When does summertime come to Minnesota, you ask? > Well, last year, I think it was a Tuesday. > > ---(end of broadcast)--- > TIP 3: Have you checked our extensive FAQ? > >http://www.postgresql.org/docs/faq -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] [HACKERS] large object regression tests
On Mon, 25 Sep 2006, Tom Lane wrote: > Jeremy Drake <[EMAIL PROTECTED]> writes: > > I just tried using the \lo_import command in a regression test, and I > > think I figured out why this will not work: > > ... > > Yes, that's the large object OID in the output there, and it is different > > each run (as I expect). > > Right. I'd suggest temporarily setting ECHO off to hide the > unpredictable part of the output. There are similar measures taken in > many of the contrib tests. I tried this: jeremyd=# \set QUIET jeremyd=# \set ECHO off jeremyd=# BEGIN; jeremyd=# \lo_import results/lotest.txt lo_import 84951 jeremyd=# ROLLBACK; >From what I could tell in the code, the message is printed regardless of setting. It looks like the large_obj.c output is missing much of the output settings handling which is in the PrintQueryStatus function in common.c, such as handling quiet mode, and html output. I will try to dig around and try to put together a patch to make it respect the settings like other commands... -- "You are old," said the youth, "and your programs don't run, And there isn't one language you like; Yet of useful suggestions for help you have none -- Have you thought about taking a hike?" "Since I never write programs," his father replied, "Every language looks equally bad; Yet the people keep paying to read all my books And don't realize that they've been had." ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] [HACKERS] large object regression tests
On Sun, 24 Sep 2006, Jeremy Drake wrote: > On Thu, 21 Sep 2006, Tom Lane wrote: > > > I suggest that instead of testing the server-side lo_import/lo_export > > functions, perhaps you could test the psql equivalents and write and > > read a file in psql's working directory. > In the mean time, I will alter the test to also test the psql backslash > commands based on how the copy equivalents are tested, since I had > forgotten them and they need to be tested also. I just tried using the \lo_import command in a regression test, and I think I figured out why this will not work: $ make check ... largeobject ... FAILED ... $ cat regression.diffs *** ./expected/largeobject.out Sun Sep 24 19:55:25 2006 --- ./results/largeobject.out Sun Sep 24 19:55:58 2006 *** *** 188,194 (1 row) \lo_import 'results/lotest.txt' ! lo_import 31138 \set newloid :LASTOID -- This is a hack to test that export/import are reversible -- This uses knowledge about the inner workings of large object mechanism --- 188,194 (1 row) \lo_import 'results/lotest.txt' ! lo_import 31199 \set newloid :LASTOID -- This is a hack to test that export/import are reversible -- This uses knowledge about the inner workings of large object mechanism == Yes, that's the large object OID in the output there, and it is different each run (as I expect). If you look at src/bin/psql/large_obj.c line 192, you see: fprintf(pset.queryFout, "lo_import %u\n", loid); Which is executed unconditionally whenever the lo_import is successful. While in a normal circumstance, it is quite necessary to know the loid, since it does change each call, in this case it serves to break the diffs, and so I guess it is impossible to use the \lo_import command in a regression test. -- The first time, it's a KLUDGE! The second, a trick. Later, it's a well-established technique! -- Mike Broido, Intermetrics ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] [HACKERS] large object regression tests
On Sun, 24 Sep 2006, Jeremy Drake wrote: > On Thu, 21 Sep 2006, Tom Lane wrote: > > > I think we could do without the Moby Dick extract too ... > > I am open to suggestions. I saw one suggestion that I use an image of an > elephant, but I suspect that was tongue-in-cheek. I am not very fond of > the idea of generating repetitious data, as I think it would be more > difficult to determine whether or not the loseek/tell functions put me in > the right place in the middle of the file. I just had the idea that I could use one of the existing data files which are used for testing COPY instead of the Moby Dick extract. They are already there, a few of them are pretty good sized, they have data in the file which is not just simple repetition so it would be pretty obvious if the seek function broke, and they are very unlikely to change. I am considering changing the test I put together to use tenk.data as the input file tomorrow and send in what I have again, since I also am doing a test of \lo_import (which also requires a patch to psql I sent in earlier to fix the output of the \lo_* commands to respect the output settings). -- When does summertime come to Minnesota, you ask? Well, last year, I think it was a Tuesday. ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] [HACKERS] large object regression tests
Jeremy Drake <[EMAIL PROTECTED]> writes: > I just tried using the \lo_import command in a regression test, and I > think I figured out why this will not work: > ... > Yes, that's the large object OID in the output there, and it is different > each run (as I expect). Right. I'd suggest temporarily setting ECHO off to hide the unpredictable part of the output. There are similar measures taken in many of the contrib tests. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] [HACKERS] large object regression tests
On Sun, 24 Sep 2006, Jeremy Drake wrote: > On Thu, 21 Sep 2006, Tom Lane wrote: > > > I suggest that instead of testing the server-side lo_import/lo_export > > functions, perhaps you could test the psql equivalents and write and > > read a file in psql's working directory. > > I did not see any precedent for that when I was looking around in the > existing tests for an example of how to do things. > When I was looking at the copy tests, it looked like the server-side ones > were tested, and then the psql ones were tested by exporting and then > importing data which was originally loaded from the server-side method. I just went back and looked at the tests again. The only time the psql \copy command was used was in the (quite recent IIRC) copyselect test, and then only via stdout (never referring to psql working directory, or to files at all). Did I misunderstand, and you are proposing a completely new way of doing things in the regression tests? I am not particularly fond of the sed substitution stuff myself, but it seems to be the only currently supported/used method in the regression tests... I do think that making the large object test and the copy test consistent would make a lot of sense, since as I said before, the functionality of file access is so similar... -- We demand rigidly defined areas of doubt and uncertainty! -- Vroomfondel ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] [HACKERS] large object regression tests
On Thu, 21 Sep 2006, Tom Lane wrote: > Jeremy Drake <[EMAIL PROTECTED]> writes: > > I put together a patch which adds a regression test for large objects, > > hopefully attached to this message. I would like some critique of it, to > > see if I have gone about it the right way. Also I would be happy to hear > > any additional tests which should be added to it. > > I'd prefer it if we could arrange not to need any absolute paths > embedded into the test, because maintaining tests that require such is > a real PITA --- instead of just committing the actual test output, one > has to reverse-convert it to a ".source" file. I just copied how the test for COPY worked, since I perceived a similarity in what I needed to do (use external files to load data). > I suggest that instead of testing the server-side lo_import/lo_export > functions, perhaps you could test the psql equivalents and write and > read a file in psql's working directory. I did not see any precedent for that when I was looking around in the existing tests for an example of how to do things. I am not even sure where the cwd of psql is, so I can put an input file there. Could you provide an example of how this might look, by telling me where to put a file in the src/test/regress tree and the path to give to \lo_import? Besides which, shouldn't both the server-side and psql versions be tested? When I was looking at the copy tests, it looked like the server-side ones were tested, and then the psql ones were tested by exporting and then importing data which was originally loaded from the server-side method. Am I correctly interpreting the precedent, or are you suggesting that the precedent be changed? I was trying to stay as close to the copy tests as possible since the functionality is so similar (transferring data to/from files in the filesystem, either via server-side functions which require absolute paths or via psql \ commands (which I forgot about for the lo funcs)). > I think we could do without the Moby Dick extract too ... I am open to suggestions. I saw one suggestion that I use an image of an elephant, but I suspect that was tongue-in-cheek. I am not very fond of the idea of generating repetitious data, as I think it would be more difficult to determine whether or not the loseek/tell functions put me in the right place in the middle of the file. Perhaps if there was a way to generate deterministic pseudo-random data, that would work (has to be deterministic so the diffs of the output come out right). Anyone have a good example of seeding a random number generator and generating a bunch of bytea which is deterministic cross-platform? > > regards, tom lane > In the mean time, I will alter the test to also test the psql backslash commands based on how the copy equivalents are tested, since I had forgotten them and they need to be tested also. -- Any sufficiently advanced technology is indistinguishable from a rigged demo. ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PATCHES] [HACKERS] large object regression tests
Jeremy Drake <[EMAIL PROTECTED]> writes: > I put together a patch which adds a regression test for large objects, > hopefully attached to this message. I would like some critique of it, to > see if I have gone about it the right way. Also I would be happy to hear > any additional tests which should be added to it. I'd prefer it if we could arrange not to need any absolute paths embedded into the test, because maintaining tests that require such is a real PITA --- instead of just committing the actual test output, one has to reverse-convert it to a ".source" file. I suggest that instead of testing the server-side lo_import/lo_export functions, perhaps you could test the psql equivalents and write and read a file in psql's working directory. I think we could do without the Moby Dick extract too ... regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
