The discussion on the backup manifest thread has gotten bogged down on the issue of the format that should be used to store the backup manifest file. I want something simple and ad-hoc; David Steele and Stephen Frost prefer JSON. That is problematic because our JSON parser does not work in frontend code, and I want to be able to validate a backup against its manifest, which involves being able to parse the manifest from frontend code. The latest development over there is that David Steele has posted the JSON parser that he wrote for pgbackrest with an offer to try to adapt it for use in front-end PostgreSQL code, an offer which I genuinely appreciate. I'll write more about that over on that thread. However, I decided to spend today doing some further investigation of an alternative approach, namely making the backend's existing JSON parser work in frontend code as well. I did not solve all the problems there, but I did come up with some patches which I think would be worth committing on independent grounds, and I think the whole series is worth posting. So here goes.
0001 moves wchar.c from src/backend/utils/mb to src/common. Unless I'm missing something, this seems like an overdue cleanup. It's long been the case that wchar.c is actually compiled and linked into both frontend and backend code. Commit 60f11b87a2349985230c08616fa8a34ffde934c8 added code into src/common that depends on wchar.c being available, but didn't actually make wchar.c part of src/common, which seems like an odd decision: the functions in the library are dependent on code that is not part of any library but whose source files get copied around where needed. Eh? 0002 does some basic header cleanup to make it possible to include the existing header file jsonapi.h in frontend code. The state of the JSON headers today looks generally poor. There seems not to have been much attempt to get the prototypes for a given source file, say foo.c, into a header file with the same name, say foo.h. Also, dependencies between various header files seem to be have added somewhat freely. This patch does not come close to fixing all that, but I consider it a modest down payment on a cleanup that probably ought to be taken further. 0003 splits json.c into two files, json.c and jsonapi.c. All the lexing and parsing stuff (whose prototypes are in jsonapi.h) goes into jsonapi.c, while the stuff that pertains to the 'json' data type remains in json.c. This also seems like a good cleanup, because to me, at least, it's not a great idea to mix together code that is used by both the json and jsonb data types as well as other things in the system that want to generate or parse json together with things that are specific to the 'json' data type. As far as I know all three of the above patches are committable as-is; review and contrary opinions welcome. On the other hand, 0004, 0005, and 0006 are charitably described as experimental or WIP. 0004 and 0005 hack up jsonapi.c so that it can still be compiled even if #include "postgres.h" is changed to #include "postgres-fe.h" and 0006 moves it into src/common. Note that I say that they make it compile, not work. It's not just untested; it's definitely broken. But it gives a feeling for what the remaining obstacles to making this code available in a frontend environment are. Since I wrote my very first email complaining about the difficulty of making the backend's JSON parser work in a frontend environment, one obstacle has been knocked down: StringInfo is now available in front-end code (commit 26aaf97b683d6258c098859e6b1268e1f5da242f). The remaining problems (that I know about) have to do with error reporting and multibyte character support; a read of the patches is suggested for those wanting further details. Suggestions welcome. Thanks, -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
0001-Move-wchar.c-to-src-common.patch
Description: Binary data
0002-Adjust-src-include-utils-jsonapi.h-so-it-s-not-backe.patch
Description: Binary data
0003-Split-JSON-lexer-parser-from-json-data-type-support.patch
Description: Binary data
0004-Introduce-json_error-macro.patch
Description: Binary data
0005-Frontendify-jsonapi.c.patch
Description: Binary data
0006-Move-jsonapi.c-to-src-common.patch
Description: Binary data