Hi Adam. Column headers in pgbench log files seem helpful. Besides programs, it seems helpful for humans to understand the column data as well. I was able to apply your patch and verify that the headers are added to the log file:
andy@MacBook-Air-4 ~/P/postgres (master)> rm pgbench_log.* andy@MacBook-Air-4 ~/P/postgres (master)> src/bin/pgbench/pgbench postgres://andy:@localhost:5432/postgres --log --log-header pgbench (17devel) .... andy@MacBook-Air-4 ~/P/postgres (master)> cat pgbench_log.* client_id transaction_no time script_no time_epoch time_us 0 1 8435 0 1699902315 902700 0 2 1130 0 1699902315 903973 ... The generated pgbench_log.62387 log file showed headers "client_id transaction_no time script_no time_epoch time_us". Hope that helps with your patch acceptance journey. Good luck! Andrew Atkinson On Mon, Nov 13, 2023 at 11:55 AM Adam Hendel <a...@tembo.io> wrote: > Hello Hackers! > > Currently, pgbench will log individual transactions to a logfile when the > `--log` parameter flag is provided. The logfile, however, does not include > column header. It has become a fairly standard expectation of users to have > column headers present in flat files. Without the header in the pgbench log > files, new users must navigate to the docs and piece together the column > headers themselves. Most industry leading frameworks have tooling built in > to read column headers though, for example python/pandas read_csv(). > > We can improve the experience for users by adding column headers to > pgbench logfiles with an optional command line flag, `--log-header`. This > will keep the API backwards compatible by making users opt-in to the column > headers. It follows the existing pattern of having conditional flags in > pgbench’s API; the `--log` option would have both –log-prefix and > –log-header if this work is accepted. > > The implementation considers the column headers only when the > `--log-header` flag is present. The values for the columns are taken > directly from the “Per-Transaction Logging” section in > https://www.postgresql.org/docs/current/pgbench.html and takes into > account the conditional columns `schedule_lag` and `retries`. > > > Below is an example of what that logfile will look like: > > > pgbench postgres://postgres:postgres@localhost:5432/postgres --log > --log-header > > client_id transaction_no time script_no time_epoch time_us > > 0 1 1863 0 1699555588 791102 > > 0 2 706 0 1699555588 791812 > > > If the interface and overall approach makes sense, I will work on adding > documentation and tests for this too. > > Respectfully, > > Adam Hendel > >
