Re: [Bacula-users] PostgreSQL: WAL and log management
On Tue, 20 Nov 2018 17:47:49 -0600 Dimitri Maziuk via Bacula-users wrote: > Check that wal archive_mode is off in > postgresql.conf and also all replication-related > stuff. grep -iE "replic|archive_mode" postgresql.conf shows only commented lines. Need I change anything? > Try `psql -U postgres -d postgres -c checkpoint` and > see if the files start disappearing after that. They don't: $ du -hs pg_wal/ 769Mpg_wal/ $ psql -U postgres -d postgres -c checkpoint CHECKPOINT $ du -hs pg_wal/ 769Mpg_wal/ -- George ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] PostgreSQL: WAL and log management
PS https://www.postgresql.org/docs/10/sql-checkpoint.html -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu signature.asc Description: OpenPGP digital signature ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Looking for a LTO autoloader (NeoT24 works)
Sorry it’s late – I don’t read the Bacula list every day. We successfully use a Neo T24, LTO-6, SAS interface, LSI SAS 9207-8e controller, on UB 18.04LTS in ESXi 6.0 (you have to use hardware passthrough on the SAS controller card to use a SAS tape drive in ESXi). We previously used a Tandberg Magnum 224 built circa 2009. Except for SCSI interface & LTO-4 speed/capacity, it was a better unit – more steel, less plastic, easier to change tapes, better front-panel interface. That’s the limit of my experience with autoloaders, so I can’t offer a brand-to-brand comparison. Good luck, Dan Stieneke IT Specialist USDA - ARS - NWISRL 3793 N 3600 E Kimberly, ID 83341 208/423-6519 From: Markus Falb Sent: Monday, November 12, 2018 4:35 PM To: bacula-users@lists.sourceforge.net Subject: [Bacula-users] Looking for a LTO autoloader Hello, I am considering a purchase of a relatively small autoloader, but am not sure about compatibility with bacula. I found this one: https://www.overlandstorage.com/products/tape-libraries-and-autoloaders/neos-t24.aspx#Overview If anyone on this list has experiences with this autoloader + bacula, please let me know. I does know there is a "supported" list in the documentation http://www.bacula.org/9.2.x-manuals/en/main/Supported_Autochangers.html but don't know how comprehensive it is, there are some Overland models listed, but not this specific model (There doesn't exist a "not supported" list, does it? ) Also, suggestions (or dis-recommendations) for other autoloaders are appreciated, of course. Best Regards, Markus P.S. the Qualstar Q24 looks pretty identical to the NEOs T24 https://www.qualstar.com/qsr-q24.php Does anyone know what the differences are? This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately. ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] PostgreSQL: WAL and log management
On 11/20/2018 04:56 PM, George Anchev via Bacula-users wrote: > On the link I read wal_keep_segments "Specifies the > *minimum* number of past log file segments kept in the > pg_wal directory". Isn't there a setting which > controls the *maximum* number/volume/retention of those > files? (like in a log rotation) Not that I know of. They keep transactions, so they're not necessarily correspond to database size. Normally they get auto-recycled unless either a) archiving is on, but archive command keeps failing, or b) replication is on but the stand-by(s) keep failing to fetch them. Check that wal archive_mode is off in postgresql.conf and also all replication-related stuff. Try `psql -U postgres -d postgres -c checkpoint` and see if the files start disappearing after that. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu signature.asc Description: OpenPGP digital signature ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] PostgreSQL: WAL and log management
On Tue, 20 Nov 2018 16:34:58 -0600 Dimitri Maziuk via Bacula-users wrote: > I've 320MB on a server that replicates a database of > a few million rows, and ~100MB on our bacula server > w/ a year's worth of backups. What do you call "too > big"? As shared in the first message: Right now, even after drop_bacula_tables and no other databases (except what psql has as default after installation) I have: $ du -hs /var/lib/pgsql/data/pg_wal/ 769M/var/lib/pgsql/data/pg_wal/ That is for an empty bacula database. Those WAL files seem to be something like a "history" for what has been done during my tests: dropping tables, importing catalog, dropping again etc. The actual data which was used during those imports and drops (also shared in my first message): # du -hs bacula-backup.sql 63M bacula-backup.sql > There is wal_keep_segments: > https://www.postgresql.org/docs/10/runtime-config-replication.html > but if replication is not on, it's normally only a > handful of 16MB files in there, like 6 or so. Right now there are 52 files like those. I am not creating a super big enterprise system, so replication is not what I am looking for. On the link I read wal_keep_segments "Specifies the *minimum* number of past log file segments kept in the pg_wal directory". Isn't there a setting which controls the *maximum* number/volume/retention of those files? (like in a log rotation) -- George ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] PostgreSQL: WAL and log management
On 11/20/2018 03:49 PM, George Anchev via Bacula-users wrote: > On Mon, 19 Nov 2018 17:16:52 -0600 Dimitri Maziuk via > Bacula-users wrote: > >> https://www.postgresql.org/docs/10/routine-vacuuming.html > > Good info. Thanks. > >> WAL: don't worry about it unless you want to archive >> them or replicate your database. > > But the pg_wal directory is way too big and I have no > idea how to decrease it (without damaging anything). > Hence the whole question: how to manage that? Is there > a way to (auto)vacuum this too? I've 320MB on a server that replicates a database of a few million rows, and ~100MB on our bacula server w/ a year's worth of backups. What do you call "too big"? There is wal_keep_segments: https://www.postgresql.org/docs/10/runtime-config-replication.html but if replication is not on, it's normally only a handful of 16MB files in there, like 6 or so. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu signature.asc Description: OpenPGP digital signature ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] PostgreSQL: WAL and log management
On Mon, 19 Nov 2018 17:16:52 -0600 Dimitri Maziuk via Bacula-users wrote: > https://www.postgresql.org/docs/10/routine-vacuuming.html Good info. Thanks. > WAL: don't worry about it unless you want to archive > them or replicate your database. But the pg_wal directory is way too big and I have no idea how to decrease it (without damaging anything). Hence the whole question: how to manage that? Is there a way to (auto)vacuum this too? -- George ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
