Re: [Bacula-users] Migrating from mariadb to postgresql
Hello Uwe I've made a script to migrate MySQL/MariaDB to PostgreSQL https://github.com/wanderleihuttel/bacula-utils/tree/master/convert_mysql_to_postgresql I've found errors only in the Log table. I've looked for chars with wrong encoding and make an update in MySQL, did a dump and import again for PostgreSQL only this table Best regards *Wanderlei Hüttel* Em ter., 30 de ago. de 2022 às 13:44, Martin Simmons escreveu: > > On Tue, 30 Aug 2022 16:04:08 +0200, Uwe Schuerkamp said: > > > > Hello Charles, > > > > thanks for "INSERT only" idea... it's worked partially. After removing > > some double quotes and other characters using sed psql didn't like, > > the import runs for a while and then stops with the following error: > > > > INSERT 0 6 > > ERROR: duplicate key value violates unique constraint "status_pkey" > > DETAIL: Key (jobstatus)=(A) already exists. > > > > > > Any idea what could be causing this? > > That looks like the JobStatus table. Don't migrate that one because the > make_postgresql_tables script inserts all of the rows that are needed. > > __Martin > > > ___ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users > ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Migrating from mariadb to postgresql
Hi Uwe, I think Martin was right with this one. You are probably going to have to edit the export file from mysqldump to remove the inserts for tables that the Postgres schema maker creates automatically. To make this easier, so you don't have to recreate the schema each time, you can use transactions. So run "BEGIN WORK;", import the dump file, if something throws an error enter "ROLLBACK WORK;" and edit out those lines, then try again from "BEGIN WORK;" Once it runs without errors do "COMMIT WORK;" to save the import state. On 2022-08-30 13:41, Martin Simmons wrote: On Tue, 30 Aug 2022 16:04:08 +0200, Uwe Schuerkamp said: Hello Charles, thanks for "INSERT only" idea... it's worked partially. After removing some double quotes and other characters using sed psql didn't like, the import runs for a while and then stops with the following error: INSERT 0 6 ERROR: duplicate key value violates unique constraint "status_pkey" DETAIL: Key (jobstatus)=(A) already exists. Any idea what could be causing this? That looks like the JobStatus table. Don't migrate that one because the make_postgresql_tables script inserts all of the rows that are needed. __Martin ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Community 11.0.6 binaries for el9
When are you going to release Bacula Community 11.0.6 binaries for EL9? We are on hold with our el9 deployment waiting for Bacula 11.0.6 binaries. OL9 (Oracle Linux 9, a redist of RHEL9) is available for free from here: https://yum.oracle.com/oracle-linux-isos.html --- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Need help with fileset declaration - what's my mistake?
Martin, you are right, my bad, I overlooked that you were not using RegexDir,
but Regex for the exclude.
I am very glad that your solution works, thank you so much. I have to admit, on
that evening I had been mucking around a lot more and also a friend, and none
of our attempts worked as intended.
So still, I find the definition of filesets very confusing, I hope I will get a
better understanding through your example.
Thank you again!
J/C
> On 30. Aug 2022, at 15:44, Martin Simmons wrote:
>
> The order of the File declarations relative to the Options declarations
> doesn't matter. However, the order of the Options declarations relative to
> each other does matter (Bacula uses the first Options that matches based on
> RegexDir etc).
>
> The problem below is that you are using RegexDir instead of Regex in the
> Exclude Options clause.
>
> __Martin
>
>
>> On Fri, 26 Aug 2022 22:37:08 +0200, Justin Case said:
>>
>> Martin, thanks, I admit that for me the fileset definition is one of the
>> more confusing things in Bacula. Thanks for the explanation, however,
>> honestly, I don’t get it.
>>
>> I adapted the fileset according to your proposal, and this is what baculum
>> makes from it:
>>
>> Fileset {
>> Name = "test-appdata"
>> EnableVss = no
>> EnableSnapshot = no
>> Include {
>> File = "/home/dockerman"
>> Options {
>> OneFs = "No"
>> RegexDir = "^/home/dockerman/containerdata-"
>> }
>> Options {
>> OneFs = "No"
>> Exclude = "Yes"
>> RegexDir = "^/home/dockerman/[^/]+$"
>> }
>> }
>>
>> Running this results in actually nearly the same result. The delta of the
>> outcome are actually 3 files (gleaned form the numbers of the jobs, I don’t
>> know which files that are).
>>
>> Does the order of the declarations (as seen above) have anything to do with
>> this result?
>>
>>> On 26. Aug 2022, at 21:24, Martin Simmons wrote:
>>>
On Fri, 26 Aug 2022 20:41:22 +0200, Justin Case said:
So actually, the fileset did back up the /home/../containerdata.. stuff,
but
also other files outside of /home/,,,/containderdata, which I did not
intend
to happen.
in the /home/dockerman there are only dot-files sich as .Xauthority and
.ssh/config Why would those have been backed up by this fileset?
And then I realized that this was also the case for the compound fileset!
The fileset does actually back up the containerdata-* folder contents both
in /home/dockerman and in /mnt. The amount of data in /home is just much
smaller and in baculum the order of the files shown suggested that files
other then /home/../containerdata-… were backed up, and the files on
containerdata couldn’t be seen in the file list.
Still the same caveat as above holds for the compound fileset, too. I don’t
understand that.
>>>
>>> It does that because your fileset definition tells it to :-)
>>>
>>> The problem is that "^/home/dockerman/.*/" only matches directories two
>>> levels
>>> below /home/dockerman, such as /home/dockerman/dir1/dir2. Everything above
>>> that is included, for example directories like /home/dockerman/dir1 and
>>> files
>>> like /home/dockerman/file1 and /home/dockerman/dir1/file1.
>>>
>>> I think this does what you want:
>>>
>>> Include {
>>> Options {
>>> OneFs = no
>>> # override the Exclude for this prefix
>>> RegexDir = "^/home/dockerman/containerdata-"
>>> }
>>> Options {
>>> OneFs = no
>>> Exclude = yes
>>> # disallow all top-level things not explicitly included above
>>> Regex = "^/home/dockerman/[^/]+$"
>>> }
>>> File = "/home/dockerman"
>>> }
>>>
>>> Note that the paths inside the /home/dockerman/containerdata-... directories
>>> will be included implicitly because they don't match the excluded Regex.
>>> The
>>> Options (except Exclude) from the last clause will be used for those paths.
>>>
>>> __Martin
>>
>>
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Migrating from mariadb to postgresql
> On Tue, 30 Aug 2022 16:04:08 +0200, Uwe Schuerkamp said: > > Hello Charles, > > thanks for "INSERT only" idea... it's worked partially. After removing > some double quotes and other characters using sed psql didn't like, > the import runs for a while and then stops with the following error: > > INSERT 0 6 > ERROR: duplicate key value violates unique constraint "status_pkey" > DETAIL: Key (jobstatus)=(A) already exists. > > > Any idea what could be causing this? That looks like the JobStatus table. Don't migrate that one because the make_postgresql_tables script inserts all of the rows that are needed. __Martin ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Q: what do JobtoVerify, WriteVerifyList ?
Hi all, I am using Baculum to manage my Bacula V11 installation. Today I wanted to define a verify job for the first time. In Baculum I find 3 fields in the Verify section: VerifyJob: AFAU this is the name of the job which should be verified (I am using Level “Data). JobToVerify: no idea what this is, sounds like the same thing as VerifyJob. I found no documentation on this. WriteVerifyList: no idea what this is. I found no documentation on this. Does somebody know what JobToVerify and WriteVerifyList are doing and would care to explain it a bit? Thanks for you time, J/C ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Migrating from mariadb to postgresql
On 2022-08-30 9:04 AM, Uwe Schuerkamp wrote: INSERT 0 6 ERROR: duplicate key value violates unique constraint "status_pkey" DETAIL: Key (jobstatus)=(A) already exists. Any idea what could be causing this? Without looking at the actual DDL, I'd say your tables are wrong and primary key should be (jobid,jobstatus). ICBW and all that. Dima ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Migrating from mariadb to postgresql
> On Mon, 29 Aug 2022 08:28:31 +0200, Uwe Schuerkamp said: > Mail-Followup-To: Uwe Schuerkamp , > > On Thu, Aug 25, 2022 at 06:57:46PM +0100, Martin Simmons wrote: > > Do you have non-ASCII characters in your volume, job or client names? If > > not, > > then I don't see why the warning would cause them to look quite funny > > (whatever that means). > > > > __Martin > > Hi folks, > > thanks for your answers. No, I don't have any non-ascii chars in my > client, job or volume names. By "looking funny" I mean that for > instance the disk volume name "zif-full-0001" turns into > \xverlongrowofrandomcharacters. > > Of course I could start with an empty catalog but it'd be nice to > transfer the existing history to the postgres db which I guess would > (or should, even) be possible somehow. In another experiment I > migrated a mariadb django backend to postgresql on the same machine > without any issues, it's just bacula that's acting up in this case. Bacula will probably not work if pgloader created the schema. I think you should do that part with Bacula's make_postgresql_tables script and configure pgloader to keep that schema (i.e. the opposite of most of the "WITH" clauses that you mentioned originally. Many of the fields in Bacula's mysql schema use BLOB types. The doc for pgloader says that blob is converted to the bytea type in postgresql, so I don't know if that will cause translation problems after using make_postgresql_tables. You might need to configure pgloader's type conversion rules. __Martin ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Migrating from mariadb to postgresql
Hello Charles, thanks for "INSERT only" idea... it's worked partially. After removing some double quotes and other characters using sed psql didn't like, the import runs for a while and then stops with the following error: INSERT 0 6 ERROR: duplicate key value violates unique constraint "status_pkey" DETAIL: Key (jobstatus)=(A) already exists. Any idea what could be causing this? Again thanks for your help! All the best, Uwe -- Uwe Schürkamp // email: ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Need help with fileset declaration - what's my mistake?
The order of the File declarations relative to the Options declarations
doesn't matter. However, the order of the Options declarations relative to
each other does matter (Bacula uses the first Options that matches based on
RegexDir etc).
The problem below is that you are using RegexDir instead of Regex in the
Exclude Options clause.
__Martin
> On Fri, 26 Aug 2022 22:37:08 +0200, Justin Case said:
>
> Martin, thanks, I admit that for me the fileset definition is one of the more
> confusing things in Bacula. Thanks for the explanation, however, honestly, I
> don’t get it.
>
> I adapted the fileset according to your proposal, and this is what baculum
> makes from it:
>
> Fileset {
> Name = "test-appdata"
> EnableVss = no
> EnableSnapshot = no
> Include {
> File = "/home/dockerman"
> Options {
> OneFs = "No"
> RegexDir = "^/home/dockerman/containerdata-"
> }
> Options {
> OneFs = "No"
> Exclude = "Yes"
> RegexDir = "^/home/dockerman/[^/]+$"
> }
> }
>
> Running this results in actually nearly the same result. The delta of the
> outcome are actually 3 files (gleaned form the numbers of the jobs, I don’t
> know which files that are).
>
> Does the order of the declarations (as seen above) have anything to do with
> this result?
>
> > On 26. Aug 2022, at 21:24, Martin Simmons wrote:
> >
> >> On Fri, 26 Aug 2022 20:41:22 +0200, Justin Case said:
> >>
> >> So actually, the fileset did back up the /home/../containerdata.. stuff,
> >> but
> >> also other files outside of /home/,,,/containderdata, which I did not
> >> intend
> >> to happen.
> >>
> >> in the /home/dockerman there are only dot-files sich as .Xauthority and
> >> .ssh/config Why would those have been backed up by this fileset?
> >>
> >> And then I realized that this was also the case for the compound fileset!
> >> The fileset does actually back up the containerdata-* folder contents both
> >> in /home/dockerman and in /mnt. The amount of data in /home is just much
> >> smaller and in baculum the order of the files shown suggested that files
> >> other then /home/../containerdata-… were backed up, and the files on
> >> containerdata couldn’t be seen in the file list.
> >>
> >> Still the same caveat as above holds for the compound fileset, too. I don’t
> >> understand that.
> >
> > It does that because your fileset definition tells it to :-)
> >
> > The problem is that "^/home/dockerman/.*/" only matches directories two
> > levels
> > below /home/dockerman, such as /home/dockerman/dir1/dir2. Everything above
> > that is included, for example directories like /home/dockerman/dir1 and
> > files
> > like /home/dockerman/file1 and /home/dockerman/dir1/file1.
> >
> > I think this does what you want:
> >
> > Include {
> >Options {
> > OneFs = no
> > # override the Exclude for this prefix
> > RegexDir = "^/home/dockerman/containerdata-"
> >}
> >Options {
> > OneFs = no
> > Exclude = yes
> > # disallow all top-level things not explicitly included above
> > Regex = "^/home/dockerman/[^/]+$"
> >}
> >File = "/home/dockerman"
> > }
> >
> > Note that the paths inside the /home/dockerman/containerdata-... directories
> > will be included implicitly because they don't match the excluded Regex.
> > The
> > Options (except Exclude) from the last clause will be used for those paths.
> >
> > __Martin
>
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Q. storage groups for a given pool using several external USB HDDs?
Greetings, I am interested in the new V13 storage groups feature. If I wanted a given pool to use file folumes on several externally connected USB HDDs: -the devices would be connected all the time, would the director fill the devices one ofter the other, or would it scatter the jobs randomly across the devices? -may I extend the list of storages in a given pool to extend the available storage space by adding USB HDD devices to the storage? (I mean distinct HDD devices, not enlarging the storage of a RAID!) -This would mean that I don’t need to use vchanger vor this? Thanks for shedding a bit more light on this new feature. Best J/C ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Bacula Status report - 30 August 2022
The topics for this Status Report are: 1. Bacula Release Status 2. Bacula Binaries 3. Windows Binaries 4. macOS Binaries 5. New Services === 1. Bacula Release Status The latest Bacula version 13.0.1 was released to Source Forge and to www.bacula.org. This version includes many enhancements to Baculum (the Web GUI interface to Bacula), new Storage Group support for the Dir and the SD, a Kubernetes plugin, a new Accurate option to update only meta data during an incremental backup job, external LDAP support for Console and many more. Note: The version 13.0.0 has a problem to start the Storage Daemon systemd service. 2. Bacula Binaries Thanks to the build support from Bacula Systems, we now have binaries for a number of popular distributions. If you need additional distributions, please notify us on the bacula-users email list. The release format is with standard packaging using debs rpms, ... that are specific to each system. However, unlike most Linux distributions, we install the Community Binaries into the directory /opt/bacula as recommended in the Bacula manual and as used by the Bacula Enterprise Edition. This Bacula file release standard is not uncommon and vastly simplifies installation, support, backup and recovery of Bacula itself. Basically, most of the release will be installed into /opt/bacula except for the few system files that must be placed in certain locations such as the start/stop scripts and the man text. To register and get access to Bacula pre-built binaries please go to www.bacula.org and on the main menu bar select Downloads -> Bacula Binary Package Download, then fill out the registration. Note, ultimately you will receive a special download link that will give you access to the binaries. Please keep that link for future access to the binary repository for updates and new releases. 3. Windows Binaries They can be found in www.bacula.org -> Downloads -> Windows Binaries 4. macOS Binaries New macOS packages can be found in www.bacula.org -> Downloads -> Deb, RPM and OSx Packages 5. Project Services Update Bacula.org server had a serious hardware issue in June. We have been able to recover most of the data, and we have moved the services to a new location. It is now hosted by Bacula Systems in a secured environment. We are in the process of converting and upgrading our different web services (cgit, mantis, cdash, wiki) to an on-premise gitlab ultimate instance. This will take a bit of time and I will keep you updated. If you have an account in Mantis, you will receive an activation email for the new Gitlab instance. Unfortunately, CDASH and the Wiki are still not available today - we anticipate these will be replaced by Gitlab. Indeed, the wiki has already been ported into GitLab. The archive of the Wiki is available on: https://web.archive.org/web/20220125230419/https://wiki.bacula.org/doku.php Thanks for using Bacula — be happy because there is much more to come! ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Gitlab migration
Dear All, As a valued member of the Bacula Community and a user of either Git or Mantis bug tracker for Bacula Community, we would like to inform you that we have started to migrate your mantis account and issues to the new GitLab instance. The Gitlab instance, like the git repository and Mantis bug tracker is hosted by Bacula Systems in Switzerland. Our IT team will be migrating issues from Mantis to GitLab and we will also redirect the old git repository to be centralized in our GitLab. During this migration, please do not create new issues on Gitlab. If you **DO NOT** want us to create an account in the new system for you, please reply to the welcome message and gitlab will record your opt-out. Thank you for using Bacula and we look forward to seeing you in the new Community Edition Gitlab shortly. Best Regards, Eric ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
