[pve-devel] backup suspend mode with guest agent enable : fsfreeze timeout

2016-09-29 Thread Alexandre DERUMIER
Hi, if we try to run backup in pause mode with guest agent, it seem that the fsfreeze qmp command is send after the suspend, so the guest agent is not responding. INFO: suspend vm INFO: snapshots found (not included into backup) INFO: creating archive '/var/lib/vz/dump/vzdump-qemu-104-2016_09_

Re: [pve-devel] backup : adding a storage option like "max concurrent backup" ?

2015-11-26 Thread Alexandre DERUMIER
x27;ll try to look at this. - Mail original - De: "Thomas Lamprecht" À: "pve-devel" Envoyé: Jeudi 26 Novembre 2015 09:42:12 Objet: Re: [pve-devel] backup : adding a storage option like "max concurrent backup" ? On 11/26/2015 09:04 AM, Dietmar Maurer wrote: >

Re: [pve-devel] backup : adding a storage option like "max concurrent backup" ?

2015-11-26 Thread Alexandre DERUMIER
x27;ll try to look at this. - Mail original - De: "Thomas Lamprecht" À: "pve-devel" Envoyé: Jeudi 26 Novembre 2015 09:42:12 Objet: Re: [pve-devel] backup : adding a storage option like "max concurrent backup" ? On 11/26/2015 09:04 AM, Dietmar Maurer wrote: >

Re: [pve-devel] backup : adding a storage option like "max concurrent backup" ?

2015-11-26 Thread Thomas Lamprecht
On 11/26/2015 09:04 AM, Dietmar Maurer wrote: My idea was to add some kind of job queue for the backup storage You could use the pmxcfs locking feature (instead of local lock)... That way only one task can be started cluster wide. The cfs_lock_domain method could be helpful here: add func

Re: [pve-devel] backup : adding a storage option like "max concurrent backup" ?

2015-11-26 Thread Dietmar Maurer
> My idea was to add some kind of job queue for the backup storage You could use the pmxcfs locking feature (instead of local lock)... That way only one task can be started cluster wide. ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.p

Re: [pve-devel] backup : adding a storage option like "max concurrent backup" ?

2015-11-25 Thread Alexandre DERUMIER
gration, backups duration can change if vms numbers is different from one day to another day) My idea was to add some kind of job queue for the backup storage - Mail original - De: "dietmar" À: "aderumier" , "pve-devel" Envoyé: Mercredi 25 Novembre 2015 19

Re: [pve-devel] backup : adding a storage option like "max concurrent backup" ?

2015-11-25 Thread Dietmar Maurer
> The problem is that if you defined a backup job for all nodes at a specific > time, > > It's launching the backup job on all nodes at the same time, We usually simply define one job per node. > and backup storage or network can be overloaded. Or you limit vzdump bwlimit on the nodes. ___

[pve-devel] backup : adding a storage option like "max concurrent backup" ?

2015-11-23 Thread Alexandre DERUMIER
Hi, during my last training sessions, I have a lot of request to add an option on backup storage, to limit the number of max concurrent backup. The problem is that if you defined a backup job for all nodes at a specific time, It's launching the backup job on all nodes at the same time, and b

Re: [pve-devel] Backup scheduler: Perl error

2015-09-26 Thread Michael Rasmussen
On Sat, 26 Sep 2015 12:52:04 +0200 (CEST) Dietmar Maurer wrote: > > But your post includes a white space at the end - why? > MUA or list server line breaks (the line is longer than 72 characters)? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp

Re: [pve-devel] Backup scheduler: Perl error

2015-09-26 Thread Dietmar Maurer
> On September 26, 2015 at 12:06 PM Michael Rasmussen wrote: > > > On Sat, 26 Sep 2015 11:49:57 +0200 (CEST) > Dietmar Maurer wrote: > > > > > Is there an white space at the end of this line? If so, remove it. > > > No white space. But your post includes a white space at the end - why?

Re: [pve-devel] Backup scheduler: Perl error

2015-09-26 Thread Michael Rasmussen
On Sat, 26 Sep 2015 11:49:57 +0200 (CEST) Dietmar Maurer wrote: > > Is there an white space at the end of this line? If so, remove it. > No white space. File is purely managed through the proxmox web gui. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc ht

Re: [pve-devel] Backup scheduler: Perl error

2015-09-26 Thread Michael Rasmussen
On Sat, 26 Sep 2015 11:46:11 +0200 (CEST) Dietmar Maurer wrote: > > You call your proxmox nodes esx1 and esx2 - interesting ;-) > Bad happit from work ;-) -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x

Re: [pve-devel] Backup scheduler: Perl error

2015-09-26 Thread Dietmar Maurer
> > Please can you post the crontab file? > > > # cat /etc/pve/vzdump.cron > # cluster wide vzdump cron schedule > # Atomatically generated file - do not edit > > PATH="/usr/sbin:/usr/bin:/sbin:/bin" > > 15 5 * * 6 root vzdump 109 114 117 144 128 --mailnotification > failure --quiet 1

Re: [pve-devel] Backup scheduler: Perl error

2015-09-26 Thread Dietmar Maurer
> 15 5 * * 6 root vzdump 109 114 117 144 128 --mailnotification > failure --quiet 1 --mode snapshot --mailto x@y.z --node esx1 --compress > lzo --storage qnap_nfs > 15 5 * * 5 root vzdump 115 125 143 148 152 102 154 156 110 > --mailnotification failure --quiet 1 --mailto x@y.z

Re: [pve-devel] Backup scheduler: Perl error

2015-09-26 Thread Michael Rasmussen
On Sat, 26 Sep 2015 10:13:54 +0200 (CEST) Dietmar Maurer wrote: > > Please can you post the crontab file? > # cat /etc/pve/vzdump.cron # cluster wide vzdump cron schedule # Atomatically generated file - do not edit PATH="/usr/sbin:/usr/bin:/sbin:/bin" 15 5 * * 6 root vzdump 109 114

Re: [pve-devel] Backup scheduler: Perl error

2015-09-26 Thread Dietmar Maurer
> Every time the backup scheduler runs is see this in log for every VM > that is backed up: > Use of uninitialized value $cmd[8] in exec > at /usr/share/perl/5.14/IPC/Open3.pm line 186. Please can you post the crontab file? ___ pve-devel mailing list pv

[pve-devel] Backup scheduler: Perl error

2015-09-25 Thread Michael Rasmussen
Hi all, Every time the backup scheduler runs is see this in log for every VM that is backed up: Use of uninitialized value $cmd[8] in exec at /usr/share/perl/5.14/IPC/Open3.pm line 186. proxmox-ve-2.6.32: 3.4-163 (running kernel: 3.10.0-11-pve) pve-manager: 3.4-11 (running version: 3.4-11/6502936

Re: [pve-devel] backup : add pigz compressor

2015-07-13 Thread Emmanuel Kasper
07/11/2015 09:25 AM, Cesar Peschiera wrote: >> Yes, that makes sense to me. > > Or maybe the PVE GUI also can has a third option for use pgzip, and the > user also can select the amount of cores to used, so in this case, also > maybe will be better add a message of caution that say "the use of man

Re: [pve-devel] backup : add pigz compressor

2015-07-11 Thread Dmitry Petuhov
ietmar Maurer" > To: "Eric Blevins" > Cc: ; > Sent: Saturday, July 11, 2015 12:27 AM > Subject: Re: [pve-devel] backup : add pigz compressor > > >>> You could even make it so using pigz requires a setting in >>> vzdump.conf. So in GUI you still c

Re: [pve-devel] backup : add pigz compressor

2015-07-11 Thread Cesar Peschiera
least for me, it would be fantastic. - Original Message - From: "Dietmar Maurer" To: "Eric Blevins" Cc: ; Sent: Saturday, July 11, 2015 12:27 AM Subject: Re: [pve-devel] backup : add pigz compressor You could even make it so using pigz requires a setting in vz

Re: [pve-devel] backup : add pigz compressor

2015-07-10 Thread Dietmar Maurer
> You could even make it so using pigz requires a setting in > vzdump.conf. So in GUI you still can only select gzip or lzop. > If set to gzip and vzdump.conf has pigz:on then pigz is used instead of gzip. > Most novice users are only going to use the GUI, this would reduce the > likelyhood of them

Re: [pve-devel] backup : add pigz compressor

2015-07-10 Thread Eric Blevins
> But pigz needs all you CPU power do be that fast! If you have running VMs, > it will be as slow as normal gzip, or it will massively slow down the VMs. > I am really afraid this will trigger many support calls ... Make the default number of pigz CPUs 2, this should not have much of a CPU impact

Re: [pve-devel] backup : add pigz compressor

2015-07-10 Thread Dietmar Maurer
> I ran a quick, informal "benchmark" to compare lzo, gzip, pigz, and > uncompressed backups of a modest 8GB VM on a host with 8 cores, all > currently at 100% from guest usage. > > None: 50 seconds, 199MB/s, output 963MB > pigz: 20 seconds, 453MB/s, output 266MB > gzip: 57 seconds, 150MB/s, out

Re: [pve-devel] backup : add pigz compressor

2015-07-09 Thread Michael Rasmussen
Problem with pigz is that it bails out on missing symbolic links. Eg. follow symlinks is on and cannot be disabled. On July 10, 2015 6:51:21 AM CEST, Alexandre DERUMIER wrote: >Hi, > >user from pve-user mailing report than using pigz (multithread gzip), >improve a lot speed of compression backu

Re: [pve-devel] backup : add pigz compressor

2015-07-09 Thread Alexandre DERUMIER
mplementation currently. - Mail original - De: "dietmar" À: "aderumier" , "pve-devel" Cc: "proxmoxve" , "Chidester, Bryce" Envoyé: Vendredi 10 Juillet 2015 07:05:07 Objet: Re: [pve-devel] backup : add pigz compressor > user from pve-user

Re: [pve-devel] backup : add pigz compressor

2015-07-09 Thread Dietmar Maurer
> user from pve-user mailing report than using pigz (multithread gzip), > improve a lot speed of compression backup. > > Maybe could we use add it ? (or maybe replace gzip ?) We already have lzop, which is as fast as pgzip and uses less resources. So what is the advantage of pgzip? IMHO a paralle

[pve-devel] backup : add pigz compressor

2015-07-09 Thread Alexandre DERUMIER
Hi, user from pve-user mailing report than using pigz (multithread gzip), improve a lot speed of compression backup. Maybe could we use add it ? (or maybe replace gzip ?) - Mail original - De: "Chidester, Bryce" À: "proxmoxve" Envoyé: Jeudi 9 Juillet 2015 20:30:30 Objet: Re: [PVE-User

Re: [pve-devel] backup lock

2014-12-24 Thread Lindsay Mathieson
On Wed, 24 Dec 2014 01:53:40 PM Dietmar Maurer wrote: > > On Wed, 24 Dec 2014 11:57:52 AM Dietmar Maurer wrote: > > > On power failure, it is impossible to remove the lock, because we are > > > already dead. > > > I guess we should remove that lock when we start a node. > > > > > > > > > > > > But

Re: [pve-devel] backup lock

2014-12-24 Thread Dietmar Maurer
> On Wed, 24 Dec 2014 11:57:52 AM Dietmar Maurer wrote: > > On power failure, it is impossible to remove the lock, because we are > > already dead. > > I guess we should remove that lock when we start a node. > > > > But this bug is about removing the lock after receiving a signal (TERM). > > Ah

Re: [pve-devel] backup lock

2014-12-24 Thread Lindsay Mathieson
On Wed, 24 Dec 2014 11:57:52 AM Dietmar Maurer wrote: > On power failure, it is impossible to remove the lock, because we are > already dead. > I guess we should remove that lock when we start a node. > > But this bug is about removing the lock after receiving a signal (TERM). Ah yes, I see your

Re: [pve-devel] backup lock

2014-12-24 Thread Dietmar Maurer
> On Wed, 24 Dec 2014 09:29:18 AM Dietmar Maurer wrote: > > > Happens when a backup fails as well - last power outage we had was in the > > > middle of backups and two VM's refused to start on node start because > > > they > > > were locked > > > > Well, this is a different problem. > > > is i

Re: [pve-devel] backup lock

2014-12-24 Thread Lindsay Mathieson
On Wed, 24 Dec 2014 09:29:18 AM Dietmar Maurer wrote: > > Happens when a backup fails as well - last power outage we had was in the > > middle of backups and two VM's refused to start on node start because > > they > > were locked > > Well, this is a different problem. is it? vm is locked beca

Re: [pve-devel] backup lock

2014-12-24 Thread Dietmar Maurer
> > I can confirm that bug. > > > TBH, I've been seeing this for months, I presumed it was well known. > > Happens when a backup fails as well - last power outage we had was in the > middle of backups and two VM's refused to start on node start because they > were locked Well, this is a diff

Re: [pve-devel] backup lock

2014-12-23 Thread Lindsay Mathieson
On Tue, 23 Dec 2014 04:35:09 PM Dietmar Maurer wrote: > > > You can cancel the job via web interface while doing backup job. And i > > > try > > > the scenario and reproduce the situation. When you use the stop button > > > and > > > try to start the vm it says locked. First you unlock it from the

Re: [pve-devel] backup lock

2014-12-23 Thread lyt_yudi
在 2014年12月23日,下午11:35,Dietmar Maurer 写道: > > >>> You can cancel the job via web interface while doing backup job. And i try >>> the scenario and reproduce the situation. When you use the stop button and >>> try to start the vm it says locked. First you unlock it from the command >>> line with qm

Re: [pve-devel] backup lock

2014-12-23 Thread Dietmar Maurer
> > You can cancel the job via web interface while doing backup job. And i try > > the scenario and reproduce the situation. When you use the stop button and > > try to start the vm it says locked. First you unlock it from the command > > line with qm unlock and then you can start the vm. > I c

Re: [pve-devel] backup lock

2014-12-23 Thread lyt_yudi
> 在 2014年12月23日,下午3:18,Gökalp Çakıcı 写道: > > You can cancel the job via web interface while doing backup job. And i try > the scenario and reproduce the situation. When you use the stop button and > try to start the vm it says locked. First you unlock it from the command line > with qm unlock

Re: [pve-devel] backup lock

2014-12-22 Thread Gökalp Çakıcı
You can cancel the job via web interface while doing backup job. And i try the scenario and reproduce the situation. When you use the stop button and try to start the vm it says locked. First you unlock it from the command line with qm unlock and then you can start the vm. On 12/23/14 9:13 AM

Re: [pve-devel] backup lock

2014-12-22 Thread Wolfgang Link
Can you pleas post me your version of PVE. pveversion -v because I can't reproduce it at my machine, not on the gui and not at cli. how do you cancel the job manually? Regrades Wolfgang > On December 23, 2014 at 6:36 AM lyt_yudi wrote: > > > > > 在 2014年12月23日,下午1:20,lyt_yudi 写道: > > >

Re: [pve-devel] backup lock

2014-12-22 Thread lyt_yudi
> 在 2014年12月23日,下午1:20,lyt_yudi 写道: > >> This indicates that something is wrong, maybe a crashed backup job? INFO: starting new backup job: vzdump 121 --remove 0 --mode snapshot --compress lzo --storage local --node t3 INFO: Starting Backup of VM 121 (qemu) INFO: status = running INFO: update

Re: [pve-devel] backup lock

2014-12-22 Thread lyt_yudi
sorry, forgot send to pve-devel :( > 在 2014年12月23日,下午12:59,Dietmar Maurer > 写道: > > This indicates that something is wrong, maybe a crashed backup job? no, it’s cancel the backup job by manual, and then start new backup job. > You should check if there is an old bac

Re: [pve-devel] backup lock

2014-12-22 Thread Dietmar Maurer
On 12/23/2014 02:58 AM, lyt_yudi wrote: hi,all INFO: starting new backup job: vzdump 121 --remove 0 --mode snapshot --compress lzo --storage local --node t3 INFO: Starting Backup of VM 121 (qemu) INFO: status = running INFO: VM is locked (backup) ERROR: Backup of VM 121 failed - command 'qm set

[pve-devel] backup lock

2014-12-22 Thread lyt_yudi
hi,all INFO: starting new backup job: vzdump 121 --remove 0 --mode snapshot --compress lzo --storage local --node t3 INFO: Starting Backup of VM 121 (qemu) INFO: status = running INFO: VM is locked (backup) ERROR: Backup of VM 121 failed - command 'qm set 121 --lock backup' failed: exit code 25

Re: [pve-devel] backup ceph high iops and slow

2014-12-09 Thread VELARTIS Philipp Dürhammer
:-( -Ursprüngliche Nachricht- Von: pve-devel [mailto:pve-devel-boun...@pve.proxmox.com] Im Auftrag von Dmitry Petuhov Gesendet: Freitag, 17. Oktober 2014 08:04 An: pve-devel@pve.proxmox.com Betreff: Re: [pve-devel] backup ceph high iops and slow 16.10.2014 22:33, VELARTIS Philipp

Re: [pve-devel] Backup-Errors - Bug?

2014-11-14 Thread Detlef Bracker
Dear, ok, with configuration the vzdump.conf and setting a temp-folder, its working fine. I will check this more intensive next days Thanks Detlef Am 15.11.2014 01:14, schrieb Detlef Bracker: > Dear, > > to backup container to nfs in suspend mode, I get many errors > > rsync chown operat

[pve-devel] Backup-Errors - Bug?

2014-11-14 Thread Detlef Bracker
Dear, to backup container to nfs in suspend mode, I get many errors rsync chown operation not permitted and the backup end with errors My question is, ist this a nfs problem or a problem of proxmox? I have longer time before create the NFS via the GUI and proxmox has create a mount link u

Re: [pve-devel] Backup hangs / no notify if second backup job can't start

2014-10-22 Thread Dietmar Maurer
> Will do so. Can you point me to the part where vzdump checks whether there is > a process already running? I was not able to find that one. In PVE/API2/VZDump.pm, search for getlock(). getlock is defined in PVE/VZDump.pm ___ pve-devel mailing list pv

Re: [pve-devel] Backup hangs / no notify if second backup job can't start

2014-10-22 Thread Stefan Priebe - Profihost AG
Am 22.10.2014 um 09:16 schrieb Dietmar Maurer: >> i had the following situation several times in the past. >> >> 1.) Backup Job started - hangs forever on a VM looping in boot "mode" as >> there >> was not an OS installed >> >> 2.) At the next day the backup job can't start as the one from before

Re: [pve-devel] Backup hangs / no notify if second backup job can't start

2014-10-22 Thread Dietmar Maurer
> i had the following situation several times in the past. > > 1.) Backup Job started - hangs forever on a VM looping in boot "mode" as there > was not an OS installed > > 2.) At the next day the backup job can't start as the one from before is still > running. > > I think it would be very impor

[pve-devel] Backup hangs / no notify if second backup job can't start

2014-10-22 Thread Stefan Priebe - Profihost AG
Hi, i had the following situation several times in the past. 1.) Backup Job started - hangs forever on a VM looping in boot "mode" as there was not an OS installed 2.) At the next day the backup job can't start as the one from before is still running. I think it would be very important to get a

Re: [pve-devel] backup

2014-10-20 Thread Cesar Peschiera
iginal Message - From: Lindsay Mathieson To: Cesar Peschiera Cc: pve-devel@pve.proxmox.com Sent: Monday, October 20, 2014 11:59 PM Subject: Re: [pve-devel] backup On 20 October 2014 14:15, Cesar Peschiera wrote: Hi Lindsay Maybe you'd better have a PVE on a

Re: [pve-devel] backup

2014-10-20 Thread Lindsay Mathieson
On 20 October 2014 14:15, Cesar Peschiera wrote: > Hi Lindsay > > Maybe you'd better have a PVE on a PC as Backup Server instead of a NAS. > Why i believe that it will be better? > For four reasons: > 1) You will can manually restore the backups that were completed > successfully on this same PC

Re: [pve-devel] backup

2014-10-20 Thread Lindsay Mathieson
On 20 October 2014 15:27, Dmitry Petuhov wrote: Yes, slow NFS servers/network is known to cause failing backup with VM > crush. As workaround, you can try enable GZIP compression. It shall slow > down NFS writes enough to let NAS server do its job. If it's not enough > then try set bwlimit in /et

Re: [pve-devel] backup ceph high iops and slow

2014-10-20 Thread Alexandre DERUMIER
mar Maurer" Cc: pve-devel@pve.proxmox.com Envoyé: Dimanche 19 Octobre 2014 20:47:04 Objet: Re: [pve-devel] backup ceph high iops and slow >>How should we do read-ahead inside qemu? manually? This is managed by the linux kernel automaticaly /sys/class/block/sda/queue/read_ahead_kb

Re: [pve-devel] backup

2014-10-19 Thread Cesar Peschiera
es: - A group of Switches Dell N2024 for connect all equipment. So, if you want to know the result of this test, that will be in a production environment, only let me know. Best regards Cesar Peschiera - Original Message - From: "Lindsay Mathieson" To: Sent: Sunday, Oct

Re: [pve-devel] backup

2014-10-19 Thread Dmitry Petuhov
20.10.2014 02:20, Lindsay Mathieson пишет: True and in my case I wonder if the problem is my NAS. Backups to locally attached USB drive are no problem, but I've always had problems backing up to my NAS over GB ethernet, using NFS or SMB. I'd often get 1 or 2 VM's failing with disk write errors,

Re: [pve-devel] backup

2014-10-19 Thread Lindsay Mathieson
On Sun, 19 Oct 2014 07:18:34 AM Dietmar Maurer wrote: > > On Sat, 18 Oct 2014 10:56:18 AM Dietmar Maurer wrote: > > > > Maybe a setting in the backup gui when checked prevents running > > > > parallel backups could solve this issue? > > > > > > It is quite easy to setup a separate job for each nod

Re: [pve-devel] backup ceph high iops and slow

2014-10-19 Thread Alexandre DERUMIER
Cc: pve-devel@pve.proxmox.com, "VELARTIS Philipp Dürhammer" , "Dmitry Petuhov" Envoyé: Dimanche 19 Octobre 2014 18:07:30 Objet: RE: [pve-devel] backup ceph high iops and slow > +RBD supports read-ahead/prefetching to optimize small, sequential reads. > +This should

Re: [pve-devel] backup ceph high iops and slow

2014-10-19 Thread Dietmar Maurer
> +RBD supports read-ahead/prefetching to optimize small, sequential reads. > +This should normally be handled by the guest OS in the case of a VM, How should we do read-ahead inside qemu? manually? ___ pve-devel mailing list pve-devel@pve.proxmox.com ht

Re: [pve-devel] backup ceph high iops and slow

2014-10-19 Thread Alexandre DERUMIER
ahead disable after bytes=0 for proxmox backups - Mail original - De: "Dietmar Maurer" À: "Alexandre DERUMIER" Cc: pve-devel@pve.proxmox.com, "VELARTIS Philipp Dürhammer" , "Dmitry Petuhov" Envoyé: Samedi 18 Octobre 2014 09:47:08 Obj

Re: [pve-devel] backup

2014-10-19 Thread Dietmar Maurer
> On Sat, 18 Oct 2014 10:56:18 AM Dietmar Maurer wrote: > > > Maybe a setting in the backup gui when checked prevents running > > > parallel backups could solve this issue? > > > > It is quite easy to setup a separate job for each node. > > > Not if you need to exclude some vm's, but migrate them

Re: [pve-devel] backup

2014-10-18 Thread Lindsay Mathieson
On Sat, 18 Oct 2014 10:56:18 AM Dietmar Maurer wrote: > > Maybe a setting in the backup gui when checked prevents running parallel > > backups could solve this issue? > > It is quite easy to setup a separate job for each node. Not if you need to exclude some vm's, but migrate them regularly for

Re: [pve-devel] backup

2014-10-18 Thread Gilberto Nunes
Oh Dear! Qnap! Pity of you! Qnap has only 128 MB of memory and low hardware... Here, I do backup locally, in an External HDD USB To one server, I saw here around 15-18 MB/s, but there's some ocasion that I saw less than 10 MB/s Compare to other server, I saw 60-80 MB/s... 2014-10-18 13

Re: [pve-devel] backup

2014-10-18 Thread Michael Rasmussen
On Sat, 18 Oct 2014 13:03:57 -0300 Gilberto Nunes wrote: > Well... > > I have just one node here, because the other server doesn't arrive yet, but > I already create the cluster... > And my backup task are extremaly slow! > It a file about 23 GB, that take almost 5 hours. > Ok! I ran backup over

Re: [pve-devel] backup

2014-10-18 Thread Gilberto Nunes
Well... I have just one node here, because the other server doesn't arrive yet, but I already create the cluster... And my backup task are extremaly slow! It a file about 23 GB, that take almost 5 hours. Ok! I ran backup over LVM-on-top-iSCSI VMIDNAMESTATUSTIMESIZEFILENAME 103Windows2012OK04:53:4

Re: [pve-devel] backup

2014-10-18 Thread Dietmar Maurer
> > It is quite easy to setup a separate job for each node. > > > I know, but maybe users are not aware of this complication. > If it can remove a > lot of "noise" on the forum from people with failing backups I think it is > worth > implementing. 1.) I don't think this is the reason for the pr

Re: [pve-devel] backup

2014-10-18 Thread Michael Rasmussen
On Sat, 18 Oct 2014 10:56:18 + Dietmar Maurer wrote: > > It is quite easy to setup a separate job for each node. > I know, but maybe users are not aware of this complication. If it can remove a lot of "noise" on the forum from people with failing backups I think it is worth implementing. -

Re: [pve-devel] backup

2014-10-18 Thread Dietmar Maurer
> I just discovered that when Proxmox initiates the configured automatic backup > it > runs backups from each node in the cluster in parallel. I wonder whether this > could be the cause for users now and then see some of the backups fails given > the backup storage is to slow storage? > > Maybe

[pve-devel] backup

2014-10-18 Thread Michael Rasmussen
Hi all, I just discovered that when Proxmox initiates the configured automatic backup it runs backups from each node in the cluster in parallel. I wonder whether this could be the cause for users now and then see some of the backups fails given the backup storage is to slow storage? Maybe a setti

Re: [pve-devel] backup ceph high iops and slow

2014-10-18 Thread Dietmar Maurer
> >>We read 64K blocks, so > > Don't for for backup, but drive-mirror have a "granulary" option to change the > block size: > > # @granularity: #optional granularity of the dirty bitmap, default is 64K. > # Must be a power of 2 between 512 and 64M. Although it would be much easie

Re: [pve-devel] backup ceph high iops and slow

2014-10-17 Thread Alexandre DERUMIER
l - De: "Dietmar Maurer" À: "VELARTIS Philipp Dürhammer" , "Dmitry Petuhov" , "Alexandre DERUMIER" Cc: pve-devel@pve.proxmox.com Envoyé: Vendredi 17 Octobre 2014 13:02:29 Objet: RE: [pve-devel] backup ceph high iops and slow > I agree. The main

Re: [pve-devel] backup ceph high iops and slow

2014-10-17 Thread Dietmar Maurer
> I agree. The main problem is with used space. > I didnt check for mirroring - but reading unused space for backup is super > fast > (around 1200mb/sec) But used space only 40-50mb/sec and too many iops... We read 64K blocks, so 4/64 ==> 625 IOPS Maybe the pending ceph read-ahead changes s

Re: [pve-devel] backup ceph high iops and slow

2014-10-17 Thread VELARTIS Philipp Dürhammer
Auftrag von Dmitry Petuhov Gesendet: Freitag, 17. Oktober 2014 09:25 An: Alexandre DERUMIER Cc: pve-devel@pve.proxmox.com Betreff: Re: [pve-devel] backup ceph high iops and slow I think that skipping free space isn't main issue: backup of used space is even slower. 17.10.2014 10:31, Alex

Re: [pve-devel] backup ceph high iops and slow

2014-10-17 Thread Dmitry Petuhov
driver to skip zero block on read. for restore no problem. For drive-mirror write to target, it's also missing a feature to skip zero-block write. - Mail original - De: "Dmitry Petuhov" À: pve-devel@pve.proxmox.com Envoyé: Vendredi 17 Octobre 2014 08:04:04 Objet: Re: [pv

Re: [pve-devel] backup ceph high iops and slow

2014-10-16 Thread Alexandre DERUMIER
ero-block write. - Mail original - De: "Dmitry Petuhov" À: pve-devel@pve.proxmox.com Envoyé: Vendredi 17 Octobre 2014 08:04:04 Objet: Re: [pve-devel] backup ceph high iops and slow 16.10.2014 22:33, VELARTIS Philipp Dürhammer пишет: > Why do backups with ceph make so high iops?

Re: [pve-devel] backup ceph high iops and slow

2014-10-16 Thread Dmitry Petuhov
16.10.2014 22:33, VELARTIS Philipp Dürhammer пишет: Why do backups with ceph make so high iops? I get around 600 iops for 40mb/sec which is by the way very slow for a backup. When I make a disk clone from local to ceph I get 120mb/sec (which is the network limit from the old proxmox nodes) and o

[pve-devel] backup ceph high iops and slow

2014-10-16 Thread VELARTIS Philipp Dürhammer
Why do backups with ceph make so high iops? I get around 600 iops for 40mb/sec which is by the way very slow for a backup. When I make a disk clone from local to ceph I get 120mb/sec (which is the network limit from the old proxmox nodes) and only around 100-120 iops which is the normal for a se

[pve-devel] Backup fails

2014-09-13 Thread Michael Rasmussen
Hi all, Recently I am beginning to see logs like can be seen below. It is the automatic backup which fails backing up some of the VMs but it seems completely random which VM will fail backup when backup is performed. Another observation is this warning given when every backup starts: Use of unini

Re: [pve-devel] Backup Scheduling

2014-07-06 Thread Dietmar Maurer
> So the WebGUI for backups basically edits a cron job? yes ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Backup Scheduling

2014-07-06 Thread Cesar Peschiera
ot;Lindsay Mathieson" To: Sent: Saturday, July 05, 2014 6:46 PM Subject: [pve-devel] Backup Scheduling Any possibility of adding per month schedules to backups? this would be very useful for our DR backups, where we rotate offsite disks once per month. Actually the ability to create a bac

Re: [pve-devel] Backup Scheduling

2014-07-05 Thread Lindsay Mathieson
On Sun, 6 Jul 2014 03:25:16 AM Dietmar Maurer wrote: > > Any possibility of adding per month schedules to backups? this would be > > very useful for our DR backups, where we rotate offsite disks once per > > month. > cron supports that, so it should not be that hard to implement it (but > someone n

Re: [pve-devel] Backup Scheduling

2014-07-05 Thread Dietmar Maurer
> Any possibility of adding per month schedules to backups? this would be very > useful for our DR backups, where we rotate offsite disks once per month. cron supports that, so it should not be that hard to implement it (but someone needs to write that patch). ___

[pve-devel] Backup Scheduling

2014-07-05 Thread Lindsay Mathieson
Any possibility of adding per month schedules to backups? this would be very useful for our DR backups, where we rotate offsite disks once per month. Actually the ability to create a backup set with no schedule that you manually trigger would be even more handy :) thanks, -- Lindsay signature

Re: [pve-devel] Backup code / part

2013-02-16 Thread Dietmar Maurer
> I can Look too should it generally work through vzdump and mode snapshot yes ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Backup code / part

2013-02-16 Thread Stefan Priebe - Profihost AG
I can Look too should it generally work through vzdump and mode snapshot Am 16.02.2013 um 15:57 schrieb Dietmar Maurer : >> A forum user have the same problem. I think it shouldn't use "vzdump -- >> mode snapshot", maybe the new backup system is not yet switched for all >> storages ? > > Will ta

Re: [pve-devel] Backup code / part

2013-02-16 Thread Dietmar Maurer
> A forum user have the same problem. I think it shouldn't use "vzdump -- > mode snapshot", maybe the new backup system is not yet switched for all > storages ? Will take a look at that on Monday. ___ pve-devel mailing list pve-devel@pve.proxmox.com http

Re: [pve-devel] Backup code / part

2013-02-16 Thread Dietmar Maurer
> Is the backup code ready for testing? Yes, please test. > Should it work with rbd? No tested so far, but it should work. > Anything special I should focus on? The whole thing is quite new, so we need to test everything. ___ pve-devel mailing lis

Re: [pve-devel] Backup code / part

2013-02-16 Thread Alexandre DERUMIER
À: pve-devel@pve.proxmox.com Envoyé: Samedi 16 Février 2013 13:16:21 Objet: Re: [pve-devel] Backup code / part No answer myself. No it's not working for rbd: - INFO: starting new backup job: vzdump 123 --remove 0 --mode snapshot --compress lzo --stor

Re: [pve-devel] Backup code / part

2013-02-16 Thread Stefan Priebe
No answer myself. No it's not working for rbd: - INFO: starting new backup job: vzdump 123 --remove 0 --mode snapshot --compress lzo --storage local --node node1283 INFO: Starting Backup of VM 123 (qemu) INFO: status = running ERROR: Backup of VM 123 fail

[pve-devel] Backup code / part

2013-02-16 Thread Stefan Priebe - Profihost AG
Is the backup code ready for testing? Should it work with rbd? Anything special I should focus on? Stefan ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] backup patches

2012-11-21 Thread Dietmar Maurer
I just posted the backup patches to the qemu-devel list. ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] backup RFC preview

2012-11-19 Thread Alexandre DERUMIER
-devel@pve.proxmox.com Envoyé: Lundi 19 Novembre 2012 15:08:48 Objet: Re: [pve-devel] backup RFC preview >>Mhm with -monitor the qm monitor shell stuff doesn't work I always get the >>failure that qmp human interface cannot be found. I can get it work in pve gui, with renam

Re: [pve-devel] backup RFC preview

2012-11-19 Thread Alexandre DERUMIER
l original - De: "Stefan Priebe - Profihost AG" À: "Dietmar Maurer" Cc: pve-devel@pve.proxmox.com Envoyé: Dimanche 18 Novembre 2012 07:52:16 Objet: Re: [pve-devel] backup RFC preview Am 18.11.2012 um 07:31 schrieb Dietmar Maurer : >>> # ./x86_64-softmmu

Re: [pve-devel] backup RFC preview

2012-11-19 Thread Alexandre DERUMIER
xmox.com Envoyé: Lundi 19 Novembre 2012 13:37:54 Objet: RE: [pve-devel] backup RFC preview My patches are against commit 6801038bc52d61f81ac8a25fbe392f1bad982887 Try to use that commit. Maybe I should expose my internal qemu-git tree? > -Original Message- > From: Alexandre DERU

Re: [pve-devel] backup RFC preview

2012-11-19 Thread Alexandre DERUMIER
yes, maybe qemu build is broken :/ #git checkout 6801038bc52d61f81ac8a25fbe392f1bad982887 and now, seem to build fine :) - Mail original - De: "Dietmar Maurer" À: "Alexandre DERUMIER" Cc: pve-devel@pve.proxmox.com Envoyé: Lundi 19 Novembre 2012 13:41:03 O

Re: [pve-devel] backup RFC preview

2012-11-19 Thread Dietmar Maurer
> osdep.o: In function `qemu_close': > /root/qemu2/qemu/osdep.c:212: undefined reference to > `monitor_fdset_dup_fd_find' > /root/qemu2/qemu/osdep.c:218: undefined reference to > `monitor_fdset_dup_fd_remove' AFAIK my patches does not touch those files - maybe current qemu build is broken? __

Re: [pve-devel] backup RFC preview

2012-11-19 Thread Dietmar Maurer
Maurer > Cc: pve-devel@pve.proxmox.com > Subject: Re: [pve-devel] backup RFC preview > > >>Just sent an updated version - does that work better? > no :/ > > Last qemu git: > > I got Hunk Failed on Makefile > > root@kvmtest1:~/qemu2/qemu# patch -p1 <

Re: [pve-devel] backup RFC preview

2012-11-19 Thread Alexandre DERUMIER
r.o vma-reader.o $(tools-obj-y) $(block-obj-y) +vma$(EXESUF): vma.o vma-reader.o $(tools-obj-y) $(block-obj-y) - Mail original - De: "Dietmar Maurer" À: "Alexandre DERUMIER" Cc: pve-devel@pve.proxmox.com Envoyé: Lundi 19 Novembre 2012 12:30:23 Objet

Re: [pve-devel] backup RFC preview

2012-11-19 Thread Dietmar Maurer
Just sent an updated version - does that work better? > -Original Message- > From: Alexandre DERUMIER [mailto:aderum...@odiso.com] > Sent: Montag, 19. November 2012 12:24 > To: Dietmar Maurer > Cc: pve-devel@pve.proxmox.com > Subject: Re: [pve-devel] backup RFC preview

Re: [pve-devel] backup RFC preview

2012-11-19 Thread Alexandre DERUMIER
d' qemu-sockets.o: In function `socket_listen': /root/qemu2/qemu/qemu-sockets.c:937: undefined reference to `monitor_get_fd' collect2: ld returned 1 exit status make: *** [vma] Error 1 any idea ? - Mail original - De: "Dietmar Maurer" À: "Alexandre DERUMI

  1   2   >