@faulpletz Do you mean with "and the guest systems continue to work
normally" that the guest does not have any read-only problems any more
with your patch?
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launc
@Tommy Eriksen, I believe the current status is that we have no idea.
Every time when I think we are getting close on the cause of the
problem, there comes proof that it is not the cause of the problems. For
us it already critical for months.
We are in the middle of moving our systems. I know @jsa
@faulpeltz I confirm the timing. I was never as specific as you are now.
I already mentioned earlier that it always happens towards the end of
the backup. I would not be surprised if the crash happens during the
merge.
--
You received this bug notification because you are a member of Kernel
Packa
@emsi I had that problem too, you should ignore the binutils upgrade. I
had a look at the CHANGELOG of binutils and there is no significant
change. So the installation should continue although there is a
dependency mismatch. I cannot recall exactly what I fix the mismatch.
--
You received this bu
@jsalisbury Maybe you can also create a 4.2.0 kernel with cd4842f4
reverted. Then I can test one.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1470250
Title:
[Hyper-V] Ubuntu 14.04.2 L
@jsalisbury I would not be surprised by that. It would explain the pain
in finding the cause of this bug. Its outside of any of our scopes.
Moreover, when you read the commit message
(https://github.com/torvalds/linux/commit/cd4842f4), it seems that there
is quite some controversial, probably thoug
@alexng-v I only had this while doing backups. Also, in the initial post
on MS Technet refers to it as a backup problem:
https://social.technet.microsoft.com/Forums/windowsserver/en-US
/8807f61c-565e-45bc-abc4-af09abf59de2/ubuntu-14042-lts-generation-2
-scsi-errors-on-vss-based-backups. We are now
@elupus This is in line with our observations. Our guess is that the
change to a read-only state occurs during/just befor/just after a FREEZE
and/or THAW.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launch
For the information, this is the commit
https://github.com/torvalds/linux/commit/c01fac1c77a00227f706a1654317023e3f4ac7f0.
It would indeed make no sense at all when this would cause the thing.
How would the backup procedure even touch those lines of code?
--
You received this bug notification bec
@jsalisbury Thanks for the work already. It's good new information to
have. We are getting closer with finding the exact cause of the issue. I
am sure we will find it any time soon now!
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux
@jsalisbury The Windows Version is the same. Regarding the hv_vss_daemon
I am using the latest recommended version of the operating system. For
Ubuntu that means I am installing tools and cloud tools. For CentOS I am
using hypervvssd.x86_64 0-0.26.20150402git.el7.
--
You received this bug notific
@trash1-z No, I believe that was never posted. Though we do know people
with 12.04 were hit too. However, I did post the version of my CentOS
machine that was never hit by the bug (to date), while all my Ubuntu
were hit frequently. I started with kernel 3.13. The kernel version of
this CentOS machi
@alexng-v Thanks for the quick reply: that sounds good! I will let you
know what our experiences are.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1470250
Title:
[Hyper-V] Ubuntu 14.04
@jsalisbury That is great, thanks. Now I can go testing for sure, but I
think the first confirmation we need is that there is no more crash when
your script to reproduce the issue is used.
Maybe @jrp can tell us what the reason why he thinks they found the
smoking gun. If he can indeed confirm the
@jrp @alexng-v Great that we have more improvements! Were you already
able to to not see any more crashes during backup while using the script
created by @jsalisbury?
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https:/
@trash1-z I also see that message when a backup succeeds. Just like I
see it when a backup fails.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1470250
Title:
[Hyper-V] Ubuntu 14.04.2 L
@trash1-z Earlier reports indicate that that is not an error. The
message is benign.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1470250
Title:
[Hyper-V] Ubuntu 14.04.2 LTS Generation
Just discovered in the CHANGELOG
(http://changelogs.ubuntu.com/changelogs/pool/main/l/linux/linux_4.2.0-30.35/changelog)
that 4.2.0-27 does not include the major storvsc update @jrp is talking
about. The referred updates were included 4.2.0-28 and distributed in
linux-image-4.2.0-30-generic. My aut
@trash1-z, thanks just disabled it. let's see what happens.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1470250
Title:
[Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Bas
@arune See attached my list of installed packages. That is correct,
right? My running kernel is (uname -a) Linux VPS-Web-Produc-Genkgo
4.2.0-27-generic. And ps -aux |grep hv says.
[hv_vmbus_con]
/usr/lib/linux-tools/4.2.0-27-generic/hv_fcopy_daemon
/usr/lib/linux-tools/4.2.0-27-generic/hv_vss_daem
Upgrading the kernel does not help this issue. Moreover, the upgrade
gives new issues. For the third time now, the network of one of the
Ubuntu VMs suddenly goes down. As so it seems. There is nothing in
/var/log/syslog that indicates that the VM has a problem. It is
continuing its work. But from t
@trash1-z Completely corrupted fs? That is not good. How long are you
chasing this bug? Was everything ok (except for the read-only problems)
before the upgrade to 4.2.0-27? Because, then I might downgrade my OS to
3.19 which was working fine.
--
You received this bug notification because you are
@trash1-z Was it still corrupted after the reboot?
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1470250
Title:
[Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based
Back
Thanks @jrp. Then the Wily kernel I installed did not help. Should
4.2.0-27 have the patches you were talking about?
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1470250
Title:
[Hyper-
@jrp Does it matter to install -virtual packages? I have never done
that. Should I?
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1470250
Title:
[Hyper-V] Ubuntu 14.04.2 LTS Generation
First backup failed immediately. We are noticing two more things.
First, the failed mount of today fails more often than other mounts. It
is not the mount of the OS, but a second added disk (mounted in
/dev/sdb1). I have attached the output for /etc/mtab and /etc/fstab of
the specific machine that
Just upgraded to 4.2.0-27-generic #32~14.04.1-Ubuntu SMP Fri Jan 22
15:32:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux. Let's see what
happens.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/
@jrp Thanks, then I will upgrade any time soon, great to hear that you
have extra people to help solving!
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1470250
Title:
[Hyper-V] Ubuntu 1
@jrp Could you confirm that if I upgrade my machines today to the wily
kernels that there are (probably) many improvements to this issue? At
this moment I have the vivid kernels installed.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to li
Thanks @lauwersm and @decui. Our comments seem to have no effect at all.
Yesterday was a read-only day again. Just two weeks after the previous
one. So now it happens two times a month, which is 2 out of 8 backups.
This is just too often, too expensive in labour and really unworkable.
Also, the wr
Is there anything I can do to help? After @jsalisbury posted the method
to reproduce the issue and there was a sign of commitment by @decui, we
decided to postpone our decision of moving away from Hyper-V.
Now we have yet again the general feeling that this issue will not be
solved soon. While I c
If those commits could just be fixes for the problem we are having, that
would be a really good start of 2016!
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1470250
Title:
[Hyper-V] Ubu
@decui Correct, ceteris paribus, the only changes were more of the same
(disk space) and some package changes through apt-get update (Ubuntu
14.04 LTS). My guess is that the downgrade is related to one of the two.
But I agree, let's postpone this one, until the read-only bug is fixed.
Afterwards, I
@decui Regarding downgrade, I think I might not have been clear on this.
The downgrade occurs only when there are backups running. Our
performance otherwise is great. This (new) problem is not necessarily
related to this specific issue, but more to backups on Hyper-V in
general. Nonetheless, as I s
@decui @jsalisbury That is indeed good news. Since we did not make any
changes to our platform yet, we can help to test possible fixes. There
were also some questions.
1. Occurrence of the issue: with the fixes of bug #1445195 the amount of
read-only errors reduced drastically. But it is still the
This problem started more than a year ago, I decided together with my
hoster to look at alternatives. Backup is the only thing that is causing
problems for us with HyperV. The problem is two fold. First, the read-
only state machines get into when generating the backup. Second, the
read/write spike
Since I was on holidays, I will give this a try in next two weeks.
Thanks for the effort @jalisbury.
> On 20 Nov 2015, at 15:06, Joseph Salisbury
> wrote:
>
> I'll see if I can figured out why the 3.10 based kernel doesn't boot.
> However, it may just be easier to test the Precise kernel, which
@stuart-luscombe That is interesting. So if we combine both our
observations, then my conclusion is: there must be something in the
hyper v daemons. And one the things to look at is where the Ubuntu
daemons might differ from the CentOS daemons. Maybe @jsalisbury knows if
there is any difference bet
@stuart-luscombe So if I understand correctly, the reason why you
installed the daemons was to get full file indexes and no complaints by
monitoring software. However, the downside is that the VMs started
crashing during backups after the install? More importantly, can you
confirm there was no prob
@jsalisbury uname -a for the CentOS machines is: Linux host.name
3.10.0-229.11.1.el7.x86_64 #1 SMP Thu Aug 6 01:06:18 UTC 2015 x86_64
x86_64 x86_64 GNU/Linux. This machine was never hit by the backup issue.
The technet link you posted contains the following quote: "As a result,
we've been hesitant
A little remark that still has no answer yet: what are the Ubuntu
specifics that cause this issue? In the 10 months we have these machines
running: many crashes for Ubuntu with VSS snapshots while the CentOS
machines have had no crash at all. Maybe @jsalisbury has a explanation
for this?
--
You r
What do you mean with: did not occur until you followed those
instructions? You mean without the daemons everything was fine?
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1470250
Title:
@jrp Thanks for that, I am happy to hear there is still being worked on. Let me
know if there is a build that I can test
@h-lbuntu-2 How are your results with Willy kernels?
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
Unfortunately, I did not have had time to test Willy kernels. But the
other 3.x stable kernels are still contain the problem.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1470250
Title:
I think I found the solution to this bug: 3.13.0-62 was not installed
correctly. This is due to the fact that fgrlx was upgraded too. During
this update the question `whether you would like to keep you config` is
asked. But you have to expand (view details) the window to actually see
that this que
Public bug reported:
Since I upgraded to linux-image-extra-3.13.0-62-generic, my personal
machine is not able to boot any longer. I have to choose Advanced
Options and boot with the 3.13.0-61 kernel. I have no idea what is wrong
at this stage.
ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package
Since this also helps on the backup issue: please merge these patches!
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1445195
Title:
[Hyper-V] Kernel patches for storvsc
Status in linux
@dander88 That sounds great, but we had the same results for a test VPS
machine. As indicated by @jrp before: it depends on your IO load if the
machine goes into read-only. And test machines usually do not have that
many IO load.
--
You received this bug notification because you are a member of
@cohn By the way, I have not installed the 4.x kernels yet. So I have no
idea how many dependency problems you will run in to. Could you let me
know?
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.n
@dander88 According to @jrp the message "Received an indication that the
operating parameters on this target have changed. The Linux SCSI layer
does not automatically adjust these parameters." is beneign. He
mentioned that in bug 1445195.
Are you just seeing that message or do your systems also g
@dander88 Are you talking on the backup bug? Please report them in bug
1470250.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1445195
Title:
[Hyper-V] Kernel patches for storvsc
Status
@jsalisbury I will start testing this week. However, I feel the latest
reports were pretty clear: the issue is there. While I was the only one
at first that still had problems, after a while more people reported (in
bug 1441595) that the new build still contains the issue. In my opinion,
it is now
That should have been bug 1445195.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1470250
Title:
[Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based
Backups
Status in l
@jsalisbury I did not find any new commits on this subject in the
current kernel master (https://github.com/torvalds/linux). And I believe
you already included all HV commits in the last test build from bug
1441595.
So testing this test kernel would mean I am testing whether another
commit (not sp
@jrp @jsalisbury I am using this kernel:
http://kernel.ubuntu.com/~jsalisbury/lp1445195/vivid/ on 14.04.2. So
that build is much more stable but NOT a complete fix of the problem.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ub
@jrp @jsalisbury I am using this kernel:
http://kernel.ubuntu.com/~jsalisbury/lp1445195/vivid/ on 14.04.2. So
that build is much more stable but a complete fix of the problem.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu
My latest report was that latest builds with patches are much more
stable but are also not a complete fix for the problem. It is still
there and occurs randomly. The error message is not changed. I have no
real indication what causes the read-only state. During the latest RO
state I noticed there w
@brainv We will do similar tests this weekend and next week. Since we
actually faced the read-only mode once more, I am not feeling
comfortable yet with the current solution. I am going to try to
reproduce the io pattern that we had when system went into read-only. It
was something like `git clone`
@stevez That means you were not able replicate the read-only issue we
had during our tests of the patched Vivid kernel?
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1445195
Title:
[Hyp
@jsalisbury I guess so, I will try it asap.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1445195
Title:
[Hyper-V] Kernel patches for storvsc
Status in linux package in Ubuntu:
Triag
@jrp How did your Vivid testing do? Were you able to find any
particularities?
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1445195
Title:
[Hyper-V] Kernel patches for storvsc
Status
@jrp I have tried to discover if there were any differences between the
backup that failed and the other ones. I noticed this one thing. Before
or during backup (cannot recall exactly atm) a 'composer install'
process was running. Composer is a php package manager. It downloads and
deploys many sm
@jrp Do you have any suggestions on my remarks above? After the machine
went intro read-only we rebooted and started the whole sequence again.
We did not have had any problems since then. As so it seems, the window
of opportunity for the bug to manifest has been decreased drastically.
However, I al
@jrp Could you indicate with which commits did you exactly tested the
bug on your side? I understand from jsalisbury that the base for the
current kernel is the master-next branch. He applied the patches
mentioned in his earlier post to that kernel. Are we missing any other
commits?
Another issue
Very unfortunate but the bug has been triggered again.
[154272.293488] sd 2:0:0:0: [storvsc] Sense Key : Unit Attention [current]
[154272.293508] sd 2:0:0:0: [storvsc] Add. Sense: Changed operating definition
[154272.293665] sd 2:0:0:0: Warning! Received an indication that the operating
parameter
Build: 3.19.0-19-generic #19~lp1445195 SMP Tue May 26 17:43:16 UTC 2015
x86_64 x86_64 x86_64 GNU/Linux
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1445195
Title:
[Hyper-V] Kernel patc
@jsalisbury: I only tested the Vivid build.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1445195
Title:
[Hyper-V] Kernel patches for storvsc
Status in linux package in Ubuntu:
Triag
@brainv We are seeing exactly the same. Backups are not crashing while
we are still creating backups every half an hour. Our perception is that
the bug is fixed, but we are going to add more boxes to see what happens
then.
--
You received this bug notification because you are a member of Kernel
P
Our tests are positive so far. Already creating backups for 19 hours,
with a backup every half an hour.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1445195
Title:
[Hyper-V] Kernel pat
@brianv, you should run dpkg --ignore-depends=binutils -i *.deb to
install. The binutils update can be ignored (if you look at the
changelog it is a minor version bump without api changes). Maybe you
should try the 3.19 kernel: we do not have any booting problems while we
are also on trusty.
--
Y
@jrp: Ok. After reboot it was fine: just one process per daemon.
However, now I do not see any logs of FREEZE and THAW anymore (while
backups are successful). I do see the warning that the operating
parameters have changed. Probably nothing to worry about.
Nonetheless, being a programmer myself, I
In addition to the previous message, there seemed to be two processes
running of the vss and kvp daemon (while there is only one instance of
fcopy). This is weird (I think), but I have seen it before with other
kernels. A reboot helps in that case. This can also be an explanation
for the "received
One thing I have noticed already, is new logs regarding received
packages.
May 27 16:30:18 Hyper-V VSS: VSS: op=FREEZE: succeeded
May 27 16:30:18 Hyper-V VSS: VSS: op=FREEZE: succeeded
May 27 16:30:18 Hyper-V VSS: Received packet from untrusted pid:2102
May 27 16:30:18 Hyper-V VSS: Received packet
We are starting backup tests with new Vivid kernel now.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1445195
Title:
[Hyper-V] Kernel patches for storvsc
Status in linux package in Ubu
@jrp: could you confirm that this might solve my issue with the 3.19
kernel? Is that specific commit required?
Because I was still seeing "Warning! Received an indication that the
operating parameters on this target have changed. The Linux SCSI layer
does not automatically adjust these parameters.
@jrp: did you test the 3.19 kernel or did you use trusty and/or utopic
build?
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1445195
Title:
[Hyper-V] Kernel patches for storvsc
Status i
Another confirmation that the reported bugs are not fixed in the vivid
build. We have had another crash this morning, same log messages. It
took quite some hours longer than yesterday though, which confirms the
occurrences are still random (we are not able to see the pattern).
--
You received thi
No luck with the current build.
Linux VPS-Genkgo 3.19.0-18-generic #18 SMP Wed May 20 17:57:26 UTC 2015
x86_64 x86_64 x86_64 GNU/Linux
First backup was succesful, second one failed.
[ 2379.049088] sd 2:0:0:0: [storvsc] Sense Key : Unit Attention [current]
[ 2379.049109] sd 2:0:0:0: [storvsc] Add
Testing in progress.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1445195
Title:
[Hyper-V] Kernel patches for storvsc
Status in linux package in Ubuntu:
Triaged
Status in linux sour
The problem we encountered with HyperV backups, filed here:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1456985, should be
solved by these patches too.
Our bug is an exact copy of the initial bug that was reported here:
https://social.technet.microsoft.com/Forums/windowsserver/en-US
/8807
@brad-figg: I just found out that this bug will be solved by
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1445195.
** Changed in: linux (Ubuntu)
Status: Incomplete => Confirmed
** Changed in: linux (Ubuntu)
Status: Confirmed => Fix Committed
--
You received this bug notifi
Public bug reported:
Because "ubuntu-bug linux" not works because I am on kernel 3.19.0-17, I
create this report manually.
Summary of problem: we create backups with HyperV of 4 linux virtual
machines. There is 1 CentOS machine (XFS) and 3 Ubuntu 14.04 machines
(EXT4). Problems occur on the Ubunt
I had the same issue when I wanted to upgrade to newer kernel. So I
first added ppa:canonical-kernel-team/ppa, upgrade my kernel and then
installed the new tools and cloud tools. Then got the issue.
I solved it by removing the ppa. This allowed me to remove the newer
tools and cloud tools, then re
83 matches
Mail list logo
