[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

If those commits could just be fixes for the problem we are having, that would be a really good start of 2016! -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V]

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

The test kernel I built in comment #78 is failing to boot. I'm going to review my backports and build another test kernel. I'll post a link to it here shortly. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu.

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

I built a Wily test kernel with those three patches. The patches also required three prerequisite commits, so the test kernel has the following six commits: 4246a0b6 block: add a bi_error field to struct bio 54efd50b block: make generic_make_request handle arbitrarily sized bios 03100aad block:

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

We should consider the following upstream storvsc commits to see if they improve the situation: storvsc_drv.c : commit 3209f9d780d137cdb54c85e0a776cb19e723a170 : scsi: storvsc: Fix a bug in the handling of SRB status flags storvsc_drv.c : commit 81988a0e6b031bc80da15257201810ddcf989e64 :

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

I honestly just went the easy way doing that: while true; do mount | grep "sda1 on / type ext4 (ro" > /dev/null if (($? == 0)); then fsck -y /dev/sda1 reboot fi sleep 60 done not the best practice for sure, but... -- You received this bug notification because you are a member

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

My update: It looks the issue is somehow related to the backup, but I tend to think there is a bug somewhere in the storvsc driver code -- it's very hard to track it down because before the ext4 read-only issue happens, the ext4 file system may have been somewhat corrupted. -- You received

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

Just wanted to chip in, we experienced the same problem twice now on a 12.04 LTS Gen 1 VM. Unlike situation described in comment #38, we haven't installed any daemons for the guest OS. Last time it happened, we could definitely correlate it to the Avamar (VSS based) backup. At the time, we were

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

I suspect the race condition may be in vss_on_msg() with the non-thread- safe variable vss_transaction.state. And I guess the below patch may have fixed the issue (the patch hasn't be in the upstream yet): http://lkml.iu.edu/hypermail/linux/kernel/1510.3/04218.html I can only test the patch

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

** No longer affects: linux (Ubuntu Utopic) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

The patch mentioned in #72 can't help -- still bad luck. :-( But I can confirm: before the issue happens, somehow athe host doesn't send us freeze/thaw commands any longer. we need further debugging... -- You received this bug notification because you are a member of Kernel Packages, which is

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@f-bosch @jsalisbury I can reproduce the issue consistently within 5~6 hours with a Ubuntu 15.10 VM. In /var/log/syslog, several minutes before the file system is remounted as read-only, the hv_vss_daemon has stopped working: the daemon just always hangs on the poll() , not receiving freeze/thaw

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

> "BTW, Since Ubuntu 15.04's " typo.. 15.04 -> 15.10. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@decui Correct, ceteris paribus, the only changes were more of the same (disk space) and some package changes through apt-get update (Ubuntu 14.04 LTS). My guess is that the downgrade is related to one of the two. But I agree, let's postpone this one, until the read-only bug is fixed. Afterwards,

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

If you let the wrapper script run long enough, it will eventually fill up the hard disk. This is because tiotest does not remove it's working files when it finishes. I just added a line after the wait to clean up the working directory. I would suggest using this new version of the script, which

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

Thanks @f-bosch for your clarification in #62. So my understanding is: the (temporary) I/O downgrade during the period of backup might be caused by the fact the disk space has been almost used up (?) recently, but it also might be somehow related to the backup. Let's focus on the backup issue at

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

Frederik, Thanks for the new information! About the I/O perf downgrade: 1. I don't think more occupied disk space should harm the perf so greatly (http resopnse time: from <100ms to several seconds). 2. The perf downgrade appeared recently but the backup issue appeared long long ago. Why do you

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@decui @jsalisbury That is indeed good news. Since we did not make any changes to our platform yet, we can help to test possible fixes. There were also some questions. 1. Occurrence of the issue: with the fixes of bug #1445195 the amount of read-only errors reduced drastically. But it is still

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@decui Regarding downgrade, I think I might not have been clear on this. The downgrade occurs only when there are backups running. Our performance otherwise is great. This (new) problem is not necessarily related to this specific issue, but more to backups on Hyper-V in general. Nonetheless, as I

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

I have some good news. I'm unable to reproduce the bug with the -proposed Xenial kernel, which is v4.3 based. I will continue to test this kernel for some time to ensure it really doesn't exhibit the bug. If needed, I'll continue to drive up the IO load. If it does not exhibit the bug, I can

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

I confirmed that the reproduce is repeatable. I can reproduce the bug within 30 to 60 minutes. The bug can be reproduce with the following steps: 1. Create a Virtual machine with 1 cpu and 2048M of memory. 2. Install Ubuntu on the VM. I installed 15.10(Wily). 3. Install the

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

Wrapper script for tiotest. ** Attachment added: "run_tiotest.sh" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+attachment/4529139/+files/run_tiotest.sh -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

Unfortunately, the 4.3 Xenial kernel eventually did hit this bug, it just took longer. I'll continue to investigate. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title:

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@f-bosch Hi Frederik, we're really sorry that this backup-can-cause- filesystem-remounted-readonly issue has been there for long time! We definitely want to get it fixed as soon as possible. Unluckily according to my understanding of the long thread here, I think we had difficulty trying to

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@f-bosch Hi Frederik, about the second issue you mentioned ("The more data getting on the disk, the higher the spikes are...requests that involve write operations tend to have these characteristics") in #54, it looks there is a storage I/O performance downgrade somehow. Do you think if it's

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

I'm now able to reproduce this bug on demand. It takes just under an hour to reproduce. I'm going to to try to reproduce a few more times to make sure it's consistent. If the reproduce is consistent, I'll list all the steps necessary. Basically I'm using the tiotest benchmark from tiobench.

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@Michele, can you confirm that you installed both the linux-image and linux-image-extra .deb packages from the link I posted in #50? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@Joseph Yes, I did, but it does not get past the "loading ramdisk" message as @Frederik says, this is a problem which could really kill ubuntu's usage on servers vm. I refused to move from ubuntu because I like it a lot, but I also refuse to increment my backup intervals. Almost every morning I

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

This problem started more than a year ago, I decided together with my hoster to look at alternatives. Backup is the only thing that is causing problems for us with HyperV. The problem is two fold. First, the read- only state machines get into when generating the backup. Second, the read/write

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

I'm on 15.10 and experiencing this problem. Windows Server 2012 r2 I'm available to test if needed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

I can confirm the kernel from #50 does not boot -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

I'll see if I can figured out why the 3.10 based kernel doesn't boot. However, it may just be easier to test the Precise kernel, which is 3.2 based. The latest Precise kernel can be downloaded from: https://launchpad.net/~ubuntu-security/+archive/ubuntu/ppa/+build/8272116 -- You received this

Re: [Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

Since I was on holidays, I will give this a try in next two weeks. Thanks for the effort @jalisbury. > On 20 Nov 2015, at 15:06, Joseph Salisbury > wrote: > > I'll see if I can figured out why the 3.10 based kernel doesn't boot. > However, it may just be easier

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

I tested the kernel from #48 and it's not booting. I am attaching the log. ** Attachment added: "icalog.txt" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+attachment/4521296/+files/icalog.txt -- You received this bug notification because you are a member of Kernel Packages,

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

I built an Ubuntu kernel based on 3.10, so it is the same base kernel version as CentOS posted in comment #42. If possible, can this kernel be tested to see if this issue is due to a regression introduced after 3.10? The test kernel can be downloaded from:

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

Sorry for the long delay in posting results but I did want to share what we've learned. 1) The failures happen during the VSS Snapshot/checkpoint creation process. You can reproduce the errors by doing manual VSS snapshots. 2) The heavier the VM load, especially I/O, the more likely the error

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@jsalisbury uname -a for the CentOS machines is: Linux host.name 3.10.0-229.11.1.el7.x86_64 #1 SMP Thu Aug 6 01:06:18 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux. This machine was never hit by the backup issue. The technet link you posted contains the following quote: "As a result, we've been

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@f-bosch (#39) I had been able to carry out backups on the VM prior to installing the daemons but I was not able to produce full file indexes and my Hyper-V monitoring software would complain about integration services not being enabled. -- You received this bug notification because you are a

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@stuart-luscombe So if I understand correctly, the reason why you installed the daemons was to get full file indexes and no complaints by monitoring software. However, the downside is that the VMs started crashing during backups after the install? More importantly, can you confirm there was no

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@stuart-luscombe That is interesting. So if we combine both our observations, then my conclusion is: there must be something in the hyper v daemons. And one the things to look at is where the Ubuntu daemons might differ from the CentOS daemons. Maybe @jsalisbury knows if there is any difference

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@f-bosch That's correct. I just checked back through my backup logs and the VM was backing up without error prior to following the technet instructions. I've attached a grab of the status windows for both jobs. ** Attachment added: "Veeam Backup status before and after installing Hyper-V tools"

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

We are currently awaiting the patches posted in comment #35. Can you post the specific CentOS kernel version you are using? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title:

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

What do you mean with: did not occur until you followed those instructions? You mean without the daemons everything was fine? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title:

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

A little remark that still has no answer yet: what are the Ubuntu specifics that cause this issue? In the 10 months we have these machines running: many crashes for Ubuntu with VSS snapshots while the CentOS machines have had no crash at all. Maybe @jsalisbury has a explanation for this? -- You

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

I am experiencing this same issue when backing up a 14.04 LTS Gen 1 VM using Veeam. The error seems to occur when the VSS snapshots are being taken. The error did not occur until I had followed Microsoft's instructions on packages to install for Ubuntu

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

Just wanted to report that I installed the Wily Kernel on Sept 3rd and the VM's ran without errors until yesterday. The errors are different then before but doing the same thing: blk_update_request: I/O error, dev sda, sector 206963000 Aborting journal on device sda2-8 EXT4-fs errors (device

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

We're getting ready for a new round of storvsc fixes that correspond to LIS 4.0.11 we'll have to see if that improves things further. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@jrp Thanks for that, I am happy to hear there is still being worked on. Let me know if there is a build that I can test @h-lbuntu-2 How are your results with Willy kernels? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

Unfortunately, I did not have had time to test Willy kernels. But the other 3.x stable kernels are still contain the problem. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title:

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

Bug is still present in Vivid 3.19.0-26-generic. Is there a workaround that avoids this problem? There's considerable pressure to move off of Hyper-V and I'd rather not do it. [57055.788468] sd 0:0:0:0: [storvsc] Sense Key : Unit Attention [current] [57055.788561] sd 0:0:0:0: [storvsc] Add.

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

To add to comment #30 I posted, the read-only bug continues to occur on both Generation 1 and Generation 2 VMs. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V]

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@ubuntu Is it possible for you to test the latest Wily kernel? The Wily kernel has been rebased to the upstream 4.2 kernel, so it should have all the latest Hyper-V updates in mainline. The Wily kernel can be downloaded from: https://launchpad.net/ubuntu/+source/linux/4.2.0-7.7/+build/7856238

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@Joseph Thank you. I upgraded the production server that is most often errors out with the Wily kernel you referenced above and will report back Please be patient, an individual VM can go several days without the error. -- You received this bug notification because you are a member of Kernel

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@Frederik, have you tested with the Wily kernel yet, posted in comment #9? The Wily kernel has since been rebased to 4.2, so testing of the latest kernel by applying the latest Wily updates would be great. -- You received this bug notification because you are a member of Kernel Packages, which

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

So far so good after the kernal update. I have been backing up a test VM 4 times a day for almost 2 weeks. It has not gone into read only mode as of yet. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu.

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@dander88 That sounds great, but we had the same results for a test VPS machine. As indicated by @jrp before: it depends on your IO load if the machine goes into read-only. And test machines usually do not have that many IO load. -- You received this bug notification because you are a member of

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@Joseph - There are not too much, we use the backup program called ALTARO. It will produce this error about every 10 or so backups. We are working on getting that new kernel into some test units. I will post results when we are done. Thanks for the follow up. -- You received this bug

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

After running the patches, we are still seeing the same error in the syslog. Options? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@Chris - 1- No Local Backup at all - All through Altaro 2 - Standard Local on Scsi 0 3- Sent to a local NAS 3 - I dont know the exact load that they currently have - I know that it is under 80% of system resources for sure. Not 100% on the services, I know MYSQL, other than that not sure. --

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

Hi Dustin, Some questions on the topic, my apologize if these got replied before or in other threads. 1. Is this repro using Windows Server Backup directly, and not through Altaro? 2. For the VM setup, is this a standard local vhdx on scsi controller 0 for a Gen2 VM for OS disk? Or there are

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@John WIlkinson, You should only need the linux-image and linux-image- extra .deb packages to install the latest kernel. The -headers .deb packages should not be needed. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu.

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@Dustin, can you run uname -a to confirm your machine is running the latest Wily kernel built from the current mainline kernel? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@cohn By the way, I have not installed the 4.x kernels yet. So I have no idea how many dependency problems you will run in to. Could you let me know? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu.

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@Joseph Salisbury Is there a specific subset of those .deb packages that need to be run, or are they all needed to patch the relevant bugs? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu.

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@dander88 According to @jrp the message Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. is beneign. He mentioned that in bug 1445195. Are you just seeing that message or do your systems also go

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@F-Bosch we did not enter the read-only mode with the one test we tried. I will keep a backup schedule going multiple times a day and see if it ever goes into read only mode. I will report back in a few days to let you know the results -- You received this bug notification because you are a

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

We have the same VSS Issues in 14.04 LTS but hyper v (2012) gen 1. Has this been seen before? We can reproduce the error on command. Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. -- You

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@Dustin it sounds like you have a reliable way to reproduce this bug? If so, can you list those steps here for others to try? Also, if you can reproduce this, would it be possible for you to test the kernel posted in comment #9? Thanks! -- You received this bug notification because you are a

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

On the other bug (1445195) somebody reported seeing the same error on Gen 1 devices. I wanted to report that we see the identical bug on both Gen 1 and Gen 2 devices. Frequency does not appear to be any different but I don't have precise data. -- You received this bug notification because you

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@jsalisbury I will start testing this week. However, I feel the latest reports were pretty clear: the issue is there. While I was the only one at first that still had problems, after a while more people reported (in bug 1441595) that the new build still contains the issue. In my opinion, it is now

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

That should have been bug 1445195. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@jsalisbury I did not find any new commits on this subject in the current kernel master (https://github.com/torvalds/linux). And I believe you already included all HV commits in the last test build from bug 1441595. So testing this test kernel would mean I am testing whether another commit (not

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@Frederik, yes that is correct. This kernel basically has all HV related commits that are currently in mainline. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title:

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

Prior comments regarding this issue can be found in bug 1445195 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

** Also affects: linux (Ubuntu Trusty) Importance: Undecided Status: New ** Also affects: linux (Ubuntu Vivid) Importance: Undecided Status: New ** Also affects: linux (Ubuntu Wily) Importance: Critical Assignee: Joseph Salisbury (jsalisbury) Status: In

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

The Wily kernel has been rebased to upstream 4.1, which has all the current Hyper-V commits in mainline. Can you give this test kernel a test to see if it still exhibits this issue, or if it is resolved. If it still exhibits the issue, we know that a new fix is needed. If this test kernel

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@jrp @jsalisbury I am using this kernel: http://kernel.ubuntu.com/~jsalisbury/lp1445195/vivid/ on 14.04.2. So that build is much more stable but NOT a complete fix of the problem. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@jrp @jsalisbury I am using this kernel: http://kernel.ubuntu.com/~jsalisbury/lp1445195/vivid/ on 14.04.2. So that build is much more stable but a complete fix of the problem. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

I believe he is running 14.04.2, which means at least the HWE kernel. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

My latest report was that latest builds with patches are much more stable but are also not a complete fix for the problem. It is still there and occurs randomly. The error message is not changed. I have no real indication what causes the read-only state. During the latest RO state I noticed there

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

Error message that happens when this bug occurs: [154272.293488] sd 2:0:0:0: [storvsc] Sense Key : Unit Attention [current] [154272.293508] sd 2:0:0:0: [storvsc] Add. Sense: Changed operating definition [154272.293665] sd 2:0:0:0: Warning! Received an indication that the operating parameters on

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

@Frederik Bosch Can you post what kernel version you are currently using? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI

[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups

Some additional details: https://social.technet.microsoft.com/Forums/windowsserver/en-US /8807f61c-565e-45bc-abc4-af09abf59de2/ubuntu-14042-lts-generation-2 -scsi-errors-on-vss-based-backups -- You received this bug notification because you are a member of Kernel Packages, which is subscribed

<    1   2   3   4