Re: [Qemu-devel] [RFC V4 00/30] QCOW2 deduplication
On Thu, Jan 03, 2013 at 01:39:48PM +0100, Stefan Hajnoczi wrote: On Wed, Jan 02, 2013 at 12:26:37PM -0600, Troy Benjegerdes wrote: The probability may be 'low' but it is not zero. Just because it's hard to calculate the hash doesn't mean you can't do it. If your input data is not random the probability of a hash collision is going to get scewed. The cost of catching hash collisions is an extra read for every write. It's possible to reduce this with a 2nd hash function and/or caching. I'm not sure it's worth it given the extremely low probability of a hash collision. Venti is an example of an existing system where hash collisions were ignored because the probability is so low. See 3.1. Choice of Hash Function section: http://plan9.bell-labs.com/sys/doc/venti/venti.html If you believe that it's 'extremely low', then please provide either: * experimental evidence to prove your claim * an insurance underwriter who will pay-out if data is lost due to a hash collision. What I have heard so far is a lot of theoretical posturing and no experimental evidence. Please google for when TCP checksums and CRC disagree for experimental evidence of problems assuming that probability is low. This is the abstract: Traces of Internet packets from the past two years show that between 1 packet in 1,100 and 1 packet in 32,000 fails the TCP checksum, even on links where link-level CRCs should catch all but 1 in 4 billion errors. For certain situations, the rate of checksum failures can be even higher: in one hour-long test we observed a checksum failure of 1 packet in 400. We investigate why so many errors are observed, when link-level CRCs should catch nearly all of them.We have collected nearly 500,000 packets which failed the TCP or UDP or IP checksum. This dataset shows the Internet has a wide variety of error sources which can not be detected by link-level checks. We describe analysis tools that have identified nearly 100 different error patterns. Categorizing packet errors, we can infer likely causes which explain roughly half the observed errors. The causes span the entire spectrum of a network stack, from memory errors to bugs in TCP.After an analysis we conclude that the checksum will fail to detect errors for roughly 1 in 16 million to 10 billion packets. From our analysis of the cause of errors, we propose simple changes to several protocols which will decrease the rate of undetected error. Even so, the highly non-random distribution of errors strongly suggests some applications should employ application-level checksums or equivalents.
Re: [Qemu-devel] [RFC V4 00/30] QCOW2 deduplication
On Wed, Jan 02, 2013 at 05:16:03PM +0100, Beno??t Canet wrote: This patchset is a cleanup of the previous QCOW2 deduplication rfc. One can compile and install https://github.com/wernerd/Skein3Fish and use the --enable-skein-dedup configure option in order to use the faster skein HASH. Images must be created with -o dedup=[skein|sha256] in order to activate the deduplication in the image. Deduplication is now fast enough to be usable. How does this code handle hash collisions, and do you have some regression tests that purposefully create a dedup hash collision, and verify that the 'right thing' happens? The next question is .. what's the right thing? It's great that this almost works, but it seems rather dangerous to put something like this into the mainline code without some regression tests. (I'm also suspecting the regression test will be a great way to find flakey hardware) -- Troy Benjegerdes'da hozer' ho...@hozed.org Somone asked my why I work on this free (http://www.fsf.org/philosophy/) software hardware (http://q3u.be) stuff and not get a real job. Charles Shultz had the best answer: Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life. -- Charles Shultz
Re: [Qemu-devel] [RFC V4 00/30] QCOW2 deduplication
The probability may be 'low' but it is not zero. Just because it's hard to calculate the hash doesn't mean you can't do it. If your input data is not random the probability of a hash collision is going to get scewed. Read about how Bitcoin uses hashes. I need a budget of around $10,000 or so for some FPGAs and/or GPU cards, and I can make a regression test that will create deduplication hash collisions on purpose. On Wed, Jan 02, 2013 at 06:33:24PM +0100, Beno?t Canet wrote: How does this code handle hash collisions, and do you have some regression tests that purposefully create a dedup hash collision, and verify that the 'right thing' happens? The two hash function that can be used are cryptographics and not broken yet. So nobody knows how to generate a collision. You can do the math to calculate the probability of collision using a 256 bit hash while processing 1EiB of data the result is so low you can consider it won't happen. The sha256 ZFS deduplication works the same way regarding collisions. I currently use qemu-io-test for testing purpose and iozone with the -w flag in the guest. I would like to find a good deduplication stress test to run in a guest. Regards Beno?t It's great that this almost works, but it seems rather dangerous to put something like this into the mainline code without some regression tests. (I'm also suspecting the regression test will be a great way to find flakey hardware) -- Troy Benjegerdes'da hozer' ho...@hozed.org Somone asked my why I work on this free (http://www.fsf.org/philosophy/) software hardware (http://q3u.be) stuff and not get a real job. Charles Shultz had the best answer: Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life. -- Charles Shultz -- -- Troy Benjegerdes'da hozer' ho...@hozed.org Somone asked my why I work on this free (http://www.fsf.org/philosophy/) software hardware (http://q3u.be) stuff and not get a real job. Charles Shultz had the best answer: Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life. -- Charles Shultz
Re: [Qemu-devel] [RFC V4 00/30] QCOW2 deduplication
If you do get a hash collision, it's a rather exceptional event, so I'd say every effort should be made to log the event and the data that created it in multiple places. There are three questions I'd ask on a hash collision: 1) was it the data? 2) was it the hardware? 3) was it a software bug? On Wed, Jan 02, 2013 at 10:47:48AM -0800, ronnie sahlberg wrote: Do you really need to resolve the conflicts? It might be easier and sufficient to just flag those hashes where a conflict has been detected as : dont dedup this hash anymore, collissions have been seen. On Wed, Jan 2, 2013 at 10:40 AM, Beno?t Canet benoit.ca...@irqsave.net wrote: Le Wednesday 02 Jan 2013 ? 12:26:37 (-0600), Troy Benjegerdes a ?crit : The probability may be 'low' but it is not zero. Just because it's hard to calculate the hash doesn't mean you can't do it. If your input data is not random the probability of a hash collision is going to get scewed. Read about how Bitcoin uses hashes. I need a budget of around $10,000 or so for some FPGAs and/or GPU cards, and I can make a regression test that will create deduplication hash collisions on purpose. It's not a problem as Eric pointed out while reviewing the previous patchset there is a small place left with zeroes on the deduplication block. A bit could be set on it when a collision is detected and an offset could point to a cluster used to resolve collisions. On Wed, Jan 02, 2013 at 06:33:24PM +0100, Beno?t Canet wrote: How does this code handle hash collisions, and do you have some regression tests that purposefully create a dedup hash collision, and verify that the 'right thing' happens? The two hash function that can be used are cryptographics and not broken yet. So nobody knows how to generate a collision. You can do the math to calculate the probability of collision using a 256 bit hash while processing 1EiB of data the result is so low you can consider it won't happen. The sha256 ZFS deduplication works the same way regarding collisions. I currently use qemu-io-test for testing purpose and iozone with the -w flag in the guest. I would like to find a good deduplication stress test to run in a guest. Regards Beno?t It's great that this almost works, but it seems rather dangerous to put something like this into the mainline code without some regression tests. (I'm also suspecting the regression test will be a great way to find flakey hardware) -- Troy Benjegerdes'da hozer' ho...@hozed.org Somone asked my why I work on this free (http://www.fsf.org/philosophy/) software hardware (http://q3u.be) stuff and not get a real job. Charles Shultz had the best answer: Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life. -- Charles Shultz -- -- Troy Benjegerdes'da hozer' ho...@hozed.org Somone asked my why I work on this free (http://www.fsf.org/philosophy/) software hardware (http://q3u.be) stuff and not get a real job. Charles Shultz had the best answer: Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life. -- Charles Shultz -- -- Troy Benjegerdes'da hozer' ho...@hozed.org Somone asked my why I work on this free (http://www.fsf.org/philosophy/) software hardware (http://q3u.be) stuff and not get a real job. Charles Shultz had the best answer: Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life. -- Charles Shultz
Re: [Qemu-devel] Connecting virtio-9p-pci to a remote 9p server
On Mon, Oct 15, 2012 at 12:36:08PM +0100, Chris Webb wrote: We're planning to implement shared filesystems for guests on our virtualized hosting platform, stored on a central fileserver separate from the hosts. Whilst we can mount the shares on each host and then use qemu's 9p passthrough/proxy support to access the mountpoint, going via the host kernel and vfs like this feels quite inefficient. We would be converting back and forth between vfs and 9p models several times needlessly. Instead, I'm wondering about the feasibility of connecting the 9p stream directly from qemu's virtio-9p-pci device to a socket opened on a 9p-over-TCP export from the fileserver. Am I right in thinking that qemu's -fsdev proxy gives me access to a file descriptor attached to the 9p stream to/from the guest, or is the protocol between virtfs-proxy-helper and qemu re-encoded within qemu first? Secondly, assuming I can somehow get at the 9p streams directly (either with an existing option or by adding a new one), I'd like to restrict guests to the relevant user's subdirectory on the fileserver, and have been thinking about doing this by filtering the 9p stream to restrict 'attach' operations. Fortunately, 9p uses client-chosen fids rather than server filesystem inode numbers which would immediately scupper any simple attempts to implement a secure chroot proxy of this kind. Looking at the 9p2000.L protocol, it doesn't look obviously difficult, but I've not really worked with 9p before, and could well be missing security complications. (I'm not sure whether there's risk of symlinks being interpreted server side rather than client side, for example.) I'd also be interested in any more general thoughts on this kind of thing. If we're going to work on it, it would be nice for us to write something that would be more widely useful to others rather than just create an in-house hack. Cheers, Chris. If scalability and security are long-term goals, I'd suggest you take a look at OpenAFS (openafs.org). There are other complications, in that you start getting into stuff like how to authenticate your users to the filesystem, but imho, it's a PITA to switch and get used to this model, but once you do, you don't have to worry about if you've got some complicated (and one-off) security filtering configured right. I've been playing with booting debian kernels initrds directly from AFS as the root filesystem ( http://bitspjoule.org/hg/initramfs-tools ). What's nice (and also a PITA) is that normally the VM client cannot modify the root filesystem, so I know there's no magic configuration on some VM disk image, but if I wanted to make a change to multiple VMs, I could authenticate to AFS as administrator on one of the nodes, make the change, and then restart the daemons on all the other VMs to load the new change. The other (potential) advantage of AFS in a virtualized environment is client side caching, so instead of VM disk image for the OS, and worrying about backing it up, you just use that disk image as the client-side cache local to the VM host machine. The actual authoritative data is stored on the AFS server, so if you have it cached locally, you never have to hit the network, and if the disk the cache is on dies, you just restart the VM on a different disk. (If you wanted to go overboard, you could modify the client-caching code to go back to the server if the cache gets a read error) I can't say that AFS has really *solved* all the hard problems with this, but there is at least some history on how to effectively deal with them. With what you are describing with 9p in a production environment, I think you'll end up re-discovering all the hard problems and have to invent new 9p-specific ways of dealing with them, and you'll end up with the same complexity as AFS. - Troy
Re: [Qemu-devel] Windows slow boot: contractor wanted
I've now triggered a very slow boot at 3x 36GB 8-core VMs on a 128GB host (i.e. 108GB on a 128GB host). It has the same profile with _raw_spin_lock_irqsave and isolate_freepages_block at the top. Then it's still memory starved. Please provide /proc/zoneinfo while this is happening. Is there a way to capture/reproduce this 'slow boot' behavior with a simple regression test? I'd like to know if it happens on a single-physical CPU socket machine, or just on dual-sockets. I'm also observing an interesting phenomenon here.. Kernel development can move so fast as to make regression testing pointless. ;)
Re: [Qemu-devel] Windows slow boot: contractor wanted
Do you have any way to determine what CPU groups the different VMs are running on? If you end up in an overcommit situation where half the 'virtual' cpus are on one AMD socket, and the other half are on a different AMD socket, then you'll be thrashing the hypertransport link. At Cray we were very carefull to never overcommit runnable processes to CPUS, and generally locked processes to a single cpu. Have a read of http://berrange.com/posts/2010/02/12/controlling-guest-cpu-numa-affinity-in-libvirt-with-qemu-kvm-xen/ I'm going to speculate that when things don't work very well you end up with memory from a booting guest scattered across many different NUMA nodes/cpus, and then it really won't matter how good the spin loop/scheduler code is because you are bound by the additional latency and bandwidth limitations of running on one socekt and accessing half the memory that's resident on a different socket. On Tue, Aug 21, 2012 at 04:21:07PM +0100, Richard Davies wrote: Avi Kivity wrote: Richard Davies wrote: We're running host kernel 3.5.1 and qemu-kvm 1.1.1. I hadn't though about it, but I agree this is related to cpu overcommit. The slow boots are intermittent (and infrequent) with cpu overcommit whereas I don't think it occurs without cpu overcommit. In addition, if there is a slow boot ongoing, and you kill some other VMs to reduce cpu overcommit then this will sometimes speed it up. I guess the question is why even with overcommit most boots are fine, but some small fraction then go slow? Could be a bug. The scheduler and the spin-loop handling code fight each other instead of working well. Please provide snapshots of 'perf top' while a slow boot is in progress. Below are two 'perf top' snapshots during a slow boot, which appear to me to support your idea of a spin-lock problem. There are a lot more unprocessable samples recorded messages at the end of each snapshot which I haven't included. I think these may be from the guest OS - the kernel is listed, and qemu-kvm itself is listed on some other traces which I did, although not these. Richard. PerfTop: 62249 irqs/sec kernel:96.9% exact: 0.0% [4000Hz cycles], (all, 16 CPUs) 35.80% [kernel] [k] _raw_spin_lock_irqsave 21.64% [kernel] [k] isolate_freepages_block 5.91% [kernel] [k] yield_to 4.95% [kernel] [k] _raw_spin_lock 3.37% [kernel] [k] kvm_vcpu_on_spin 2.74% [kernel] [k] add_preempt_count 2.45% [kernel] [k] _raw_spin_unlock 2.33% [kernel] [k] sub_preempt_count 2.18% [kernel] [k] svm_vcpu_run 2.17% [kernel] [k] kvm_vcpu_yield_to 1.89% [kernel] [k] memcmp 1.50% [kernel] [k] get_pid_task 1.26% [kernel] [k] kvm_arch_vcpu_ioctl_run 1.16% [kernel] [k] pid_task 0.70% [kernel] [k] rcu_note_context_switch 0.70% [kernel] [k] trace_hardirqs_on 0.52% [kernel] [k] __rcu_read_unlock 0.51% [kernel] [k] trace_preempt_on 0.47% [kernel] [k] __srcu_read_lock 0.43% [kernel] [k] get_parent_ip 0.42% [kernel] [k] get_pageblock_flags_group 0.38% [kernel] [k] in_lock_functions 0.34% [kernel] [k] trace_preempt_off 0.34% [kernel] [k] trace_hardirqs_off 0.29% [kernel] [k] clear_page_c 0.23% [kernel] [k] __srcu_read_unlock 0.20% [kernel] [k] __rcu_read_lock 0.14% [kernel] [k] handle_exit 0.11% libc-2.10.1.so[.] strcmp 0.11% [kernel] [k] _raw_spin_unlock_irqrestore 0.11% [kernel] [k] _raw_spin_lock_irq 0.11% [kernel] [k] find_highest_vector 0.09% [kernel] [k] ktime_get 0.08% [kernel] [k] copy_page_c 0.08% [kernel] [k] pause_interception 0.08% [kernel] [k] kmem_cache_alloc 0.08% [kernel] [k] resched_task 0.08% perf [.] dso__find_symbol 0.06% [kernel] [k] compaction_alloc 0.06% libc-2.10.1.so[.] 0x00076dab 0.06% [kernel] [k] read_tsc 0.06% perf [.] add_hist_entry 0.05% [kernel] [k] svm_read_l1_tsc 0.05% [kernel] [k] native_read_tsc 0.05% perf [.] sort__dso_cmp 0.05% [kernel] [k] copy_user_generic_string 0.05% [kernel] [k] ktime_get_update_offsets 0.04% [kernel] [k] kvm_check_async_pf_completion 0.04% [kernel] [k] __schedule 0.04% [kernel] [k] __rcu_pending 0.04%
Re: [Qemu-devel] Windows slow boot: contractor wanted
I'd be interested in working on this.. What I'd like to propose is to write an automated regression test harness that will reboot the host hardware, and start booting up guest VMs and report the time-to-boot, as well as relative performance of the running VMs. For best results, I'd need access to the specific hardware you are using. I'd also like to release the test harness back to the community, so I would like some feedback from the mailing list on what kinds of tests should be written that would provide the best information for the KVM developers. What do you want to know, and what is the most usefull data to record to debug this and future performance regressions? On Thu, Aug 16, 2012 at 11:47:27AM +0100, Richard Davies wrote: Hi, We run a cloud hosting provider using qemu-kvm 1.1, and are keen to find a contractor to track down and fix problems we have with large memory Windows guests booting very slowly - they can take several hours. We previously reported these problems in July (copied below) and they are still present with Linux kernel 3.5.1 and qemu-kvm 1.1.1. This is a serious issue for us which is causing significant pain to our larger Windows VM customers when their servers are offline for many hours during boot. If anyone knowledgeable in the area would be interested in being paid to work on this, or if you know someone who might be, I would be delighted to hear from you. Cheers, Richard. = Previous bug report http://marc.info/?l=qemu-develm=134304194329745 We have been experiencing this problem for a while now too, using qemu-kvm (currently at 1.1.1). Unfortunately, hv_relaxed doesn't seem to fix it. The following command line produces the issue: qemu-kvm -nodefaults -m 4096 -smp 8 -cpu host,hv_relaxed -vga cirrus -usbdevice tablet -vnc :99 -monitor stdio -hda test.img The hardware consists of dual AMD Opteron 6128 processors (16 cores in total) and 64GB of memory. This command line was tested on kernel 3.1.4. I've also tested with -no-hpet. What I have seen is much as described: the memory fills out slowly, and top on the host will show the process using 100% on all allocated CPU cores. The most extreme case was a machine which took something between 6 and 8 hours to boot. This seems to be related to the assigned memory, as described, but also the number of processor cores (which makes sense if we believe it's a timing issue?). I have seen slow-booting guests improved by switching down to a single or even two cores. Matthew, I agree that this seems to be linked to the number of VMs running - in fact, shutting down other VMs on a dedicated test host caused the machine to start booting at a normal speed (with no reboot required). However, the level of contention is never such that this could be explained by the host simply being overcommitted. If it helps anyone, there's an image of the hard drive I've been using to test at: http://46.20.114.253/ It's 5G of gzip file containing a fairly standard Windows 2008 trial installation. Since it's in the trial period, anyone who wants to use it may have to re-arm the trial: http://support.microsoft.com/kb/948472 Please let me know if I can provide any more information, or test anything. Best wishes, Owen Tuz
[Qemu-devel] OSX 10.4.7 build
I am trying to build qemu-cvs on OSX 10.4.7, and I am getting the following error: gcc -DQEMU_TOOL -Wall -O2 -g -fno-strict-aliasing -I. -mdynamic-no-pic -g -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -o qemu-img qemu-img.c block.c block-raw.c block-cow.c block-qcow.c aes.c block-vmdk.c block-cloop.c block-dmg.c block-bochs.c block-vpc.c block-vvfat.c block-qcow2.c -lz block.c: In function 'bdrv_commit': block.c:448: error: 'ENOMEDIUM' undeclared (first use in this function) -- -- Troy Benjegerdes'da hozer'[EMAIL PROTECTED] Somone asked me why I work on this free (http://www.fsf.org/philosophy/) software stuff and not get a real job. Charles Shultz had the best answer: Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life. -- Charles Shultz ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] qemu vnc.c vnchextile.h
On Sun, May 14, 2006 at 06:11:50PM +, Fabrice Bellard wrote: CVSROOT: /sources/qemu Module name: qemu Branch: Changes by: Fabrice Bellard [EMAIL PROTECTED] 06/05/14 18:11:49 Modified files: . : vnc.c vnchextile.h Log message: support for all VNC pixel formats Chicken of the vnc on MacOS X http://sourceforge.net/projects/cotvnc/ seems to have some issues with hextile encoding with a linux-x86-64 host system running qemu. Xvncviewer on a big-endian PPC system works just find though. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] qemu vnc.c vnchextile.h
Modified files: . : vnc.c vnchextile.h Log message: support for all VNC pixel formats Chicken of the vnc on MacOS X http://sourceforge.net/projects/cotvnc/ seems to have some issues with hextile encoding with a linux-x86-64 host system running qemu. Xvncviewer on a big-endian PPC system works just find though. I have ethereal output from this, although it makes little sense since there does not seem to be a decent VNC protocol dissector available. Please contact me directly if you enjoy wading through VNC logs.. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] qemu vnc.c vnchextile.h
Chicken of the vnc on MacOS X http://sourceforge.net/projects/cotvnc/ Can you provide a bit more info on what you mean by troubles? Regards, Anthony Liguori I found a wonderfull little tool called 'rfbproxy'. It, and saved sessions of the bogus behavior with chickenofthevnc are at: http://scl.ameslab.gov/~troy/qemu Also, can I make a feature request that 'info vnc' in the qemu monitor return all the appropriate connection information, including bit depth, encoding, etc? ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] PATCH: Support for multi-file raw images
On Thu, May 11, 2006 at 02:30:45AM -0400, Ryan Lortie wrote: Hello. Attached is a C file (and small patch) to add support for multi-file raw images to QEMU. The rationale (for me at least) is as follows: I use rsync to backup my home directory. The act of starting up QEMU changes a 20GB file on my drive. This causes 20GB of extra copying next time I do backups. If I could split the drive image into smaller parts (maybe 2048 10MB files) then the amount of extra copying is drastically reduced (since only a few of these files are modified). There are definitely other reasons that this may be useful. Have you tried making a read-only 'base' image and using qcow images instead? I'm not convinced that splitting things up is going to help a lot. You might end up writing 1 512 byte block each to 500 files.. in the qcow image case, that is writing 256K, and with 10mb files, that's 5GB. o If the files comprising the device are deleted (for example) while QEMU is running then this is quite bad. Currently this will result in read/write requests returning -1. Maybe it makes sense to panic and cause QEMU to exit. at the very least, the console should print an error. If you can keep all the files open, deleting the file won't be a problem. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
[Qemu-devel] VNC cross-endian failures
The VNC protocol says the server is is supposed to send the data in the format the client wants, however the current implementation sends vnc data in the server native format. What is the best way to fix this? Using -bgr is not right since that will mess up same-endian clients. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] VNC cross-endian failures
Much better.
I did have one bogon after running xvncview on a big-endian host,
closing it, then starting it on a little-endian host that resulted in
this from realvnc:
Rect too big: 24832x33024 at 8448,16640 exceeds 640x480
main:Rect too big
On Sat, May 13, 2006 at 01:02:47AM +0200, Fabrice Bellard wrote:
Troy Benjegerdes wrote:
The VNC protocol says the server is is supposed to send the data in the
format the client wants, however the current implementation sends vnc
data in the server native format.
What is the best way to fix this? Using -bgr is not right since that
will mess up same-endian clients.
Try the attached patch.
Fabrice.
Index: vnc.c
===
RCS file: /sources/qemu/qemu/vnc.c,v
retrieving revision 1.5
diff -u -w -r1.5 vnc.c
--- vnc.c 3 May 2006 21:18:59 - 1.5
+++ vnc.c 12 May 2006 22:54:21 -
@@ -42,6 +42,14 @@
typedef int VncReadEvent(VncState *vs, char *data, size_t len);
+typedef void VncWritePixels(VncState *vs, void *data, int size);
+
+typedef void VncSendHextileTile(VncState *vs,
+int x, int y, int w, int h,
+uint32_t *last_bg,
+uint32_t *last_fg,
+int *has_bg, int *has_fg);
+
struct VncState
{
QEMUTimer *timer;
@@ -53,12 +61,19 @@
int height;
uint64_t dirty_row[768];
char *old_data;
-int depth;
+int depth; /* internal VNC frame buffer byte per pixel */
int has_resize;
int has_hextile;
Buffer output;
Buffer input;
kbd_layout_t *kbd_layout;
+/* current output mode information */
+VncWritePixels *write_pixels;
+VncSendHextileTile *send_hextile_tile;
+int pix_bpp, pix_big_endian;
+int red_shift, red_max, red_shift1;
+int green_shift, green_max, green_shift1;
+int blue_shift, blue_max, blue_shift1;
VncReadEvent *read_handler;
size_t read_handler_expect;
@@ -130,6 +145,66 @@
}
}
+/* fastest code */
+static void vnc_write_pixels_copy(VncState *vs, void *pixels, int size)
+{
+vnc_write(vs, pixels, size);
+}
+
+/* slowest but generic code. */
+static void vnc_convert_pixel(VncState *vs, uint8_t *buf, uint32_t v)
+{
+unsigned int r, g, b;
+
+r = (v vs-red_shift1) vs-red_max;
+g = (v vs-green_shift1) vs-green_max;
+b = (v vs-blue_shift1) vs-blue_max;
+v = (r vs-red_shift) |
+(g vs-green_shift) |
+(b vs-blue_shift);
+switch(vs-pix_bpp) {
+case 1:
+buf[0] = v;
+break;
+case 2:
+if (vs-pix_big_endian) {
+buf[0] = v 8;
+buf[1] = v;
+} else {
+buf[1] = v 8;
+buf[0] = v;
+}
+break;
+default:
+case 4:
+if (vs-pix_big_endian) {
+buf[0] = v 24;
+buf[1] = v 16;
+buf[2] = v 8;
+buf[3] = v;
+} else {
+buf[3] = v 24;
+buf[2] = v 16;
+buf[1] = v 8;
+buf[0] = v;
+}
+break;
+}
+}
+
+static void vnc_write_pixels_generic(VncState *vs, void *pixels1, int size)
+{
+uint32_t *pixels = pixels1;
+uint8_t buf[4];
+int n, i;
+
+n = size 2;
+for(i = 0; i n; i++) {
+vnc_convert_pixel(vs, buf, pixels[i]);
+vnc_write(vs, buf, vs-pix_bpp);
+}
+}
+
static void send_framebuffer_update_raw(VncState *vs, int x, int y, int w,
int h)
{
int i;
@@ -139,7 +214,7 @@
row = vs-ds-data + y * vs-ds-linesize + x * vs-depth;
for (i = 0; i h; i++) {
- vnc_write(vs, row, w * vs-depth);
+ vs-write_pixels(vs, row, w * vs-depth);
row += vs-ds-linesize;
}
}
@@ -162,35 +237,26 @@
#include vnchextile.h
#undef BPP
+#define GENERIC
+#define BPP 32
+#include vnchextile.h
+#undef BPP
+#undef GENERIC
+
static void send_framebuffer_update_hextile(VncState *vs, int x, int y, int
w, int h)
{
int i, j;
int has_fg, has_bg;
uint32_t last_fg32, last_bg32;
-uint16_t last_fg16, last_bg16;
-uint8_t last_fg8, last_bg8;
vnc_framebuffer_update(vs, x, y, w, h, 5);
has_fg = has_bg = 0;
for (j = y; j (y + h); j += 16) {
for (i = x; i (x + w); i += 16) {
- switch (vs-depth) {
- case 1:
- send_hextile_tile_8(vs, i, j, MIN(16, x + w - i), MIN(16, y + h
- j),
- last_bg8, last_fg8, has_bg, has_fg);
- break;
- case 2:
- send_hextile_tile_16(vs, i, j, MIN(16, x + w - i), MIN(16, y +
h - j),
- last_bg16, last_fg16, has_bg, has_fg);
- break;
- case 4:
- send_hextile_tile_32(vs, i, j, MIN
Re: [Qemu-devel] Re: QEMU 0.8.1 and vnc
On Fri, May 05, 2006 at 09:06:20AM -0500, Anthony Liguori wrote: Ben Taylor wrote: I'm seeing quite a few bugs on Qemu 0.8.1 with the vnc feature 1) Sparc based system comes up in distored colors (foreground of a Damn Small linux iso comes up in yellow, instead of white) This is a know problem. qemu doesn't give any indication that the guest is storing pixels in big endian mode. A proper fix is on my TODO list. 2) When it bounces from the initial syslinux text to the grahpical screen, it leaves the text in the top left corner not cleared. (to the boot: at the bottom) Yeah, I've noticed something similar myself. It's on the TODO list (see vnc.c). 3) screen autoresize is not working. 2 examples with DSL. a) initial screen is 80x25, but qemu comes up in 80x24 and stays there and I can't see boot: prompt at the bottom. b) when it goes into graphical mode, it's not resizing so I'm missing some portion of the screen TightVNC doesn't support the desktop resize encoding. Try RealVNC. I am running the current CVS code and seeing endian color problems with a x86 machine running qemu and a PPC linux machine running xrealvncviewer. This is the debian xvncviewer package version 3.3.7-7. Also, how does one get to the qemu console with the vnc? The vnc server also seems to occasionally send invalid vnc packets, and the screen resize does not seem to work. Included is a log of starting up and exiting because a bad message.. The bad message problem occurs with Chicken of the VNC on MacOS X as well. VNC viewer version 3.3.7 - built Sep 25 2004 21:02:46 Copyright (C) 2002-2003 RealVNC Ltd. Copyright (C) 1994-2000 ATT Laboratories Cambridge. See http://www.realvnc.com for information on VNC. VNC server supports protocol version 3.3 (viewer 3.3) No authentication needed Desktop name QEMU Connected to VNC server, using protocol version 3.3 VNC server default format: 32 bits per pixel. Least significant byte first in each pixel. True colour: max red 255 green 255 blue 255, shift red 16 green 8 blue 0 Using default colormap and visual, TrueColor, depth 24. Got 256 exact BGR233 colours out of 256 Using BGR233 pixel format: 8 bits per pixel. True colour: max red 7 green 7 blue 3, shift red 0 green 3 blue 6 Throughput 2 kbit/s - changing to Hextile Throughput 2 kbit/s - changing from 8bit Using viewer's native pixel format: 32 bits per pixel. Most significant byte first in each pixel. True colour: max red 255 green 255 blue 255, shift red 16 green 8 blue 0 Rect too large: 69x1 at (705, 577) ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] vnc patch
-usbdevice tablet emulates a absolute position USB HID device instead of a PS/2 mouse, so there is no need for messy mouse calibration when using VNC. It also allows you to get rid of the need to have the SDL version 'grab' the mouse. On Sat, Apr 29, 2006 at 07:36:01PM +0200, Christian MICHON wrote: did I understand well ? 1) there's a rework of the original vnc patch 2) there are no problems of sync using the usb tablet what is this usb tablet stuff? do I really need a usb tablet connected to the host or this is yet another emulated hardware inside the qem guest ? If this is the case, it means the client OS must have the proper drivers, etc... I'm a bit lost. Christian On 4/29/06, Anthony Liguori [EMAIL PROTECTED] wrote: Lonnie Mendez wrote: Lonnie Mendez wrote: There is another vnc patch by Anthony Liguori (same person involved with the tablet along with others). From what I've heard it will reduce the bandwidth usage greatly. Also, afaik this is still under development. The source was posted on the irc channel a few days ago: http://www.cs.utexas.edu/users/aliguori/vnc.diff The mercurial tree is located at http://hg.codemonkey.ws/qemu-vnc If you go to Manifest - Changeset you can download a static tarball if you're unfamiliar with mercurial. It's in a pretty good state now. I don't think there's any significant bugs and all the features I plan on implementing for the first rev are there. I'll have a patch ready for submission in a day or so. Regards, Anthony Liguori ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel -- -- Troy Benjegerdes'da hozer'[EMAIL PROTECTED] Somone asked me why I work on this free (http://www.fsf.org/philosophy/) software stuff and not get a real job. Charles Shultz had the best answer: Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life. -- Charles Shultz ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
[Qemu-devel] ppc32 linux host compile failure
Can someone educate me on what might be wrong here? [EMAIL PROTECTED]:/scratch/troy/qemu-cvs-patched$ gcc-3.4 --version gcc-3.4 (GCC) 3.4.4 20050314 (prerelease) (Debian 3.4.3-13) [EMAIL PROTECTED]:/scratch/troy/qemu-cvs-patched$ ld --version GNU ld version 2.16.91 20060118 Debian GNU/Linux make[1]: Entering directory `/scratch/troy/qemu-cvs-patched/i386-user' gcc-3.4 -g -Wl,-T,/scratch/troy/qemu-cvs-patched/ppc.ld -o qemu-i386 elfload.o main.o syscall.o mmap.o signal.o path.o osdep.o thunk.o vm86.o libqemu.a gdbstub.o -lm /usr/bin/ld: qemu-i386: Not enough room for program headers (allocated 8, need 9) /usr/bin/ld: final link failed: Bad value collect2: ld returned 1 exit status make[1]: *** [qemu-i386] ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
[Qemu-devel] Feature request: -hda image file locks, and warnings for readonly
I managed to make the mistake of starting up two instances of qemu using the same '-hda' win2k guest. Needless to say this rendered the image unbootable. What I'd like is for qemu to do some sort of optional (but default behavior) file lock on the disk image, and spit out a warning and refuse to start if another qemu process already locked the image. This would be something you should be able to override if you really know what you are doing. I also managed to cause myself confusion by having an earlier image file set mode '444' and the windows error messages didn't make much sense, and I never got any sort of error indication out of qemu that the writes had been failing. -- -- Troy Benjegerdes'da hozer'[EMAIL PROTECTED] Somone asked me why I work on this free (http://www.fsf.org/philosophy/) software stuff and not get a real job. Charles Shultz had the best answer: Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life. -- Charles Shultz ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [PATCH] VNC display support for QEMU
On Sun, Apr 30, 2006 at 03:03:55AM +0200, Johannes Schindelin wrote: Hi, On Sat, 29 Apr 2006, Anthony Liguori wrote: One thing you may notice is that RealVNC has some issues with being disconnected. This is because it likes to switch from 8bit to 32bit depths automatically at startup. Unfortunately, there is a race condition in the VNC protocol and since this implementation is asynchronous, we seem to be much more prone to exposing this. This, along with other problems, has been solved with LibVNCServer. But of course, you are welcome to solve them again. I have noticed that interactive performance with the current libvncserver based patches is quite bad compared to doing the same thing on SDL. Maybe re-implementing vnc will uncover why it is slow. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [PATCH] VNC display support for QEMU
Where 1 is the first display (port 5901). This syntax may change in the near future to support binding to a particular interface. It's very useful to use an absolute mouse with VNC as the relative support is quite poor. It may be useful to adapt the libvncserver patch's calibration code here but I've not attempted to do that yet. This patch is still experimental. I've tested it with RealVNC and TightVNC under a variety of depths but I won't be suprised if there are still problems. I only implement Raw, CopyRect, and Hextile encodings too. Any sort of palette color mode or pixel format that QEMU doesn't support will not work either. One thing you may notice is that RealVNC has some issues with being disconnected. This is because it likes to switch from 8bit to 32bit depths automatically at startup. Unfortunately, there is a race condition in the VNC protocol and since this implementation is asynchronous, we seem to be much more prone to exposing this. A short near-term TODO list is: 1) More testing 2) Support switching between monitor/serial 3) Support a better encoding (like TightEncoding or ZRLE) 4) Support a vnc password (and perhaps stuff like TLS) Any feedback is greatly appreciated (especially with how it works with clients I've not tested). realvnc on debian-ppc (xvncviewer-3.3.7) has endian issues (the colors are wrong). However, mouse tracking and dragging windows actually *works*. (This is with the version currently in cvs) ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] USB Tablet Emulation + VNC patch..
Now, does anyone have instructions on how to get Win2k installed and updated to the latest set of security patches? I can get service pack 4 installed, but running windowsupdate seems to never work right. If I run it up with -win2k-hack then windowsupdate works fine.. Here is what I did though (although variants have worked) Install win2k-sp4 run windowsupdate and ignore the warning about ie6, let it install the updated windows-update component. Reboot, install ie6, reboot, then run windows-update and install all the patches. All of this with win2k-hack enabled or it won't work.. and the updated win2k-hack from Leo so it works with dma.. I'm still stuck with windowsupdate not working.. I am finding now that after I go through installing win2k, and running ie5, I can see it has 56 bit encryption. But after installing service pack 3 and service pack 4, IE says '0-bit encryption'. I think this means there is a DLL or something corrupted somewhere, but I have no idea how to find out where. I am wondering if delaying the interrupt every time instead of every 16th time really is needed.. Is there any good way to detect the behavior that needs to have the delay in qemu and print out a warning on the console? I have also found that haveing '-usbdevice tablet' enabled during the install sometimes results in the mouse stopping working halfway through the install, although it's never consistent when it stops. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] Windows 2000 issues questions
On Mon, Apr 10, 2006 at 08:16:17PM +0400, Brad Campbell wrote: Hetz Ben Hamo wrote: I'm sorry to bring this issues back from the dead: * Full disk issues * Service pack issues I Do know that both these issues have been dealt before, but yet, there is no fix from the QEMU application itself, compared to the competitors.. One thing that I don't understand is: are these issues related to DMA implementation in QEMU or rather to a specific chipset implementation? Just to back up what Leo said.. I have his %16 -win2k-hack patch installed (which is required as you need to add the delay to DMA IO also) and if I leave -win2k-hack enabled I find very little performance degradation.. in addition to it enabling windows update to work. I'm sure it's either slowing down the VM or wasting cpu cycles, but it does work and does not knock things about noticeably (and I test on a PIII-M 1Ghz running at 750Mhz to give kinda worst case performance testing) I do recall not having these issues with an async-io patch installed, but the last async-io patch I used changed the virtual ide controller and made all my vm's useless. I'm somewhat reticent to install another patch that is going to require me to re-install all my vm's.. but if there is one floating about that does not change the virtual hardware and is pretty self contained I'd be more than happy to run it through my test suite. I'd be happy to test a patch that changes the virtual IDE controller to one that works better, since I can't seem to get a win2k vm installed and updated at all anyway ;) ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] Patch for kqemu-1.3.0pre5 on Fedora Core 4 x86_64
On Wed, Apr 26, 2006 at 12:50:16AM +0200, Fabrice Bellard wrote: Well, there is a change log in the archive and here it is: version 1.3.0pre6: - compile fix for Linux kernel version = 2.6.16 - better null LDT handling (aka Plan9 and ReactOS bug) - moved monitor code to another address (aka win2k 256 MB bug) FYI, the windows 2000 install blows up right away with -kernel-kqemu on an x86_64 host with qemu-system-x86_64 . an install with kqemu-user seems to be going okay so far ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] gnu-c99-math.h file
On Tue, Apr 25, 2006 at 09:04:33PM -0500, Chris Bagwell wrote: I just upgraded to current CVS to get all the changes submitted today. The solaris port commits seems to have added a #include gnu-c99-math.h file to fpu/softfloat-native.h. This file doesn't exist on my Fedora Core 5 system with gcc32 installed for compatibility. The mailing archive mentions something about this missing file being a custom file for the solaris port. Should that file only be referenced under solaris? Qemu appears to compile fine on linux if I just comment that line out. The same issue occurs on Debian as well. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] VNC terminal server?
On Sat, Apr 08, 2006 at 10:12:07PM +0400, Brad Campbell wrote: Samuel Hunt wrote: It occurs to me that this program would make an excellent basis for a VNC terminal server. [...] If you use the vnc patch you kinda get a large part of this already. Major issue is still mouse synch, but to be honest if you turn of all acceleration in the guest it stays pretty well synced up now as it is. I use it all the time on my server to host a win2k session when I need to access windows only stuff.. Coupled with kqemu it makes for a pretty quick combination. [...] Currently I run gentoo, freebsd-6 and win2k sessions on my server.. they just sit there idle until I connect to them with vnc.. works a treat. (server is debian) Have you tried the vnc patch with current CVS? I'm seeing some issues with -vnc-and-sdl, and with -vnc only, it looks like something is not getting initialized, and I only see the qemu console in the vnc window. It appears the guest is running, but no video is going to VNC. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
[Qemu-devel] current CVS + kqemu + win2kpro problems?
I am running the current CVS (with the vnc and ide-dma win2k-hack patches applied), and kqemu 1.3.0-pre5 on a debian-sarge (3.1) release with kernel 2.6.12-1-k7. I have a (very old) original win2k pro install iso, and after I get service pack 3 and 4 installed, I've gotten into a situation where I can download the win2k service pack rollup 1, and then the guest OS/machine will spontaneously reboot. How do I go about sending in a usefull bug report for this? Has anyone done a win2k install on the current cvs? -- -- Troy Benjegerdes'da hozer'[EMAIL PROTECTED] Somone asked me why I work on this free (http://www.fsf.org/philosophy/) software stuff and not get a real job. Charles Shultz had the best answer: Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life. -- Charles Shultz ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
[Qemu-devel] USB Tablet Emulation + VNC patch..
On Mon, Apr 10, 2006 at 12:23:05PM +0400, Brad Campbell wrote: Anthony Liguori wrote: I spent some time cleaning this all up. The following integrates Brad's patches and the patch from http://gnome.dnsalias.net/patches/qemu-hidmousexp.patch It adds a new emulated USB device that reports absolute coordinates. It also modifies SDL to operate in grabless mode when an absolute input device is enabled. I think it's pretty close to apply-able. To use, just specify: -usbdevice tablet With Xorg from CVS, the evdev driver segfaults. This is apparently expected behavior. Hopefully it will be fixed soon. It works quite nicely under Win2k. Here is the vnc patch to go on top of that and the latest cvs. This works beautifully so far.. I applied the VNC patch to the latest qemu CVS (which has the usb tablet emulation already). Now, does anyone have instructions on how to get Win2k installed and updated to the latest set of security patches? I can get service pack 4 installed, but running windowsupdate seems to never work right. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] -kernel-kqemu
On Thu, Feb 09, 2006 at 04:01:34PM -0600, Anthony Liguori wrote: Jim C. Brown wrote: -kernel-kqemu virtualizes ring 0 code. So it basically makes qemu do what VMware does. IIRC someone reported a 33% speedup with the new option. That was me. That was a 33% speedup on win2k startup time. kqemu (user only) has a negligible impact on win2k startup time which suggests this is mostly ring 0 code running which would make it a good benchmark for kernel-kqemu performance. This was a terribly unscientific benchmarking so don't read too much into it. Regards, Anthony Liguori My win2k guest (with SP4, but not any updates) seemed to hang on startup with -kernel-kqemu. (This is today's qemu-cvs with kqemu-1.3.0pre5) ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [PATCH] -win2k-hack performance+DMA support
On Thu, Feb 09, 2006 at 06:03:53PM +0400, Brad Campbell wrote: Leonardo E. Reiter wrote: Attached is a patch which greatly speeds up disk writes when using -win2k-hack to install Windows 2000. It only delays every 16th interrupt, which after rigorous testing is still enough to overcome the Windows 2000 disk full bug. If you are really adventurous you can try it every 32nd time, but for me that caused problems during some installations, so I thought it was too aggressive. This patch also adds -win2k-hack functionality to DMA writes, since DMA now works in the latest CVS QEMU. Using todays CVS, and kqemu in user mode only I tried to install Win2k-SP4 25 times all with disk full errors until I applied this patch. It would appear the installer will use DMA mode if it's available and therefore the existing win2k-hack does not work. Using this patch 5 out of 5 installs worked perfectly. Thanks Leo :) Regards, Brad with current CVS, I am unable to install win2k.. This patch appears to be needed for win2k installs to work. With this patch, I am able to install w2k with user-kqemu. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] build problems with linux usb support?
On Sat, Mar 11, 2006 at 02:57:03PM -0500, Daniel Jacobowitz wrote: On Wed, Mar 08, 2006 at 10:55:21AM -0600, Troy Benjegerdes wrote: The only think I can track down so far is that BITS_PER_LONG is only defined in /usr/include/asm-x86_64/types.h if __KERNEL__ is also set. What other linux distros/headers work with the current qemu cvs usb code? I've just been disabling USB support by a local patch and hoping someone else fixed this :-) Would you mind posting the 'disable usb' patch? Right now I am resorting to gross things like building on a redhat box and installing the binaries on debian. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] make '-smb $HOME' work
Which smbd are you using? The one on debian sarge wants to have write access to some /var/run and /var/lib directories to coordinate locking. Because it gets run as a regular user, (and is not suid root), it winds up spitting out an error to the logfile and dying. It took me a while to figure this out, since there's no error message display by qemu. On Sat, Oct 08, 2005 at 03:30:36PM -0400, John Coiner wrote: The most common use case for the '-smb' option may be '-smb $HOME'. There is a problem with this case: Windows attempts to connect as user nobody. Smbd allows the connection -- unfortunately, it also maps the nobody accesses to the host's nobody account, so all write accesses fail. How are people using '-smb'? Am I the only person that runs into this? One lame workaround is to point '-smb' at an area on /tmp that everybody, including nobody, has access to. The problem happens with a Windows 2000 guest, and maybe other NT derivatives. This patch sets up smbd to only allow guest access from Windows, and no other access. (I suspect and hope that smbd can coax any version of Windows into doing a guest access, by rejecting everything else. This is only tested with Win2K.) When smbd receives a guest access, it maps that onto the account of the same user who is running qemu. This fixes the common, personal use, '-smb $HOME' case. For more complicated cases, for example if you don't trust the guest, you may want to craft your own 'smb.conf' rather than relying on '-smb'. From a security standpoint, the patched '-smb' has no authentication to break, and it constrains smb access to a single user on the host. So while the gates are wide open to whatever directory you share, you at least know what you're getting. -- John --- qemu-0.7.2-dmapatch/vl.c2005-09-04 13:11:31.0 -0400 +++ qemu-0.7.2-broken/vl.c 2005-10-08 14:41:55.0 -0400 @@ -29,6 +29,8 @@ #include time.h #include errno.h #include sys/time.h +#include sys/types.h +#include pwd.h #ifndef _WIN32 #include sys/times.h @@ -1605,15 +1607,17 @@ log file=%s/log.smbd\n smb passwd file=%s/smbpasswd\n security = share\n + guest account=%s\n [qemu]\n path=%s\n read only=no\n -guest ok=yes\n, +guest only=yes\n, smb_dir, smb_dir, smb_dir, smb_dir, smb_dir, + getpwuid( geteuid( ) )-pw_name, exported_dir ); fclose(f); ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel -- -- Troy Benjegerdes'da hozer'[EMAIL PROTECTED] Somone asked me why I work on this free (http://www.fsf.org/philosophy/) software stuff and not get a real job. Charles Shultz had the best answer: Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life. -- Charles Shultz ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] VNC patch status?
On Sat, Oct 08, 2005 at 08:29:27PM +0200, Johannes Schindelin wrote: Hi, On Sat, 8 Oct 2005, Iain McFarlane wrote: VNC support has now been added to cvs Am I missing something? I don't see it. Furthermore, the main problem still prevails: There is no sane way to get the mouse running yet. I played around with a tablet patch someone sent me, but could not get it to run with a Win98 guest. Since this is my main use for qemu+vnc, I need a solution for this case. I gave up and installed VNC in the guest OS, and then used '-redir'. But GUI/mouse responsiveness seemed pretty slow that way. I think the best solution would be for someone to write a special qemu mouse driver for windows. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
On Mon, Oct 03, 2005 at 11:16:18PM -0400, John Coiner wrote: Troy Benjegerdes wrote: I am also haveing trouble getting a fresh win2k install under qemu to actually be able to run windows update. I had to download and install Win2k SP4, then Win2k SP4 Hotfixes, and also an IE6 upgrade, before windows update ran. Are you running with a tun/tap device, or -user-net ? I'm still unable to get windowsupdate to work. Getting it to work in vmware seems to work on the first try. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
On Mon, Oct 03, 2005 at 08:41:46AM -0400, John Coiner wrote: Magnus, I don't think the Windows 2000 install hack will ever be obsolete. The installer assumes that a hard disk will take nonzero time to read some data. QEMU always services a read in zero-guest-time. (With the nonblocking IO patch, zero-guest-time reads still occur, when the requested data is in the host's file cache.) I doubt the IDE spec allows Windows to make this assumption... but the assumption is there, and we work around it by adding a delay that's visible to the guest. There are other bugs in the IDE emulation.. I need to try this latest patch, but with the previous DMA patch (without non-blocking support) having a disk image on an AFS filesystem just did not work at all. I am also haveing trouble getting a fresh win2k install under qemu to actually be able to run windows update. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] non-blocking disk IO
If you anyway plan on having Posix AIO support then go for the Posix AIO interface. The performance reasons why Linux AIO exists is very unlikely to be an issue to qemu as you need to be quite I/O intensive to see any performance difference. Ideally, we should be able to use a Posix AIO support libary that uses native Linux AIO underneath. What we want is to be able to have the guest OS request some DMA I/O operation, and have qemu be able to use AIO so that the actual disk hardware can dump the data directly in the pages the userspace process on the guest OS ends up wanting it in, avoiding several expensive memcopy and context switch operations. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] [patch] IDE busmaster DMA support
On Mon, Sep 26, 2005 at 09:48:25PM -0400, John Coiner wrote: Hi, Here's another patch to enable IDE DMA: http://people.brandeis.edu/~jcoiner/qemu_idedma/qemu_dma_patch.html This patch, like Juergen Lock's earlier patch from which it is derived, replaces the virtual PIIX3 IDE controller with a PIIX4. It applies cleanly to 0.7.2. This patch extends the Windows 2000 install hack to work for DMA also -- I installed and ran Windows 2000 with DMA enabled. It'd be cool if QEMU supported nonblocking disk access -- if it would let the guest resume running currently, while waiting for the disk to finish seeking. This patch is a first step toward concurrent IO, which would probably have to use busmastering DMA transfers to be efficient, if it's going to be efficient at all. Do people have ideas about how to implement concurrent IO? I have a couple hazy ideas... Can anyone shed light on the potential performance to be realized? For example, how long does a context switch take in the guest? If it takes 10ms (one disk seek) then there's probably no benefit to concurrent disk IO. If a context switch takes 1ms, then the guest has opportunity to make forward progress while awaiting an IDE interrupt. I would strongly suggest attempting to make use of the Linux Async I/O support.. this way the data from disk could be placed directly in address space of the guest OS. Personally, I'd love to see nonblocking disk IO. I've tried running Qemu using a disk image on an AFS filesystem, and having the guest be able to continue while the filesystem is fetching data from the network would make things much nicer. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
Re: [Qemu-devel] qemu and AFS filesystem bugs?
I have had similiar problems with and without kqemu. Initially, I thought kqemu was causing the problem. What exactly does the win2k install hack do anyway? Does it change the disk emulation somehow? On Thu, Sep 15, 2005 at 08:00:56AM +0200, Hetz Ben Hamo wrote: Do u run with or without KQEMU? if you're running without kqemu, please retry with kqemu. As for Windows 2000 guest installation, please use the win2000 hack parameter (running the command qemu alone should show you the parameter) Thanks, Hetz On 9/15/05, Troy Benjegerdes [EMAIL PROTECTED] wrote: I'm running qemu 0.7.2 on a Fedora Core 4 system (Opteron system), with OpenAFS 1.4.0-rc3. When I try to install either Windows 2000 or Red Hat WS, I am finding that if the guest install ISO image is on an AFS filesystem, I get strange errors during the install. On Windows, I would get 'file corrupted' when copying files in the initial install. Once the ISO image is in the AFS cache, it seems to work better. How would I go about debugging something like this? For what it's worth, other people have run VMware with ISO and disk images on AFS with no problems. ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
[Qemu-devel] qemu and AFS filesystem bugs?
I'm running qemu 0.7.2 on a Fedora Core 4 system (Opteron system), with OpenAFS 1.4.0-rc3. When I try to install either Windows 2000 or Red Hat WS, I am finding that if the guest install ISO image is on an AFS filesystem, I get strange errors during the install. On Windows, I would get 'file corrupted' when copying files in the initial install. Once the ISO image is in the AFS cache, it seems to work better. How would I go about debugging something like this? For what it's worth, other people have run VMware with ISO and disk images on AFS with no problems. -- -- Troy Benjegerdes'da hozer'[EMAIL PROTECTED] Somone asked me why I work on this free (http://www.fsf.org/philosophy/) software stuff and not get a real job. Charles Shultz had the best answer: Why do musicians compose symphonies and poets write poems? They do it because life wouldn't have any meaning for them if they didn't. That's why I draw cartoons. It's my life. -- Charles Shultz ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
