Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-12-04 Thread Stefan Hajnoczi
On Fri, Dec 01, 2017 at 07:03:15PM +, Peter Maydell wrote: > On 1 December 2017 at 17:03, Paolo Bonzini wrote: > > Same here. Just like fixing the C code provides a good foundation for a > > language switch, some more battle-tested code could be converted from > > QEMU to Rust, in order to ge

Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-12-04 Thread Peter Maydell
On 4 December 2017 at 10:16, Stefan Hajnoczi wrote: > Yes, please! __attribute__((cleanup)) is great and we should use it in > QEMU. > > I haven't looked at compiler support though... :) Looks like it came in in gcc 3.6, which is earlier than our earliest required version. For clang, it's harder

Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-12-04 Thread Stefan Hajnoczi
On Fri, Dec 01, 2017 at 01:27:37PM -0600, Eric Blake wrote: > On 12/01/2017 11:03 AM, Paolo Bonzini wrote: > > Just my 2 cents on the language topic, as in general I agree completely > > with Stefan. > > > > > Luckily, several benefits don't require a full rewrite or language switch: > > > > - r

Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-12-01 Thread Eric Blake
On 12/01/2017 11:03 AM, Paolo Bonzini wrote: Just my 2 cents on the language topic, as in general I agree completely with Stefan. Luckily, several benefits don't require a full rewrite or language switch: - readability from RAII-style code. If this is important enough, we could actually use

Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-12-01 Thread Peter Maydell
On 1 December 2017 at 17:03, Paolo Bonzini wrote: > Same here. Just like fixing the C code provides a good foundation for a > language switch, some more battle-tested code could be converted from > QEMU to Rust, in order to get familiar with it and probe whether the > benefits are real. Maybe th

Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-12-01 Thread Paolo Bonzini
Just my 2 cents on the language topic, as in general I agree completely with Stefan. On 01/12/2017 15:08, Stefan Hajnoczi wrote: > Rust's threading model is 1:1. Besides mutexes it also has channels > (looks similar to Go and communicating sequential processes-style > channels). > > It is probab

Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-12-01 Thread Stefan Hajnoczi
On Fri, Dec 01, 2017 at 06:16:44PM +0800, Fam Zheng wrote: > On Thu, 11/30 14:19, Stefan Hajnoczi wrote: > > On Thu, Nov 30, 2017 at 05:47:09PM +0800, Fam Zheng wrote: > > > On Wed, 11/29 12:00, Stefan Hajnoczi wrote: > > > > On Wed, Nov 29, 2017 at 11:55:02AM +0800, Fam Zheng wrote: > > > > > > >

Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-12-01 Thread Fam Zheng
On Thu, 11/30 14:19, Stefan Hajnoczi wrote: > On Thu, Nov 30, 2017 at 05:47:09PM +0800, Fam Zheng wrote: > > On Wed, 11/29 12:00, Stefan Hajnoczi wrote: > > > On Wed, Nov 29, 2017 at 11:55:02AM +0800, Fam Zheng wrote: > > > > > > Event loops and coroutines are good but they should not be used dire

Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-12-01 Thread Fam Zheng
On Fri, 12/01 14:08, Stefan Hajnoczi wrote: > What I'm getting at is that the essential complexity of a parallel I/O > engine with a runtime reconfigurable graph, background operations, and > non-trivial disk image file formats sets a certain floor (minimum) > complexity. The consequence is that o

Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-11-30 Thread Stefan Hajnoczi
On Thu, Nov 30, 2017 at 05:47:09PM +0800, Fam Zheng wrote: > On Wed, 11/29 12:00, Stefan Hajnoczi wrote: > > On Wed, Nov 29, 2017 at 11:55:02AM +0800, Fam Zheng wrote: > > > > Event loops and coroutines are good but they should not be used directly > > by block drivers and block jobs. We need saf

Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-11-30 Thread Fam Zheng
On Wed, 11/29 12:00, Stefan Hajnoczi wrote: > On Wed, Nov 29, 2017 at 11:55:02AM +0800, Fam Zheng wrote: > > As we move forwards with new features in the block layer, the chances of > > tricky > > bugs happening have been increasing alongside - block jobs, coroutines, > > throttling, AioContext, o

Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-11-29 Thread Dr. David Alan Gilbert
* Paolo Bonzini (pbonz...@redhat.com) wrote: > On 29/11/2017 13:00, Stefan Hajnoczi wrote: > > We are at a point where code review isn't finding certain bugs because > > no single person knows all the assumptions. Previously the problem was > > contained because maintainers spotted problems before

Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-11-29 Thread Kevin Wolf
Am 29.11.2017 um 13:24 hat Paolo Bonzini geschrieben: > On 29/11/2017 13:00, Stefan Hajnoczi wrote: > > We are at a point where code review isn't finding certain bugs because > > no single person knows all the assumptions. Previously the problem was > > contained because maintainers spotted proble

Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-11-29 Thread Daniel P. Berrange
On Wed, Nov 29, 2017 at 11:55:02AM +0800, Fam Zheng wrote: > So in principle, what should we do to make the block layer easy to understand, > develop with and debug? I think we have opportunities in these aspects: > > - Documentation > > There is no central developer doc about block layer, espe

Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-11-29 Thread Paolo Bonzini
On 29/11/2017 13:00, Stefan Hajnoczi wrote: > We are at a point where code review isn't finding certain bugs because > no single person knows all the assumptions. Previously the problem was > contained because maintainers spotted problems before patches were > merged. > > This is not primarily a

Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-11-29 Thread Paolo Bonzini
On 29/11/2017 13:16, Stefan Hajnoczi wrote: > While the atomics documentation is good, atomics themselves have been a > source of difficult bugs. > > They should be used as little as possible and only where they can be > encapsulated in a composable abstraction (i.e. don't expect users of > your a

Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-11-29 Thread Stefan Hajnoczi
On Wed, Nov 29, 2017 at 01:30:06AM -0500, Jeff Cody wrote: > On Wed, Nov 29, 2017 at 11:55:02AM +0800, Fam Zheng wrote: > > Hi all, > > > > As we move forwards with new features in the block layer, the chances of > > tricky > > bugs happening have been increasing alongside - block jobs, coroutine

Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-11-29 Thread Stefan Hajnoczi
On Wed, Nov 29, 2017 at 11:55:02AM +0800, Fam Zheng wrote: > As we move forwards with new features in the block layer, the chances of > tricky > bugs happening have been increasing alongside - block jobs, coroutines, > throttling, AioContext, op blockers and image locking combined together make a

Re: [Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-11-28 Thread Jeff Cody
On Wed, Nov 29, 2017 at 11:55:02AM +0800, Fam Zheng wrote: > Hi all, > > As we move forwards with new features in the block layer, the chances of > tricky > bugs happening have been increasing alongside - block jobs, coroutines, > throttling, AioContext, op blockers and image locking combined tog

[Qemu-devel] Block layer complexity: what to do to keep it under control?

2017-11-28 Thread Fam Zheng
Hi all, As we move forwards with new features in the block layer, the chances of tricky bugs happening have been increasing alongside - block jobs, coroutines, throttling, AioContext, op blockers and image locking combined together make a large and complex picture that is hard to fully understand