Re: [Qemu-devel] [PATCH v9 0/8] Optimize VMDK I/O by allocating multiple clusters

On Fri, Oct 20, 2017 at 11:58 Fam Zheng <f...@redhat.com> wrote: > On Mon, 10/09 22:12, Fam Zheng wrote: > > On Mon, 10/09 18:29, Ashijeet Acharya wrote: > > > Optimization test results: > > > > > > This patch series improves 128 KB sequential write per

[Qemu-devel] [PATCH v9 8/8] vmdk: Make vmdk_get_cluster_offset() return cluster offset only

-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 56 1 file changed, 12 insertions(+), 44 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index cbeffb1552..497e3

[Qemu-devel] [PATCH v9 6/8] vmdk: New functions to assist allocating multiple clusters

the offset of the first of the many newly allocated clusters. Also, provide proper documentation for both. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 201 +++

[Qemu-devel] [PATCH v9 7/8] vmdk: Update metadata for multiple clusters

Include a next pointer in VmdkMetaData struct to point to the previous allocated L2 table. Modify vmdk_L2update to start updating metadata for allocation of multiple clusters at once. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com>

[Qemu-devel] [PATCH v9 4/8] vmdk: Factor out metadata loading code out of vmdk_get_cluster_offset()

Move the cluster tables loading code out of the existing vmdk_get_cluster_offset() function and implement it in separate get_cluster_table() and vmdk_l2load() functions. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/

[Qemu-devel] [PATCH v9 5/8] vmdk: Set maximum bytes allocated in one cycle

Set the maximum bytes allowed to get allocated at once to be not more than the extent size boundary to handle writes at two separate extents appropriately. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 13

[Qemu-devel] [PATCH v9 2/8] vmdk: Rename get_whole_cluster() to vmdk_perform_cow()

Rename the existing function get_whole_cluster() to vmdk_perform_cow() as its sole purpose is to perform COW for the first and the last allocated clusters if needed. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block

[Qemu-devel] [PATCH v9 3/8] vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset()

Rename the existing get_cluster_offset() to vmdk_get_cluster_offset() and update name in all the callers accordingly. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 46 +++---

[Qemu-devel] [PATCH v9 1/8] vmdk: Move vmdk_find_offset_in_cluster() to the top

Move the existing vmdk_find_offset_in_cluster() function to the top of the driver. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 24 1 file changed, 12 insertions(+), 12 deletions(-)

[Qemu-devel] [PATCH v9 0/8] Optimize VMDK I/O by allocating multiple clusters

hmark test results in v2 Ashijeet Acharya (8): vmdk: Move vmdk_find_offset_in_cluster() to the top vmdk: Rename get_whole_cluster() to vmdk_perform_cow() vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset() vmdk: Factor out metadata loading code out of vmdk_get_cluster_offse

Re: [Qemu-devel] [PATCH v8 0/8] Optimize VMDK I/O by allocating multiple clusters

On Wed, Oct 4, 2017 at 1:58 PM, Fam Zheng <f...@redhat.com> wrote: > On Wed, 10/04 13:47, Ashijeet Acharya wrote: > > Fam: Ping? > > Hi Ashijeet, looks like this patch doesn't apply to current master, could > you > rebase and post another version? > Hello

Re: [Qemu-devel] [PATCH v8 0/8] Optimize VMDK I/O by allocating multiple clusters

On Thu, Aug 10, 2017 at 11:13 PM, Stefan Hajnoczi <stefa...@gmail.com> wrote: > On Thu, Aug 10, 2017 at 9:18 AM, Ashijeet Acharya > <ashijeetacha...@gmail.com> wrote: > > On Thu, Aug 10, 2017 at 1:41 PM, Stefan Hajnoczi <stefa...@gmail.com> > wrote: > >

Re: [Qemu-devel] [PATCH v2 0/7] Refactor DMG driver to have chunk size independence

On Tue, Sep 5, 2017 at 4:28 PM, Stefan Hajnoczi <stefa...@gmail.com> wrote: > On Wed, Aug 30, 2017 at 06:32:52PM +0530, Ashijeet Acharya wrote: > > On Tue, Aug 29, 2017 at 8:55 PM, Stefan Hajnoczi <stefa...@gmail.com> > wrote: > > > > > On Sun, Au

Re: [Qemu-devel] [PATCH v2 0/7] Refactor DMG driver to have chunk size independence

On Tue, Aug 29, 2017 at 8:55 PM, Stefan Hajnoczi <stefa...@gmail.com> wrote: > On Sun, Aug 20, 2017 at 1:47 PM, Ashijeet Acharya > <ashijeetacha...@gmail.com> wrote: > > On Fri, May 5, 2017 at 7:29 PM, Stefan Hajnoczi <stefa...@gmail.com> > wrote: > >&g

Re: [Qemu-devel] [PATCH v2 0/7] Refactor DMG driver to have chunk size independence

On Fri, May 5, 2017 at 7:29 PM, Stefan Hajnoczi <stefa...@gmail.com> wrote: > On Thu, Apr 27, 2017 at 01:36:30PM +0530, Ashijeet Acharya wrote: > > This series helps to provide chunk size independence for DMG driver to > prevent > > denial-of-service in cases where u

Re: [Qemu-devel] [PATCH v8 0/8] Optimize VMDK I/O by allocating multiple clusters

On Thu, Aug 10, 2017 at 1:41 PM, Stefan Hajnoczi <stefa...@gmail.com> wrote: > On Thu, Jul 27, 2017 at 3:33 PM, Ashijeet Acharya > <ashijeetacha...@gmail.com> wrote: > > Previously posted series patches: > > v1 - http://lists.nongnu.org/archive/html/qemu-devel/2

[Qemu-devel] [PATCH v8 5/8] vmdk: Set maximum bytes allocated in one cycle

Set the maximum bytes allowed to get allocated at once to be not more than the extent size boundary to handle writes at two separate extents appropriately. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 13

[Qemu-devel] [PATCH v8 6/8] vmdk: New functions to assist allocating multiple clusters

the offset of the first of the many newly allocated clusters. Also, provide proper documentation for both. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 201 +++

[Qemu-devel] [PATCH v8 7/8] vmdk: Update metadata for multiple clusters

Include a next pointer in VmdkMetaData struct to point to the previous allocated L2 table. Modify vmdk_L2update to start updating metadata for allocation of multiple clusters at once. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com>

[Qemu-devel] [PATCH v8 0/8] Optimize VMDK I/O by allocating multiple clusters

lable (fam) - call bdrv_pwrite_sync() for batches of atmost 512 clusters at once (fam) Changes in v2: - segregate the ugly Patch 1 in v1 into 6 readable and sensible patches - include benchmark test results in v2 Ashijeet Acharya (8): vmdk: Move vmdk_find_offset_in_cluster() to the top vm

[Qemu-devel] [PATCH v8 8/8] vmdk: Make vmdk_get_cluster_offset() return cluster offset only

-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 56 1 file changed, 12 insertions(+), 44 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index 4a59ca4..a84b26c 100644

[Qemu-devel] [PATCH v8 1/8] vmdk: Move vmdk_find_offset_in_cluster() to the top

Move the existing vmdk_find_offset_in_cluster() function to the top of the driver. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 24 1 file changed, 12 insertions(+), 12 deletions(-)

[Qemu-devel] [PATCH v8 4/8] vmdk: Factor out metadata loading code out of vmdk_get_cluster_offset()

Move the cluster tables loading code out of the existing vmdk_get_cluster_offset() function and implement it in separate get_cluster_table() and vmdk_l2load() functions. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/

[Qemu-devel] [PATCH v8 3/8] vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset()

Rename the existing get_cluster_offset() to vmdk_get_cluster_offset() and update name in all the callers accordingly. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 46 +++---

[Qemu-devel] [PATCH v8 2/8] vmdk: Rename get_whole_cluster() to vmdk_perform_cow()

Rename the existing function get_whole_cluster() to vmdk_perform_cow() as its sole purpose is to perform COW for the first and the last allocated clusters if needed. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block

[Qemu-devel] [PATCH v7 8/8] vmdk: Make vmdk_get_cluster_offset() return cluster offset only

-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 56 1 file changed, 12 insertions(+), 44 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index 60b8adc..d41fde9 100644

[Qemu-devel] [PATCH v7 5/8] vmdk: Set maximum bytes allocated in one cycle

Set the maximum bytes allowed to get allocated at once to be not more than the extent size boundary to handle writes at two separate extents appropriately. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 13

[Qemu-devel] [PATCH v7 4/8] vmdk: Factor out metadata loading code out of vmdk_get_cluster_offset()

Move the cluster tables loading code out of the existing vmdk_get_cluster_offset() function and implement it in separate get_cluster_table() and vmdk_l2load() functions. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/

[Qemu-devel] [PATCH v7 6/8] vmdk: New functions to assist allocating multiple clusters

the offset of the first of the many newly allocated clusters. Also, provide proper documentation for both. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/vmdk.c | 200 --- 1 file changed, 190 insertions(+), 10 del

[Qemu-devel] [PATCH v7 3/8] vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset()

Rename the existing get_cluster_offset() to vmdk_get_cluster_offset() and update name in all the callers accordingly. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 46 +++---

[Qemu-devel] [PATCH v7 2/8] vmdk: Rename get_whole_cluster() to vmdk_perform_cow()

Rename the existing function get_whole_cluster() to vmdk_perform_cow() as its sole purpose is to perform COW for the first and the last allocated clusters if needed. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block

[Qemu-devel] [PATCH v7 7/8] vmdk: Update metadata for multiple clusters

Include a next pointer in VmdkMetaData struct to point to the previous allocated L2 table. Modify vmdk_L2update to start updating metadata for allocation of multiple clusters at once. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/vmdk.c

[Qemu-devel] [PATCH v7 0/8] Optimize VMDK I/O by allocating multiple clusters

v1 into 6 readable and sensible patches - include benchmark test results in v2 Ashijeet Acharya (8): vmdk: Move vmdk_find_offset_in_cluster() to the top vmdk: Rename get_whole_cluster() to vmdk_perform_cow() vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset() vmdk: Factor ou

[Qemu-devel] [PATCH v7 1/8] vmdk: Move vmdk_find_offset_in_cluster() to the top

Move the existing vmdk_find_offset_in_cluster() function to the top of the driver. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 24 1 file changed, 12 insertions(+), 12 deletions(-)

Re: [Qemu-devel] [PATCH v6 7/8] vmdk: Update metadata for multiple clusters

On Tue, Jun 27, 2017 at 1:34 PM, Fam Zheng <f...@redhat.com> wrote: > On Mon, 06/05 13:22, Ashijeet Acharya wrote: >> @@ -1876,6 +1942,13 @@ static int vmdk_pwritev(BlockDriverState *bs, >> uint64_t offset, >> offset += n_bytes; >> bytes_done

Re: [Qemu-devel] [PATCH v6 6/8] vmdk: New functions to assist allocating multiple clusters

On Tue, Jun 27, 2017 at 1:32 PM, Fam Zheng <f...@redhat.com> wrote: > On Mon, 06/05 13:22, Ashijeet Acharya wrote: >> +/* >> + * vmdk_handle_alloc >> + * >> + * Allocate new clusters for an area that either is yet unallocated or >> needs a >> + *

[Qemu-devel] [PATCH v6 8/8] vmdk: Make vmdk_get_cluster_offset() return cluster offset only

-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/vmdk.c | 56 1 file changed, 12 insertions(+), 44 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index 9fa2414..accf1c3 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -1

[Qemu-devel] [PATCH v6 4/8] vmdk: Factor out metadata loading code out of vmdk_get_cluster_offset()

Move the cluster tables loading code out of the existing vmdk_get_cluster_offset() function and implement it in separate get_cluster_table() and vmdk_l2load() functions. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/

[Qemu-devel] [PATCH v6 6/8] vmdk: New functions to assist allocating multiple clusters

the offset of the first of the many newly allocated clusters. Also, provide proper documentation for both. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/vmdk.c | 192 +++ 1 file changed, 182 insertions(+), 10 del

[Qemu-devel] [PATCH v6 7/8] vmdk: Update metadata for multiple clusters

Include a next pointer in VmdkMetaData struct to point to the previous allocated L2 table. Modify vmdk_L2update to start updating metadata for allocation of multiple clusters at once. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/vmdk.c

[Qemu-devel] [PATCH v6 3/8] vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset()

Rename the existing get_cluster_offset() to vmdk_get_cluster_offset() and update name in all the callers accordingly. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 46 +++---

[Qemu-devel] [PATCH v6 5/8] vmdk: Set maximum bytes allocated in one cycle

Set the maximum bytes allowed to get allocated at once to be not more than the extent size boundary to handle writes at two separate extents appropriately. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 13

[Qemu-devel] [PATCH v6 1/8] vmdk: Move vmdk_find_offset_in_cluster() to the top

Move the existing vmdk_find_offset_in_cluster() function to the top of the driver. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 24 1 file changed, 12 insertions(+), 12 deletions(-)

[Qemu-devel] [PATCH v6 0/8] Optimize VMDK I/O by allocating multiple clusters

1 in v1 into 6 readable and sensible patches - include benchmark test results in v2 Ashijeet Acharya (8): vmdk: Move vmdk_find_offset_in_cluster() to the top vmdk: Rename get_whole_cluster() to vmdk_perform_cow() vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset() vmdk: Factor out m

[Qemu-devel] [PATCH v6 2/8] vmdk: Rename get_whole_cluster() to vmdk_perform_cow()

Rename the existing function get_whole_cluster() to vmdk_perform_cow() as its sole purpose is to perform COW for the first and the last allocated clusters if needed. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block

Re: [Qemu-devel] [PATCH v5 6/8] vmdk: New functions to assist allocating multiple clusters

On Mon, Jun 5, 2017 at 11:23 AM, Ashijeet Acharya <ashijeetacha...@gmail.com> wrote: > Introduce two new helper functions handle_alloc() and > vmdk_alloc_cluster_offset(). handle_alloc() helps to allocate multiple > clusters at once starting from a given offset on disk a

[Qemu-devel] [PATCH v5 8/8] vmdk: Make vmdk_get_cluster_offset() return cluster offset only

-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/vmdk.c | 56 1 file changed, 12 insertions(+), 44 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index 9fa2414..accf1c3 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -1

[Qemu-devel] [PATCH v5 4/8] vmdk: Factor out metadata loading code out of vmdk_get_cluster_offset()

Move the cluster tables loading code out of the existing vmdk_get_cluster_offset() function and implement it in separate get_cluster_table() and vmdk_l2load() functions. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/

[Qemu-devel] [PATCH v5 5/8] vmdk: Set maximum bytes allocated in one cycle

Set the maximum bytes allowed to get allocated at once to be not more than the extent size boundary to handle writes at two separate extents appropriately. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 13

[Qemu-devel] [PATCH v5 7/8] vmdk: Update metadata for multiple clusters

Include a next pointer in VmdkMetaData struct to point to the previous allocated L2 table. Modify vmdk_L2update to start updating metadata for allocation of multiple clusters at once. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/vmdk.c

[Qemu-devel] [PATCH v5 3/8] vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset()

Rename the existing get_cluster_offset() to vmdk_get_cluster_offset() and update name in all the callers accordingly. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 46 +++---

[Qemu-devel] [PATCH v5 1/8] vmdk: Move vmdk_find_offset_in_cluster() to the top

Move the existing vmdk_find_offset_in_cluster() function to the top of the driver. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 24 1 file changed, 12 insertions(+), 12 deletions(-)

[Qemu-devel] [PATCH v5 6/8] vmdk: New functions to assist allocating multiple clusters

the offset of the first of the many newly allocated clusters. Also, provide proper documentation for both. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/vmdk.c | 192 +++ 1 file changed, 182 insertions(+), 10 del

[Qemu-devel] [PATCH v5 0/8] Optimize VMDK I/O by allocating multiple clusters

ke them compilable (fam) - call bdrv_pwrite_sync() for batches of atmost 512 clusters at once (fam) Changes in v2: - segregate the ugly Patch 1 in v1 into 6 readable and sensible patches - include benchmark test results in v2 Ashijeet Acharya (8): vmdk: Move vmdk_find_offset_in_cluster() to

[Qemu-devel] [PATCH v5 2/8] vmdk: Rename get_whole_cluster() to vmdk_perform_cow()

Rename the existing function get_whole_cluster() to vmdk_perform_cow() as its sole purpose is to perform COW for the first and the last allocated clusters if needed. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block

Re: [Qemu-devel] [PATCH v4 6/8] vmdk: New functions to assist allocating multiple clusters

On Thu, Jun 1, 2017 at 7:27 PM, Fam Zheng <f...@redhat.com> wrote: > On Sat, 04/22 10:43, Ashijeet Acharya wrote: >> Introduce two new helper functions handle_alloc() and >> vmdk_alloc_cluster_offset(). handle_alloc() helps to allocate multiple >> clusters at once s

Re: [Qemu-devel] [PATCH v2 0/7] Refactor DMG driver to have chunk size independence

On Thu, Apr 27, 2017 at 1:36 PM, Ashijeet Acharya <ashijeetacha...@gmail.com> wrote: > Previously posted series patches: > v1: http://lists.nongnu.org/archive/html/qemu-devel/2017-04/msg04641.html > > This series helps to provide chunk size independence for DMG driver t

[Qemu-devel] [PATCH v2 5/7] dmg: Handle bz2 compressed/raw/zeroed chunks

We do not need to cache the access point for these chunks but need to update our various supporting variables like chunk, sectors_read etc. to keep maintaining our code structure. Call cache_access_point() after reading chunks of these types. Signed-off-by: Ashijeet Acharya <ashijeeta

[Qemu-devel] [PATCH v2 3/7] dmg: Refactor and prepare dmg_read_chunk() to cache random access points

ces inside dmg_read_chunk() Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/dmg.c | 159 +++- 1 file changed, 94 insertions(+), 65 deletions(-) diff --git a/block/dmg.c b/block/dmg.c index c6fe8b0..32623e2 100644 --- a/b

[Qemu-devel] [PATCH v2 2/7] dmg: New function to help us cache random access point

Introduce a new cache_access_point() function which will help us first cache a random access point inside a compressed stream and then keep updating it according to our requirement at appropriate times. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/dmg.

[Qemu-devel] [PATCH v2 7/7] dmg: Limit the output buffer size to a max of 2MB

is. Remove the error messages to prevent denial-of-service in cases where untrusted files are being accessed by the user. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/dmg.c | 24 +--- 1 file changed, 1 insertion(+), 23 deletions(-) diff --git a/blo

[Qemu-devel] [PATCH v2 1/7] dmg: Introduce a new struct to cache random access points

We need to cache the random access point while performing partial decompression so that we can resume decompression from that point onwards in our next sequential read request. Introduce a new struct DMGReadState which will help us do this. Signed-off-by: Ashijeet Acharya <ashijeeta

[Qemu-devel] [PATCH v2 6/7] dmg: Refactor dmg_co_preadv() to start reading multiple sectors

ata' variable depending on our cached access point situation to align our read requests appropriately. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/dmg.c | 23 +++ 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/block/dmg.c b/block/

[Qemu-devel] [PATCH v2 4/7] dmg: Handle zlib compressed chunks

Set the output buffer size to be equal to the size of number of sectors stored in @sectors_read. Start inflating to a max output buffer size of 2MB and cache our access point to aid random access later if required. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/dmg.

[Qemu-devel] [PATCH v2 0/7] Refactor DMG driver to have chunk size independence

s in v2: - limit the buffer size to 2MB after fixing the buffering problems (john/fam) Ashijeet Acharya (7): dmg: Introduce a new struct to cache random access points dmg: New function to help us cache random access point dmg: Refactor and prepare dmg_read_chunk() to cache random access points

Re: [Qemu-devel] [PATCH v1 3/8] dmg: Limit the output buffer size to a max of 2MB

On Thu, Apr 27, 2017 at 12:56 PM, Fam Zheng wrote: > On Wed, 04/26 17:30, John Snow wrote: >> Seems OK otherwise, but I would normally expect you to fix the buffering >> problems first, and then reduce the size of the buffer -- not the other >> way around. This version introduces

Re: [Qemu-devel] [PATCH v1 3/8] dmg: Limit the output buffer size to a max of 2MB

On Thu, Apr 27, 2017 at 3:00 AM, John Snow <js...@redhat.com> wrote: > > > On 04/25/2017 03:59 PM, Ashijeet Acharya wrote: >> The size of the output buffer is limited to a maximum of 2MB so that >> QEMU doesn't end up allocating huge amounts of memory while >&g

[Qemu-devel] [PATCH v1 7/8] dmg: Refactor dmg_co_preadv() to start reading multiple sectors

ata' variable depending on our cached access point situation to align our read requests appropriately. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/dmg.c | 23 +++ 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/block/dmg.c b/block/

[Qemu-devel] [PATCH v1 6/8] dmg: Handle bz2 compressed/raw/zeroed chunks

We do not need to cache the access point for these chunks but need to update our various supporting variables like chunk, sectors_read etc. to keep maintaining our code structure. Call cache_access_point() after reading chunks of these types. Signed-off-by: Ashijeet Acharya <ashijeeta

[Qemu-devel] [PATCH v1 8/8] dmg: Remove the error messages to allow wild images

in cases where untrusted files are being accessed by the user. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/dmg.c | 22 -- 1 file changed, 22 deletions(-) diff --git a/block/dmg.c b/block/dmg.c index b0f3c84..01ec40e 100644 --- a/block/dmg.c +++ b

[Qemu-devel] [PATCH v1 5/8] dmg: Handle zlib compressed chunks

Set the output buffer size to be equal to the size of number of sectors stored in @sectors_read. Start inflating to a max output buffer size of 2MB and cache our access point to aid random access later if required. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/dmg.

[Qemu-devel] [PATCH v1 3/8] dmg: Limit the output buffer size to a max of 2MB

DMG files is to be compatible with qemu-img convert. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/dmg.c | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/block/dmg.c b/block/dmg.c index c6fe8b0..7ae30e3 100644 --- a/block/dmg.c +

[Qemu-devel] [PATCH v1 2/8] dmg: New function to help us cache random access point

Introduce a new cache_access_point() function which will help us first cache a random access point inside a compressed stream and then keep updating it according to our requirement at appropriate times. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/dmg.

[Qemu-devel] [PATCH v1 1/8] dmg: Introduce a new struct to cache random access points

We need to cache the random access point while performing partial decompression so that we can resume decompression from that point onwards in our next sequential read request. Introduce a new struct DMGReadState which will help us do this. Signed-off-by: Ashijeet Acharya <ashijeeta

[Qemu-devel] [PATCH v1 4/8] dmg: Refactor and prepare dmg_read_chunk() to cache random access points

Refactor dmg_read_chunk() to start making use of the new DMGReadState structure and do chunk and sector related calculations based on it. Add a new argument "DMGReadState *drs" to it. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- b

[Qemu-devel] [PATCH v1 0/8] Refactor DMG driver to have chunk size independence

ccess-to-bzip2.html [3] http://linux.math.tifr.res.in/manuals/html/manual_3.html#SEC17 Special thanks to Peter Wu for helping me understand and tackle the bz2 compressed chunks. Ashijeet Acharya (8): dmg: Introduce a new struct to cache random access points dmg: New function to help us cach

Re: [Qemu-devel] DMG chunk size independence

Hi, cc'ing Peter Wu in... Currently I have completed the task for zlib, uncompressed and zeroed chunks in a DMG file using the approach we discussed earlier. Unfortunately, this approach is not appropriate for bz2 chunks since we cannot restart our decompression from the access point we cached

Re: [Qemu-devel] DMG chunk size independence

>> For testing I am first converting the images to raw format and then >> comparing the resulting image with the one converted using v2.9.0 DMG >> driver and after battling for 2 days with my code, it finally prints >> "Images are identical." According to John, that should be pretty >> conclusive

Re: [Qemu-devel] DMG chunk size independence

Hi, Great news! I have almost completed this task and the results are looking promising. I have not yet attended to the DMG files having bz2 compressed chunks but that should be easy and pretty similar to my approach for zlib compressed files. So, no worries there. For testing I am first

[Qemu-devel] [PATCH v4 7/8] vmdk: Update metadata for multiple clusters

Include a next pointer in VmdkMetaData struct to point to the previous allocated L2 table. Modify vmdk_L2update to start updating metadata for allocation of multiple clusters at once. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/vmdk.c

[Qemu-devel] [PATCH v4 8/8] vmdk: Make vmdk_get_cluster_offset() return cluster offset only

-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/vmdk.c | 56 1 file changed, 12 insertions(+), 44 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index e52c373..be08bde 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -1

[Qemu-devel] [PATCH v4 6/8] vmdk: New functions to assist allocating multiple clusters

the offset of the first of the many newly allocated clusters. Also, provide proper documentation for both. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/vmdk.c | 192 +++ 1 file changed, 182 insertions(+), 10 del

[Qemu-devel] [PATCH v4 5/8] vmdk: Set maximum bytes allocated in one cycle

Set the maximum bytes allowed to get allocated at once to be not more than the extent size boundary to handle writes at two separate extents appropriately. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/vmdk.c | 13 +++-- 1 file changed, 11 insertions

[Qemu-devel] [PATCH v4 4/8] vmdk: Factor out metadata loading code out of vmdk_get_cluster_offset()

Move the cluster tables loading code out of the existing vmdk_get_cluster_offset() function and implement it in separate get_cluster_table() and vmdk_L2load() functions. This patch will help us avoid code duplication in future patches of this series. Signed-off-by: Ashijeet Acharya <ashijeeta

[Qemu-devel] [PATCH v4 1/8] vmdk: Move vmdk_find_offset_in_cluster() to the top

Move the existing vmdk_find_offset_in_cluster() function to the top of the driver. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block/vmdk.c | 24 1 file changed, 12 insertions(+), 12 deletions(-)

[Qemu-devel] [PATCH v4 2/8] vmdk: Rename get_whole_cluster() to vmdk_perform_cow()

Rename the existing function get_whole_cluster() to vmdk_perform_cow() as its sole purpose is to perform COW for the first and the last allocated clusters if needed. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> Reviewed-by: Fam Zheng <f...@redhat.com> --- block

[Qemu-devel] [PATCH v4 0/8] Optimize VMDK I/O by allocating multiple clusters

able (fam) - call bdrv_pwrite_sync() for batches of atmost 512 clusters at once (fam) Changes in v2: - segregate the ugly Patch 1 in v1 into 6 readable and sensible patches - include benchmark test results in v2 Ashijeet Acharya (8): vmdk: Move vmdk_find_offset_in_cluster() to the top vmdk: Rename

[Qemu-devel] [PATCH v4 3/8] vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset()

Rename the existing get_cluster_offset() to vmdk_get_cluster_offset() and update name in all the callers accordingly. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/vmdk.c | 46 +++--- 1 file changed, 23 insertions(+), 23 del

Re: [Qemu-devel] [PATCH v3 5/6] vmdk: Set maximum bytes allocated in one cycle

On Fri, Apr 21, 2017 at 8:23 PM, Ashijeet Acharya <ashijeetacha...@gmail.com> wrote: > On Wed, Apr 19, 2017 at 6:30 PM, Fam Zheng <f...@redhat.com> wrote: >> On Sat, 04/01 20:14, Ashijeet Acharya wrote: >>> Set the maximum bytes allowed to get allocated at once to b

Re: [Qemu-devel] [PATCH v3 6/6] vmdk: Update metadata for multiple clusters

On Fri, Apr 21, 2017 at 1:45 PM, Fam Zheng <f...@redhat.com> wrote: > On Sat, 04/01 20:14, Ashijeet Acharya wrote: >> Include a next pointer in VmdkMetaData struct to point to the previous >> allocated L2 table. Modify vmdk_L2update to start updating metadata for >> allo

Re: [Qemu-devel] [PATCH v3 5/6] vmdk: Set maximum bytes allocated in one cycle

On Wed, Apr 19, 2017 at 6:30 PM, Fam Zheng <f...@redhat.com> wrote: > On Sat, 04/01 20:14, Ashijeet Acharya wrote: >> Set the maximum bytes allowed to get allocated at once to be not more >> than the extent size boundary to handle writes at two separate extents >> appr

Re: [Qemu-devel] [PATCH v3 4/6] vmdk: Rename get_cluster_offset() to vmdk_get_cluster_offset()

On Wed, Apr 19, 2017 at 18:27 Fam Zheng <f...@redhat.com> wrote: > On Sat, 04/01 20:14, Ashijeet Acharya wrote: > > Rename the existing get_cluster_offset() function to > > vmdk_get_cluster_offset() and have it make use of the new > > get_cluster_table() to lo

Re: [Qemu-devel] [PATCH v3 3/6] vmdk: New functions to assist allocating multiple clusters

On Wed, Apr 19, 2017 at 18:26 Fam Zheng <f...@redhat.com> wrote: > On Sat, 04/01 20:14, Ashijeet Acharya wrote: > > Move the cluster tables loading code out of the existing > > get_cluster_offset() function to avoid code duplication and implement it > > in

Re: [Qemu-devel] DMG chunk size independence

On Tue, Apr 18, 2017 at 10:35 PM, John Snow wrote: > >> >> I could be wrong, but that's my hunch. >> >> Maybe you can cache the state of the INFLATE process such that once you >> fill the cache with data, we can simply resume the INFLATE procedure >> when the

Re: [Qemu-devel] DMG chunk size independence

On Tue, Apr 18, 2017 at 01:59 John Snow <js...@redhat.com> wrote: > > > On 04/15/2017 04:38 AM, Ashijeet Acharya wrote: > > Hi, > > > > Some of you are already aware but for the benefit of the open list, > > this mail is regarding the task mentioned > &

[Qemu-devel] DMG chunk size independence

Hi, Some of you are already aware but for the benefit of the open list, this mail is regarding the task mentioned Here -> http://wiki.qemu-project.org/ToDo/Block/DmgChunkSizeIndependence I had a chat with Fam regarding this and he suggested a solution where we fix the output buffer size to a max

Re: [Qemu-devel] [PATCH v3 1/6] vmdk: Move vmdk_find_offset_in_cluster() to the top

On Sat, Apr 1, 2017 at 8:14 PM, Ashijeet Acharya <ashijeetacha...@gmail.com> wrote: > Move the existing vmdk_find_offset_in_cluster() function to the top of > the driver. Also, introduce a new helper function size_to_clusters() > which returns the number of clusters for a given size

[Qemu-devel] [PATCH v3 6/6] vmdk: Update metadata for multiple clusters

Include a next pointer in VmdkMetaData struct to point to the previous allocated L2 table. Modify vmdk_L2update to start updating metadata for allocation of multiple clusters at once. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/vmdk.c

[Qemu-devel] [PATCH v3 5/6] vmdk: Set maximum bytes allocated in one cycle

Set the maximum bytes allowed to get allocated at once to be not more than the extent size boundary to handle writes at two separate extents appropriately. Signed-off-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/vmdk.c | 13 +++-- 1 file changed, 11 insertions

[Qemu-devel] [PATCH v3 1/6] vmdk: Move vmdk_find_offset_in_cluster() to the top

-by: Ashijeet Acharya <ashijeetacha...@gmail.com> --- block/vmdk.c | 24 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index a9bd22b..22be887 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -242,6 +242,18 @@ stati

  1   2   3   4   >