Re: [2/8] d3dx9: Check the size of a DDS file in D3DXGetImageInfoFromFileInMemory.

2012-05-12 Thread Henri Verbeet
On 10 May 2012 20:39, Stefan Dösinger  wrote:
> Some test suggestions:
>
> *) See if the pitch_or_linearsize member is used at all
> *) Non-DWORD aligned pitches
> *) Pitches that are not pixel-size aligned
> *) pitch < width * byte_per_pixel
> *) Negative pitches / pitches > 2^31(Remember the LONG vs DWORD)
> *) A too small linear size for compressed surfaces(e.g. < block size)
>
If we're testing things anyway, integer overflows are always cute to
test for as well, in particular for these kinds of image handling
functions where you end up multiplying potentially untrusted width,
height and bpp. We probably don't handle that correctly at all.




Re: [2/8] d3dx9: Check the size of a DDS file in D3DXGetImageInfoFromFileInMemory.

2012-05-11 Thread Stefan Dösinger
Am Donnerstag, 10. Mai 2012, 21:44:56 schrieb Józef Kucia:
> Do you think this patch series can get in before tests for pitch
> handling? The patch series is big enough and I don't want to make it
> any bigger.
That's fine with me if it makes your work easier.


signature.asc
Description: This is a digitally signed message part.



Re: [2/8] d3dx9: Check the size of a DDS file in D3DXGetImageInfoFromFileInMemory.

2012-05-10 Thread Józef Kucia
On Thu, May 10, 2012 at 8:39 PM, Stefan Dösinger  wrote:
> Since I've burned my hands with DDSURFACEDESC2 pitch handling in the past(e.g.
> bug 21238), I highly recommend to test it. Knowing msdn, the above page makes
> me less confident, not more :-\
>
> The most likely situation where an error occurs is when a different API or
> tool(e.g. gdi32 or ddraw) uses a specific pitch and the application developers
> are not aware of it and just forward the data 1:1.
>
> Some test suggestions:
>
> *) See if the pitch_or_linearsize member is used at all
> *) Non-DWORD aligned pitches
> *) Pitches that are not pixel-size aligned
> *) pitch < width * byte_per_pixel
> *) Negative pitches / pitches > 2^31(Remember the LONG vs DWORD)
> *) A too small linear size for compressed surfaces(e.g. < block size)

Thanks for the suggestions.

Do you think this patch series can get in before tests for pitch
handling? The patch series is big enough and I don't want to make it
any bigger. The patch series also makes d3dx9 to pass almost all tests
for DDS file format support. Besides that, there are many things left
and improvements to be done until the DDS support in d3dx9 will be
completed. Amongst things to do the most problematic seems to be DXTn
compressor and decompressor. So I think these tests could be added in
the next series of patches. What do you think?




Re: [2/8] d3dx9: Check the size of a DDS file in D3DXGetImageInfoFromFileInMemory.

2012-05-10 Thread Stefan Dösinger
Am Donnerstag, 10. Mai 2012, 11:12:06 schrieb Józef Kucia:
> Surfaces pitches doesn't seem to be 4 or 8 byte aligned in DDS files.
> MSDN recommends to not use pitch_or_linear_size and gives formulas for
> computing pitches [1]. When it comes to formats mentioned by you,
> D3DFMT_P8 doesn't seem to be supported in DDS files. The formula from
> this patch also works for the dds_16bit file which stores pixels in
> D3DFMT_X1R5G5B5 format. Besides that, the patches series seems to work
> pretty well with at least two real applications.
Since I've burned my hands with DDSURFACEDESC2 pitch handling in the past(e.g.
bug 21238), I highly recommend to test it. Knowing msdn, the above page makes
me less confident, not more :-\

The most likely situation where an error occurs is when a different API or
tool(e.g. gdi32 or ddraw) uses a specific pitch and the application developers
are not aware of it and just forward the data 1:1.

Some test suggestions:

*) See if the pitch_or_linearsize member is used at all
*) Non-DWORD aligned pitches
*) Pitches that are not pixel-size aligned
*) pitch < width * byte_per_pixel
*) Negative pitches / pitches > 2^31(Remember the LONG vs DWORD)
*) A too small linear size for compressed surfaces(e.g. < block size)


signature.asc
Description: This is a digitally signed message part.



Re: [2/8] d3dx9: Check the size of a DDS file in D3DXGetImageInfoFromFileInMemory.

2012-05-10 Thread Józef Kucia
On Thu, May 10, 2012 at 4:27 PM, Matteo Bruni  wrote:
> It looks like you're not actually using the faces variable.

Thanks. I should multiply expected_size by faces.




Re: [2/8] d3dx9: Check the size of a DDS file in D3DXGetImageInfoFromFileInMemory.

2012-05-10 Thread Matteo Bruni
2012/5/9 Józef Kucia :
> +static HRESULT get_image_info_from_dds(const void *buffer, UINT length, 
> D3DXIMAGE_INFO *info)
>  {
> +   UINT i;
> +   UINT faces = 0;
...
> +   /* calculate the expected length */
> +   width = info->Width;
> +   height = info->Height;
> +   for (i = 0; i < info->MipLevels; i++)
> +   {
> +       UINT pitch, size = 0;
> +       calculate_dds_surface_size(info, width, height, &pitch, &size);
> +       expected_length += size;
> +       width = max(1, width / 2);
> +       height = max(1, width / 2);
> +   }
> +
> +   if (length < expected_length)
> +   return D3DXERR_INVALIDDATA;

It looks like you're not actually using the faces variable.




Re: [2/8] d3dx9: Check the size of a DDS file in D3DXGetImageInfoFromFileInMemory.

2012-05-10 Thread Józef Kucia
On Thu, May 10, 2012 at 10:32 AM, Stefan Dösinger
 wrote:
> Am Mittwoch, 9. Mai 2012, 23:32:05 schrieb Józef Kucia:
>> +        *pitch = width * format_desc->bytes_per_pixel;
>> +        *size = *pitch * height;
> Usually surface pitches are either 4 or 8 byte aligned(depending on the API
> and pool). I recommend to write some tests for this. The most likely
> problematic formats are D3DFMT_P8, any 16 bit format like R5G6B5 and 24 bit
> R8G8B8. Maybe you're supposed to honor the pitch_or_linear_size member of the
> DDSURFACEDESC.

Surfaces pitches doesn't seem to be 4 or 8 byte aligned in DDS files.
MSDN recommends to not use pitch_or_linear_size and gives formulas for
computing pitches [1]. When it comes to formats mentioned by you,
D3DFMT_P8 doesn't seem to be supported in DDS files. The formula from
this patch also works for the dds_16bit file which stores pixels in
D3DFMT_X1R5G5B5 format. Besides that, the patches series seems to work
pretty well with at least two real applications.

On the other hand, writing more tests is a good idea.

[1] - 
http://msdn.microsoft.com/en-us/library/windows/desktop/bb943991%28v=vs.85%29.aspx




Re: [2/8] d3dx9: Check the size of a DDS file in D3DXGetImageInfoFromFileInMemory.

2012-05-10 Thread Stefan Dösinger
Am Mittwoch, 9. Mai 2012, 23:32:05 schrieb Józef Kucia:
> +*pitch = width * format_desc->bytes_per_pixel;
> +*size = *pitch * height;
Usually surface pitches are either 4 or 8 byte aligned(depending on the API
and pool). I recommend to write some tests for this. The most likely
problematic formats are D3DFMT_P8, any 16 bit format like R5G6B5 and 24 bit
R8G8B8. Maybe you're supposed to honor the pitch_or_linear_size member of the
DDSURFACEDESC.


signature.asc
Description: This is a digitally signed message part.