Re: [PATCH] comedi: integer overflow in do_insnlist_ioctl()

2011-11-25 Thread Dan Carpenter
On Thu, Nov 24, 2011 at 02:07:49PM -0500, Xi Wang wrote: > Using kcalloc looks good to me. Do you want to redo the patch in that way? > It's your choice. The other fix you wrote is valid as well. regards, dan carpenter signature.asc Description: Digital signature ___

Re: [PATCH] comedi: integer overflow in do_insnlist_ioctl()

2011-11-24 Thread Xi Wang
Using kcalloc looks good to me. Do you want to redo the patch in that way? - xi On Nov 23, 2011, at 4:51 PM, Dan Carpenter wrote: > > I think the reason why I didn't do that in my original patch is that > kcalloc() has a memset(..., 0, ...) in it so it's a slow down. But > this isn't performan

Re: [PATCH] comedi: integer overflow in do_insnlist_ioctl()

2011-11-23 Thread Dan Carpenter
On Wed, Nov 23, 2011 at 10:41:07PM +0100, Lars-Peter Clausen wrote: > > if (insnlist.n_insns <= ULONG_MAX / sizeof(struct comedi_insn)) > > insns = > > kmalloc(sizeof(struct comedi_insn) * insnlist.n_insns, > > GFP_KERNEL); > > if (!insns) > > ...

Re: [PATCH] comedi: integer overflow in do_insnlist_ioctl()

2011-11-23 Thread Lars-Peter Clausen
On 11/23/2011 05:06 PM, Ian Abbott wrote: > On 2011-11-23 14:50, Dan Carpenter wrote: >> On Wed, Nov 23, 2011 at 08:59:52AM -0500, Xi Wang wrote: >>> Thanks for the pointer. However you cannot do the overflow check using >>> >>>if (sizeof(struct comedi_insn) * insnlist.n_insns < >>> insnlist.

Re: [PATCH] comedi: integer overflow in do_insnlist_ioctl()

2011-11-23 Thread Ian Abbott
On 2011-11-23 14:50, Dan Carpenter wrote: On Wed, Nov 23, 2011 at 08:59:52AM -0500, Xi Wang wrote: Thanks for the pointer. However you cannot do the overflow check using if (sizeof(struct comedi_insn) * insnlist.n_insns < insnlist.n_insns) Let's assume 32-bit system, sizeof(struct comedi_

Re: [PATCH] comedi: integer overflow in do_insnlist_ioctl()

2011-11-23 Thread Dan Carpenter
On Wed, Nov 23, 2011 at 08:59:52AM -0500, Xi Wang wrote: > Thanks for the pointer. However you cannot do the overflow check using > > if (sizeof(struct comedi_insn) * insnlist.n_insns < insnlist.n_insns) > > Let's assume 32-bit system, sizeof(struct comedi_insn) = 32, and > insnlist.n_insns =

Re: [PATCH] comedi: integer overflow in do_insnlist_ioctl()

2011-11-23 Thread Xi Wang
Thanks for the pointer. However you cannot do the overflow check using if (sizeof(struct comedi_insn) * insnlist.n_insns < insnlist.n_insns) Let's assume 32-bit system, sizeof(struct comedi_insn) = 32, and insnlist.n_insns = 0x7fff. Note that 32 * 0x7fff = 0xffe0 overflows but byp

Re: [PATCH] comedi: integer overflow in do_insnlist_ioctl()

2011-11-23 Thread Dan Carpenter
I sent a patch for this already. http://driverdev.linuxdriverproject.org/pipermail/devel/2011-November/022469.html regards, dan carpenter signature.asc Description: Digital signature ___ devel mailing list devel@linuxdriverproject.org http://driverd

[PATCH] comedi: integer overflow in do_insnlist_ioctl()

2011-11-22 Thread Xi Wang
There is a potential integer overflow in do_insnlist_ioctl() if userspace passes in a large insnlist.n_insns. The call to kmalloc() would allocate a small buffer, which would result in a memory corruption. Reported-by: Haogang Chen Signed-off-by: Xi Wang --- drivers/staging/comedi/comedi_fop