Re: Disabling block layer
On Mon, Mar 26 2007, Mark Lobo wrote: Hello! I had a question about disabling the block layer for SCSI devices. We have an embedded device, and it runs 2.4.30. We need to be able to support a lot of SCSI devices (in the thousands) for our device, and we talk to the devices via SG. We are facing a memory allocation problem after discovering a few thousand devices. For every device, there seems to be a lot of memory allocated in the block layer. This memory includes cache memory (which IIRC is reclaimable by the kernel memory subsystem when it needs it) and also pages that are used for the alloc_pages pool. A much easier approach would be to limit the memory used for each device in the block layer. Since SCSI uses the block layer as a transport for commands, you cannot disable the block layer in any easy manner. But your memory is likely being eated by the queue freelist. So edit drivers/block/ll_rw_blk.c and hardcode nr_requests to a low number (like 2). -- Jens Axboe - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Disabling block layer
Hello! I had a question about disabling the block layer for SCSI devices. We have an embedded device, and it runs 2.4.30. We need to be able to support a lot of SCSI devices (in the thousands) for our device, and we talk to the devices via SG. We are facing a memory allocation problem after discovering a few thousand devices. For every device, there seems to be a lot of memory allocated in the block layer. This memory includes cache memory (which IIRC is reclaimable by the kernel memory subsystem when it needs it) and also pages that are used for the alloc_pages pool. My questions were relating to disabling the block layer for the devices. We always talk direct passthrough to the storage(except the local hard disk), and do not need the block layer at all. 1. Is there a way to disable the block layer for specific devices? 2. If yes, how can that be done, and are there any gotchas associated with that? Thanks! M Never miss an email again! Yahoo! Toolbar alerts you the instant new Mail arrives. http://tools.search.yahoo.com/toolbar/features/mail/ - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Disabling block layer
Mark Lobo wrote: Hello! I had a question about disabling the block layer for SCSI devices. We have an embedded device, and it runs 2.4.30. We need to be able to support a lot of SCSI devices (in the thousands) for our device, and we talk to the devices via SG. We are facing a memory allocation problem after discovering a few thousand devices. For every device, there seems to be a lot of memory allocated in the block layer. This memory includes cache memory (which IIRC is reclaimable by the kernel memory subsystem when it needs it) and also pages that are used for the alloc_pages pool. My questions were relating to disabling the block layer for the devices. We always talk direct passthrough to the storage(except the local hard disk), and do not need the block layer at all. 1. Is there a way to disable the block layer for specific devices? 2. If yes, how can that be done, and are there any gotchas associated with that? Mark, Tempting thought that: linux without a block layer. I think you have no hope in the lk 2.4 series and even less in the lk 2.6 series. Now for some thoughts. If you don't need to mount any SCSI disks, you could build a kernel with sd as a module and remove/hide sd_mod.o . A more invasive method would be to modify the sd driver so that it was no longer interested in SCSI devices whose peripheral device type was zero (i.e. disks). On the sg driver side, if lots of sg file descriptors are open to those thousands of SCSI devices, then reducing the per fd SG_DEF_RESERVED_SIZE from 32 KB may help. This could be reduced by editing include/scsi/sg.h . Doug Gilbert - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Disabling block layer
Mark Lobo wrote: I had a question about disabling the block layer for SCSI devices. We have an embedded device, and it runs 2.4.30. We need to be able to support a lot of SCSI devices (in the thousands) for our device, and we talk to the devices via SG. We are facing a memory allocation problem after discovering a few thousand devices. For every device, there seems to be a lot of memory allocated in the block layer. This memory includes cache memory (which IIRC is reclaimable by the kernel memory subsystem when it needs it) and also pages that are used for the alloc_pages pool. My questions were relating to disabling the block layer for the devices. We always talk direct passthrough to the storage(except the local hard disk), and do not need the block layer at all. You may consider something we experimented with here (for performance reasons).. We basically recompiled one of the scsi drivers to call our own version of scsi_host_alloc() and then made calls to the queuecommand() routine directly. You then allow the kernel probe routines to only discover the first target with the local disk. I assume you know ahead of time which scsi cards your using in your system. The point is that you could just build a heavily modified scsi driver with application specific hooks. BTW we aren't currently doing this because in the end we got most of what we needed by writing a driver which replaces sg and bypasses most of the kernel without being as invasive. In the long run we may still use a modified LLDD since the interfaces we depend on are changing a little to fast for our liking and we are not running any system devices on the interface cards we need to directly access. I'm not sure how you would go about tearing down enough of the system that the device doesn't consume any resources, yet leave enough of it around to be accessible. I will be interested to find out what you end up doing. - To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
