Re: About bio_endio
On Wed, Jun 25, 2014 at 10:31 AM, Alvin Abitria wrote: > Hello Pranay, > > Thanks for your very helpful insights! I hope you don’t mind if I continue > with more questions on block layer :-) Surely. > > On Jun 25, 2014 2:09 AM, "Pranay Srivastava" wrote: >> >> Hello Alvin, >> >> On Tue, Jun 24, 2014 at 9:53 PM, Alvin Abitria >> wrote: >> > Hello Pranay! >> > >> > Thanks for your reply. I apologize for my very late reply, I was very >> > preoccupied earlier at work. >> > >> > >> > On Tue, Jun 24, 2014 at 1:07 PM, Pranay Srivastava >> > wrote: >> >> >> >> Hello Alvin, >> >> >> >> On Mon, Jun 23, 2014 at 10:39 PM, Alvin Abitria >> >> >> >> wrote: >> >> > Hello, >> >> > >> >> > I'm developing a block driver using the make_request method, >> >> > effectively >> >> > bypassing existing scsi or request stack in block layer. So that >> >> > means >> >> > im >> >> > directly working with bios. As prescribed in linux documentation and >> >> > from >> >> > referring to similar drivers in kernel, you close a session with a >> >> > bio >> >> > with >> >> > the bio_endio function. >> >> >> >> So it means you are just passing on the bios without the request >> >> structure if I'm correct? >> >> I don't know how you are handling blk_finish_plug without having >> >> request or request queue, >> >> I maybe wrong in understanding how you are handling it. >> >> >> > Yes, I'm working on bio's level. No struct requests, and I haven't used >> > blk_finish_plug yet. >> > The block driver method I'm implementing is somewhat along the same line >> > with nvme and mtip2xxx >> > drivers in drivers/block directory (but differing in hardware specific >> > level >> > of course). >> >> Ok I got it now. But why not use request structures? You can use at >> least one for chaining bio(s). See below what I mean.. >> >> >> >> >> > >> >> > I usually invoke bio_endio during successful I/O completion, meaning >> >> > with an >> >> > error code of zero. But there are cases that this is not fulfilled >> >> > or >> >> > there >> >> > are error cases. My question is, what are the valid error codes that >> >> > can be >> >> > used with it? My initial impression is that other than zero as error >> >> > code, >> >> -EIO is the one that you should use I think, >> >> > bio_endio will fail. I've read somewhere that -EBUSY is not >> >> > recognized, >> >> > and >> >> > I tried -EIO but my driver crashed. I got a panic in some dio_xxx >> >> > function >> >> > leading from bio_endio(bio,-EIO). I would like to block subsequent >> >> > bios >> >> > sent >> >> >> >> If it's okay for you to post the error then can you do that? I was >> >> seeing the code for >> >> dio_end_io but it would be good if you can post the exact crash >> >> backtrace if you've got that. >> > >> > >> > Here you go: >> > >> > BUG: unable to handle kernel NULL pointer dereference at (null) >> > IP: [] bio_check_pages_dirty+0x50/0xe0 >> > PGD 42e5e4067 PUD 42e6e7067 PMD 0 >> > Oops: [#1] SMP >> > last sysfs file: >> > /sys/devices/system/cpu/cpu0/cache/index0/coherency_line_size >> > CPU 7 >> > Modules linked in: block_module(U) fuse ip6table_filter ip6_tables >> > ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 >> > nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_CHECKSUM >> > iptable_mangle >> > iptable_filter ip_tables bridge autofs4 sunrpc 8021q garp stp llc >> > cpufreq_ondemand freq_table pcc_cpufreq ipv6 vhost_net macvtap macvlan >> > tun >> > kvm_intel kvm uinput power_meter hpilo hpwdt sg tg3 microcode serio_raw >> > iTCO_wdt iTCO_vendor_support ioatdma dca shpchp ext4 mbcache jbd2 sd_mod >> > crc_t10dif hpsa pata_acpi ata_generic ata_piix dm_mirror dm_region_hash >> > dm_log dm_mod [last unloaded: scsi_wait_scan] >> > >> > Pid: 3740, comm: fio Not tainted 2.6.32-358.el6.x86_64 #1 HP ProLiant >> > DL380p >> > Gen8 >> > RIP: 0010:[] [] >> > bio_check_pages_dirty+0x50/0xe0 >> > RSP: 0018:8804191618c8 EFLAGS: 00010046 >> > RAX: 2000 RBX: 88041909f0c0 RCX: 11ae >> > RDX: RSI: RDI: >> > RBP: 8804191618f8 R08: 81c07728 R09: 0040 >> > R10: 0002 R11: 0002 R12: >> > R13: 8804191b9b80 R14: 8804191b9b80 R15: >> > FS: 7fcd43e2d720() GS:8800366e() >> > knlGS: >> > CS: 0010 DS: ES: CR0: 80050033 >> > CR2: CR3: 00041e69f000 CR4: 000407e0 >> > DR0: DR1: DR2: >> > DR3: DR6: 0ff0 DR7: 0400 >> > Process fio (pid: 3740, threadinfo 88041916, task >> > 88043341f500) >> > Stack: >> > 88043433f400 88043433f520 88041909f0c0 >> > 88041909f0c0 8804191b9b80 880419161948 811bdc38 >> > 880419161968 34236400 fffb 88043433f400 >
Re: About bio_endio
Hello Pranay, Thanks for your very helpful insights! I hope you don’t mind if I continue with more questions on block layer :-) On Jun 25, 2014 2:09 AM, "Pranay Srivastava" wrote: > > Hello Alvin, > > On Tue, Jun 24, 2014 at 9:53 PM, Alvin Abitria wrote: > > Hello Pranay! > > > > Thanks for your reply. I apologize for my very late reply, I was very > > preoccupied earlier at work. > > > > > > On Tue, Jun 24, 2014 at 1:07 PM, Pranay Srivastava > > wrote: > >> > >> Hello Alvin, > >> > >> On Mon, Jun 23, 2014 at 10:39 PM, Alvin Abitria < abitria.al...@gmail.com> > >> wrote: > >> > Hello, > >> > > >> > I'm developing a block driver using the make_request method, effectively > >> > bypassing existing scsi or request stack in block layer. So that means > >> > im > >> > directly working with bios. As prescribed in linux documentation and > >> > from > >> > referring to similar drivers in kernel, you close a session with a bio > >> > with > >> > the bio_endio function. > >> > >> So it means you are just passing on the bios without the request > >> structure if I'm correct? > >> I don't know how you are handling blk_finish_plug without having > >> request or request queue, > >> I maybe wrong in understanding how you are handling it. > >> > > Yes, I'm working on bio's level. No struct requests, and I haven't used > > blk_finish_plug yet. > > The block driver method I'm implementing is somewhat along the same line > > with nvme and mtip2xxx > > drivers in drivers/block directory (but differing in hardware specific level > > of course). > > Ok I got it now. But why not use request structures? You can use at > least one for chaining bio(s). See below what I mean.. > > >> > >> > > >> > I usually invoke bio_endio during successful I/O completion, meaning > >> > with an > >> > error code of zero. But there are cases that this is not fulfilled or > >> > there > >> > are error cases. My question is, what are the valid error codes that > >> > can be > >> > used with it? My initial impression is that other than zero as error > >> > code, > >> -EIO is the one that you should use I think, > >> > bio_endio will fail. I've read somewhere that -EBUSY is not recognized, > >> > and > >> > I tried -EIO but my driver crashed. I got a panic in some dio_xxx > >> > function > >> > leading from bio_endio(bio,-EIO). I would like to block subsequent bios > >> > sent > >> > >> If it's okay for you to post the error then can you do that? I was > >> seeing the code for > >> dio_end_io but it would be good if you can post the exact crash > >> backtrace if you've got that. > > > > > > Here you go: > > > > BUG: unable to handle kernel NULL pointer dereference at (null) > > IP: [] bio_check_pages_dirty+0x50/0xe0 > > PGD 42e5e4067 PUD 42e6e7067 PMD 0 > > Oops: [#1] SMP > > last sysfs file: > > /sys/devices/system/cpu/cpu0/cache/index0/coherency_line_size > > CPU 7 > > Modules linked in: block_module(U) fuse ip6table_filter ip6_tables > > ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 > > nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_CHECKSUM iptable_mangle > > iptable_filter ip_tables bridge autofs4 sunrpc 8021q garp stp llc > > cpufreq_ondemand freq_table pcc_cpufreq ipv6 vhost_net macvtap macvlan tun > > kvm_intel kvm uinput power_meter hpilo hpwdt sg tg3 microcode serio_raw > > iTCO_wdt iTCO_vendor_support ioatdma dca shpchp ext4 mbcache jbd2 sd_mod > > crc_t10dif hpsa pata_acpi ata_generic ata_piix dm_mirror dm_region_hash > > dm_log dm_mod [last unloaded: scsi_wait_scan] > > > > Pid: 3740, comm: fio Not tainted 2.6.32-358.el6.x86_64 #1 HP ProLiant DL380p > > Gen8 > > RIP: 0010:[] [] > > bio_check_pages_dirty+0x50/0xe0 > > RSP: 0018:8804191618c8 EFLAGS: 00010046 > > RAX: 2000 RBX: 88041909f0c0 RCX: 11ae > > RDX: RSI: RDI: > > RBP: 8804191618f8 R08: 81c07728 R09: 0040 > > R10: 0002 R11: 0002 R12: > > R13: 8804191b9b80 R14: 8804191b9b80 R15: > > FS: 7fcd43e2d720() GS:8800366e() knlGS: > > CS: 0010 DS: ES: CR0: 80050033 > > CR2: CR3: 00041e69f000 CR4: 000407e0 > > DR0: DR1: DR2: > > DR3: DR6: 0ff0 DR7: 0400 > > Process fio (pid: 3740, threadinfo 88041916, task 88043341f500) > > Stack: > > 88043433f400 88043433f520 88041909f0c0 > > 88041909f0c0 8804191b9b80 880419161948 811bdc38 > > 880419161968 34236400 fffb 88043433f400 > > Call Trace: > > [] dio_bio_complete+0xc8/0xd0 > > You are using a different kernel then mine. See if the offset within > this function leads you to > > spin_lock_irqsave(&bio_dirty_lock, flags); > bio->bi_pri
Re: About bio_endio
Hello Alvin, On Tue, Jun 24, 2014 at 9:53 PM, Alvin Abitria wrote: > Hello Pranay! > > Thanks for your reply. I apologize for my very late reply, I was very > preoccupied earlier at work. > > > On Tue, Jun 24, 2014 at 1:07 PM, Pranay Srivastava > wrote: >> >> Hello Alvin, >> >> On Mon, Jun 23, 2014 at 10:39 PM, Alvin Abitria >> wrote: >> > Hello, >> > >> > I'm developing a block driver using the make_request method, effectively >> > bypassing existing scsi or request stack in block layer. So that means >> > im >> > directly working with bios. As prescribed in linux documentation and >> > from >> > referring to similar drivers in kernel, you close a session with a bio >> > with >> > the bio_endio function. >> >> So it means you are just passing on the bios without the request >> structure if I'm correct? >> I don't know how you are handling blk_finish_plug without having >> request or request queue, >> I maybe wrong in understanding how you are handling it. >> > Yes, I'm working on bio's level. No struct requests, and I haven't used > blk_finish_plug yet. > The block driver method I'm implementing is somewhat along the same line > with nvme and mtip2xxx > drivers in drivers/block directory (but differing in hardware specific level > of course). Ok I got it now. But why not use request structures? You can use at least one for chaining bio(s). See below what I mean.. >> >> > >> > I usually invoke bio_endio during successful I/O completion, meaning >> > with an >> > error code of zero. But there are cases that this is not fulfilled or >> > there >> > are error cases. My question is, what are the valid error codes that >> > can be >> > used with it? My initial impression is that other than zero as error >> > code, >> -EIO is the one that you should use I think, >> > bio_endio will fail. I've read somewhere that -EBUSY is not recognized, >> > and >> > I tried -EIO but my driver crashed. I got a panic in some dio_xxx >> > function >> > leading from bio_endio(bio,-EIO). I would like to block subsequent bios >> > sent >> >> If it's okay for you to post the error then can you do that? I was >> seeing the code for >> dio_end_io but it would be good if you can post the exact crash >> backtrace if you've got that. > > > Here you go: > > BUG: unable to handle kernel NULL pointer dereference at (null) > IP: [] bio_check_pages_dirty+0x50/0xe0 > PGD 42e5e4067 PUD 42e6e7067 PMD 0 > Oops: [#1] SMP > last sysfs file: > /sys/devices/system/cpu/cpu0/cache/index0/coherency_line_size > CPU 7 > Modules linked in: block_module(U) fuse ip6table_filter ip6_tables > ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 > nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_CHECKSUM iptable_mangle > iptable_filter ip_tables bridge autofs4 sunrpc 8021q garp stp llc > cpufreq_ondemand freq_table pcc_cpufreq ipv6 vhost_net macvtap macvlan tun > kvm_intel kvm uinput power_meter hpilo hpwdt sg tg3 microcode serio_raw > iTCO_wdt iTCO_vendor_support ioatdma dca shpchp ext4 mbcache jbd2 sd_mod > crc_t10dif hpsa pata_acpi ata_generic ata_piix dm_mirror dm_region_hash > dm_log dm_mod [last unloaded: scsi_wait_scan] > > Pid: 3740, comm: fio Not tainted 2.6.32-358.el6.x86_64 #1 HP ProLiant DL380p > Gen8 > RIP: 0010:[] [] > bio_check_pages_dirty+0x50/0xe0 > RSP: 0018:8804191618c8 EFLAGS: 00010046 > RAX: 2000 RBX: 88041909f0c0 RCX: 11ae > RDX: RSI: RDI: > RBP: 8804191618f8 R08: 81c07728 R09: 0040 > R10: 0002 R11: 0002 R12: > R13: 8804191b9b80 R14: 8804191b9b80 R15: > FS: 7fcd43e2d720() GS:8800366e() knlGS: > CS: 0010 DS: ES: CR0: 80050033 > CR2: CR3: 00041e69f000 CR4: 000407e0 > DR0: DR1: DR2: > DR3: DR6: 0ff0 DR7: 0400 > Process fio (pid: 3740, threadinfo 88041916, task 88043341f500) > Stack: > 88043433f400 88043433f520 88041909f0c0 > 88041909f0c0 8804191b9b80 880419161948 811bdc38 > 880419161968 34236400 fffb 88043433f400 > Call Trace: > [] dio_bio_complete+0xc8/0xd0 You are using a different kernel then mine. See if the offset within this function leads you to spin_lock_irqsave(&bio_dirty_lock, flags); bio->bi_private = bio_dirty_list; /*This is the line number I got from the offset*/ bio_dirty_list = bio; spin_unlock_irqrestore(&bio_dirty_lock, flags); You are not doing bio_put in the driver code right? Assuming you are not doing bio_get also. On which filesystem you are testing this? Can you see if it happens on a simpler one like ext2 since it uses the generic direct_IO call. So we can further narrow it d
Re: About bio_endio
Hello Pranay! Thanks for your reply. I apologize for my very late reply, I was very preoccupied earlier at work. On Tue, Jun 24, 2014 at 1:07 PM, Pranay Srivastava wrote: > Hello Alvin, > > On Mon, Jun 23, 2014 at 10:39 PM, Alvin Abitria > wrote: > > Hello, > > > > I'm developing a block driver using the make_request method, effectively > > bypassing existing scsi or request stack in block layer. So that means > im > > directly working with bios. As prescribed in linux documentation and > from > > referring to similar drivers in kernel, you close a session with a bio > with > > the bio_endio function. > > So it means you are just passing on the bios without the request > structure if I'm correct? > I don't know how you are handling blk_finish_plug without having > request or request queue, > I maybe wrong in understanding how you are handling it. > > Yes, I'm working on bio's level. No struct requests, and I haven't used blk_finish_plug yet. The block driver method I'm implementing is somewhat along the same line with nvme and mtip2xxx drivers in drivers/block directory (but differing in hardware specific level of course). > > > > I usually invoke bio_endio during successful I/O completion, meaning > with an > > error code of zero. But there are cases that this is not fulfilled or > there > > are error cases. My question is, what are the valid error codes that > can be > > used with it? My initial impression is that other than zero as error > code, > -EIO is the one that you should use I think, > > bio_endio will fail. I've read somewhere that -EBUSY is not recognized, > and > > I tried -EIO but my driver crashed. I got a panic in some dio_xxx > function > > leading from bio_endio(bio,-EIO). I would like to block subsequent bios > sent > > If it's okay for you to post the error then can you do that? I was > seeing the code for > dio_end_io but it would be good if you can post the exact crash > backtrace if you've got that. > Here you go: BUG: unable to handle kernel NULL pointer dereference at (null) IP: [] bio_check_pages_dirty+0x50/0xe0 PGD 42e5e4067 PUD 42e6e7067 PMD 0 Oops: [#1] SMP last sysfs file: /sys/devices/system/cpu/cpu0/cache/index0/coherency_line_size CPU 7 Modules linked in: block_module(U) fuse ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_CHECKSUM iptable_mangle iptable_filter ip_tables bridge autofs4 sunrpc 8021q garp stp llc cpufreq_ondemand freq_table pcc_cpufreq ipv6 vhost_net macvtap macvlan tun kvm_intel kvm uinput power_meter hpilo hpwdt sg tg3 microcode serio_raw iTCO_wdt iTCO_vendor_support ioatdma dca shpchp ext4 mbcache jbd2 sd_mod crc_t10dif hpsa pata_acpi ata_generic ata_piix dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan] Pid: 3740, comm: fio Not tainted 2.6.32-358.el6.x86_64 #1 HP ProLiant DL380p Gen8 RIP: 0010:[] [] bio_check_pages_dirty+0x50/0xe0 RSP: 0018:8804191618c8 EFLAGS: 00010046 RAX: 2000 RBX: 88041909f0c0 RCX: 11ae RDX: RSI: RDI: RBP: 8804191618f8 R08: 81c07728 R09: 0040 R10: 0002 R11: 0002 R12: R13: 8804191b9b80 R14: 8804191b9b80 R15: FS: 7fcd43e2d720() GS:8800366e() knlGS: CS: 0010 DS: ES: CR0: 80050033 CR2: CR3: 00041e69f000 CR4: 000407e0 DR0: DR1: DR2: DR3: DR6: 0ff0 DR7: 0400 Process fio (pid: 3740, threadinfo 88041916, task 88043341f500) Stack: 88043433f400 88043433f520 88041909f0c0 88041909f0c0 8804191b9b80 880419161948 811bdc38 880419161968 34236400 fffb 88043433f400 Call Trace: [] dio_bio_complete+0xc8/0xd0 [] dio_bio_end_aio+0x2f/0xd0 [] bio_endio+0x1d/0x40 [] block_make_request+0xe1/0x150 [block_module] [] generic_make_request+0x25e/0x530 [] ? bvec_alloc_bs+0x62/0x110 [] submit_bio+0x8d/0x120 [] dio_bio_submit+0xbc/0xc0 [] __blockdev_direct_IO_newtrunc+0x631/0xb30 [] ? filemap_fault+0xd3/0x500 [] __blockdev_direct_IO+0x5e/0xd0 [] ? blkdev_get_blocks+0x0/0xc0 [] blkdev_direct_IO+0x57/0x60 [] ? blkdev_get_blocks+0x0/0xc0 [] generic_file_aio_read+0x6bb/0x700 [] ? kmem_getpages+0xba/0x170 [] ? cache_grow+0x217/0x320 [] blkdev_aio_read+0x53/0xc0 [] ? mempool_alloc+0x63/0x140 [] ? blkdev_aio_read+0x0/0xc0 [] aio_rw_vect_retry+0x84/0x200 [] aio_run_iocb+0x64/0x170 [] do_io_submit+0x291/0x920 [] sys_io_submit+0x10/0x20 [] system_call_fastpath+0x16/0x1b Code: 31 e4 45 31 ff eb 15 0f 1f 40 00 0f b7 43 28 41 83 c4 01 41 83 c7 01 44 39 e0 7e 36 4d 63 ec 49 c1 e5 04 4f 8d 2c 2e 49 8b 7d 00 <48> 8b 07 a8 10 75 06 66 a9 00 c0 74 d3 e8 5e 59 f7 ff 49 c7 45 RIP
Re: About bio_endio
Hello Alvin, On Mon, Jun 23, 2014 at 10:39 PM, Alvin Abitria wrote: > Hello, > > I'm developing a block driver using the make_request method, effectively > bypassing existing scsi or request stack in block layer. So that means im > directly working with bios. As prescribed in linux documentation and from > referring to similar drivers in kernel, you close a session with a bio with > the bio_endio function. So it means you are just passing on the bios without the request structure if I'm correct? I don't know how you are handling blk_finish_plug without having request or request queue, I maybe wrong in understanding how you are handling it. > > I usually invoke bio_endio during successful I/O completion, meaning with an > error code of zero. But there are cases that this is not fulfilled or there > are error cases. My question is, what are the valid error codes that can be > used with it? My initial impression is that other than zero as error code, -EIO is the one that you should use I think, > bio_endio will fail. I've read somewhere that -EBUSY is not recognized, and > I tried -EIO but my driver crashed. I got a panic in some dio_xxx function > leading from bio_endio(bio,-EIO). I would like to block subsequent bios sent If it's okay for you to post the error then can you do that? I was seeing the code for dio_end_io but it would be good if you can post the exact crash backtrace if you've got that. > to me after reaching my queue depth and with no tags left, and so I want to > use bio_endio with an error code. If you have a request queue then you could call blk_stop_queue and blk_start_queue but I don't know if this is relevant to your case. > > What are those error codes, and will they work for my intended function? > Thanks! -EIO should work, but first let's find out why you got the crash. > > > > Sent from Samsung Mobile > > ___ > Kernelnewbies mailing list > Kernelnewbies@kernelnewbies.org > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies > -- ---P.K.S ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
