Re: Question about calculate trbs in xhci
On 10/06/2014 11:37 AM, vichy wrote: > hi Mathias: >> As I understood it, it is ok to cross page boundaries as long as the buffer >> is physically contiguous. As our dma case should be. >> xhci specification section 3.2.8 says that: >> >> "Note that no constraints are placed on the TRB Length fields in a >> Scatter/Gather list. Classically all the >> buffers pointed to by a scatter gather list were required to be “page size” >> in length except for the first and >> last (as illustrated by the example above). The xHCI does not require this >> constraint. Any buffer pointed to >> by a Normal, Data Stage, or Isoch TRB in a TD may be any size between 0 and >> 64K bytes in size. For >> instance, if when an OS translates a virtual memory buffer into a list of >> physical pages, some of the entries >> in the list reference multiple contiguous pages, the flexible Length fields >> of TRBs allow a 1:1 mapping of list >> entries to TRBs, i.e. a multi-page list entry does not need to be defined as >> multiple page sized TRBs." >> " > Yes, you are correct. ^^ > there is no page boundary limitation for Scatter/Gather list, but how > about iso transfer? > for iso TRB, it seems a limitation for not crossing page boundary in > section 3.2.11. > > appreciate all your kind help, > As I interpreted it the page crossing limitation is all about buffers in virtual memory that are not contiguous in physical memory. In that case we have to dig out the physical address of each page, and use chained TRBs to represent each non-continuous physical memory chunk. The limitaition says: "If the data required by an Isoch TD is not physically contiguous (e.g. crosses a page boundary), then one or more additional Normal TRBs shall be chained to the Isoch TRB by the host." But we have physically contiguous memory. Data Buffer Pointer Hi and Lo in the TRB structure point to physical memory. For isoc transfers xhci uses urb->transfer_dma for the data buffer pointers. urb->transfer_dma is DMA mapped memory of type dma_addr_t and should be physically contiguous. So we don't need to worry about page sizes. But I'm not an expert on memory management, this is how I understand it and how xhci driver is implemented, and it seems to work. -Mathias -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Question about calculate trbs in xhci
From: vichy > hi Mathias: > > As I understood it, it is ok to cross page boundaries as long as the buffer > > is physically > contiguous. As our dma case should be. > > xhci specification section 3.2.8 says that: > > > > "Note that no constraints are placed on the TRB Length fields in a > > Scatter/Gather list. Classically > all the > > buffers pointed to by a scatter gather list were required to be “page size” > > in length except for the > first and > > last (as illustrated by the example above). The xHCI does not require this > > constraint. Any buffer > pointed to > > by a Normal, Data Stage, or Isoch TRB in a TD may be any size between 0 and > > 64K bytes in size. For > > instance, if when an OS translates a virtual memory buffer into a list of > > physical pages, some of > the entries > > in the list reference multiple contiguous pages, the flexible Length fields > > of TRBs allow a 1:1 > mapping of list > > entries to TRBs, i.e. a multi-page list entry does not need to be defined > > as multiple page sized > TRBs." > > " > Yes, you are correct. ^^ > there is no page boundary limitation for Scatter/Gather list, but how > about iso transfer? > for iso TRB, it seems a limitation for not crossing page boundary in > section 3.2.11. > > appreciate all your kind help, The important section is 4.11.7.1, if those requirements are also met the it will go wrong in obscure ways. David N�r��yb�X��ǧv�^�){.n�+{��^n�r���z���h�&���G���h�(�階�ݢj"���m��z�ޖ���f���h���~�m�
RE: Question about calculate trbs in xhci
From: Paul Zimmerman ... > > The xhi hardware doesn't have that constraint, but has the different > > constraints: > > 1) A buffer must not cross a 64k address boundary. > > 2) A LINK TRB (at the end of the ring) must only occur at a USB message > > boundary > >(actually it is a slightly tighter constraint). > > In combination those two are a PITA. > > Hi David, > > May I ask where in the xHCI spec is 2) stated? I'm not finding it > (looking at "Revision 1.0 with errata to 7/12/12"). > > I can't find "message boundary" anywhere in the spec. Did you mean packet > boundary? Yes... the ones that are 1kB for USB3 I can't remember what every protocol calls each data unit :-) > If so, there is some text in the spec which contradicts that. In 4.11.7.1, > it says: > > "In Figure 26 the IOC flag only may be set in TRBs 5 and 10 ... > Note that THE LINK TRB DOES NOT LAND ON A PACKET BOUNDARY relative > to the start of TD Fragment 1, so its IOC flag may not be set." Understanding section 4.11.7.1 is the problem here. It talks about the 'Max Burst Payload' (MBP) which is actually set to 16 packets (16kB). Ignore the references to IOB and status TRBs, the issue is the second bullet point on the first list 'A TD fragment shall not span Transfer Ring Segments'. In effect this means that a TD fragment will terminate at a LINK TRB and a new TD fragment will start in the new ring. Which is (more or less) equivalent to saying that the 'Chain bit' is ignored on the last TRB in the ring (for bulk data). So a link TRB must occur on a packet boundary in order for the target to treat the following TD fragment as part of the same bulk data message. David N�r��yb�X��ǧv�^�){.n�+{��^n�r���z���h�&���G���h�(�階�ݢj"���m��z�ޖ���f���h���~�m�
Re: Question about calculate trbs in xhci
hi Mathias: > As I understood it, it is ok to cross page boundaries as long as the buffer > is physically contiguous. As our dma case should be. > xhci specification section 3.2.8 says that: > > "Note that no constraints are placed on the TRB Length fields in a > Scatter/Gather list. Classically all the > buffers pointed to by a scatter gather list were required to be “page size” > in length except for the first and > last (as illustrated by the example above). The xHCI does not require this > constraint. Any buffer pointed to > by a Normal, Data Stage, or Isoch TRB in a TD may be any size between 0 and > 64K bytes in size. For > instance, if when an OS translates a virtual memory buffer into a list of > physical pages, some of the entries > in the list reference multiple contiguous pages, the flexible Length fields > of TRBs allow a 1:1 mapping of list > entries to TRBs, i.e. a multi-page list entry does not need to be defined as > multiple page sized TRBs." > " Yes, you are correct. ^^ there is no page boundary limitation for Scatter/Gather list, but how about iso transfer? for iso TRB, it seems a limitation for not crossing page boundary in section 3.2.11. appreciate all your kind help, -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Question about calculate trbs in xhci
> From: linux-usb-ow...@vger.kernel.org > [mailto:linux-usb-ow...@vger.kernel.org] On Behalf Of David Laight > Sent: Friday, October 03, 2014 6:08 AM > > From: Mathias Nyman > > On 10/01/2014 05:50 PM, vichy wrote: > > > hi Mathias: > > > > > >> > > >> In xhci-ring.c, static int xhci_queue_isoc_tx(): > > >> > > >> /* Calculate TRB length */ > > >> trb_buff_len = TRB_MAX_BUFF_SIZE - (addr & ((1 << TRB_MAX_BUFF_SHIFT) > > >> - 1)); > > >> > > >> where addr = start_addr + urb->iso_frame_desc[i].offset; > > >> > > >> this limits the trb_buff_len to stop the trb buffer at TRB_MAX_BUFF_SIZE > > >> boundary > > > Yes, you are right. > > > the above is make sure trb buffer will not cross over > > > TRB_MAX_BUFF_SIZE boundary. > > > But it still possible to cross the page boundary for ISO trb, right? > > > (suppose page boundary is 4KB, TRB_MAX_BUFF_SIZE is 64KB) > > > > > > appreciate your kind help, > > > > > > > Hi > > > > As I understood it, it is ok to cross page boundaries as long as the buffer > > is physically contiguous. > > As our dma case should be. > > xhci specification section 3.2.8 says that: > > > > "Note that no constraints are placed on the TRB Length fields in a > > Scatter/Gather list. Classically > > all the > > buffers pointed to by a scatter gather list were required to be “page size” > > in length except for the > > first and > > last (as illustrated by the example above). The xHCI does not require this > > constraint. Any buffer > > pointed to > > by a Normal, Data Stage, or Isoch TRB in a TD may be any size between 0 and > > 64K bytes in size. For > > instance, if when an OS translates a virtual memory buffer into a list of > > physical pages, some of the > > entries > > in the list reference multiple contiguous pages, the flexible Length fields > > of TRBs allow a 1:1 > > mapping of list > > entries to TRBs, i.e. a multi-page list entry does not need to be defined > > as multiple page sized > > TRBs." > > " > > The ohci hardware requires that scatter-gather boundaries occur on USB > message boundaries. > > The xhi hardware doesn't have that constraint, but has the different > constraints: > 1) A buffer must not cross a 64k address boundary. > 2) A LINK TRB (at the end of the ring) must only occur at a USB message > boundary >(actually it is a slightly tighter constraint). > In combination those two are a PITA. Hi David, May I ask where in the xHCI spec is 2) stated? I'm not finding it (looking at "Revision 1.0 with errata to 7/12/12"). I can't find "message boundary" anywhere in the spec. Did you mean packet boundary? If so, there is some text in the spec which contradicts that. In 4.11.7.1, it says: "In Figure 26 the IOC flag only may be set in TRBs 5 and 10 ... Note that THE LINK TRB DOES NOT LAND ON A PACKET BOUNDARY relative to the start of TD Fragment 1, so its IOC flag may not be set." (emphasis mine) -- Paul N�r��yb�X��ǧv�^�){.n�+{��^n�r���z���h�&���G���h�(�階�ݢj"���m��z�ޖ���f���h���~�m�
RE: Question about calculate trbs in xhci
From: Mathias Nyman > On 10/01/2014 05:50 PM, vichy wrote: > > hi Mathias: > > > >> > >> In xhci-ring.c, static int xhci_queue_isoc_tx(): > >> > >> /* Calculate TRB length */ > >> trb_buff_len = TRB_MAX_BUFF_SIZE - (addr & ((1 << TRB_MAX_BUFF_SHIFT) - > >> 1)); > >> > >> where addr = start_addr + urb->iso_frame_desc[i].offset; > >> > >> this limits the trb_buff_len to stop the trb buffer at TRB_MAX_BUFF_SIZE > >> boundary > > Yes, you are right. > > the above is make sure trb buffer will not cross over > > TRB_MAX_BUFF_SIZE boundary. > > But it still possible to cross the page boundary for ISO trb, right? > > (suppose page boundary is 4KB, TRB_MAX_BUFF_SIZE is 64KB) > > > > appreciate your kind help, > > > > Hi > > As I understood it, it is ok to cross page boundaries as long as the buffer > is physically contiguous. > As our dma case should be. > xhci specification section 3.2.8 says that: > > "Note that no constraints are placed on the TRB Length fields in a > Scatter/Gather list. Classically > all the > buffers pointed to by a scatter gather list were required to be “page size” > in length except for the > first and > last (as illustrated by the example above). The xHCI does not require this > constraint. Any buffer > pointed to > by a Normal, Data Stage, or Isoch TRB in a TD may be any size between 0 and > 64K bytes in size. For > instance, if when an OS translates a virtual memory buffer into a list of > physical pages, some of the > entries > in the list reference multiple contiguous pages, the flexible Length fields > of TRBs allow a 1:1 > mapping of list > entries to TRBs, i.e. a multi-page list entry does not need to be defined as > multiple page sized > TRBs." > " The ohci hardware requires that scatter-gather boundaries occur on USB message boundaries. The xhi hardware doesn't have that constraint, but has the different constraints: 1) A buffer must not cross a 64k address boundary. 2) A LINK TRB (at the end of the ring) must only occur at a USB message boundary (actually it is a slightly tighter constraint). In combination those two are a PITA. David N�r��yb�X��ǧv�^�){.n�+{��^n�r���z���h�&���G���h�(�階�ݢj"���m��z�ޖ���f���h���~�m�
Re: Question about calculate trbs in xhci
On 10/01/2014 05:50 PM, vichy wrote: > hi Mathias: > >> >> In xhci-ring.c, static int xhci_queue_isoc_tx(): >> >> /* Calculate TRB length */ >> trb_buff_len = TRB_MAX_BUFF_SIZE - (addr & ((1 << TRB_MAX_BUFF_SHIFT) - >> 1)); >> >> where addr = start_addr + urb->iso_frame_desc[i].offset; >> >> this limits the trb_buff_len to stop the trb buffer at TRB_MAX_BUFF_SIZE >> boundary > Yes, you are right. > the above is make sure trb buffer will not cross over > TRB_MAX_BUFF_SIZE boundary. > But it still possible to cross the page boundary for ISO trb, right? > (suppose page boundary is 4KB, TRB_MAX_BUFF_SIZE is 64KB) > > appreciate your kind help, > Hi As I understood it, it is ok to cross page boundaries as long as the buffer is physically contiguous. As our dma case should be. xhci specification section 3.2.8 says that: "Note that no constraints are placed on the TRB Length fields in a Scatter/Gather list. Classically all the buffers pointed to by a scatter gather list were required to be “page size” in length except for the first and last (as illustrated by the example above). The xHCI does not require this constraint. Any buffer pointed to by a Normal, Data Stage, or Isoch TRB in a TD may be any size between 0 and 64K bytes in size. For instance, if when an OS translates a virtual memory buffer into a list of physical pages, some of the entries in the list reference multiple contiguous pages, the flexible Length fields of TRBs allow a 1:1 mapping of list entries to TRBs, i.e. a multi-page list entry does not need to be defined as multiple page sized TRBs." " Same section defines a scatter/gather transfer as: "Transfers that are comprised of non-contiguous data (e.g. cross memory Page boundaries) are referred to as Scatter/Gather Transfers." -Mathias -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Question about calculate trbs in xhci
hi Mathias: > > In xhci-ring.c, static int xhci_queue_isoc_tx(): > > /* Calculate TRB length */ > trb_buff_len = TRB_MAX_BUFF_SIZE - (addr & ((1 << TRB_MAX_BUFF_SHIFT) - 1)); > > where addr = start_addr + urb->iso_frame_desc[i].offset; > > this limits the trb_buff_len to stop the trb buffer at TRB_MAX_BUFF_SIZE > boundary Yes, you are right. the above is make sure trb buffer will not cross over TRB_MAX_BUFF_SIZE boundary. But it still possible to cross the page boundary for ISO trb, right? (suppose page boundary is 4KB, TRB_MAX_BUFF_SIZE is 64KB) appreciate your kind help, -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Question about calculate trbs in xhci
From: vichy > hi David and All: > >> > Because the trb buffers can't cross a 64k physical address boundary. > >> > >> Is it only specific to iso scheduling or bulk also need to take care? > >> from your kind explanation, the trb buffer for Bulk also need to take > >> care crossing 64k physical boundary. > >> But i don't find it for bulk transfer. > >> Below is excerpted from xhci_queue_bulk_tx > >> while (running_total < urb->transfer_buffer_length) { > >> num_trbs++; > >> running_total += TRB_MAX_BUFF_SIZE; > >> } > > > > Hmmm... that looks like a division by a power of 2! > I am not quite understand what you mean "a division by a power of 2" > Would you please give me more hint? ^^ That code is approximately equal to: num_trbs += urb->transfer_buffer_length / TRB_MAX_BUFF_SIZE; running_total += urb->transfer_buffer_length & ~(TRB_MAX_BUFF_SIZE-1); David N�r��yb�X��ǧv�^�){.n�+{��^n�r���z���h�&���G���h�(�階�ݢj"���m��z�ޖ���f���h���~�m�
Re: Question about calculate trbs in xhci
On 10/01/2014 03:16 PM, vichy wrote: > hi David and All: Because the trb buffers can't cross a 64k physical address boundary. >>> >>> Is it only specific to iso scheduling or bulk also need to take care? >>> from your kind explanation, the trb buffer for Bulk also need to take >>> care crossing 64k physical boundary. >>> But i don't find it for bulk transfer. >>> Below is excerpted from xhci_queue_bulk_tx >>> while (running_total < urb->transfer_buffer_length) { >>> num_trbs++; >>> running_total += TRB_MAX_BUFF_SIZE; >>> } >> >> Hmmm... that looks like a division by a power of 2! > I am not quite understand what you mean "a division by a power of 2" > Would you please give me more hint? ^^ > >> The constraint applies to all TRB. > i found where bulk handle 64k physical address boundary as you explain. > it just a couple lines ahead of the source code I captured in the > previous mail :P > Meanwhile, I have another question: > for iso transfer, the first iso trb cannot cross page boundary, > but I cannot find this restriction in xhci driver. > > appreciate all your kind help, > -- In xhci-ring.c, static int xhci_queue_isoc_tx(): /* Calculate TRB length */ trb_buff_len = TRB_MAX_BUFF_SIZE - (addr & ((1 << TRB_MAX_BUFF_SHIFT) - 1)); where addr = start_addr + urb->iso_frame_desc[i].offset; this limits the trb_buff_len to stop the trb buffer at TRB_MAX_BUFF_SIZE boundary -Mathias -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Question about calculate trbs in xhci
On 10/01/2014 11:28 AM, vichy wrote: > hi all: > for iso transactions, we use below function to calculate trbs >addr = (u64) (urb->transfer_dma + urb->iso_frame_desc[i].offset); > td_len = urb->iso_frame_desc[i].length; > num_trbs = DIV_ROUND_UP(td_len + (addr & (TRB_MAX_BUFF_SIZE - 1)), > TRB_MAX_BUFF_SIZE); > > Could we use below method to calculate it instead? >addr = (u64) (urb->transfer_dma + urb->iso_frame_desc[i].offset); > td_len = urb->iso_frame_desc[i].length; > -num_trbs = DIV_ROUND_UP(td_len + (addr & (TRB_MAX_BUFF_SIZE - 1)), > TRB_MAX_BUFF_SIZE); > + num_trbs = DIV_ROUND_UP(td_len ,TRB_MAX_BUFF_SIZE); > > Why the trb calculation for iso is related to dma address? > > Appreciate your kind help in advance, > Because the TRB buffers must be aligned at TRB_MAX_BUFF_SIZE boundary, (except for the first TRB) If the starting addr is not aligned at this boundary it may cut the first TRB buffer short, and we need one extra TRB to cover the data |---|---|---|---|, | = TRB_MAX_BUFF_SIZE boundaries, in this example 8 "chars" per trb, |---|, d = data, 20 chars Without any alignment restrictions we would fit 20 data chars to 3 TRBs. But in this case we need 4 trbs, because the first TRB can only hold two d's before it hits a boundary. The two middle ones are full (8 d's per TRB), and last TRB has remaining 2 d's Alan's original explanation is here: http://marc.info/?l=linux-usb&m=131307343023488&w=2 -Mathias -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Question about calculate trbs in xhci
hi David and All: >> > Because the trb buffers can't cross a 64k physical address boundary. >> >> Is it only specific to iso scheduling or bulk also need to take care? >> from your kind explanation, the trb buffer for Bulk also need to take >> care crossing 64k physical boundary. >> But i don't find it for bulk transfer. >> Below is excerpted from xhci_queue_bulk_tx >> while (running_total < urb->transfer_buffer_length) { >> num_trbs++; >> running_total += TRB_MAX_BUFF_SIZE; >> } > > Hmmm... that looks like a division by a power of 2! I am not quite understand what you mean "a division by a power of 2" Would you please give me more hint? ^^ > The constraint applies to all TRB. i found where bulk handle 64k physical address boundary as you explain. it just a couple lines ahead of the source code I captured in the previous mail :P Meanwhile, I have another question: for iso transfer, the first iso trb cannot cross page boundary, but I cannot find this restriction in xhci driver. appreciate all your kind help, -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Question about calculate trbs in xhci
From: vichy [mailto:vichy@gmail.com] > hi David: > > > Because the trb buffers can't cross a 64k physical address boundary. > > Is it only specific to iso scheduling or bulk also need to take care? > from your kind explanation, the trb buffer for Bulk also need to take > care crossing 64k physical boundary. > But i don't find it for bulk transfer. > Below is excerpted from xhci_queue_bulk_tx > while (running_total < urb->transfer_buffer_length) { > num_trbs++; > running_total += TRB_MAX_BUFF_SIZE; > } Hmmm... that looks like a division by a power of 2! The constraint applies to all TRB. I've not looked at the code recently, it certainly does eventually split on 64k physical boundaries. It might not be detected until the entries are written to the ring, you don't really want to do all the work twice. David
Re: Question about calculate trbs in xhci
hi David: > Because the trb buffers can't cross a 64k physical address boundary. Is it only specific to iso scheduling or bulk also need to take care? from your kind explanation, the trb buffer for Bulk also need to take care crossing 64k physical boundary. But i don't find it for bulk transfer. Below is excerpted from xhci_queue_bulk_tx while (running_total < urb->transfer_buffer_length) { num_trbs++; running_total += TRB_MAX_BUFF_SIZE; } appreciate your kind help, -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Question about calculate trbs in xhci
From: vichy > hi all: > for iso transactions, we use below function to calculate trbs >addr = (u64) (urb->transfer_dma + urb->iso_frame_desc[i].offset); > td_len = urb->iso_frame_desc[i].length; > num_trbs = DIV_ROUND_UP(td_len + (addr & (TRB_MAX_BUFF_SIZE - 1)), > TRB_MAX_BUFF_SIZE); > > Could we use below method to calculate it instead? >addr = (u64) (urb->transfer_dma + urb->iso_frame_desc[i].offset); > td_len = urb->iso_frame_desc[i].length; > -num_trbs = DIV_ROUND_UP(td_len + (addr & (TRB_MAX_BUFF_SIZE - 1)), > TRB_MAX_BUFF_SIZE); > + num_trbs = DIV_ROUND_UP(td_len ,TRB_MAX_BUFF_SIZE); > > Why the trb calculation for iso is related to dma address? Because the trb buffers can't cross a 64k physical address boundary. Whichever hardware engineer decided to save a few gates on the address counter really made it hard work for the device driver writer. Modern logic has no problems generating counters that would be fast enough and wide enough. Even a constraint at 32bit boundaries would be a pain. David N�r��yb�X��ǧv�^�){.n�+{��^n�r���z���h�&���G���h�(�階�ݢj"���m��z�ޖ���f���h���~�m�
Question about calculate trbs in xhci
hi all: for iso transactions, we use below function to calculate trbs addr = (u64) (urb->transfer_dma + urb->iso_frame_desc[i].offset); td_len = urb->iso_frame_desc[i].length; num_trbs = DIV_ROUND_UP(td_len + (addr & (TRB_MAX_BUFF_SIZE - 1)), TRB_MAX_BUFF_SIZE); Could we use below method to calculate it instead? addr = (u64) (urb->transfer_dma + urb->iso_frame_desc[i].offset); td_len = urb->iso_frame_desc[i].length; -num_trbs = DIV_ROUND_UP(td_len + (addr & (TRB_MAX_BUFF_SIZE - 1)), TRB_MAX_BUFF_SIZE); + num_trbs = DIV_ROUND_UP(td_len ,TRB_MAX_BUFF_SIZE); Why the trb calculation for iso is related to dma address? Appreciate your kind help in advance, -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html