On Wed, 2019-06-12 at 09:25 +0300, Oded Gabbay wrote: > > > You can't. Your device is broken. Devices that don't support DMAing to > > the full 64-bit deserve to be added to the trash pile. > > > > Hmm... right know they are added to customers data-centers but what do I know > ;)
Well, some customers don't know they are being sold a lemon :) > > As a result, getting it to work will require hacks. Some GPUs have > > similar issues and require similar hacks, it's unfortunate. > > > > Added a couple of guys on CC who might be able to help get those hacks > > right. > > Thanks :) > > > > It's still very fishy .. the idea is to detect the case where setting a > > 64-bit mask will give your system memory mapped at a fixed high address > > (1 << 59 in our case) and program that in your chip in the "Fixed high > > bits" register that you seem to have (also make sure it doesn't affect > > MSIs or it will break them). > > MSI-X are working. The set of bit 59 doesn't apply to MSI-X > transactions (AFAICS from the PCIe controller spec we have). Ok. > > This will only work as long as all of the system memory can be > > addressed at an offset from that fixed address that itself fits your > > device addressing capabilities (50 bits in this case). It may or may > > not be the case but there's no way to check since the DMA mask logic > > won't really apply. > > Understood. In the specific system we are integrated to, that is the > case - we have less then 48 bits. But, as you pointed out, it is not a > generic solution but with my H/W I can't give a generic fit-all > solution for POWER9. I'll settle for the best that I can do. > > > > > You might want to consider fixing your HW in the next iteration... This > > is going to bite you when x86 increases the max physical memory for > > example, or on other architectures. > > Understood and taken care of. Cheers, Ben. > > > > Cheers, > > Ben. > > > > > > > >
