HI Fiona So I understand we're moving away from mbufs because of its size limitation (64k) and cacheline overhead and their more suitability to n/w applications. Given that, I understand benefit of having another structure to input data but then what is proposal for ipcomp like application where mbuf usage may be a better option? Should we keep support for both (mbuf and this structure) so that apps can use appropriate data structure depending on their requirement.
Further comments, on github. Thanks Shally >-----Original Message----- >From: Trahe, Fiona [mailto:[email protected]] >Sent: 12 March 2018 21:31 >To: Ahmed Mansour <[email protected]>; Verma, Shally ><[email protected]>; [email protected] >Cc: De Lara Guarch, Pablo <[email protected]>; Athreya, Narayana >Prasad <[email protected]>; >Gupta, Ashish <[email protected]>; Sahu, Sunila ><[email protected]>; Challa, Mahipal ><[email protected]>; Jain, Deepak K <[email protected]>; Hemant >Agrawal <[email protected]>; Roy >Pledge <[email protected]>; Youri Querry <[email protected]>; >[email protected]; Daly, Lee <[email protected]>; >Jozwiak, TomaszX <[email protected]> >Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative > >Hi Shally, Ahmed, and anyone else interested in compressdev, > >I mentioned last week that we've been exploring using something other than >mbufs to pass src/dst buffers to compressdev PMDs. > >Reasons: > - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data for > compression > can be greater and it would add cycles to have to break up into smaller > segments. > - data may originate in mbufs, but is more likely, particularly for storage > use-cases, to > originate in other data structures. > - There's a 2 cache-line overhead for every segment in a chain, most of this > data > is network-related, not needed by compressdev >So moving to a custom structure would minimise memory overhead, remove >restriction on 64k-1 size and give more flexibility if >compressdev ever needs any comp-specific meta-data. > >We've come up with a compressdev-specific structure using the struct iovec >from sys/uio.h, which is commonly used by storage >applications. This would replace the src and dest mbufs in the op. >I'll not include the code here - Pablo will push that to github shortly and >we'd appreciate review comments there. >https://github.com/pablodelara/dpdk-draft-compressdev >Just posting on the mailing list to give a heads-up and ensure this reaches a >wider audience than may see it on github. > >Note : We also considered having no data structures in the op, instead the >application >would supply a callback which the PMD would use to retrieve meta-data (virt >address, iova, length) >for each next segment as needed. While this is quite flexible and allow the >application >to keep its data in its native structures, it's likely to cost more cycles. >So we're not proposing this at the moment, but hope to benchmark it later >while the API is still experimental. > >General feedback on direction is welcome here on the mailing list. >For feedback on the details of implementation we would appreciate comments on >github. > >Regards, >Fiona.
