>-----Original Message----- >From: Ahmed Mansour [mailto:[email protected]] >Sent: 14 February 2018 22:25 >To: Verma, Shally <[email protected]>; Trahe, Fiona ><[email protected]>; [email protected] >Cc: Athreya, Narayana Prasad <[email protected]>; Gupta, >Ashish <[email protected]>; Sahu, Sunila ><[email protected]>; De Lara Guarch, Pablo ><[email protected]>; Challa, Mahipal ><[email protected]>; Jain, Deepak K <[email protected]>; Hemant >Agrawal <[email protected]>; Roy >Pledge <[email protected]>; Youri Querry <[email protected]> >Subject: Re: [RFC v2] doc compression API for DPDK > >On 2/14/2018 12:41 AM, Verma, Shally wrote: >> Hi Ahmed >> >>> -----Original Message----- >>> From: Ahmed Mansour [mailto:[email protected]] >>> Sent: 02 February 2018 02:20 >>> To: Trahe, Fiona <[email protected]>; Verma, Shally >>> <[email protected]>; [email protected] >>> Cc: Athreya, Narayana Prasad <[email protected]>; Gupta, >>> Ashish <[email protected]>; Sahu, Sunila >>> <[email protected]>; De Lara Guarch, Pablo >>> <[email protected]>; Challa, Mahipal >>> <[email protected]>; Jain, Deepak K <[email protected]>; >>> Hemant Agrawal <[email protected]>; Roy >>> Pledge <[email protected]>; Youri Querry <[email protected]> >>> Subject: Re: [RFC v2] doc compression API for DPDK >>> >>>>>> [Fiona] I propose if BFINAL bit is detected before end of input >>>>>> the decompression should stop. In this case consumed will be < >>>>>> src.length. >>>>>> produced will be < dst buffer size. Do we need an extra STATUS response? >>>>>> STATUS_BFINAL_DETECTED ? >>>>> [Shally] @fiona, I assume you mean here decompressor stop after >>>>> processing Final block right? >>>> [Fiona] Yes. >>>> >>>> And if yes, >>>>> and if it can process that final block successfully/unsuccessfully, then >>>>> status could simply be >>>>> SUCCESS/FAILED. >>>>> I don't see need of specific return code for this use case. Just to >>>>> share, in past, we have practically run into >>>>> such cases with boost lib, and decompressor has simply worked this way. >>>> [Fiona] I'm ok with this. >>>> >>>>>> Only thing I don't like this is it can impact on performance, as normally >>>>>> we can just look for STATUS == SUCCESS. Anything else should be an >>>>>> exception. >>>>>> Now the application would have to check for SUCCESS || BFINAL_DETECTED >>>>>> every time. >>>>>> Do you have a suggestion on how we should handle this? >>>>>> >>> [Ahmed] This makes sense. So in all cases the PMD should assume that it >>> should stop as soon as a BFINAL is observed. >>> >>> A question. What happens ins stateful vs stateless modes when >>> decompressing an op that encompasses multiple BFINALs. I assume the >>> caller in that case will use the consumed=x bytes to find out how far in >>> to the input is the end of the first stream and start from the next >>> byte. Is this correct? >> [Shally] As per my understanding, each op can be tied up to only one stream >> as we have only one stream pointer per op and one >stream can have only one BFINAL (as stream is one complete compressed data) >but looks like you're suggesting a case where one op >can carry multiple independent streams? and thus multiple BFINAL?! , such as, >below here is op pointing to more than one streams >> >> -------------------------------------------- >> op --> |stream1|stream2| |stream3| >> -------------------------------------------- >> >> Could you confirm if I understand your question correct? >[Ahmed] Correct. We found that in some storage applications the user >does not know where exactly the BFINAL is. They rely on zlib software >today. zlib.net software halts at the first BFINAL. Users put multiple >streams in one op and rely on zlib to stop and inform them of the end >location of the first stream.
[Shally] Then this is practically case possible on decompressor and decompressor doesn't regard flush flag. So in that case, I expect PMD to internally reset themselves (say in case of zlib going through cycle of deflateEnd and deflateInit or deflateReset) and return with status = SUCCESS with updated produced and consumed. Now in such case, if previous stream also has some footer followed by start of next stream, then I am not sure how PMD / lib can support that case. Have you had practically run of such use-case on zlib? If yes, how then such application handle it in your experience? I can imagine for such input zlib would return with Z_FLUSH_END after 1st BFINAL is processed to the user. Then application doing deflateReset() or Init-End() cycle before starting with next. But if it starts with input that doesn't have valid zlib header, then likely it will throw an error. >> >> Thanks >> Shally >>
