Hi Lalaji, In REQUESTHASHGenerator, we excluded Date and User-Agent headers when generating the request hash. If there are any other or custom headers which need to be excluded, we can write a new Hash Generator implementation by extending REQUESTHASHGenerator and override 'getDigest()' method. Then we can use that in the cache mediator.
Thanks, Lakmali On 22 October 2014 22:59, Lalaji Sureshika <lal...@wso2.com> wrote: > Hi, > > > On Fri, Dec 6, 2013 at 2:49 AM, Lakmali Baminiwatta <lakm...@wso2.com> > wrote: > >> Hi Sanjeewa, >> >> >> On 6 December 2013 07:48, Sanjeewa Malalgoda <sanje...@wso2.com> wrote: >> >>> >>> >>> >>> On Mon, Dec 2, 2013 at 6:23 PM, Lakmali Baminiwatta <lakm...@wso2.com> >>> wrote: >>> >>>> Hi, >>>> >>>> >>>> On 2 December 2013 18:02, Sumedha Rubasinghe <sume...@wso2.com> wrote: >>>> >>>>> >>>>> >>>>> >>>>> On Mon, Dec 2, 2013 at 5:24 PM, Lakmali Baminiwatta <lakm...@wso2.com> >>>>> wrote: >>>>> >>>>>> Hi Sanjeewa, >>>>>> >>>>>> >>>>>> On 2 December 2013 17:00, Sanjeewa Malalgoda <sanje...@wso2.com> >>>>>> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Sun, Dec 1, 2013 at 11:01 PM, Lakmali Baminiwatta < >>>>>>> lakm...@wso2.com> wrote: >>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> When processing the response, if the request is REST then soap >>>>>>>> format, transport headers and message type are also stored. Similarly >>>>>>>> when >>>>>>>> retrieving the cached response, if the request is REST then stored >>>>>>>> values >>>>>>>> are used to compose the response message. With these modifications >>>>>>>> cache >>>>>>>> mediator returns the response correctly for non SOAP requests. >>>>>>>> >>>>>>>> But we have another issue here. Currently the request Hash value is >>>>>>>> derived from the SOAP message body. So for REST calls it gets the same >>>>>>>> hash >>>>>>>> value for all requests which sent without a payload. >>>>>>>> >>>>>>>> ex: Both below requests are recognized as the same request >>>>>>>> according to the hash value derived from DOMHASHGenerator. >>>>>>>> >>>>>>>> curl -v -H "Authorization: Bearer 1EV6Qqa_DboaBzNj2JfiyXoO1Ysa" >>>>>>>> http://localhost:8280/test/1.0.0?regNo=001 >>>>>>>> curl -v -H "Authorization: Bearer 1EV6Qqa_DboaBzNj2JfiyXoO1Ysa" >>>>>>>> http://localhost:8280/test/1.0.0?regNo=002 >>>>>>>> <http://localhost:8280/test/1.0.0?regNo=001> >>>>>>>> >>>>>>>> So right now we need to figure out a mechanism to generate hash >>>>>>>> value for REST calls. >>>>>>>> >>>>>>> How if we consider full request path(including query params and etc) >>>>>>> + headers. For above example something like follows (full request path >>>>>>> and >>>>>>> "," separated transport headers list). We can compute hash value of that >>>>>>> string and store it. >>>>>>> >>>>>>> *"http://localhost:8280/test/1.0.0?regNo=002,Authorization >>>>>>> <http://localhost:8280/test/1.0.0?regNo=002,Authorization>: Bearer >>>>>>> 1EV6Qqa_DboaBzNj2JfiyXoO1Ysa"* >>>>>>> >>>>>>> Here we cannot only consider full request path as we take some >>>>>>> decisions at backend based on headers. WDYT? >>>>>>> >>>>>> >>>>>> Yeah, need to consider request path + headers. But I think we don't >>>>>> need hash Authorization headers, as request hits the cache mediator after >>>>>> going through the handlers. So in default case, at the time of hashing we >>>>>> don't have the authorization header. So AFAIU, the only concern which >>>>>> comes >>>>>> up here is, same request invoked using different keys may get the >>>>>> response >>>>>> from the cache. I believe that is ok. WDYT? >>>>>> >>>>> >>>>> >>>>> We simply store response values against a given request pattern. But >>>>> Authorisation header should not be part of it. Otherwise it will become a >>>>> token based cache. But are whole lot of other header parameter like >>>>> content-type, accept header, accept language ( >>>>> http://en.wikipedia.org/wiki/List_of_HTTP_header_fields). >>>>> >>>>> Let's ignore Authorisation header for time being. >>>>> >>>>> +1. So for now let's generate the hash based on request path, Accept & >>>> Content-Type headers and payload (if available). >>>> >>> +1. IMO we should use all other headers except authentication headers >>> (Ex. sometimes response may depend on client type or sometimes it may >>> depend on any other custom header). >>> >> >> We can use all the headers [1]. But there are some headers which seems >> not correct to use for hashing the request. >> >> ex: >> Date - This is unique for each request. If this is used for generting the >> hash, cache will be never used. >> User-Agent - This is specific to the user client/browser who sent the >> request. So different caches for different users. >> >> So I think we need to figure out which headers to skip or which headers >> to use for hash generation. >> > > Did we consider , above unique request headers to include in hashing > the request or are we generating the cache digest based on all transport > headers..? > >> >> >> [1]http://en.wikipedia.org/wiki/List_of_HTTP_header_fields >> >> Thanks, >> Lakmali >> >>> >>> Thanks, >>> sanjeewa. >>> >>>> >>>> Thanks, >>>> Lakmali >>>> >>>>> >>>>> I am writing a new Digest Generator implementation by extending >>>>> DOMHASHGenerator. This basically does what DOMHASHGenerator had been doing >>>>> and additionally it digests request path and available transport headers >>>>> as >>>>> well. >>>>> >>>>> Thanks, >>>>> Lakmali >>>>> >>>>>> >>>>>> Thanks, >>>>>> sanjeewa. >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Lakmali >>>>>>> >>>>>>> >>>>>>> On 29 November 2013 00:18, Lakmali Baminiwatta <lakm...@wso2.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> Currently Cache mediator stores the response message as a byte >>>>>>>> array and builds the soap envelope from that when sending the cached >>>>>>>> response[1]. When building the response, it checks whether the request >>>>>>>> format is in SOAP11 or SOAP12 and do the build accordingly. >>>>>>>> >>>>>>>> For SOAP requests, the SOAP format for response and request >>>>>>>> messages are same since most of the time a WSDL is used. So above logic >>>>>>>> works. >>>>>>>> >>>>>>>> But the response message for REST calls may be in different SOAP >>>>>>>> formats than the request. This results in errors when building the >>>>>>>> message. >>>>>>>> >>>>>>>> ex: >>>>>>>> >>>>>>>> Below REST call's request message context is SOAP12. But the >>>>>>>> response SOAP envelope is SOAP11. Hence building the message throws an >>>>>>>> exception. >>>>>>>> >>>>>>>> curl -v -H "Authorization: Bearer 1EV6Qqa_DboaBzNj2JfiyXoO1Ysa" >>>>>>>> http://localhost:8280/test/1.0.0?regNo=001 >>>>>>>> >>>>>>>> Further Content-Type of the responses can be different (ex: >>>>>>>> application/xml, application/json). So I think we should store the >>>>>>>> actual >>>>>>>> SOAP format and the Content-Type of the response than just storing the >>>>>>>> response envelope. One option would be to store them in >>>>>>>> CachableResponse >>>>>>>> object[2] (May be in a property map?) >>>>>>>> >>>>>>>> Any better way to fix this? >>>>>>>> >>>>>>>> [1] >>>>>>>> https://svn.wso2.org/repos/wso2/carbon/platform/branches/turing/dependencies/synapse/2.1.2-wso2v2/modules/core/src/main/java/org/apache/synapse/mediators/builtin/CacheMediator.java >>>>>>>> [2] >>>>>>>> https://svn.wso2.org/repos/wso2/carbon/platform/branches/turing/dependencies/commons/caching/4.0.2/modules/core/src/main/java/org/wso2/caching/CachableResponse.java >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Lakmali >>>>>>>> >>>>>>>> >>>>>>>> On 22 November 2013 15:59, Sumedha Rubasinghe <sume...@wso2.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Dinusha, >>>>>>>>> Did you identify what needs to be fixed? I think ESB team is busy >>>>>>>>> with a release. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Nov 21, 2013 at 7:09 PM, Dinusha Senanayaka < >>>>>>>>> dinu...@wso2.com> wrote: >>>>>>>>> >>>>>>>>>> Hi All, >>>>>>>>>> >>>>>>>>>> We are trying to add response caching as a default feature in API >>>>>>>>>> Manager using Cache mediator. There, we found that Cache mediator >>>>>>>>>> could >>>>>>>>>> handle only SOAP-1.1 or SOAP-1.2 messages. >>>>>>>>>> >>>>>>>>>> - If backend returns a POX response, it throws the following >>>>>>>>>> exception [1] when request is served from the cache. >>>>>>>>>> - If backend returns a JSON response, and once the response get >>>>>>>>>> added to cache, client does not receive a response until cache >>>>>>>>>> expires. (No >>>>>>>>>> backend errors) >>>>>>>>>> >>>>>>>>>> Is it possible to fix those for API Manager-1.6.0 release ? >>>>>>>>>> >>>>>>>>>> [1] [2013-11-21 13:33:18,044] ERROR - NativeWorkerPool Uncaught >>>>>>>>>> exception >>>>>>>>>> org.apache.axiom.soap.SOAPProcessingException: Transport level >>>>>>>>>> information does not match with SOAP Message namespace URI >>>>>>>>>> at >>>>>>>>>> org.apache.axiom.soap.impl.builder.StAXSOAPModelBuilder.identifySOAPVersion(StAXSOAPModelBuilder.java:187) >>>>>>>>>> at >>>>>>>>>> org.apache.axiom.soap.impl.builder.StAXSOAPModelBuilder.<init>(StAXSOAPModelBuilder.java:170) >>>>>>>>>> at >>>>>>>>>> org.apache.axis2.saaj.SOAPPartImpl.<init>(SOAPPartImpl.java:194) >>>>>>>>>> at >>>>>>>>>> org.apache.axis2.saaj.SOAPMessageImpl.<init>(SOAPMessageImpl.java:112) >>>>>>>>>> at >>>>>>>>>> org.apache.axis2.saaj.MessageFactoryImpl.createMessage(MessageFactoryImpl.java:132) >>>>>>>>>> at >>>>>>>>>> org.wso2.caching.util.SOAPMessageHelper.buildSOAPEnvelopeFromBytes(SOAPMessageHelper.java:77) >>>>>>>>>> at >>>>>>>>>> org.apache.synapse.mediators.builtin.CacheMediator.processRequestMessage(CacheMediator.java:319) >>>>>>>>>> at >>>>>>>>>> org.apache.synapse.mediators.builtin.CacheMediator.mediate(CacheMediator.java:170) >>>>>>>>>> at >>>>>>>>>> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:77) >>>>>>>>>> at >>>>>>>>>> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:47) >>>>>>>>>> at >>>>>>>>>> org.apache.synapse.mediators.filters.FilterMediator.mediate(FilterMediator.java:160) >>>>>>>>>> at >>>>>>>>>> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:77) >>>>>>>>>> at >>>>>>>>>> org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:47) >>>>>>>>>> at >>>>>>>>>> org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:131) >>>>>>>>>> at org.apache.synapse.rest.Resource.process(Resource.java:297) >>>>>>>>>> at org.apache.synapse.rest.API.process(API.java:341) >>>>>>>>>> at >>>>>>>>>> org.apache.synapse.rest.RESTRequestHandler.dispatchToAPI(RESTRequestHandler.java:76) >>>>>>>>>> at >>>>>>>>>> org.apache.synapse.rest.RESTRequestHandler.process(RESTRequestHandler.java:63) >>>>>>>>>> at >>>>>>>>>> org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:220) >>>>>>>>>> at >>>>>>>>>> org.apache.synapse.core.axis2.SynapseMessageReceiver.receive(SynapseMessageReceiver.java:83) >>>>>>>>>> at >>>>>>>>>> org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180) >>>>>>>>>> at >>>>>>>>>> org.apache.synapse.transport.passthru.ServerWorker.processNonEntityEnclosingRESTHandler(ServerWorker.java:337) >>>>>>>>>> at >>>>>>>>>> org.apache.synapse.transport.passthru.ServerWorker.processEntityEnclosingRequest(ServerWorker.java:378) >>>>>>>>>> at >>>>>>>>>> org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:184) >>>>>>>>>> at >>>>>>>>>> org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>>>>>>>>> at java.lang.Thread.run(Thread.java:619) >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> Dinusha. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Dinusha Dilrukshi >>>>>>>>>> Senior Software Engineer >>>>>>>>>> WSO2 Inc.: http://wso2.com/ >>>>>>>>>> Mobile: +94725255071 >>>>>>>>>> Blog: http://dinushasblog.blogspot.com/ >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> /sumedha >>>>>>>>> m: +94 773017743 >>>>>>>>> b : bit.ly/sumedha >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Dev mailing list >>>>>>>>> Dev@wso2.org >>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Lakmali Baminiwatta >>>>>>>> Software Engineer >>>>>>>> WSO2, Inc.: http://wso2.com >>>>>>>> lean.enterprise.middleware >>>>>>>> mobile: +94 71 2335936 >>>>>>>> blog : lakmali.com >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Lakmali Baminiwatta >>>>>>> Software Engineer >>>>>>> WSO2, Inc.: http://wso2.com >>>>>>> lean.enterprise.middleware >>>>>>> mobile: +94 71 2335936 >>>>>>> blog : lakmali.com >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> Dev@wso2.org >>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> *Sanjeewa Malalgoda* >>>>>> Senior Software Engineer >>>>>> WSO2 Inc. >>>>>> Mobile : +94713068779 >>>>>> >>>>>> <http://sanjeewamalalgoda.blogspot.com/>blog >>>>>> :http://sanjeewamalalgoda.blogspot.com/ >>>>>> <http://sanjeewamalalgoda.blogspot.com/> >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Lakmali Baminiwatta >>>>> Software Engineer >>>>> WSO2, Inc.: http://wso2.com >>>>> lean.enterprise.middleware >>>>> mobile: +94 71 2335936 >>>>> blog : lakmali.com >>>>> >>>>> >>>>> _______________________________________________ >>>>> Dev mailing list >>>>> Dev@wso2.org >>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> /sumedha >>>>> b : bit.ly/sumedha >>>>> >>>> >>>> >>>> >>>> -- >>>> Lakmali Baminiwatta >>>> Software Engineer >>>> WSO2, Inc.: http://wso2.com >>>> lean.enterprise.middleware >>>> mobile: +94 71 2335936 >>>> blog : lakmali.com >>>> >>>> >>> >>> >>> -- >>> >>> *Sanjeewa Malalgoda* >>> Senior Software Engineer >>> WSO2 Inc. >>> Mobile : +94713068779 >>> >>> <http://sanjeewamalalgoda.blogspot.com/>blog >>> :http://sanjeewamalalgoda.blogspot.com/ >>> <http://sanjeewamalalgoda.blogspot.com/> >>> >>> >>> >> >> >> -- >> Lakmali Baminiwatta >> Software Engineer >> WSO2, Inc.: http://wso2.com >> lean.enterprise.middleware >> mobile: +94 71 2335936 >> blog : lakmali.com >> >> >> _______________________________________________ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > Lalaji Sureshika > WSO2, Inc.; http://wso2.com/ > email: lal...@wso2.com; cell: +94 71 608 6811 > blog: http://lalajisureshika.blogspot.com > > > -- Lakmali Baminiwatta Senior Software Engineer WSO2, Inc.: http://wso2.com lean.enterprise.middleware mobile: +94 71 2335936 blog : lakmali.com
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
