Re: How to get stream from CachedStream ?
Hi thanks, well, I guess there is some specific code sequence there, when ReplaceOutInterceptor2 is run, which exposes a bug. As far as I know CachedOutputStream was created to support some typical request/response situations and I'm not sure it was ever meant to be used publicly - it is indeed possible that the reason you see it failing in ReplaceOutInterceptor2 is that something is not called or called in the wrong order. Please debug a working WriterInterceptor sequence and mark how CachedOutputStream is used under the hood, method call by method call. That may help identify why ReplaceOutInterceptor2 is failing Cheers, Sergey On 14/10/16 07:32, Vjacheslav V. Borisov wrote: Hi! Please see https://github.com/slavb18/cachedstreamtest There is 2 interceptors My first variant based on AbstractPhaseInterceptor https://github.com/slavb18/cachedstreamtest/blob/master/src/main/java/ru/ilb/common/jaxrs/interceptors/ReplaceOutInterceptor2.java is not working with large payloads: this returns empty string curl http://localhost:8080/cachedstreamtest/web/test?limit=10 then i rewrite it to WriterInterceptor as you suggested: https://github.com/slavb18/cachedstreamtest/blob/master/src/main/java/ru/ilb/common/jaxrs/interceptors/ReplaceOutInterceptor.java and, when i write this test, surprisingly, CachedStream working here So may be i just found bad example in first place. Don't now how it is related to CachedStream in detail There is both two beans in this test https://github.com/slavb18/cachedstreamtest/blob/master/src/main/webapp/WEB-INF/beans.xml with working one commented 2016-10-13 17:11 GMT+04:00 Sergey Beryozkin : Hi So far we have not seen any side-effects and this class is used a lot. I guess there is indeed some code in that class which is problematic. Can you please consider creating a simple test which does an output stream replacement from the JAX-RS filter and hits this problem ? Thanks, Sergey On 08/10/16 20:19, Vjacheslav V. Borisov wrote: I have used this class istead of CachedOutputStream https://www.whitebyte.info/programming/java/convert-a-bytear rayoutputstream-into-a-bytearrayinputstream and also rewirte interceptor to WriterInterceptor interface, now my ReplaceOutInterceptor is working fine Thanks! 2016-10-08 1:14 GMT+04:00 Vjacheslav V. Borisov : Hi! I see WriterInterceptorContext could be used, but it also requires CachedOutputStream to intercept OutputStream, I see this in JweJsonWriterInterceptor... So CachedOutputStream bug will be repeated in WriterInterceptorContext 2016-09-20 13:13 GMT+04:00 Sergey Beryozkin : Hi, JAX-RS ContainerRequestContext and ContainerResponseContext, as well as WriterInterceptorContext and ReaderInterceptorContext filter contexts all allow resetting the entity streams or on the way out - entity objects - perhaps that can offer a simpler option for replacing the content... Cheers, Sergey On 19/09/16 11:30, Vjacheslav V. Borisov wrote: During message.getInterceptorChain().doIntercept(message); I see call to maybeDeleteTempFile() which resets inmem = true; and later csnew.getInputStream() execution leads to if (inmem) { } 2016-09-19 13:05 GMT+04:00 Vjacheslav V. Borisov : Hi! I have following code (this is interceptror which does regular expression replacements) https://github.com/ilb/common/blob/master/common-jaxrs/src/ main/java/ru/ilb/common/jaxrs/interceptors/ReplaceOutInterceptor.java private void replaceContents(Message message) { OutputStream os = message.getContent(OutputStream.class); CachedStream cs = new CachedStream(); message.setContent(OutputStream.class, cs); message.getInterceptorChain().doIntercept(message); try { cs.flush(); CachedOutputStream csnew = (CachedOutputStream) message.getContent(OutputStream.class); String contents = IOUtils.toString(csnew.getInpu tStream()); for (Map.Entry keyValue : replacements.entrySet()) { contents = contents.replaceAll(keyValue.getKey(), keyValue.getValue()); } //String replaced=contents.replaceAll(regex, replacement); os.write(contents.getBytes(Charset.forName(encoding))); os.flush(); message.setContent(OutputStream.class, os); } catch (IOException ioe) { throw new RuntimeException(ioe); } } Howewer, when response contents length of if less than threshold (128 * 1024 bytes) i can get csnew.getInputStream() and get its contents, but when content is more than treshold, i see that in enforceLimits() -> createFileOutputStream() temp file is created, but using csnew.getInputStream() I am getting empty stream. -- Sergey Beryozkin Talend Community Coders http://coders.talend.com/ -- Sergey Beryozkin Talend Community Coders http://coders.talend.com/ -- Sergey Beryozkin Talend Community Coders http://coders.talend.com/
Re: How to get stream from CachedStream ?
Hi! Please see https://github.com/slavb18/cachedstreamtest There is 2 interceptors My first variant based on AbstractPhaseInterceptor https://github.com/slavb18/cachedstreamtest/blob/master/src/main/java/ru/ilb/common/jaxrs/interceptors/ReplaceOutInterceptor2.java is not working with large payloads: this returns empty string curl http://localhost:8080/cachedstreamtest/web/test?limit=10 then i rewrite it to WriterInterceptor as you suggested: https://github.com/slavb18/cachedstreamtest/blob/master/src/main/java/ru/ilb/common/jaxrs/interceptors/ReplaceOutInterceptor.java and, when i write this test, surprisingly, CachedStream working here So may be i just found bad example in first place. Don't now how it is related to CachedStream in detail There is both two beans in this test https://github.com/slavb18/cachedstreamtest/blob/master/src/main/webapp/WEB-INF/beans.xml with working one commented 2016-10-13 17:11 GMT+04:00 Sergey Beryozkin : > Hi > > So far we have not seen any side-effects and this class is used a lot. I > guess there is indeed some code in that class which is problematic. > > Can you please consider creating a simple test which does an output stream > replacement from the JAX-RS filter and hits this problem ? > > Thanks, Sergey > > On 08/10/16 20:19, Vjacheslav V. Borisov wrote: > >> I have used this class istead of CachedOutputStream >> https://www.whitebyte.info/programming/java/convert-a-bytear >> rayoutputstream-into-a-bytearrayinputstream >> and also rewirte interceptor to WriterInterceptor interface, now my >> ReplaceOutInterceptor is working fine >> Thanks! >> >> 2016-10-08 1:14 GMT+04:00 Vjacheslav V. Borisov : >> >> Hi! >>> >>> I see WriterInterceptorContext could be used, but it also requires >>> CachedOutputStream to intercept OutputStream, I see this in >>> JweJsonWriterInterceptor... >>> So CachedOutputStream bug will be repeated in WriterInterceptorContext >>> >>> >>> >>> >>> 2016-09-20 13:13 GMT+04:00 Sergey Beryozkin : >>> >>> Hi, JAX-RS ContainerRequestContext and ContainerResponseContext, as well as WriterInterceptorContext and ReaderInterceptorContext filter contexts all allow resetting the entity streams or on the way out - entity objects - perhaps that can offer a simpler option for replacing the content... Cheers, Sergey On 19/09/16 11:30, Vjacheslav V. Borisov wrote: During message.getInterceptorChain().doIntercept(message); I see call > to > maybeDeleteTempFile() which resets inmem = true; > and later csnew.getInputStream() execution leads to if (inmem) { } > > 2016-09-19 13:05 GMT+04:00 Vjacheslav V. Borisov : > > Hi! > >> >> I have following code (this is interceptror which does regular >> expression >> replacements) >> >> >> https://github.com/ilb/common/blob/master/common-jaxrs/src/ >> main/java/ru/ilb/common/jaxrs/interceptors/ReplaceOutInterceptor.java >> >> private void replaceContents(Message message) { >> OutputStream os = message.getContent(OutputStream.class); >> CachedStream cs = new CachedStream(); >> message.setContent(OutputStream.class, cs); >> >> message.getInterceptorChain().doIntercept(message); >> >> try { >> cs.flush(); >> CachedOutputStream csnew = (CachedOutputStream) >> message.getContent(OutputStream.class); >> >> String contents = IOUtils.toString(csnew.getInpu >> tStream()); >> for (Map.Entry keyValue : >> replacements.entrySet()) { >> contents = contents.replaceAll(keyValue.getKey(), >> keyValue.getValue()); >> } >> //String replaced=contents.replaceAll(regex, >> replacement); >> os.write(contents.getBytes(Charset.forName(encoding))); >> os.flush(); >> >> message.setContent(OutputStream.class, os); >> >> } catch (IOException ioe) { >> throw new RuntimeException(ioe); >> } >> } >> >> >> >> Howewer, when response contents length of if less than threshold >> (128 * >> 1024 bytes) >> i can get csnew.getInputStream() and get its contents, but when >> content >> is >> more than treshold, >> i see that in enforceLimits() -> createFileOutputStream() temp file is >> created, >> but using csnew.getInputStream() I am getting empty stream. >> >> >> > -- Sergey Beryozkin Talend Community Coders http://coders.talend.com/ >>> >>> >> > > -- > Sergey Beryozkin > > Talend Community Coders > http://coders.talend.com/ >
Re: How to get stream from CachedStream ?
Hi So far we have not seen any side-effects and this class is used a lot. I guess there is indeed some code in that class which is problematic. Can you please consider creating a simple test which does an output stream replacement from the JAX-RS filter and hits this problem ? Thanks, Sergey On 08/10/16 20:19, Vjacheslav V. Borisov wrote: I have used this class istead of CachedOutputStream https://www.whitebyte.info/programming/java/convert-a-bytearrayoutputstream-into-a-bytearrayinputstream and also rewirte interceptor to WriterInterceptor interface, now my ReplaceOutInterceptor is working fine Thanks! 2016-10-08 1:14 GMT+04:00 Vjacheslav V. Borisov : Hi! I see WriterInterceptorContext could be used, but it also requires CachedOutputStream to intercept OutputStream, I see this in JweJsonWriterInterceptor... So CachedOutputStream bug will be repeated in WriterInterceptorContext 2016-09-20 13:13 GMT+04:00 Sergey Beryozkin : Hi, JAX-RS ContainerRequestContext and ContainerResponseContext, as well as WriterInterceptorContext and ReaderInterceptorContext filter contexts all allow resetting the entity streams or on the way out - entity objects - perhaps that can offer a simpler option for replacing the content... Cheers, Sergey On 19/09/16 11:30, Vjacheslav V. Borisov wrote: During message.getInterceptorChain().doIntercept(message); I see call to maybeDeleteTempFile() which resets inmem = true; and later csnew.getInputStream() execution leads to if (inmem) { } 2016-09-19 13:05 GMT+04:00 Vjacheslav V. Borisov : Hi! I have following code (this is interceptror which does regular expression replacements) https://github.com/ilb/common/blob/master/common-jaxrs/src/ main/java/ru/ilb/common/jaxrs/interceptors/ReplaceOutInterceptor.java private void replaceContents(Message message) { OutputStream os = message.getContent(OutputStream.class); CachedStream cs = new CachedStream(); message.setContent(OutputStream.class, cs); message.getInterceptorChain().doIntercept(message); try { cs.flush(); CachedOutputStream csnew = (CachedOutputStream) message.getContent(OutputStream.class); String contents = IOUtils.toString(csnew.getInputStream()); for (Map.Entry keyValue : replacements.entrySet()) { contents = contents.replaceAll(keyValue.getKey(), keyValue.getValue()); } //String replaced=contents.replaceAll(regex, replacement); os.write(contents.getBytes(Charset.forName(encoding))); os.flush(); message.setContent(OutputStream.class, os); } catch (IOException ioe) { throw new RuntimeException(ioe); } } Howewer, when response contents length of if less than threshold (128 * 1024 bytes) i can get csnew.getInputStream() and get its contents, but when content is more than treshold, i see that in enforceLimits() -> createFileOutputStream() temp file is created, but using csnew.getInputStream() I am getting empty stream. -- Sergey Beryozkin Talend Community Coders http://coders.talend.com/ -- Sergey Beryozkin Talend Community Coders http://coders.talend.com/
Re: How to get stream from CachedStream ?
I have used this class istead of CachedOutputStream https://www.whitebyte.info/programming/java/convert-a-bytearrayoutputstream-into-a-bytearrayinputstream and also rewirte interceptor to WriterInterceptor interface, now my ReplaceOutInterceptor is working fine Thanks! 2016-10-08 1:14 GMT+04:00 Vjacheslav V. Borisov : > Hi! > > I see WriterInterceptorContext could be used, but it also requires > CachedOutputStream to intercept OutputStream, I see this in > JweJsonWriterInterceptor... > So CachedOutputStream bug will be repeated in WriterInterceptorContext > > > > > 2016-09-20 13:13 GMT+04:00 Sergey Beryozkin : > >> Hi, >> JAX-RS ContainerRequestContext and ContainerResponseContext, as well as >> WriterInterceptorContext and ReaderInterceptorContext filter contexts all >> allow resetting the entity streams or on the way out - entity objects - >> perhaps that can offer a simpler option for replacing the content... >> >> Cheers, Sergey >> >> On 19/09/16 11:30, Vjacheslav V. Borisov wrote: >> >>> During message.getInterceptorChain().doIntercept(message); I see call to >>> maybeDeleteTempFile() which resets inmem = true; >>> and later csnew.getInputStream() execution leads to if (inmem) { } >>> >>> 2016-09-19 13:05 GMT+04:00 Vjacheslav V. Borisov : >>> >>> Hi! I have following code (this is interceptror which does regular expression replacements) https://github.com/ilb/common/blob/master/common-jaxrs/src/ main/java/ru/ilb/common/jaxrs/interceptors/ReplaceOutInterceptor.java private void replaceContents(Message message) { OutputStream os = message.getContent(OutputStream.class); CachedStream cs = new CachedStream(); message.setContent(OutputStream.class, cs); message.getInterceptorChain().doIntercept(message); try { cs.flush(); CachedOutputStream csnew = (CachedOutputStream) message.getContent(OutputStream.class); String contents = IOUtils.toString(csnew.getInputStream()); for (Map.Entry keyValue : replacements.entrySet()) { contents = contents.replaceAll(keyValue.getKey(), keyValue.getValue()); } //String replaced=contents.replaceAll(regex, replacement); os.write(contents.getBytes(Charset.forName(encoding))); os.flush(); message.setContent(OutputStream.class, os); } catch (IOException ioe) { throw new RuntimeException(ioe); } } Howewer, when response contents length of if less than threshold (128 * 1024 bytes) i can get csnew.getInputStream() and get its contents, but when content is more than treshold, i see that in enforceLimits() -> createFileOutputStream() temp file is created, but using csnew.getInputStream() I am getting empty stream. >>> >> >> -- >> Sergey Beryozkin >> >> Talend Community Coders >> http://coders.talend.com/ >> > >
Re: How to get stream from CachedStream ?
Hi! I see WriterInterceptorContext could be used, but it also requires CachedOutputStream to intercept OutputStream, I see this in JweJsonWriterInterceptor... So CachedOutputStream bug will be repeated in WriterInterceptorContext 2016-09-20 13:13 GMT+04:00 Sergey Beryozkin : > Hi, > JAX-RS ContainerRequestContext and ContainerResponseContext, as well as > WriterInterceptorContext and ReaderInterceptorContext filter contexts all > allow resetting the entity streams or on the way out - entity objects - > perhaps that can offer a simpler option for replacing the content... > > Cheers, Sergey > > On 19/09/16 11:30, Vjacheslav V. Borisov wrote: > >> During message.getInterceptorChain().doIntercept(message); I see call to >> maybeDeleteTempFile() which resets inmem = true; >> and later csnew.getInputStream() execution leads to if (inmem) { } >> >> 2016-09-19 13:05 GMT+04:00 Vjacheslav V. Borisov : >> >> Hi! >>> >>> I have following code (this is interceptror which does regular expression >>> replacements) >>> >>> >>> https://github.com/ilb/common/blob/master/common-jaxrs/src/ >>> main/java/ru/ilb/common/jaxrs/interceptors/ReplaceOutInterceptor.java >>> >>> private void replaceContents(Message message) { >>> OutputStream os = message.getContent(OutputStream.class); >>> CachedStream cs = new CachedStream(); >>> message.setContent(OutputStream.class, cs); >>> >>> message.getInterceptorChain().doIntercept(message); >>> >>> try { >>> cs.flush(); >>> CachedOutputStream csnew = (CachedOutputStream) >>> message.getContent(OutputStream.class); >>> >>> String contents = IOUtils.toString(csnew.getInputStream()); >>> for (Map.Entry keyValue : >>> replacements.entrySet()) { >>> contents = contents.replaceAll(keyValue.getKey(), >>> keyValue.getValue()); >>> } >>> //String replaced=contents.replaceAll(regex, replacement); >>> os.write(contents.getBytes(Charset.forName(encoding))); >>> os.flush(); >>> >>> message.setContent(OutputStream.class, os); >>> >>> } catch (IOException ioe) { >>> throw new RuntimeException(ioe); >>> } >>> } >>> >>> >>> >>> Howewer, when response contents length of if less than threshold (128 * >>> 1024 bytes) >>> i can get csnew.getInputStream() and get its contents, but when content >>> is >>> more than treshold, >>> i see that in enforceLimits() -> createFileOutputStream() temp file is >>> created, >>> but using csnew.getInputStream() I am getting empty stream. >>> >>> >> > > -- > Sergey Beryozkin > > Talend Community Coders > http://coders.talend.com/ >
Re: How to get stream from CachedStream ?
Hi, JAX-RS ContainerRequestContext and ContainerResponseContext, as well as WriterInterceptorContext and ReaderInterceptorContext filter contexts all allow resetting the entity streams or on the way out - entity objects - perhaps that can offer a simpler option for replacing the content... Cheers, Sergey On 19/09/16 11:30, Vjacheslav V. Borisov wrote: During message.getInterceptorChain().doIntercept(message); I see call to maybeDeleteTempFile() which resets inmem = true; and later csnew.getInputStream() execution leads to if (inmem) { } 2016-09-19 13:05 GMT+04:00 Vjacheslav V. Borisov : Hi! I have following code (this is interceptror which does regular expression replacements) https://github.com/ilb/common/blob/master/common-jaxrs/src/ main/java/ru/ilb/common/jaxrs/interceptors/ReplaceOutInterceptor.java private void replaceContents(Message message) { OutputStream os = message.getContent(OutputStream.class); CachedStream cs = new CachedStream(); message.setContent(OutputStream.class, cs); message.getInterceptorChain().doIntercept(message); try { cs.flush(); CachedOutputStream csnew = (CachedOutputStream) message.getContent(OutputStream.class); String contents = IOUtils.toString(csnew.getInputStream()); for (Map.Entry keyValue : replacements.entrySet()) { contents = contents.replaceAll(keyValue.getKey(), keyValue.getValue()); } //String replaced=contents.replaceAll(regex, replacement); os.write(contents.getBytes(Charset.forName(encoding))); os.flush(); message.setContent(OutputStream.class, os); } catch (IOException ioe) { throw new RuntimeException(ioe); } } Howewer, when response contents length of if less than threshold (128 * 1024 bytes) i can get csnew.getInputStream() and get its contents, but when content is more than treshold, i see that in enforceLimits() -> createFileOutputStream() temp file is created, but using csnew.getInputStream() I am getting empty stream. -- Sergey Beryozkin Talend Community Coders http://coders.talend.com/
Re: How to get stream from CachedStream ?
During message.getInterceptorChain().doIntercept(message); I see call to maybeDeleteTempFile() which resets inmem = true; and later csnew.getInputStream() execution leads to if (inmem) { } 2016-09-19 13:05 GMT+04:00 Vjacheslav V. Borisov : > Hi! > > I have following code (this is interceptror which does regular expression > replacements) > > > https://github.com/ilb/common/blob/master/common-jaxrs/src/ > main/java/ru/ilb/common/jaxrs/interceptors/ReplaceOutInterceptor.java > > private void replaceContents(Message message) { > OutputStream os = message.getContent(OutputStream.class); > CachedStream cs = new CachedStream(); > message.setContent(OutputStream.class, cs); > > message.getInterceptorChain().doIntercept(message); > > try { > cs.flush(); > CachedOutputStream csnew = (CachedOutputStream) > message.getContent(OutputStream.class); > > String contents = IOUtils.toString(csnew.getInputStream()); > for (Map.Entry keyValue : > replacements.entrySet()) { > contents = contents.replaceAll(keyValue.getKey(), > keyValue.getValue()); > } > //String replaced=contents.replaceAll(regex, replacement); > os.write(contents.getBytes(Charset.forName(encoding))); > os.flush(); > > message.setContent(OutputStream.class, os); > > } catch (IOException ioe) { > throw new RuntimeException(ioe); > } > } > > > > Howewer, when response contents length of if less than threshold (128 * > 1024 bytes) > i can get csnew.getInputStream() and get its contents, but when content is > more than treshold, > i see that in enforceLimits() -> createFileOutputStream() temp file is > created, > but using csnew.getInputStream() I am getting empty stream. >