Hi,
Here I am attaching the patch which does the check of the content size. And
if it is <1MB then we load it to memory and pass a ByteArray data source
for the data handler. And if >1MB we proceed as previously.
@Danushka,
Please use the following patch and verify your scenario.
On Tue, Jun 10, 2014 at 8:34 PM, Ajith Vitharana <aji...@wso2.com> wrote:
> Please find the blog post [1].
>
> [1]http://ajithvblogs.blogspot.de/2013/11/jmeter-java-request-to-test.html
>
> Thanks.
> Ajith
>
>
> On Tue, Jun 10, 2014 at 8:28 PM, Subash Chaturanga <sub...@wso2.com>
> wrote:
>
>> Hi ,
>> This is regarding issue on WS Registry Service Client in high
>> concurrency which seems blocks the AF release.
>>
>> When reading the content with several threads in AF, i.e two resources
>>
>> path = /foo/r1 content = 12345
>> path = /foo/r2 content = 67
>>
>> For AF guys, when they retrieve r1 and r2 resources, i.e for
>> r1 - returns 12345
>> but r2 returns 67*45*
>>
>> ( r2 returns with some values from old resource content. )
>>
>> When dig deep with Janaka, found that, in server side
>> org.wso2.carbon.registry.ws.api.WSRegistry maintains single tempFile per
>> instance. And it get used to maintain the content to pass to client side
>> inside the DataHolder.
>>
>> Hence obviously we can expect this issue in a but high concurrency. So as
>> a solution when creating the data holder we create a copy of the
>> aforementioned file as follows. And then for multiple requests on the same
>> ws connection it deals with separate files and issue went away.
>>
>> But this leads to grow the tmp files in the tmp directory but upon server
>> downtime those will get vanished.
>>
>> With the following fix, since files are growing, one option is to do an
>> additional ws call inside ws-client to server side to clean up the
>> particular file. But it seems not a best solution.
>>
>>
>> *So @Senaka @Ajith, Any better solution you can see ? *
>>
>>
>> Index:
>> src/main/java/org/wso2/carbon/registry/ws/api/utils/CommonUtil.java
>> ===================================================================
>> --- src/main/java/org/wso2/carbon/registry/ws/api/utils/CommonUtil.java
>> (revision 204294)
>> +++ src/main/java/org/wso2/carbon/registry/ws/api/utils/CommonUtil.java
>> (working copy)
>> @@ -193,6 +193,9 @@
>> }
>>
>> public static DataHandler makeDataHandler(Resource resource, File
>> tempFile) throws IOException, RegistryException{
>> + tempFile = File.createTempFile("wsresource", ".tmp");
>> + tempFile.deleteOnExit();
>> +
>> if (resource.getContent() == null) {
>> return null;
>> }
>>
>>
>> --
>> Thanks
>> /subash
>>
>> *Subash Chaturanga*
>> Senior Software Engineer :Platform TG; WSO2 Inc. http://wso2.com
>>
>> email: sub...@wso2.com
>> blog: http://subashsdm.blogspot.com/
>> twitter: @subash89
>> phone: +9477 2225922
>> Lean . Enterprise . Middleware
>>
>
>
>
> --
> Ajith Vitharana.
> WSO2 Inc. - http://wso2.org
> Email : aji...@wso2.com
> Mobile : +94772217350
>
>
--
Thanks
/subash
*Subash Chaturanga*
Senior Software Engineer & Lead WSO2 Governance Registry
Platform TG; WSO2 Inc. http://wso2.com
Contact:
email: sub...@wso2.com
blog: http://subashsdm.blogspot.com/
twitter: @subash89
phone: +9477 2225922
Lean . Enterprise . Middleware
Index: src/main/java/org/wso2/carbon/registry/ws/api/utils/CommonUtil.java
===================================================================
--- src/main/java/org/wso2/carbon/registry/ws/api/utils/CommonUtil.java (revision 191396)
+++ src/main/java/org/wso2/carbon/registry/ws/api/utils/CommonUtil.java (working copy)
@@ -46,9 +46,11 @@
import javax.activation.DataHandler;
import javax.activation.FileDataSource;
+import javax.mail.util.ByteArrayDataSource;
import javax.servlet.http.HttpServletRequest;
import org.apache.axis2.context.MessageContext;
+import org.apache.commons.io.IOUtils;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.wso2.carbon.registry.core.*;
@@ -192,32 +194,39 @@
return output.toByteArray();
}
- public static DataHandler makeDataHandler(Resource resource, File tempFile) throws IOException, RegistryException{
+ public static DataHandler makeDataHandler(Resource resource, File tempFile) throws IOException, RegistryException {
if (resource.getContent() == null) {
return null;
}
InputStream is = null;
OutputStream os = null;
+
+
try {
- os = new FileOutputStream(tempFile);
if (resource.getContent() instanceof String[]) {
String[] strArray = (String[]) resource.getContent();
ObjectOutputStream oos = new ObjectOutputStream(os);
oos.writeObject(strArray);
} else {
- try {
- is = resource.getContentStream();
-// os = new FileOutputStream(tempFile);
+ is = resource.getContentStream();
+ if (isContentLarge(resource.getContentStream())) {
+ try {
+ os = new FileOutputStream(tempFile);
- byte[] buffer = new byte[4096];
- for (int n; (n = is.read(buffer)) != -1; )
- os.write(buffer, 0, n);
- os.flush();
- } finally {
- if (is != null) {
- is.close();
+ byte[] buffer = new byte[4096];
+ for (int n; (n = is.read(buffer)) != -1; )
+ os.write(buffer, 0, n);
+ os.flush();
+ } finally {
+ if (is != null) {
+ is.close();
+ }
}
+ return new DataHandler(new FileDataSource(tempFile));
+
+ } else {
+ return new DataHandler(new ByteArrayDataSource(is, "application/octet-stream"));
}
}
} finally {
@@ -225,11 +234,28 @@
os.close();
}
}
+ return null;
// Base64Binary base64Binary = new Base64Binary();
- return new DataHandler(new FileDataSource(tempFile));
}
+ /*
+ If the content is larger than 1MB, it is considered as a large file and will not load in to memory
+ */
+ private static boolean isContentLarge(InputStream is) throws IOException {
+ int i=0;
+ byte[] buffer = new byte[1024];
+ for (int n; (n = is.read(buffer)) != -1; ){
+ i++;
+ }
+// Checks if content is > 1MB
+ if(i >= 1024) {
+ return true;
+ } else {
+ return false;
+ }
+ }
+
public static WSResource newResourcetoWSResource(Resource resource) {
WSResource wsResource = new WSResource();
// wsResource.setAuthorUserName(resource.getAuthorUserName());
_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
