It seems your 1st approach provides much better performance than 2nd approach, can you incorporate improvements that I mentioned here[1] ? I believe applying those modification will solve performance issue with 2nd approach.
[1] - http://markmail.org/message/xyfxzn2clc3r5zfg Thanks ! On Sat, Aug 11, 2012 at 4:07 AM, Shameera Rathnayaka <[email protected]> wrote: > Hi devs, > > I did performance test with 1. SOAP support , 2 JSON Badgerfish , 3 JSON > first approach(pure json) and 4. JSON second > approach(XMLStreamReader/XMLStreamWriter implementation). Here i have > summarize the results i got. > I used same POJO service and send the exact same request in relevent > convention. > > Environment: > OS : Ubuntu 11.04 > JDK 1.6_30 > memory 4 GB > > SOAP JSON badgerfish > JSON 1st approach JSON 2nd approach > > Concurrency Level: 50 50 > 50 50 > Time taken for tests: 243.13 sec 287.18 sec > 118.99 200.0 > Complete requests: 500000 500000 > 500000 500000 > Total transferred: 2.2 GB 34.42 GB > 0.61 GB 0.63 GB > Requests per second: 2056.53 #/sec 1741.04 #/sec 4201.69 > #/sec 2499.91 #/sec > Time per request: 24.3 ms 28.7 ms > 11.90 ms 20.00 ms > Time per request:(mean) 0.486 ms 0.544 ms 0.238 > ms 0.4 ms > > Thanks, > Shameera. > > > On Wed, Aug 8, 2012 at 12:17 PM, Shameera Rathnayaka > <[email protected]> wrote: >> >> HI devs, >> >> I finished the implementation and did a performance test with existing >> SOAP and implemented JSON. Here is the summary of the test results. >> For this i used same service and send the same request in both SOAP and >> JSON ways, size of JSON message is lower than SOAP message but requests are >> exactly same. >> >> Environment : >> OS - Ubuntu 11.10 >> Axis2 1.7(current trunk) >> Tool - Java benchmark >> >> num of requests : 50000 >> concurrency level: 10 (Total num of requests are 50000*10 = 500000 ) >> Time taken for test : with SOAP 178 sec and with JSON 140 sec >> Total transferred : with SOAP 2371000000 bytes and with JSON 674000000 >> bytes >> Requests per second: with SOAP 2,801.76 [#/sec] (mean) and with JSON >> 3,568.89 [#/sec] (mean) >> >> will do a performance test with concurrency level 50 and num of requests >> 500000 and give the summary of results. >> >> Thanks, >> Shameera. >> >> >> >> >> On Tue, Jul 31, 2012 at 10:50 AM, Shameera Rathnayaka >> <[email protected]> wrote: >>> >>> HI devs, >>> >>> I have implemented XMLStreamWriter as well as XMLStreamReader to convert >>> XML <----> JSON with XMLSchema, >>> Basically it works fine, but still need few test to be done after that I >>> am willing to write performance test with implemented JSON support with >>> existing SOAP support. >>> >>> Thanks, >>> Shameera. >>> >>> >>> >>> On Mon, Jul 16, 2012 at 10:43 AM, Amila Suriarachchi >>> <[email protected]> wrote: >>>> >>>> >>>> >>>> On Sun, Jul 15, 2012 at 10:53 PM, Shameera Rathnayaka >>>> <[email protected]> wrote: >>>>> >>>>> Hi devs, >>>>> >>>>> I implemented this feature completely to convert XML to JSON with the >>>>> help of XmlSchema of that XML document. I tested this with different >>>>> Schemas, it is working fine. I implemented this as a separate project, so >>>>> i >>>>> will integrate this with axis2 trunk and try with actual scenario. >>>> >>>> >>>> Can you use this technique at the reader level as well to support >>>> namespaces? >>>> >>>> Then do a performance comparison using a POJO service with the exiting >>>> json support and based the new gson based XMLstream API. According to the >>>> result we need to optimise the later to have better performance. >>>> >>>> thanks, >>>> Amila. >>>> >>>>> >>>>> >>>>> Yes, This may be a standard way to convert XML to JSON conversion as >>>>> other implementations don't convert JSON message correctly when it has one >>>>> value array, unless implementation supports special XML structure where >>>>> Json-lib does. >>>>> >>>>> Definitely we can improve the performance of this implementation. May >>>>> be in the way that Sagara had suggested in his reply or in another way. >>>>> >>>>> Thanks, >>>>> Shameera. >>>>> >>>>> On Sun, Jul 15, 2012 at 6:56 PM, Amila Suriarachchi >>>>> <[email protected]> wrote: >>>>>> >>>>>> >>>>>> >>>>>> On Sun, Jul 15, 2012 at 2:19 PM, Sagara Gunathunga >>>>>> <[email protected]> wrote: >>>>>>> >>>>>>> On Sun, Jul 15, 2012 at 1:27 PM, Amila Suriarachchi >>>>>>> <[email protected]> wrote: >>>>>>> > >>>>>>> > >>>>>>> > On Sat, Jul 14, 2012 at 1:14 PM, Sagara Gunathunga >>>>>>> > <[email protected]> wrote: >>>>>>> >> >>>>>>> >> On Sat, Jul 14, 2012 at 12:42 PM, Shameera Rathnayaka >>>>>>> >> <[email protected]> wrote: >>>>>>> >> > Hi devs, >>>>>>> >> > >>>>>>> >> > As Amila(Project mentor) suggested in previous thread[0] , I >>>>>>> >> > have >>>>>>> >> > implemented >>>>>>> >> > XMLStreamWriter API to use schema of the processing xml and gson >>>>>>> >> > together >>>>>>> >> > to convert XML ---> JSON. I have made my progress up to support >>>>>>> >> > xml >>>>>>> >> > which >>>>>>> >> > don't have complex type elements with maxOccur >1, but may have >>>>>>> >> > simple >>>>>>> >> > type elements with maxOccur >=1 . That means In the JSON point >>>>>>> >> > of view, >>>>>>> >> > my >>>>>>> >> > present implementation supports XML --> JSON where there is not >>>>>>> >> > any JSON >>>>>>> >> > Arrays which has JSON objects in the converted JSON String. >>>>>>> >> > >>>>>>> >> > Here is a summary of the implementation. >>>>>>> >> > >>>>>>> >> > GsonXmlStreamWriter which implements XMLStreamWriter has a >>>>>>> >> > constructor >>>>>>> >> > to get JsonWriter and XmlSchema object. Then it process >>>>>>> >> > XmlSchema and >>>>>>> >> >>>>>>> >> Are you plan to process XmlSchema object with every response ? If >>>>>>> >> so >>>>>>> >> processing XmlSchema object will add some overhead and result >>>>>>> >> into >>>>>>> >> performance. Axis2 already has JSON support main objective of >>>>>>> >> this >>>>>>> >> project is to provide a high performance JSON implementation, >>>>>>> >> processing XmlSchema object with every response does not help to >>>>>>> >> meet >>>>>>> >> your expectations. Instead I suggest to process XmlSchema object >>>>>>> >> only >>>>>>> >> one-time during the deployment after the AxisService creation and >>>>>>> >> store result of XmlSchema processing with AxisService object as >>>>>>> >> a >>>>>>> >> light-weight object ( Let's say a map) and use this light weight >>>>>>> >> map >>>>>>> >> with response. In this approach overhead if relatively low. >>>>>>> > >>>>>>> > >>>>>>> > This is a POC level approach to convert xml stream to json using >>>>>>> > xml schema. >>>>>>> > For an example there are some techniques like >>>>>>> > bagger fish and others to address this problem. But those things >>>>>>> > complicated >>>>>>> > the produced json stream. >>>>>>> > >>>>>>> > Once it proves at the POC level we can think of improving >>>>>>> > performance. >>>>>>> > >>>>>>> > If you look at the current axis2 json support does that properly >>>>>>> > address the >>>>>>> > problem shameera mentioned earlier? i.e serialising the >>>>>>> > array elements when there is on element. And also can that be used >>>>>>> > with some >>>>>>> > thing like ADB or WSO2 Data Services,rules etc.. >>>>>>> > Idea here to develop a technique to address those problems properly >>>>>>> > while >>>>>>> > converting an xml stream to json stream. >>>>>>> >>>>>>> I don't have any issue with the technique proposed here (i.e. >>>>>>> processing XMLSchema to identify structure ). My only concern is why >>>>>>> we should process XMLSchema with every response because in each and >>>>>>> every response we end up getting same result, why shouldn't we >>>>>>> process >>>>>>> XMLSchema only one time and reuse results ? May be as last step of >>>>>>> deployment or with very first response processing ? >>>>>>> >>>>>>> Of course we can keep it as future improvement for the moment but >>>>>>> Axis2 having good history of forgetting those improvement after >>>>>>> initial development every time when I dig into code I found couple of >>>>>>> such TODOs. >>>>>> >>>>>> >>>>>> :) I am not telling to do it after GSoc project. But first we need to >>>>>> have a proper POC. >>>>>> >>>>>> thanks, >>>>>> Amila. >>>>>>> >>>>>>> >>>>>>> Thanks ! >>>>>>> >>>>>>> >>>>>>> >>>>>>> > >>>>>>> > thanks, >>>>>>> > Amila. >>>>>>> > >>>>>>> >> >>>>>>> >> >>>>>>> >> Thanks ! >>>>>>> >> >>>>>>> >> > use a queue to keep the schema structure. At the moment i have >>>>>>> >> > only >>>>>>> >> > implemented >>>>>>> >> > writeStartElement(String localName) , writeCharaters(String >>>>>>> >> > text) , >>>>>>> >> > writeEndElement() >>>>>>> >> > and writeEndDocument() functions. I am processing the queue >>>>>>> >> > which has >>>>>>> >> > the schema structure and put it to another stack to identify the >>>>>>> >> > scope >>>>>>> >> > of >>>>>>> >> > end elements. >>>>>>> >> > >>>>>>> >> > I am now working on, improving this to support xml which has >>>>>>> >> > complex >>>>>>> >> > type >>>>>>> >> > elements >>>>>>> >> > with maxOccour > 1 . I'll keep updating my progress mean while. >>>>>>> >> > >>>>>>> >> > Thanks, >>>>>>> >> > Shameera. >>>>>>> >> > >>>>>>> >> > -- >>>>>>> >> > Shameera Rathnayaka >>>>>>> >> > Undergraduate >>>>>>> >> > Department of Computer Science and Engineering >>>>>>> >> > University of Moratuwa. >>>>>>> >> > Sri Lanka. >>>>>>> >> > >>>>>>> >> > Blog : http://shameerarathnayaka.blogspot.com/ >>>>>>> >> > >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> -- >>>>>>> >> Sagara Gunathunga >>>>>>> >> >>>>>>> >> Blog - http://ssagara.blogspot.com >>>>>>> >> Web - http://people.apache.org/~sagara/ >>>>>>> >> LinkedIn - http://www.linkedin.com/in/ssagara >>>>>>> >> >>>>>>> >> >>>>>>> >> --------------------------------------------------------------------- >>>>>>> >> To unsubscribe, e-mail: [email protected] >>>>>>> >> For additional commands, e-mail: [email protected] >>>>>>> >> >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > -- >>>>>>> > Amila Suriarachchi >>>>>>> > WSO2 Inc. >>>>>>> > blog: http://amilachinthaka.blogspot.com/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Sagara Gunathunga >>>>>>> >>>>>>> Blog - http://ssagara.blogspot.com >>>>>>> Web - http://people.apache.org/~sagara/ >>>>>>> LinkedIn - http://www.linkedin.com/in/ssagara >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>> For additional commands, e-mail: [email protected] >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Amila Suriarachchi >>>>>> WSO2 Inc. >>>>>> blog: http://amilachinthaka.blogspot.com/ >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Shameera Rathnayaka >>>>> Undergraduate >>>>> Department of Computer Science and Engineering >>>>> University of Moratuwa. >>>>> Sri Lanka. >>>>> >>>>> Blog : http://shameerarathnayaka.blogspot.com/ >>>>> >>>> >>>> >>>> >>>> -- >>>> Amila Suriarachchi >>>> WSO2 Inc. >>>> blog: http://amilachinthaka.blogspot.com/ >>> >>> >>> >>> >>> -- >>> Shameera Rathnayaka >>> Undergraduate >>> Department of Computer Science and Engineering >>> University of Moratuwa. >>> Sri Lanka. >>> >>> Blog : http://shameerarathnayaka.blogspot.com/ >>> >> >> >> >> -- >> Shameera Rathnayaka >> Undergraduate >> Department of Computer Science and Engineering >> University of Moratuwa. >> Sri Lanka. >> >> Blog : http://shameerarathnayaka.blogspot.com/ >> > > > > -- > Shameera Rathnayaka > Undergraduate > Department of Computer Science and Engineering > University of Moratuwa. > Sri Lanka. > > Blog : http://shameerarathnayaka.blogspot.com/ > -- Sagara Gunathunga Blog - http://ssagara.blogspot.com Web - http://people.apache.org/~sagara/ LinkedIn - http://www.linkedin.com/in/ssagara --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
