If you want to handle large chunks of data, why not use MTOM? Also, there is a data source mode in Axis2/C to stream the chunk through.
We use used Axis2/C to send mega bytes nearing giga range and it works for us for MTOM. For service case, I am not sure if it is a good idea to use AXIOM to build a mega range doc - that means the service design is busted. Samisa... On Mon, Jul 5, 2010 at 6:20 PM, Stadelmann Josef < [email protected]> wrote: > Hope this helps you?`! > > > > Yes, how did I strike it out? I am using WSIT from HP to integrate legacy > systems. In short, big (huge) OMElement is arriving at the JAVA side, (could > well contain Bas64 encoded data) it is then serialized to a very large > (huge) xml string; which is then pass to an out-of-process server via some > sort of WsiIpcContext (IPC Interprocess Communication Channel); JNI is > involved too. > > > > …. > > > > When my large data arrives at the C-Legacy Server passed by WSIT ToolKit, I > get a OpenVMS Descriptor which has a Length Field and a Pointer to the data. > > > > > …. > > > > And now comes what I did not understand so far but since I try to help you, > I was able to fix my own problem today. > > So what I did not understand is as follow. > > > > My data arrives at WSI$$fktmap() in *pbIndata with a inLen of Bystes set to > 91344 > > The code you see below is HP-WSIT-generated-code, and it works fine. > > > > The second green code line shows how this data is passed to my own routine > fktmap() which is declared as > > ‘__int32 fktmap (char * p1, int inLen, dsc_d * p2);’ > > > > Given that pbInData can be passed further on via a P1 of type char* then > please explain me the problem I am faced with. > > > > 1. In one case when I do > > Unsigned __int16 scnt = strlen(P1); > > I get scnt back as 25808 bytes > > BUT > > inLen shows me 91344 > > Why? > > > > 2. When I save P1 in a file, all bytes are there > BUT > When I pass it to > > xml_reader = axiom_xml_reader_create_for_memory(env, payload_inp, scnt, > encoding, AXIS2_XML_PARSER_TYPE_BUFFER); > > Then I get the following errors in my log file for one transaction, at > different call levels. > > > > [Mon Jul 5 14:10:28 2010] [error] > DKB3:[SW-PROJEKTE.webservices.axis2.trunk.c.axiom.src.parser]libxml2_reader_wrapper.c;36(886) > > > expected '>' > > *************** > > -- SEVERITY_ERROR > > *************** > > [Mon Jul 5 14:10:28 2010] [error] > DKB3:[SW-PROJEKTE.webservices.axis2.trunk.c.axiom.src.parser]libxml2_reader_wrapper.c;36(886) > > > Opening and ending tag mismatch: Fldnam line 0 and F > > *************** > > -- SEVERITY_ERROR > > *************** > > [Mon Jul 5 14:10:28 2010] [error] > DKB3:[SW-PROJEKTE.webservices.axis2.trunk.c.axiom.src.parser]libxml2_reader_wrapper.c;36(886) > > > Extra content at the end of the document > > *************** > > -- SEVERITY_ERROR > > *************** > > [Mon Jul 5 14:10:28 2010] [error] > DKB3:[SW-PROJEKTE.webservices.axis2.trunk.c.axiom.src.parser]libxml2_reader_wrapper.c;36(427) > > > error occured in reading xml stream > > [Mon Jul 5 14:10:28 2010] [debug] > DKB3:[SW-PROJEKTE.SPEZSRVW.axawl.spezpla.servers.SPgServer]SPg-legacy.c;100(1593) > > *************** > > > > > > The source of the problem is now fixed; > > > > scnt receiving from strlen(P1) the byte count was of unsigned __int16, too > short to keep a value bigger then 65535!!!!! > > > > scnt must be of type unsigned __int32 and only then it is able to keep the > number 91344 ! > > > > One can take the calculator and enter 91344 as decimal number, then convert > to hex and subtract 10000 Hex, then convert back to decimal. > > A too short type of an integer maskes out what strlen() returns correctly. > > > > > > BUT when the proper length of bytes is passed to > axiom_xml_reader_create_for_memory() > > the routine works fine and does not bail out with a -- SEVERITY ERROR > > > > Conclusion of the day! If you work with very long strings (longer then FFFF > === 65535) it is essential that you use > > “unsigned __int32 OR unsigned __int64 as type for then “scnt” = strlen(P1)” > with P1 pointing to a string longer than 65535 characters. > > > > > > Now back to you: How do you detect that your XML is broken? Which routine > informs you about that? > > > > Sepp > > > > > > /////////////////////////////////////////////////////////// > > // > > // function: WSI$$fktmap > > // > > // > > // > > /////////////////////////////////////////////////////////// > > static unsigned int WSI$$fktmap (unsigned int sessionID, > > *MSGB *pbInData,* > > *int inLen,* > > MSGB **ppbOutData, > > int *pOutLen) > > { > > #ifdef __DEBUG__ > > lib$signal(SS$_DEBUG); > > #endif > > char wsi$ret [4]; > > dsc_sd *wsi$p_dx; > > dsc_a *wsi$arydsc; > > int wsi$pnum = 2; > > int wsi$size = 0; > > encoder wsi$enc; > > dsc_sd wsi$ret_dx = {4, DSC$K_DTYPE_L, DSC$K_CLASS_S, wsi$ret, 0, 0, > 0}; > > internal_context_t *wsi$context = (internal_context_t *) sessionID; > > > > wsi$p_dx = (dsc_sd *) pbInData; > > > > // All pointers are converted from relative to absolute addresses > here. > > wsi$p_dx[0].dsc$a_pointer += (int) wsi$p_dx; > > wsi$p_dx[1].dsc$a_pointer += (int) wsi$p_dx; > > > > // Special parameters, such as Floats & Dynamic Strings, are fixup > here. > > wsi$dstr_fix( &wsi$p_dx[1] ); > > > > // Structure parameters are fixed up here. > > > > #pragma names restore // we escape to Spg-Legacy and Axis2 code here > which is compiled AS_IS & SHORTENED > > // user-side function call > > *(__int32 *)wsi$ret_dx.dsc$a_pointer = fktmap((char *) > wsi$p_dx[0].dsc$a_pointer, inLen, (dsc_d *) &wsi$p_dx[1]); > > #pragma names save > > #pragma names uppercase, shortened // and back in HP WSIt world go > for UPPERCASE & SHORTENED > > > > // OUT & IN/OUT Floats get converted back to IEEE here. > > // OUT & IN/OUT structures get fixup up here. > > // OUT & IN/OUT strings get fixed up here > > > > // Determine size of output buffer, size of static + dynamic out params > > wsi$size = 28 + wsi$p_dx[1].dsc$w_length; > > wsi$context->mem_disp->malloc( wsi$context->mem_disp->cnxid, wsi$size, > (void **) ppbOutData ); > > wsi$init_encoder( &wsi$enc, *ppbOutData, wsi$pnum ); > > > > // Package up the parameters for returning here. > > wsi$add_param( &wsi$ret_dx, &wsi$enc ); > > wsi$add_param( &wsi$p_dx[1], &wsi$enc ); > > > > // Deallocate any memory used by Dynamic Strings here. > > wsi$dstr_free( &wsi$p_dx[1] ); > > > > // Return here > > *pOutLen = wsi$size; > > return error_status_ok; > > } > > > > > > -----Ursprüngliche Nachricht----- > Von: Akos Marton [mailto:[email protected]] > Gesendet: Montag, 5. Juli 2010 11:13 > > An: Apache AXIS C User List > Betreff: Re: AW: transfer limitations > > > > Hi Sepp, > > > > On Mon, Jul 5, 2010 at 9:24 AM, Stadelmann Josef > > <[email protected]> wrote: > > > Given you send ALL your XML data into a file and parse it. Does the > parser bail out somewhere? > > I also thought it, but varying the sent amount of data, thus XML is > > not broken at the same point in every case. > > > > > > > > I just have similar situation. There is (in my mind) too much data > delivered; AND - Are you working with some kind of Strings through char*? > > Yes, axis2_char_t*. > > > > > > > > How long can a char* String be? How far does this pointer reach? Writing > all my XML data into a file and checking the XML Structure shows a very well > formed XML file; But any attempt to parse it through the > axis2_libxml2_reader_wrapper is a no go and it bails out with various errors > toward the end of the files-content.- no I memory. > > I may not got what you mean above about the long of char*. You can > > allocate arbitrary (depends on your OS) amount of space, using a char* > > if you use dynamic memory management. How did you strike it out? > > > > > > > > Pointer size? 16, 32, 64? > > The size of pointer depends on your hardware and operating system, so > > that on 32bit operating system pointer has size of 4bytes. > > > > Now, I'm experimenting and learning how to configure the .Net side > > using WCF-technologies and set it to send data in chunks using MTOM. > > It seems for me, .Net-.Net can communicate with each other very well > > (of course), but needs more portability configuration to make it to be > > able to handle MTOM. > > In spite this we don't understand the reason of broken XMLs. > > > > Akos > > > > > > > > Sepp > > > > > > -----Ursprüngliche Nachricht----- > > > Von: Akos Marton [mailto:[email protected]] > > > Gesendet: Freitag, 18. Juni 2010 10:20 > > > An: Apache AXIS C User List > > > Betreff: Re: AW: transfer limitations > > > > > > The problem is still not solved, so that any idea and answer is > appreciated. > > > > > > You can read a mail on c-user list about the same problem by Jan-fon, > > > except that he found the same situation on client-side. > > > subject: 'Client to receive large data problem' > > > date: Thu, 25 Feb 2010 11:07:53 GMT > > > > http://mail-archives.apache.org/mod_mbox/axis-c-user/201002.mbox/%3ccc8ff067f7f4e54796afdf21339ba0fc134a9...@orsmsx505.amr.corp.intel.com%3e > > > > > > Hope it helps, > > > Akos Marton > > > > > > > > > Stadelmann Josef wrote: > > >> Have you setup to send in chunks from CL to SV or do you expect that > more than 4 MB will go at once, within one request object? > > >> Sepp > > >> > > >> -----Ursprüngliche Nachricht----- > > >> Von: Akos Marton [mailto:[email protected]] > > >> Gesendet: Freitag, 11. Juni 2010 15:31 > > >> An: Apache AXIS C User List > > >> Betreff: Re: transfer limitations > > >> > > >> The XML is broken every time at different part. I tried to configure > httpd server with switching out many known size limitation and run axis2c > under this, but it also doesn't send the whole XML. > > >> > > >> The sent data is just broken somewhere. I watched it with tcpdump. > > >> Smaller XMLs can be transferred. > > >> > > >> Well, written a short tester code, which reads the necessary XML from > file and parses, decodes it perfectly, so that the algorithm and code must > do the right. > > >> > > >> Any idea why the sending process is being broken? > > >> > > >> Thanks, > > >> mAkos > > >> > > >> > > >> Akos Marton wrote: > > >>> Hi List, > > >>> > > >>> Is there any limitation about the amount of data in a certain XML-tag? > > >>> In case of true, could I increase it somehow? > > >>> For instance <image> ... </image> by the size of 4Megabytes of base64 > > >>> code. The whole XML data does not reach the server (have cut at > > >>> certain part), but a smaller amount of data reaches it fully. > > >>> > > >>> Could be httpd.conf a solution? I do not know how to locate it for > > >>> axis2c binary version. > > >>> > > >>> Thanks, > > >>> mAkos > > >>> > > >>> > > >> > > >> > > >> > > >> --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: [email protected] > > >> For additional commands, e-mail: [email protected] > > >> > > >> > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > -- > > People seldom notice clothes, if you wear a big smile. > > My grandmother uses Ubuntu, don't you think so?! > > OLVASD: http://www.xkcd.org/ > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > Samisa Abeysinghe VP Engineering WSO2 Inc. http://wso2.com http://wso2.org
