java.lang.IndexOutOfBoundsException on XML Struct?
I'm currently parsing a number of xml files. In attempt to speed up processing I'm running each file in its own parse thread (may be relevant, I'm not sure). Inside of each I have a loop that is causing the error java.lang.IndexOutOfBoundsException. cfif StructKeyExists(stcXML.country,cities) cfloop index=i from=1 to=#ArrayLen(stcXml.country.cities.xmlChildren)# cfset CityData = stcXml.country.cities.xmlChildren[i] / cfsavecontent variable=STUFF'#CityData.url.xmlText#'/cfsavecontent cfset writeToFile(STUFF) / /cfloop /cfif --- The line that the log file says is the problem is: cfset CityData = stcXml.country.cities.xmlChildren[i] / How can that statement with 'i' be throwing a java.lang.IndexOutOfBoundsException when I just set 'i' to increment from 1 to the length of xmlChildren? ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:327508 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: java.lang.IndexOutOfBoundsException on XML Struct?
How can that statement with 'i' be throwing a java.lang.IndexOutOfBoundsException when I just set 'i' to increment from 1 to the length of xmlChildren? Since you mentioned threads, are all of the variables properly scoped? Assuming the array has not changed, it sounds like you might be getting the wrong value because the variable is shared by other threads. -Leigh ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:327509 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
re: java.lang.IndexOutOfBoundsException on XML Struct?
If stcXml.country.cities.xmlChildren is empty, then the first pass at index=1 would be invalid. Can you verify that the the XML has at least 1 cities child? ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:327510 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
re: java.lang.IndexOutOfBoundsException on XML Struct?
If stcXml.country.cities.xmlChildren is empty, then the first pass at index=1 would be invalid. Can you verify that the the XML has at least 1 cities child? Duh! Good point ;-) -Leigh ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:327512 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: java.lang.IndexOutOfBoundsException on XML Struct?
If stcXml.country.cities.xmlChildren is empty, then the first pass at index=1 would be invalid. Can you verify that the the XML has at least 1 cities child? I do need to verify it, but if stcXml.country.cities.xmlChildren was empty wouldn't the cfloop be equivalent to: cfloop index=i from=1 to=0 In that case the contents of the loop wouldn't execute at all, would they? I also doubt scoping. i is not declared outside the cfthread tag. Even if it was cfthread performs a deep copy of all parent thread variables; meaning they are held in different memory than the other thread 'copies' (or parent) and at that point are independent of each other. Also, all variables declared inside the thread are scoped independently (I could declare soupTaNuts inside a dozen threads and each one would occupy its own memory). ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:327522 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: java.lang.IndexOutOfBoundsException on XML Struct?
Did you add some debugging? What is the value of i and the array length XMLChildren when the error occurs? __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:327523 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Re: java.lang.IndexOutOfBoundsException on XML Struct?
@Matthew, Good call on the from=1 to=0 ... you're right that it wouldn't execute at all ... hm ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:327524 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: java.lang.IndexOutOfBoundsException on XML Struct?
Did you add some debugging? What is the value of i and the array length XMLChildren when the error occurs? So the bizarre thing is a test page, like below, does not process the contents of the loop: Hello - Startbr / cfloop index=i from=1 to=0 Howdy-Do Dah! cfoutput#i#/cfoutputbr / /cfloop End However, changing the previous code example to: cfif StructKeyExists(stcXML.country,cities) cfif ArrayLen(stcXml.country.cities.xmlChildren) gt 0 cfloop index=i from=1 to=#ArrayLen(stcXml.country.cities.xmlChildren)# cfset CityData = stcXml.country.cities.xmlChildren[i] / cfsavecontent variable=STUFF'#CityData.url.xmlText#'/cfsavecontent cfset writeToFile(STUFF) / /cfloop /cfif /cfif --- appears to fix the problem. all I can say is wtf? this is still happening even when I limit the processing to one spawned thread - so there isn't another child thread competing against it; just a parent waiting for the child to finish processing with a cfthread action=join statement. Now that that error is handled I'm getting GC overhead limit exceeded (java.lang.OutOfMemoryError) while parsing the larger file (approx 187MB). I'm reading in the xml one line at a time, suppressing whitespace anywhere I can (so as not to add to the cfthread[#mythread#] output total), etc. Ugh. ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:327525 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: java.lang.IndexOutOfBoundsException on XML Struct?
Sounds like it may just be trying to load the entire 187 MB into memory and choking on that? Perhaps overall memory usage is the real issue and the OutOfBounds message is only a false report of a problem related to empty memory spaces. Good luck debugging that! ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:327526 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: java.lang.IndexOutOfBoundsException on XML Struct?
However, changing the previous code example to: cfif ArrayLen(stcXml.country.cities.xmlChildren) gt 0 ... appears to fix the problem. all I can say is wtf? Weird. That is like saying zero does not equal 0 anymore. Maybe there is something special about the XMLChildren array? ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:327527 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: java.lang.IndexOutOfBoundsException on XML Struct?
Ah, Leigh may be on to it: If the XML is blank or even if just the 'country' element is empty, than 0 is still 0, but 0 can't be determined, since 'cities' wouldn't even exist in the arrayLen(stcXml.country.cities.xmlChildren) test is 'country' is empty. That would be why your new first test (cfif StructKeyExists(stcXML.country, cities) ) is the key ... ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:327528 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: java.lang.IndexOutOfBoundsException on XML Struct?
Sounds like it may just be trying to load the entire 187 MB into memory No. My initial approach was to attempt to load the entire file. Obviously, that would consume a lot of memory and cause problems. Instead, I'm using the cfloop file= process to load one line of xml into memory at a time for processing (thankfully each child element of the parent exists on its own unique line). Reading http://forums.adobe.com/thread/43968 makes it seem as if the GCOverheadLimit error is being thrown because the java garbage collector is taking to long to clean up memory; this is consistent with the behavior I have seen. If I load up multiple files to be simultaneously processed (with parallel threads) the smaller files finish fine but the filesize for the large file outputs slows to a crawl the closer jrun.exe gets to the maximum heap size. Finally the GCOverheadLimit error is thrown, the cfthread action=join ops runs, and I see the dumped error. It's as if the garbage collector can't effectively clean up memory allocated to threaded processes; or at least can't release it until all threads are joined together? Belch. I had so much hope for the parallel processing but it just seems to be obscuring and creating more problems than if I just did a straightforward, linear processing of files. ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:327531 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: java.lang.IndexOutOfBoundsException on XML Struct?
But before you can use CFLOOP the file had to be read into memory ... something had to be there for CFLOOP to take action *on* ... Matthew Reinbold wrote: Sounds like it may just be trying to load the entire 187 MB into memory No. My initial approach was to attempt to load the entire file. Obviously, that would consume a lot of memory and cause problems. Instead, I'm using the cfloop file= process to load one line of xml into memory at a time for processing (thankfully each child element of the parent exists on its own unique line). ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:327535 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: java.lang.IndexOutOfBoundsException on XML Struct?
Well, that clinches it. The *exact* same code with the threading pulled out (so now the xml files are processed sequentially, not parallel) and the memory problems disappear (the crude method of watching the memory allocation in windows task manager shows a slight uptick, but no where near to hitting the wall of the java heap size). On a whim I also pulled out the extra check for the xmlChildren size. Guess what? The unexplainable indexing error is gone now too. Of course, processing things will take longer - one file has to wait for the other to finish. But considering I've wasted a day chasing unexplainable ghost problems caused by threading, I'll have to figure out some other processing method to speed things up. ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:327536 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Re: java.lang.IndexOutOfBoundsException on XML Struct?
But before you can use CFLOOP the file had to be read into memory ... No, that is not how the cfloop file= row= / tag works. Using that syntax a person processes one line at a time. See this post (under Step 3 main header starting with 1. Reading Lines): http://coldfused.blogspot.com/2007/07/new-file-io-in-coldfusion-8.html ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:327538 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Re: java.lang.IndexOutOfBoundsException on XML Struct?
... On a whim I also pulled out the extra check for the xmlChildren size. Guess what? The unexplainable indexing error is gone now too. If you can consistently reproduce it, I would consider reporting it as a bug. ~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:327552 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
