Let me see if I'm following along. 1. You save the xml for every transmision? At what point do you hit write-the-file? (And delete the file?)
2. You put only one item in a submission? The communication overhead is of no concern to you? How thick is your pipe! :) On Dec 14, 5:44 pm, Tom Wilson <[email protected]> wrote: > Writing the xml message to a temp file check it size and then from > there decide to send it was the default precaution i took as i do with > other projects that fire messages between servers. Then you have a > handy reference of failed messages. > > I stick to an item a submital, and its never failed me yet. Even with > over 500,000 items if its spread well and the system that drive it is > efficient its always been more than enough. > > On Dec 12, 4:33 pm, icebackhaz <[email protected]> wrote: > > > Good question. Yes it is clear (and provable :) ) that one may only > > put 1Mb in the xml payload, but it is not at all clear to me how one > > either a) checks how full the payload is and b) detects after the fact > > that the problem was an overly large payload. > > > We have a lot to send and want to do it efficiently and correctly. > > Not checking before hand means we would need a the issues (742, 921) > > taken care of or we will not know the exact problem. If I could check > > before hand I would never encounter overhead of the double failure > > (921). > > > Alternatives abound but don't really appeal. Continuously calculate > > the xml based on known overhead and data lengths: fraught with > > miscalculation errors and seriously exposed google api/xml changes. > > Send ultra-conservative batch sizes (30 items might be the upper limit > > of perfectly full (1000 char) values if I've done the arithmetic > > correctly): seriously under uses the payload and increases the number > > of submissions, traffic overhead. Send less conservatively (perhaps > > aggressively) sized batches and on failure assume the payload was too > > large and split it (recursively): many possible other reasons for > > failure. > > > So yes, I'm trying to solidify our end and both these, um, ah features > > of the API get in the way. Btw, what we've decided to do on failure > > is to generate the xml with on our own Writer and test the size of the > > generated xml and react accordingly. Seems a reasonable compromise, > > no? > > > Keep in mind, I'm not sure 921 will be accepted as a bug. I do > > believe the java.io.IOException is more the result of mis- > > communication (using a closed connection) than anything else. Pretty > > sure they thought they would be sending back a ServiceException. > > InvalidEntryException really, and that's also wrong. HTTP_BAD_REQUEST > > is the http error, and in this case it the result of payload > 1 Mb, > > not that a particular entry is malformed. > > > On Dec 11, 5:37 pm, Tom Wilson <[email protected]> wrote: > > > > Can i ask why exactly your looking at this ? > > > > The documentation states that it accepts batch requests up to 1MB so > > > why are you checking the size before posting ? > > > > There only so much the API will do for you but building a solid system/ > > > app relies on checks on both ends. > > > > Tom Wilson > > > Freelance Google Base Developer and Consultantwww.tomthedeveloper.com > > > > Google Base Tools -http://dev.tomthedeveloper.com/googlebase > > > Featured Project > > > :http://google-code-featured.blogspot.com/2008/02/google-base-competit... > > > > On Dec 11, 11:28 pm, icebackhaz <[email protected]> wrote: > > > > > Trying to see what happens when one stuffs too much into the xml > > > > payload, we have discovered that there is no facility for detecting > > > > the exact problem either before or after the send. > > > > > The repsonse from the server is HttpURLConnection.HTTP_BAD_REQUEST. > > > > It looks like the intension was to generate a InvalidEntryException > > > > and that looks bogus too, but at least the getReason() might be useful > > > > > Unfortunately the built-in second attempt to send the payload chokes > > > > big time and we get a java.io.IOException from deep in Sun's code and > > > > the 401 floats off into the either. > > > > > I've sent the to the bugs forum, but not sure it will be recognized as > > > > a bug:http://code.google.com/p/gdata-issues/issues/detail?id=921 > > > > > This is further compounded by the fact that even if we hadn't been > > > > blown out of the water there is no access to the httpRequest which has > > > > the all important header: "Content-Length" (This has been recognized > > > > as a > > > > bug:http://code.google.com/p/gdata-issues/issues/detail?id=742&sort=-id&c...) --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google Base Data API" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/Google-Base-data-API?hl=en -~----------~----~----~----~------~----~------~--~---
