Re: [google-appengine] SocketTimeoutException from one AppEngine Server to another
Hi, FYI: As you have suggested, we deployed the Server B as backend module and Server A as frontend module in the same App ID and used URLFetch in module A(Server A) to communicate with module B (Server B) of the same app. Still the same problem for one user who is using the application and that's me. So there is no question of exceeding the per minute quota limits or any other for that matter. As you have quoted: Longer-term, the even better option would be to refactor your application code so shared services are available as libraries and not as individual endpoints. Libraries will increase the size of the application or the parent module if you opt for modules. Deploying the app with libraries is old fashion...Its not good to follow if you consider the growth of the application. REST based service is the current trend. All this together, does that mean SOA (Service Oriented Architecture) or REST API Services will fluctuate in Google AppEngine (ie) from AppEngine Server to another AppEngine Server very often? Thanks. Regards, Anan On Wednesday, December 11, 2013 12:13:06 PM UTC+5:30, Ananthakrishnan Venkatasubramanian wrote: Hi, Definitely Server A is not exceeding the per minute quota for URLFetch API Calls (32,000 calls per min). SocketTimeOutExceptions are coming even there is less traffic in both the Servers. Note: We have idle instances in both the servers Definitely there is no question of queueing the requests also. Requests should be served at any point of time. As far as AppStats is concerned, we already knew how to do it. Its a matter of enabling. We will enable it and give you the information about it in private. As you have suggested, we deployed the Server B as backend module and Server A as frontend module in the same App ID and used URLFetch in module A(Server A) to communicate with module B (Server B) of the same app. Still the same problem. Those exceptions are repeating. The only thing we need to test now with modules is cut down the URLFetch in module A(Server A) and communicate directly with module B(Server B). Technically we need to integrate the codebase of Server B inside module A(Server A). This we can do it and try out from our end. But my question is, imagine there are two different organisation using AppEngine and one of the sever in the organisation running in AppEngine needs to communicate with the server in the other organisation running same in AppEngine, they have to communicate with URLFetch calls only. We cannot force the organisations to create one common App and use modules to avoid SocketTimeOutExceptions. Can you please give the appropriate reasons/solutions for the same? Please let me know if you need any other information regarding this. Thanks. Regards, Anantha Krishnan. On Wednesday, December 11, 2013 10:24:14 AM UTC+5:30, Vinny P wrote: On Tue, Dec 10, 2013 at 2:52 AM, Ananthakrishnan Venkatasubramanian ananthakrishnan.venkatasubraman...@a-cti.com wrote: Server A B represent different application IDs and the error rate depends on traffic. Most likely you are hitting App Engine per-minute rate limits then. Your traffic goes up, which leads to increased requests between your AB servers, which causes App Engine throttling to kick in to protect resource usage. The best way to fix this is to consolidate your applications; instead of running applications A and B, run them within a single application ID as separate versions or as different modules, then communicate intra-app using task queues and other methods. Longer-term, the even better option would be to refactor your application code so shared services are available as libraries and not as individual endpoints. On Tue, Dec 10, 2013 at 2:52 AM, Ananthakrishnan Venkatasubramanian ananthakrishnan.venkatasubraman...@a-cti.com wrote: We don't have appstats available at the moment. Why not? Install it using the guide available at: https://developers.google.com/appengine/docs/java/tools/appstats - -Vinny P Technology Media Advisor Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] SocketTimeoutException from one AppEngine Server to another
On Wed, Dec 11, 2013 at 2:33 AM, Ananthakrishnan Venkatasubramanian ananthakrishnan.venkatasubraman...@a-cti.com wrote: Libraries will increase the size of the application or the parent module if you opt for modules. Deploying the app with libraries is old fashion...Its not good to follow if you consider the growth of the application. REST based service is the current trend. Regardless of the newest trends (that's an entirely different fight) the issue at hand is that intra-app and inter-app communications have often experienced difficulties on App Engine. This is not a new topic: see here for a discussion on intra-app urlfetches being subject to rate limiting: https://groups.google.com/forum/#!topic/google-appengine/iTMUY1TA8BU and the associated issue: http://code.google.com/p/googleappengine/issues/detail?id=8052 . It's better, at least for the short term until Google fixes these issues, to consolidate shared libraries or to communicate in other manners such as Task Queues. On Wed, Dec 11, 2013 at 2:33 AM, Ananthakrishnan Venkatasubramanian ananthakrishnan.venkatasubraman...@a-cti.com wrote: So there is no question of exceeding the per minute quota limits or any other for that matter. There are other quotas, throttling, and limitations internal to App Engine other than the per-minute quotas listed on the quotas page. It's been discussed before on this forum. On Wed, Dec 11, 2013 at 2:33 AM, Ananthakrishnan Venkatasubramanian ananthakrishnan.venkatasubraman...@a-cti.com wrote: All this together, does that mean SOA (Service Oriented Architecture) or REST API Services will fluctuate in Google AppEngine (ie) from AppEngine Server to another AppEngine Server very often? Depends on the use case, the profile of the requests (do the requests spike, or are they constant), the size of the request, size of the response, and many other factors. - -Vinny P Technology Media Advisor Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] SocketTimeoutException from one AppEngine Server to another
On Tue, Dec 10, 2013 at 2:52 AM, Ananthakrishnan Venkatasubramanian ananthakrishnan.venkatasubraman...@a-cti.com wrote: Server A B represent different application IDs and the error rate depends on traffic. Most likely you are hitting App Engine per-minute rate limits then. Your traffic goes up, which leads to increased requests between your AB servers, which causes App Engine throttling to kick in to protect resource usage. The best way to fix this is to consolidate your applications; instead of running applications A and B, run them within a single application ID as separate versions or as different modules, then communicate intra-app using task queues and other methods. Longer-term, the even better option would be to refactor your application code so shared services are available as libraries and not as individual endpoints. On Tue, Dec 10, 2013 at 2:52 AM, Ananthakrishnan Venkatasubramanian ananthakrishnan.venkatasubraman...@a-cti.com wrote: We don't have appstats available at the moment. Why not? Install it using the guide available at: https://developers.google.com/appengine/docs/java/tools/appstats - -Vinny P Technology Media Advisor Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] SocketTimeoutException from one AppEngine Server to another
Hi, Definitely Server A is not exceeding the per minute quota for URLFetch API Calls (32,000 calls per min). SocketTimeOutExceptions are coming even there is less traffic in both the Servers. Note: We have idle instances in both the servers Definitely there is no question of queueing the requests also. Requests should be served at any point of time. As far as AppStats is concerned, we already knew how to do it. Its a matter of enabling. We will enable it and give you the information about it in private. As you have suggested, we deployed the Server B as backend module and Server A as frontend module in the same App ID and used URLFetch in module A(Server A) to communicate with module B (Server B) of the same app. Still the same problem. Those exceptions are repeating. The only thing we need to test now with modules is cut down the URLFetch in module A(Server A) and communicate directly with module B(Server B). Technically we need to integrate the codebase of Server B inside module A(Server A). This we can do it and try out from our end. But my question is, imagine there are two different organisation using AppEngine and one of the sever in the organisation running in AppEngine needs to communicate with the server in the other organisation running same in AppEngine, they have to communicate with URLFetch calls only. We cannot force the organisations to create one common App and use modules to avoid SocketTimeOutExceptions. Can you please give the appropriate reasons/solutions for the same? Please let me know if you need any other information regarding this. Thanks. Regards, Anantha Krishnan. On Wednesday, December 11, 2013 10:24:14 AM UTC+5:30, Vinny P wrote: On Tue, Dec 10, 2013 at 2:52 AM, Ananthakrishnan Venkatasubramanian ananthakrishnan.venkatasubraman...@a-cti.com javascript: wrote: Server A B represent different application IDs and the error rate depends on traffic. Most likely you are hitting App Engine per-minute rate limits then. Your traffic goes up, which leads to increased requests between your AB servers, which causes App Engine throttling to kick in to protect resource usage. The best way to fix this is to consolidate your applications; instead of running applications A and B, run them within a single application ID as separate versions or as different modules, then communicate intra-app using task queues and other methods. Longer-term, the even better option would be to refactor your application code so shared services are available as libraries and not as individual endpoints. On Tue, Dec 10, 2013 at 2:52 AM, Ananthakrishnan Venkatasubramanian ananthakrishnan.venkatasubraman...@a-cti.com javascript: wrote: We don't have appstats available at the moment. Why not? Install it using the guide available at: https://developers.google.com/appengine/docs/java/tools/appstats - -Vinny P Technology Media Advisor Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] SocketTimeoutException from one AppEngine Server to another
Hi, We are a Premier Account User in GoogleAppEngine. We have two front-end servers running on Google AppEngine developed using Java. Front-end server A communicates with another front-end server B using URLFetch(java.net) request. We have set the maximum connectionTimeOut and readTimeOuts (6ms) also for URLFetch in server A. All the requests from server A to B works successfully 70% of the time. But we are finding weird scenarios like 30% of the time where server A would have given URLFetch request to Server B and throwing java.net.SocketTimeoutException: Timeout while fetching URL. Note: The request from Server A to B remains dead air for 60 seconds and there aren't any request being received at Server B for the same. There are no instance restarts on both the servers. But when the client retries the request again, it runs successfully. We have 2 idle instances and instance classes configured to F4 on both the servers. Can we please know why the request from Server A timeouts out without a request being received at the other server end (ie) Server B? Thanks. Regards, Anantha Krishnan -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] SocketTimeoutException from one AppEngine Server to another
On Mon, Dec 9, 2013 at 10:57 AM, Ananthakrishnan Venkatasubramanian ananthakrishnan.venkatasubraman...@a-cti.com wrote: Can we please know why the request from Server A timeouts out without a request being received at the other server end (ie) Server B? There are multiple possible reasons for this. First of all, do servers A and B represent different application IDs, different modules within the same application ID, or something else? Secondly, is the error rate always a constant 30%, or does it seem to fluctuate depending on traffic levels and/or the time of day? Can you post an AppStats ( https://developers.google.com/appengine/docs/java/tools/appstats ) analysis of a failed request, plus any exception/error notices that are coming up ( including a full stack trace of the SocketTimeoutException referred to in your title )? If you see the error rate fluctuate, you may be hitting the per-minute quotas governing resource usage. On the other hand if your error rate is a constant 30% while your traffic is fluctuating, you may have an issue with your application code. - -Vinny P Technology Media Advisor Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
