[google-appengine] Re: Socket Timeout errors on URLFetch calls

[Hugo Visser]

I think you are misunderstanding what I'm saying. On the Java 8 runtime you 
can do without URLFetch because the limitations that were in effect on the 
older runtimes are lifted. Because of that I can work around it by using 
any other http library, which is what the documentation is referring to. 
I'm glad that this is possible but it's still a workaround.

>From the errors I was encountering, I still believe the actual URLFetch 
service has an issue and has flaky behaviour (as Joshua also mentions), 
which should be resolved by Google as it may affect other apps using that 
service, especially older App Engine apps migrating from the previous 
runtime(s). 

On Thursday, March 29, 2018 at 10:42:18 PM UTC+2, George (Cloud Platform 
Support) wrote:
>
> In the "Issuing an HTTP request" sub-chapter of the "Issuing HTTP(S) 
> Requests" document 
> , 
> this possibility is indicated: "You can also use the third-party requests 
> library as long as you configure it to use URLFetch." There is no doubt 
> that other people will be interested in details about your successful 
> implementation. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/00547d40-4b24-42a4-bf6b-a410093dea79%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Socket Timeout errors on URLFetch calls

[Hugo Visser]

Not using URLFetchService seems to have worked, I'm not seeing any 
SocketTimeoutExceptions anymore since I've deployed a version of my app 
that uses OkHttp to replace the URLFetchService. Of course that might be a 
coincidence but the errors that have been occurring throughout the day are 
gone now. I actually implemented the URLFetchService interface for my 
OkHttp implementation, which I'm happy to share if someone needs it too.

On Thursday, March 29, 2018 at 8:23:11 PM UTC+2, Hugo Visser wrote:
>
> I'm literally getting java.net.SocketTimeoutException, meaning that the 
> connection to the host times out. I have billing enabled so that can't be 
> the cause, but maybe it's related.
>
> I thought the error was because that particular host was having troubles, 
> but I'm seeing it on other hosts too now. I'm did a quick change to replace 
> my URLFetch calls with normal socket based http calls through OkHttp, and 
> that seems to be better, though it's to soon to really tell.
>
> I'm using Java 8, but since this is an older app it's still using the 
> URLFetchService as that was the only way to do a request in the past. I 
> don't really need it to be replaced, I just need it to work like it used to 
> and not fail at random :-)
>
> On Thursday, March 29, 2018 at 8:06:32 PM UTC+2, George (Cloud Platform 
> Support) wrote:
>>
>> Hi Hugo, 
>>
>> What type of time out exceptions do you refer to? Which errors are the 
>> most frequently encountered? You are surely not the only one facing this 
>> situation. How would you like to see URLFetch replaced? Do you use Java 7 
>> or 8? Which features described on the "Issuing HTTP(S) Requests" 
>> documentation page 
>> <https://cloud.google.com/appengine/docs/standard/java/issue-requests> 
>> do you actually use? 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/8fb36292-ca06-48ca-a278-2d22f9969799%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Socket Timeout errors on URLFetch calls

[Hugo Visser]

I'm literally getting java.net.SocketTimeoutException, meaning that the 
connection to the host times out. I have billing enabled so that can't be 
the cause, but maybe it's related.

I thought the error was because that particular host was having troubles, 
but I'm seeing it on other hosts too now. I'm did a quick change to replace 
my URLFetch calls with normal socket based http calls through OkHttp, and 
that seems to be better, though it's to soon to really tell.

I'm using Java 8, but since this is an older app it's still using the 
URLFetchService as that was the only way to do a request in the past. I 
don't really need it to be replaced, I just need it to work like it used to 
and not fail at random :-)

On Thursday, March 29, 2018 at 8:06:32 PM UTC+2, George (Cloud Platform 
Support) wrote:
>
> Hi Hugo, 
>
> What type of time out exceptions do you refer to? Which errors are the 
> most frequently encountered? You are surely not the only one facing this 
> situation. How would you like to see URLFetch replaced? Do you use Java 7 
> or 8? Which features described on the "Issuing HTTP(S) Requests" 
> documentation page 
>  do 
> you actually use? 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/b61beab0-4d06-4438-ac51-40fa62873cc0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Socket Timeout errors on URLFetch calls

[Hugo Visser]

My app makes heavy use of URLFetch to get content and I'm seeing socket 
time out exceptions on App Engine standard (Java). Am I the only one 
experiencing this? Would ditching URLFetch for normal url connections fix 
this?

Thanks,

Hugo

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/7c5a6697-9a73-4a83-8ce6-8b5953a5b69a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: Task queue not executing tasks / backlogged

[Hugo Visser]

This has been happening on and off but today it's really off the charts. 
Currently have 28k tasks and none of them are executed. I filed #13118 for 
that.

On Friday, May 13, 2016 at 8:44:57 PM UTC+2, Nick (Cloud Platform Support) 
wrote:
>
> Thanks for the update. Barring a very low setting for "max concurrent 
> tasks", I don't see why this could occur, so it appears this could be a 
> production issue. In that case, I'd recommend opening a Public Issue 
> Tracker <http://code.google.com/p/googleappengine/issues/list> thread 
> with label "Restrict-View-EditIssue" (to protect your information) and the 
> following information: 
>
> * your app id
> * the name of the queue
> * the queue settings
> * some queue statistics
>
> We may be able to determine the root cause of this at that point. 
>
> Sincerely,
>
> Nick
> Cloud Platform Community Support
>
> On Friday, May 13, 2016 at 2:02:04 PM UTC-4, Hugo Visser wrote:
>>
>> Just got an alert again. The task rate is 25 tasks/sec with a bucket size 
>> of 10. Average latency of the handlers is varying depending on the task, 
>> but anywhere between 1 and 8 seconds.
>>
>> The thing that I'm seeing is that the task queue shows 120 tasks in the 
>> queue, 0 running, oldest task ETA 6 minutes ago. That doesn't seem right to 
>> me, and it looks like these tasks are just not being executed. I see 
>> nothing in the logs of my task handlers that indicate that a request is 
>> being executed. 
>>
>> Obviously this task execution thing is pretty essential to my app.
>>
>> Hugo
>>
>> On Wednesday, May 11, 2016 at 11:34:34 PM UTC+2, Nick (Cloud Platform 
>> Support) wrote:
>>>
>>> Thanks for coming back to update. Feel free to post again with the 
>>> information mentioned if you notice anything in the future. 
>>>
>>> Best wishes,
>>>
>>> Nick
>>> Cloud Platform Community Support
>>>
>>> On Wednesday, May 11, 2016 at 4:33:36 PM UTC-4, Hugo Visser wrote:
>>>>
>>>> It looks like the queues are back to normal. I've setup alerts in stack 
>>>> driver that triggered for a while but are resolved now. Will report back 
>>>> if 
>>>> that changes.
>>>>
>>>> On Wed, May 11, 2016, 21:08 'Nick (Cloud Platform Support)' via Google 
>>>> App Engine <google-a...@googlegroups.com> wrote:
>>>>
>>>>> Hey Hugo,
>>>>>
>>>>> Without knowing more about A) the configuration of the queue, B) the 
>>>>> rate of task creation, C) the latency statistics on the task handler, 
>>>>> it's 
>>>>> difficult to say much on why this might be happening. If you could gather 
>>>>> all those types of information, myself or someone else may be able to 
>>>>> leave 
>>>>> a more insightful comment.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Nick
>>>>> Cloud Platform Community Support
>>>>>
>>>>>
>>>>> On Tuesday, May 10, 2016 at 4:54:00 AM UTC-4, Hugo Visser wrote:
>>>>>>
>>>>>> I'm seeing tasks in my queue that are there for 6 minutes without any 
>>>>>> tasks running at all. There doesn't seem to be much I can do about this 
>>>>>> which is really frustrating.
>>>>>>
>>>>>> On Tuesday, May 10, 2016 at 8:54:33 AM UTC+2, Hugo Visser wrote:
>>>>>>>
>>>>>>> Since last night or so I'm seeing this issue where tasks are being 
>>>>>>> queued up for some time and not being executed on an app that has been 
>>>>>>> running like this for quite a while. This results in a huge pile of 
>>>>>>> tasks 
>>>>>>> that get backlogged for my app.
>>>>>>>
>>>>>>> Cloud status dashboard says everything is OK. Anyone else seeing 
>>>>>>> this?
>>>>>>>
>>>>>>> Hugo
>>>>>>>
>>>>>> -- 
>>>>> You received this message because you are subscribed to a topic in the 
>>>>> Google Groups "Google App Engine" group.
>>>>> To unsubscribe from this topic, visit 
>>>>> https://groups.google.com/d/topic/google-appengine/MYxDL8sihN4/unsubscribe
>>>>> .
>>>>> To unsubscribe from this group and all its topics, send an email to 
>>>>

Re: [google-appengine] Re: Task queue not executing tasks / backlogged

[Hugo Visser]

Hitting the "run now" button makes the task run by the way. I don't see any 
relation to error rates on the task handler instances either. Looks like 
they are just stuck there waiting for execution.

On Friday, May 13, 2016 at 8:02:04 PM UTC+2, Hugo Visser wrote:
>
> Just got an alert again. The task rate is 25 tasks/sec with a bucket size 
> of 10. Average latency of the handlers is varying depending on the task, 
> but anywhere between 1 and 8 seconds.
>
> The thing that I'm seeing is that the task queue shows 120 tasks in the 
> queue, 0 running, oldest task ETA 6 minutes ago. That doesn't seem right to 
> me, and it looks like these tasks are just not being executed. I see 
> nothing in the logs of my task handlers that indicate that a request is 
> being executed. 
>
> Obviously this task execution thing is pretty essential to my app.
>
> Hugo
>
> On Wednesday, May 11, 2016 at 11:34:34 PM UTC+2, Nick (Cloud Platform 
> Support) wrote:
>>
>> Thanks for coming back to update. Feel free to post again with the 
>> information mentioned if you notice anything in the future. 
>>
>> Best wishes,
>>
>> Nick
>> Cloud Platform Community Support
>>
>> On Wednesday, May 11, 2016 at 4:33:36 PM UTC-4, Hugo Visser wrote:
>>>
>>> It looks like the queues are back to normal. I've setup alerts in stack 
>>> driver that triggered for a while but are resolved now. Will report back if 
>>> that changes.
>>>
>>> On Wed, May 11, 2016, 21:08 'Nick (Cloud Platform Support)' via Google 
>>> App Engine <google-a...@googlegroups.com> wrote:
>>>
>>>> Hey Hugo,
>>>>
>>>> Without knowing more about A) the configuration of the queue, B) the 
>>>> rate of task creation, C) the latency statistics on the task handler, it's 
>>>> difficult to say much on why this might be happening. If you could gather 
>>>> all those types of information, myself or someone else may be able to 
>>>> leave 
>>>> a more insightful comment.
>>>>
>>>> Cheers,
>>>>
>>>> Nick
>>>> Cloud Platform Community Support
>>>>
>>>>
>>>> On Tuesday, May 10, 2016 at 4:54:00 AM UTC-4, Hugo Visser wrote:
>>>>>
>>>>> I'm seeing tasks in my queue that are there for 6 minutes without any 
>>>>> tasks running at all. There doesn't seem to be much I can do about this 
>>>>> which is really frustrating.
>>>>>
>>>>> On Tuesday, May 10, 2016 at 8:54:33 AM UTC+2, Hugo Visser wrote:
>>>>>>
>>>>>> Since last night or so I'm seeing this issue where tasks are being 
>>>>>> queued up for some time and not being executed on an app that has been 
>>>>>> running like this for quite a while. This results in a huge pile of 
>>>>>> tasks 
>>>>>> that get backlogged for my app.
>>>>>>
>>>>>> Cloud status dashboard says everything is OK. Anyone else seeing this?
>>>>>>
>>>>>> Hugo
>>>>>>
>>>>> -- 
>>>> You received this message because you are subscribed to a topic in the 
>>>> Google Groups "Google App Engine" group.
>>>> To unsubscribe from this topic, visit 
>>>> https://groups.google.com/d/topic/google-appengine/MYxDL8sihN4/unsubscribe
>>>> .
>>>> To unsubscribe from this group and all its topics, send an email to 
>>>> google-appengi...@googlegroups.com.
>>>> To post to this group, send email to google-a...@googlegroups.com.
>>>> Visit this group at https://groups.google.com/group/google-appengine.
>>>> To view this discussion on the web visit 
>>>> https://groups.google.com/d/msgid/google-appengine/370b31f3-e1fa-4a2f-9287-652aeac0500c%40googlegroups.com
>>>>  
>>>> <https://groups.google.com/d/msgid/google-appengine/370b31f3-e1fa-4a2f-9287-652aeac0500c%40googlegroups.com?utm_medium=email_source=footer>
>>>> .
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/25f4084e-0759-4c46-aaba-63bb90f9541b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: Task queue not executing tasks / backlogged

[Hugo Visser]

Just got an alert again. The task rate is 25 tasks/sec with a bucket size 
of 10. Average latency of the handlers is varying depending on the task, 
but anywhere between 1 and 8 seconds.

The thing that I'm seeing is that the task queue shows 120 tasks in the 
queue, 0 running, oldest task ETA 6 minutes ago. That doesn't seem right to 
me, and it looks like these tasks are just not being executed. I see 
nothing in the logs of my task handlers that indicate that a request is 
being executed. 

Obviously this task execution thing is pretty essential to my app.

Hugo

On Wednesday, May 11, 2016 at 11:34:34 PM UTC+2, Nick (Cloud Platform 
Support) wrote:
>
> Thanks for coming back to update. Feel free to post again with the 
> information mentioned if you notice anything in the future. 
>
> Best wishes,
>
> Nick
> Cloud Platform Community Support
>
> On Wednesday, May 11, 2016 at 4:33:36 PM UTC-4, Hugo Visser wrote:
>>
>> It looks like the queues are back to normal. I've setup alerts in stack 
>> driver that triggered for a while but are resolved now. Will report back if 
>> that changes.
>>
>> On Wed, May 11, 2016, 21:08 'Nick (Cloud Platform Support)' via Google 
>> App Engine <google-a...@googlegroups.com > wrote:
>>
>>> Hey Hugo,
>>>
>>> Without knowing more about A) the configuration of the queue, B) the 
>>> rate of task creation, C) the latency statistics on the task handler, it's 
>>> difficult to say much on why this might be happening. If you could gather 
>>> all those types of information, myself or someone else may be able to leave 
>>> a more insightful comment.
>>>
>>> Cheers,
>>>
>>> Nick
>>> Cloud Platform Community Support
>>>
>>>
>>> On Tuesday, May 10, 2016 at 4:54:00 AM UTC-4, Hugo Visser wrote:
>>>>
>>>> I'm seeing tasks in my queue that are there for 6 minutes without any 
>>>> tasks running at all. There doesn't seem to be much I can do about this 
>>>> which is really frustrating.
>>>>
>>>> On Tuesday, May 10, 2016 at 8:54:33 AM UTC+2, Hugo Visser wrote:
>>>>>
>>>>> Since last night or so I'm seeing this issue where tasks are being 
>>>>> queued up for some time and not being executed on an app that has been 
>>>>> running like this for quite a while. This results in a huge pile of tasks 
>>>>> that get backlogged for my app.
>>>>>
>>>>> Cloud status dashboard says everything is OK. Anyone else seeing this?
>>>>>
>>>>> Hugo
>>>>>
>>>> -- 
>>> You received this message because you are subscribed to a topic in the 
>>> Google Groups "Google App Engine" group.
>>> To unsubscribe from this topic, visit 
>>> https://groups.google.com/d/topic/google-appengine/MYxDL8sihN4/unsubscribe
>>> .
>>> To unsubscribe from this group and all its topics, send an email to 
>>> google-appengi...@googlegroups.com .
>>> To post to this group, send email to google-a...@googlegroups.com 
>>> .
>>> Visit this group at https://groups.google.com/group/google-appengine.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/google-appengine/370b31f3-e1fa-4a2f-9287-652aeac0500c%40googlegroups.com
>>>  
>>> <https://groups.google.com/d/msgid/google-appengine/370b31f3-e1fa-4a2f-9287-652aeac0500c%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/29d999c4-10fd-4c5c-9f13-fb185aa3be32%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: Task queue not executing tasks / backlogged

[Hugo Visser]

It looks like the queues are back to normal. I've setup alerts in stack
driver that triggered for a while but are resolved now. Will report back if
that changes.

On Wed, May 11, 2016, 21:08 'Nick (Cloud Platform Support)' via Google App
Engine <google-appengine@googlegroups.com> wrote:

> Hey Hugo,
>
> Without knowing more about A) the configuration of the queue, B) the rate
> of task creation, C) the latency statistics on the task handler, it's
> difficult to say much on why this might be happening. If you could gather
> all those types of information, myself or someone else may be able to leave
> a more insightful comment.
>
> Cheers,
>
> Nick
> Cloud Platform Community Support
>
>
> On Tuesday, May 10, 2016 at 4:54:00 AM UTC-4, Hugo Visser wrote:
>>
>> I'm seeing tasks in my queue that are there for 6 minutes without any
>> tasks running at all. There doesn't seem to be much I can do about this
>> which is really frustrating.
>>
>> On Tuesday, May 10, 2016 at 8:54:33 AM UTC+2, Hugo Visser wrote:
>>>
>>> Since last night or so I'm seeing this issue where tasks are being
>>> queued up for some time and not being executed on an app that has been
>>> running like this for quite a while. This results in a huge pile of tasks
>>> that get backlogged for my app.
>>>
>>> Cloud status dashboard says everything is OK. Anyone else seeing this?
>>>
>>> Hugo
>>>
>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Google App Engine" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/google-appengine/MYxDL8sihN4/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at https://groups.google.com/group/google-appengine.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/google-appengine/370b31f3-e1fa-4a2f-9287-652aeac0500c%40googlegroups.com
> <https://groups.google.com/d/msgid/google-appengine/370b31f3-e1fa-4a2f-9287-652aeac0500c%40googlegroups.com?utm_medium=email_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/CAMSc4X2ntazGDjxzG4yqS9_ENECNtFRVrAbopW6Jrf4b_SeAeg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Task queue not executing tasks / backlogged

[Hugo Visser]

I'm seeing tasks in my queue that are there for 6 minutes without any tasks 
running at all. There doesn't seem to be much I can do about this which is 
really frustrating.

On Tuesday, May 10, 2016 at 8:54:33 AM UTC+2, Hugo Visser wrote:
>
> Since last night or so I'm seeing this issue where tasks are being queued 
> up for some time and not being executed on an app that has been running 
> like this for quite a while. This results in a huge pile of tasks that get 
> backlogged for my app.
>
> Cloud status dashboard says everything is OK. Anyone else seeing this?
>
> Hugo
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/49fb1901-a198-4ef2-8879-331802189c0b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Task queue not executing tasks / backlogged

[Hugo Visser]

Since last night or so I'm seeing this issue where tasks are being queued 
up for some time and not being executed on an app that has been running 
like this for quite a while. This results in a huge pile of tasks that get 
backlogged for my app.

Cloud status dashboard says everything is OK. Anyone else seeing this?

Hugo

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/8f32f6ca-ba45-4965-bdb2-bb59671bbd86%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Problems with SSL on a custom domain

[Hugo Visser]

OK, got another one on Sky. I'll ask for the info and will try to reach out 
to the other ones that have contacted me.

Hugo

On Wednesday, September 2, 2015 at 12:16:28 AM UTC+2, Jon Travers wrote:
>
> Sorry, in point 2 of my explanation above I should have said the 
> "ClientHello" message, not "ServerHello".
>
> On Tuesday, 1 September 2015 23:06:51 UTC+1, Jon Travers wrote:
>>
>> Hi Patrice
>>
>> I can confirm that our office broadband, where we've consistently seen 
>> this problem, is indeed a Sky connection. Since I last posted, I've done 
>> some further investigation using Wireshark to capture and decode the TSL 
>> protocol traffic being sent by Safari, Chrome, and Curl. My conclusion was 
>> that the connection failure is caused by a very specific combination of 
>> factors:
>>
>>1. There is some specific property of the TLS client setup that must 
>>be present. I'm unable to identify exactly what, but on my machine it 
>> fails 
>>with up-to-date versions of Safari and Chrome, and also with the built-in 
>>OpenSSL (0.9.8zg), but not with the version of Curl that I have, nor with 
>>an updated version of OpenSSL. I can see differences in the connection 
>>parameters between these clients, but it's unclear which one is the cause.
>>2. The Initial TLS "ServerHello" message is being modified in a 
>>specific way by some IP routing node between my computer, and the 
>> endpoint 
>>at ghs.googlehosted.com. Based on all the evidence, we believe this 
>>is definitely happening inside our ISP, Sky Broadband, but the same issue 
>>could also occur on other routes. Whether this is the result of a 
>>configuration error at Sky, or the result of something they're doing 
>>deliberately, and how widespread this problem is, is all unclear.
>>3. The server at the destination of the SNI TLS connection, 
>>ghs.googlehosted.com, has been configured in such a way that the 
>>combination of 1 and 2 is regarded as a terminal error, and the 
>> connection 
>>is immediately terminated with an 'Illegal parameter' error. Connecting 
>> to 
>>another SNI SSL server (our own proxy) does not trigger the same error 
>>response, but since I can't see the traffic as it arrives at the Google, 
>>and I don't have access to any further debug information from the server, 
>> I 
>>can't determine whether the rejection is valid. Most likely it is, and 
>> our 
>>own proxy works simply because it has a less strict security setup.
>>
>> Now with these conclusions established, the only feasible solution I 
>> could see was to try running the connection via a dedicated virtual IP 
>> address (which costs $39 per month), rather than using the (free) SNI 
>> setup. I finally gave in, paid our $39, updated our DNS records, and it 
>> worked perfectly, no more connection errors. It's not exactly clear why 
>> this works, but I'm actually very happy with this as a solution (other than 
>> having to pay), especially since even if Sky fixes their routing error, I 
>> can still be confident that if the same problem crops up somewhere else on 
>> the Internet we won't be affected.
>>
>> *Bottom line: I would strongly advise anybody reading this to regard the 
>> Google Apps SNI SSL service as inherently unreliable. Pay the money and go 
>> with a virtual IP if at all possible, because the last thing you want is 
>> for some of your customers to be unable to access your app depending on 
>> where they are. I also think this advice should be clearly given in the 
>> Google documentation.*
>>
>> Now Patrice, if you still want to investigate the specific issue at Sky, 
>> then I'm very happy to provide you with additional debug information, once 
>> I get into the office tomorrow morning (it's 11pm here now and I'm at 
>> home). Can you be more specific about what you want? When you talk about 
>> tcpdump output, are you looking for a capture of the network packets? So my 
>> Wireshark capture would also be OK?
>>
>> Cheers
>> Jon
>>
>> On Tuesday, 1 September 2015 20:03:50 UTC+1, Patrice (Cloud Platform 
>> Support) wrote:
>>>
>>> Hi Hugo,
>>>
>>> Exactly what I was expecting. Whoever reports these always seem to be on 
>>> Sky, which could be the problem.
>>>
>>> I don't know if you'll be able to get this since it's your users and not 
>>> you directly, but if you could get a tcpdump and a print screen of the page 
>>> "chro

[google-appengine] Re: Problems with SSL on a custom domain

[Hugo Visser]

The latest report came from a user on Sky. He also mentioned that sometimes 
it works, and sometimes it doesn't. I can't verify if that's true though. 
I've asked both users to open a browser to my custom domain app url. I've 
now resorted to updating the app and setting the endpoint to the appspot 
domain, which is not what I'd want ideally, but I don't want to lose any 
users over it either.

The user that contacted me stated that it broke several weeks ago.

Hugo

On Tuesday, September 1, 2015 at 5:16:34 PM UTC+2, Patrice (Cloud Platform 
Support) wrote:
>
> Hi Hugo, 
>
> Continuing to look into this, I realized that a lot of people who have 
> similar issues are all using a precise ISP. Do you know the ISP that your 
> users have?
>
> Cheers!
>
> On Saturday, August 29, 2015 at 2:37:01 PM UTC-4, Hugo Visser wrote:
>>
>> Like I mentioned previously, today I got another user from the UK 
>> reporting a similar issue. For some reason the SNI SSL served version of my 
>> app isn't reachable or working for them, but going to 
>> https://myapp.appspot.com works. I don't see a huge drop in usage in the 
>> app so I can't really tell what is causing it, just that those users can't 
>> establish a SSL connection to my custom domain url from their devices.
>>
>> Hugo
>>
>> On Wednesday, August 26, 2015 at 9:09:52 PM UTC+2, Patrice (Cloud 
>> Platform Support) wrote:
>>>
>>> Hi again Jon,
>>>
>>> I continued trying to find what is exactly happening here. So here goes :
>>>
>>> Running this in San Francisco :
>>>
>>> openssl s_client -debug -msg -servername dashboard.geospock.com 
>>> -connect dashboard.geospock.com:443 -showcerts 
>>>
>>> works perfectly and consistently. So I think we can all see that SF 
>>> won't have issues connecting here.
>>>
>>> I then connected through an European location, and from there, trying 
>>> this also succeeds:
>>>
>>> openssl s_client -servername dashboard.geospock.com -connect 
>>> 64.233.167.121:443 -showcerts 
>>> curl -k -I --resolve dashboard.geospock.com:443:64.233.167.121 
>>> https://dashboard.geospock.com/ 
>>>
>>> From your error and your message, I get you're using TLS 1.0, while I am 
>>> running with TLS 1.2. I started looking into the version of openSSL we're 
>>> both using, and while you're using the 0.98zg, I am on 1.0.1f. 
>>>
>>> I believe since 0.9.8j SNI support was part of OpenSSL, so you using 
>>> 0.98zg should work fine, but as a test, do you mind trying to run the same 
>>> with 1.01 latest? I believe they are up to 1.0.1p. If I remember correctly, 
>>> there was a backport patch implemented in 0.9.8j to let openSSL work with 
>>> SNI (https://en.wikipedia.org/wiki/Server_Name_Indication ).
>>>
>>> The basis of the issue we have here is that I'm incapable of reproducing 
>>> any of the behaviors you're experiencing, so it's hard to investigate 
>>> further. I'm definitely interested in continuing this, but without a 
>>> reliable way for me to replicate, it's impossible to send it up the chain 
>>> to get it looked at and possibly fixed. If you have a specific test case 
>>> that consistently fail, that I can then reproduce on my side as 
>>> consistently, I'll be able to get some traction on this. 
>>>
>>> Thanks
>>>
>>> On Wednesday, August 26, 2015 at 11:16:54 AM UTC-4, Jon Travers wrote:
>>>>
>>>> Here is some more detailed debugging information from the failing 
>>>> openssl SSL negotiation. Perhaps this would give you a clue what's 
>>>> actually 
>>>> going on?:
>>>>
>>>> $ openssl s_client -debug -msg -servername dashboard.geospock.com 
>>>> -connect dashboard.geospock.com:443 -showcerts
>>>> CONNECTED(0003)
>>>> write to 0x7f96b8d006a0 [0x7f96b9802000] (131 bytes => 131 (0x83))
>>>>  - 16 03 01 00 7e 01 00 00-7a 03 01 55 dd d7 93 c7   
>>>> ~...z..U
>>>> 0010 - f0 b2 0d ee ea f4 1c 2b-ee 50 b6 ff 0f e6 8f 59   
>>>> ...+.P.Y
>>>> 0020 - 8b 81 9e 05 2f 17 84 e2-20 ed b7 00 00 2e 00 39   /... 
>>>> ..9
>>>> 0030 - 00 38 00 35 00 16 00 13-00 0a 00 33 00 32 00 2f   
>>>> .8.5...3.2./
>>>> 0040 - 00 9a 00 99 00 96 00 05-00 04 00 15 00 12 00 09   
>>>> 
>>>> 0050 - 00 14 00 11 00 08 00 06-00 03 00 ff 01 00 00 23   
>>>> ...#
>>>> 0

[google-appengine] Re: Problems with SSL on a custom domain

[Hugo Visser]

Like I mentioned previously, today I got another user from the UK reporting 
a similar issue. For some reason the SNI SSL served version of my app isn't 
reachable or working for them, but going to https://myapp.appspot.com 
works. I don't see a huge drop in usage in the app so I can't really tell 
what is causing it, just that those users can't establish a SSL connection 
to my custom domain url from their devices.

Hugo

On Wednesday, August 26, 2015 at 9:09:52 PM UTC+2, Patrice (Cloud Platform 
Support) wrote:

 Hi again Jon,

 I continued trying to find what is exactly happening here. So here goes :

 Running this in San Francisco :

 openssl s_client -debug -msg -servername dashboard.geospock.com -connect 
 dashboard.geospock.com:443 -showcerts 

 works perfectly and consistently. So I think we can all see that SF won't 
 have issues connecting here.

 I then connected through an European location, and from there, trying this 
 also succeeds:

 openssl s_client -servername dashboard.geospock.com -connect 
 64.233.167.121:443 -showcerts 
 curl -k -I --resolve dashboard.geospock.com:443:64.233.167.121 
 https://dashboard.geospock.com/ 

 From your error and your message, I get you're using TLS 1.0, while I am 
 running with TLS 1.2. I started looking into the version of openSSL we're 
 both using, and while you're using the 0.98zg, I am on 1.0.1f. 

 I believe since 0.9.8j SNI support was part of OpenSSL, so you using 
 0.98zg should work fine, but as a test, do you mind trying to run the same 
 with 1.01 latest? I believe they are up to 1.0.1p. If I remember correctly, 
 there was a backport patch implemented in 0.9.8j to let openSSL work with 
 SNI (https://en.wikipedia.org/wiki/Server_Name_Indication ).

 The basis of the issue we have here is that I'm incapable of reproducing 
 any of the behaviors you're experiencing, so it's hard to investigate 
 further. I'm definitely interested in continuing this, but without a 
 reliable way for me to replicate, it's impossible to send it up the chain 
 to get it looked at and possibly fixed. If you have a specific test case 
 that consistently fail, that I can then reproduce on my side as 
 consistently, I'll be able to get some traction on this. 

 Thanks

 On Wednesday, August 26, 2015 at 11:16:54 AM UTC-4, Jon Travers wrote:

 Here is some more detailed debugging information from the failing openssl 
 SSL negotiation. Perhaps this would give you a clue what's actually going 
 on?:

 $ openssl s_client -debug -msg -servername dashboard.geospock.com 
 -connect dashboard.geospock.com:443 -showcerts
 CONNECTED(0003)
 write to 0x7f96b8d006a0 [0x7f96b9802000] (131 bytes = 131 (0x83))
  - 16 03 01 00 7e 01 00 00-7a 03 01 55 dd d7 93 c7   ~...z..U
 0010 - f0 b2 0d ee ea f4 1c 2b-ee 50 b6 ff 0f e6 8f 59   ...+.P.Y
 0020 - 8b 81 9e 05 2f 17 84 e2-20 ed b7 00 00 2e 00 39   /... ..9
 0030 - 00 38 00 35 00 16 00 13-00 0a 00 33 00 32 00 2f   .8.5...3.2./
 0040 - 00 9a 00 99 00 96 00 05-00 04 00 15 00 12 00 09   
 0050 - 00 14 00 11 00 08 00 06-00 03 00 ff 01 00 00 23   ...#
 0060 - 00 00 00 1b 00 19 00 00-16 64 61 73 68 62 6f 61   .dashboa
 0070 - 72 64 2e 67 65 6f 73 70-6f 63 6b 2e 63 6f 6d 00   rd.geospock.com.
 0080 - 23#
 0083 - SPACES/NULS
  TLS 1.0 Handshake [length 007e], ClientHello
 01 00 00 7a 03 01 55 dd d7 93 c7 f0 b2 0d ee ea
 f4 1c 2b ee 50 b6 ff 0f e6 8f 59 8b 81 9e 05 2f
 17 84 e2 20 ed b7 00 00 2e 00 39 00 38 00 35 00
 16 00 13 00 0a 00 33 00 32 00 2f 00 9a 00 99 00
 96 00 05 00 04 00 15 00 12 00 09 00 14 00 11 00
 08 00 06 00 03 00 ff 01 00 00 23 00 00 00 1b 00
 19 00 00 16 64 61 73 68 62 6f 61 72 64 2e 67 65
 6f 73 70 6f 63 6b 2e 63 6f 6d 00 23 00 00
 read from 0x7f96b8d006a0 [0x7f96b9807600] (7 bytes = 7 (0x7))
  - 15 03 01 00 02 02 2f  ../
  TLS 1.0 Alert [length 0002], fatal illegal_parameter
 02 2f
 8175:error:14077417:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert 
 illegal 
 parameter:/SourceCache/OpenSSL098/OpenSSL098-52.40.1/src/ssl/s23_clnt.c:593:



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/a5153c60-8281-4606-816f-8f345aec2edc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Configuring SSL for custom domain failing with error 1000

[Hugo Visser]

OK, to follow up: I removed the SSL billing and enabled it again. Then the 
error was gone. However, now a different problem occurs in that I can't get 
my RapidSSL certificate working. 

I've included the issued certificate + all of the intermediate certificates 
in the chain. When the certificate is shown in Google Apps it shows a 
strange OU attribute which is not in the certificate as far as I can tell. 
The Google Apps console doesn't show me the info I need to verify the 
certificate is imported correctly. Does anybody else use RapidSSL certs?

On Thursday, March 6, 2014 1:24:23 PM UTC+1, Hugo Visser wrote:

 I'm trying to setup ssl for a custom domain. Billing is enabled and I'm 
 actually being charged for SSL already, but I'm unable to associate the SSL 
 certificate with the domain. When I try to save the changes, all I get is a 
 generic error 1000 error. 
 I've read on the list that others have experienced the same, so I waited a 
 day to see if that would make any difference, but it doesn't appear to be 
 the case. Also I don't like being billed for stuff I can't use.

 Who do I need to contact to resolve this issue?

 Thanks,

 Hugo


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Configuring SSL for custom domain failing with error 1000

[Hugo Visser]

OK, sorry to spam you all, but apparently it took 10 minutes or so since 
now SSL appears to be working fine.

On Friday, March 7, 2014 1:57:18 PM UTC+1, Hugo Visser wrote:

 OK, to follow up: I removed the SSL billing and enabled it again. Then the 
 error was gone. However, now a different problem occurs in that I can't get 
 my RapidSSL certificate working. 

 I've included the issued certificate + all of the intermediate 
 certificates in the chain. When the certificate is shown in Google Apps it 
 shows a strange OU attribute which is not in the certificate as far as I 
 can tell. The Google Apps console doesn't show me the info I need to verify 
 the certificate is imported correctly. Does anybody else use RapidSSL certs?

 On Thursday, March 6, 2014 1:24:23 PM UTC+1, Hugo Visser wrote:

 I'm trying to setup ssl for a custom domain. Billing is enabled and I'm 
 actually being charged for SSL already, but I'm unable to associate the SSL 
 certificate with the domain. When I try to save the changes, all I get is a 
 generic error 1000 error. 
 I've read on the list that others have experienced the same, so I waited 
 a day to see if that would make any difference, but it doesn't appear to be 
 the case. Also I don't like being billed for stuff I can't use.

 Who do I need to contact to resolve this issue?

 Thanks,

 Hugo



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Configuring SSL for custom domain failing with error 1000

[Hugo Visser]

I'm trying to setup ssl for a custom domain. Billing is enabled and I'm 
actually being charged for SSL already, but I'm unable to associate the SSL 
certificate with the domain. When I try to save the changes, all I get is a 
generic error 1000 error. 
I've read on the list that others have experienced the same, so I waited a 
day to see if that would make any difference, but it doesn't appear to be 
the case. Also I don't like being billed for stuff I can't use.

Who do I need to contact to resolve this issue?

Thanks,

Hugo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: SSL VIP

[Hugo Visser]

The docs state Each VIP only supports one certificate. That implies a VIP 
for each SSL domain you want to deploy, unless you are using a wildcard SSL 
certificate of some kind.

On Saturday, August 17, 2013 4:08:25 PM UTC+2, James Gilliam wrote:

 I noticed that the SSL VIPs cost was reduced (a while ago) from $99 per 
 month to $39 per month.  I realize you have to get certificates for each 
 SSL as well.

 My question is can you use one SSL VIP to support multiple domains?  Or do 
 you need a VIP for each different domain you want https ?

 If this question is dealt with somewhere else, please refer me.  I 
 couldn't find the answer on this doc page --

 https://developers.google.com/appengine/docs/ssl?hl=en

 Thanks so much


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Flood of warm up requests!

[Hugo Visser]

Same issue yesterday on the Java runtime. Now I'm seeing a lot of warm up 
requests too, almost one for every request I do, despite the fact I have 
resident instances. I've filed a billing issue to request a refund for 
charged resources.

Hugo

On Tuesday, June 25, 2013 10:49:00 PM UTC+2, chilang wrote:

 Hi,

 I'm seeing 15 warm up requests per secs causing 60+ instances to be spin 
 up, despite having little traffic!

 I haven't changed anything (code nor settings) recently. 
 This has started in the last 2 hours. 

 I've only noticed this due to app suddenly running out of quota. 
 I'm trying to cap Idle Instances to 1-1 to no avail. I've increased the 
 quota but at this rate it is going to burn through it very soon.

 Anyone else seeing this ?

 Regards,
 Chi Lang


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: New Google+ sign-in on App Engine, bug or feature?

[Hugo Visser]

My guess would be feature, since you are basically using external 
authentication that App Engine knows nothing about. So you are actually not 
using the user service in this case.

Hugo

On Thursday, February 28, 2013 12:32:03 AM UTC+1, Antonio Zugaldia wrote:

 I tried the Python 
 quickstarthttps://developers.google.com/+/quickstart/pythonfor the new 
 Google+ sign-in on App Engine and I'm glad to report it works 
 flawlessly (you just need to manually add Flask).

 However, once I've got an user logged in, if I call 
 users.get_current_user() from google.appengine.api I still get a None 
 response.

 So I have a Google user logged in but AppEngine's API is not seeing it. 
 Bug or feature?


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Google Apps No Longer Free

[Hugo Visser]

I think allowing to send from an alias would be very helpful. When running 
from an app, you most of the time don't need the actual inbox, you just 
need to be able to send from a email address originating from your domain. 
Currently every sender has to be a real account, but it would be nice if 
you can just add that account and with that also allow the aliases fo that 
account to send email.

On Friday, December 7, 2012 8:11:24 AM UTC+1, Thomas Wiradikusuma wrote:

 Hi Greg (of Google),

 I agree of what the other Greg said. It's very common to send 
 transactional emails from a dedicated address (e.g. noreply). It's not 
 professional (and even raise suspicion) if the Click here to reset your 
 password email comes from j...@startupname.com javascript: for example.

 If it's not possible to increase the account from 1 to n, at least please 
 allow the use of alias.


 On Friday, 7 December 2012 10:42:42 UTC+8, Greg wrote:

 Just saw that Google Apps is no longer free for 
 businesseshttp://googleenterprise.blogspot.co.nz/2012/12/changes-to-google-apps-for-businesses.html
 . 

 I have no problem paying for a Google Apps account where I actually use 
 Google apps, but at the moment you have to have a Google Apps account to 
 link a domain to an Appengine app. Some of our apps have two or three 
 domains showing the same app, and because you need to have an account for 
 each email address that Appengine sends email from, we have three or four 
 accounts per domain. So this is potentially going to add $600 per year to 
 our costs - all for virtual accounts that don't actually use Google Apps at 
 all.

 Can someone from Google comment please? Either Google Apps accounts need 
 to remain free if they are associated with Appengine apps, or there needs 
 to be another way to link domains (and authorise email addresses) for 
 Appengine.




-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/RB8eMtbyGKYJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Issues associated with enabling multi-threading via threadsafe

[Hugo Visser]

Hi Andrew,

To me, the whole point is kinda moot, since threadsafe is a required 
setting for Java apps (which is a good thing).

Also, the points that you raise in the post are not different from the 
gotcha's as they are present in a normal Java servlet stack. There's not 
much App Engine specific in there.

The statements like static initializers are safe are technically not 
correct; while it's true for a single instance running multiple threads, 
your app will still run on multiple instances. So if you are doing 
something funky in a static initializer (like writing to the datastore) 
there's still the possibility of race conditions or data corruption.
Same holds true for writing to memcache and the data store that you 
describe in the post: even if your app is single threaded, a production app 
will run on multiple instances which could all concurrently write to the 
same entity or memcache entry.

I hope you find this useful :)

Hugo

On Monday, September 24, 2012 2:41:25 PM UTC+2, Andrew Mackenzie wrote:

 I have published an analysis of using multi-threading for GAE/J here 
 http://devcon5.blogspot.com for a project I am working on.

 I would very much appreciate any comments, corrections, ommissions or 
 additional questions I should cover. 

 Thanks.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/lHqF2_MQTAIJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Issues associated with enabling multi-threading via threadsafe

[Hugo Visser]

Yes, you're right, I thought it was required, but supplying the element is 
now required, thanks for pointing that out.
I don't see why you wouldn't want to use thread safe though, especially for 
Java apps and libraries that are in other environments also built with 
thread safety in mind.

Hugo

On Tuesday, September 25, 2012 8:45:42 PM UTC+2, Andrew Mackenzie wrote:

 Hi Hugo,

 Thanks for the comments.

 But its not moot. The element in appengine-web.xml is obligatory, but can 
 be set to false. Current production environment respects it. I have done 
 many tests to verify that with overlapped requests and the changes in 
 behaviour when changing that setting.

 Point taken about multiple parallel instances and ds/mc. I thought I 
 discussed that but maybe left it out in final version - not 
 multi-threading within a VM or affected by setting in question 
 though.we could call that instance-safe :-)

 Thanks

 Andrew
  On 25 Sep 2012 18:52, Hugo Visser bott...@gmail.com javascript: 
 wrote:

 Hi Andrew,

 To me, the whole point is kinda moot, since threadsafe is a required 
 setting for Java apps (which is a good thing).

 Also, the points that you raise in the post are not different from the 
 gotcha's as they are present in a normal Java servlet stack. There's not 
 much App Engine specific in there.

 The statements like static initializers are safe are technically not 
 correct; while it's true for a single instance running multiple threads, 
 your app will still run on multiple instances. So if you are doing 
 something funky in a static initializer (like writing to the datastore) 
 there's still the possibility of race conditions or data corruption.
 Same holds true for writing to memcache and the data store that you 
 describe in the post: even if your app is single threaded, a production app 
 will run on multiple instances which could all concurrently write to the 
 same entity or memcache entry.

 I hope you find this useful :)

 Hugo

 On Monday, September 24, 2012 2:41:25 PM UTC+2, Andrew Mackenzie wrote:

 I have published an analysis of using multi-threading for GAE/J here 
 http://devcon5.blogspot.com for a project I am working on.

 I would very much appreciate any comments, corrections, ommissions or 
 additional questions I should cover. 

 Thanks.

  -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To view this discussion on the web visit 
 https://groups.google.com/d/msg/google-appengine/-/lHqF2_MQTAIJ.
 To post to this group, send email to 
 google-a...@googlegroups.comjavascript:
 .
 To unsubscribe from this group, send email to 
 google-appengi...@googlegroups.com javascript:.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/WDs3o9-C-14J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] File finalization

[Hugo Visser]

Hi,

Can someone clarify how the finalization works for blobstore and gs files? 
In particular, how long can I leave a file in the unfinalized state (if I 
need to append to the file later) and how do I find files that aren't 
finalized properly?

Thanks,

Hugo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/T8M4ySGJVycJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Datastore and index stats not updated, who to contact?

[Hugo Visser]

Hi,

For one of our apps the datastore stats does not list all namespaces in de 
admin console and the index page does not show the index entry count and 
index storage data.

I'm not sure if that qualifies as a production issue, but it is annoying 
for us because we don't have the insight that we would like managing the 
app.
The app id is qbustalento.

I already tried to report the issue using the screenshot tool, but there's 
no response whatsoever, so I'm wondering how we can get this resolved?

Thanks,

Hugo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/gnGBl5OOatYJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: C2DM gives 401 UNATHORIZED

[Hugo Visser]

Not really clear what you mean or if it's related to app engine or c2dm. If 
your users get this it looks like authentication with app engine fails. I 
had a few problems authenticating while deploying yesterday, which could be 
related, but those are solved now.

If sending the message to the c2dm server fails, you should refresh the 
client-login tokens.

Hugo

On Wednesday, July 25, 2012 9:32:40 PM UTC+2, Theis Borg wrote:

 OK - I am kind of lost here. 

 I have a site with several thousand users and it has been running fine 
 with C2DM for more than a year now. But suddenly it comes back with a: 

 HTMLHEADTITLEUnauthorized/TITLE/HEADBODY BGCOLOR=#FF 
 TEXT=#00H1Unauthorized/H1H2Error 401/H2/BODY/ 
 HTML 

 For all my users. So I guess that it is my sender account that has 
 been messed up? How can I solve this? 

 I know that google are changing C2DM to GCM but as far as I can read I 
 should still be able to run my old code. 

 I hope some of you guys has a hint for me... 

 BR. 

 q:o)Theis 

 androidlost.com 


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/0Zp3ugAdRPUJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Startup time exceeded...on F4?!

[Hugo Visser]

Yeah but where this whole discussion started was that when there's no 
available instance you can't defer anything, since a cold instance will 
apparently get the request and due to the change dynamic instances aren't 
warmed up like they used to be. You're bound to need to do 
some initialization at that point. And while you could build every single 
component you're app needs yourself to let it be super duper optimized for 
App Engine, that will just not justify the amount of work it will take to 
run a app on App Engine.

IMHO the warm up requests serve their purpose; they warm up the app so that 
a slightly longer start up time of 10 secs or so isn't that bad. The issue 
is just that there are now situations where user facing requests end up 
warming up a instance which sounds like a bad thing, especially if the app 
is receiving spiky loads.


On Saturday, July 21, 2012 11:00:53 PM UTC+2, Brandon Wirtz wrote:

 When I said 1.5 I meant PMF was adding 3.5 you don’t need. PMF takes 5s 
 the API takes under 1s.  I think 2.5s is probably the lowest you can get 
 and do something truly useful, but it seems that I can do a LOT in 2.5s  
 (assuming the F4 this thread was about when we started) (well actually 
 assuming F2, F4’s seem to need 3s no matter what you do)

 But also you don’t have to do anything useful on warmup. So you shouldn’t 
 serve a ColdStart very often.

  

 I have mentioned before I work to keep everything to under 4s. If my App 
 Gets to 8s I start slicing it up 

  

 I am also 100% confident that if all you do is say “I am up” your warm up 
 will take under 1.5s  for 95% of starts. And that should really do “I am 
 up” and defer lazyloads of everything you need to be ready to do real work.

  

  

 *From:* google-appengine@googlegroups.com [mailto:
 google-appengine@googlegroups.com] *On Behalf Of *hyperflame
 *Sent:* Saturday, July 21, 2012 1:47 PM
 *To:* google-appengine@googlegroups.com
 *Subject:* Re: [google-appengine] Re: Startup time exceeded...on F4?!

  

 I just looked over the logs of a corporate GAE application. It's a very 
 simple heartbeat application; essentially our production applications 
 have to send a message to it periodically. The message sent is about 500 
 characters; the application does some checksum-ing and sends back a JSON 
 string about 100 characters long. The only external libraries are jars of 
 twitter4j and the org.json parser. It's really not much more complex than a 
 Hello World app.

 Looking over the logs, the fastest start time from cold-instance to 
 first-request-served is 1695 ms (although on average, it needs 2.2 to 2.5 
 seconds to cold-start) on a F1 instance. Once the instance is warm, 
 subsequent requests take, on average, ~150ms to process and send back a 
 reply. I could probably get that lower if I upgraded to F4.

 On Saturday, July 21, 2012 3:19:21 PM UTC-5, André Pankraz wrote:

 I use getRessourceAsStream an Stream-copy - but you miss the point...it's 
 a Hello World, done in 5 minutes as test for your post, stripping away all 
 excuses.
 The calls after startup are answered in 80 ms.
 That mans that I'm still far above 3 seconds for initializing of an empty 
 project. I could simply write Hello World into the Stream.

 I could use static...yes...but thats not the point. Some people have 
 dynamic and user dependand data on each page.

 -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To view this discussion on the web visit 
 https://groups.google.com/d/msg/google-appengine/-/TDuwyumyZAIJ.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/yM2ywgzd4PIJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Startup time exceeded...on F4?!

[Hugo Visser]

I did, and used Objectify (Jeff is kinda familiar with that I think :)). A 
plain Hello World with only registering the entities will get you around 
the 4-5 sec mark, which is hardly a real application. 

I have a highly optimized app as a backend for my Android app, and that 
even takes around the 6-8 seconds mark to start up and it varies too.
Add in some third party frameworks, like Jersey for JAXRS and Guice to keep 
your code sane and you're at the 12 secs.

So kool aid or not, Java start up overhead will always be more than it is 
for python and I think the 5 seconds mentioned is far from typical if you 
are running a non-trivial app.

On Friday, July 20, 2012 6:20:56 AM UTC+2, Brandon Wirtz wrote:

 Dump PMF use the low level API 
 https://developers.google.com/appengine/docs/java/javadoc/com/google/appengi 

 ne/api/datastore/package-summaryhttps://developers.google.com/appengine/docs/java/javadoc/com/google/appengine/api/datastore/package-summary
  

 Either drink the Kool Aid, or quit complaining :-) 

 5s will be 1.5 






-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/4icNAFZtGKwJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Jersey 1.8 issue

[Hugo Visser]

Hi Yann,

I'm running an older version of Jersey on app engine (1.3) which runs
fine.

One tip that I can give you is not to use classpath scanning as you do
now. Just create an Application and register that in your web xml
(it's easier to debug too). That will give you better start up
performance on App Engine.

If you do use classpath scanning, I hope you understand that your
resources should be in the package net.whatever.rest for Jersey to
find them?

Hugo

On 26 jul, 11:57, Yann Lambret yann.lamb...@gmail.com wrote:
 Hello,

 I'm trying to deploy a restful web service on GAE with Jersey 1.8
 implementation. On the page 'will it run on appengine', there is no
 information about this specific release. I've set up the web.xml as
 indicated in the Jersey documentation :

 servlet
     servlet-nameJersey Web Application/servlet-name
     
 servlet-classcom.sun.jersey.spi.container.servlet.ServletContainer/servl 
 et-class
     init-param
         param-namecom.sun.jersey.config.property.packages/param-name
         param-valuenet.whatever.rest/param-value
     /init-param
 /servlet

 My war/WEB-INF/lib directory contains the following files:

 asm-3.1.jar
 jackson-core-asl-1.7.1.jar
 jackson-jaxrs-1.7.1.jar
 jackson-mapper-asl-1.7.1.jar
 jackson-xc-1.7.1.jar
 jersey-client-1.8.jar
 jersey-core-1.8.jar
 jersey-json-1.8.jar
 jersey-server-1.8.jar
 jettison-1.1.jar
 jsr311-api-1.1.1.jar

 My REST class is annotated with @Path btu I still get these errors when 
 accessing the application (deploying is ok):

 Uncaught exception from servlet
 javax.servlet.UnavailableException: 
 com.sun.jersey.api.container.ContainerException: The ResourceConfig instance 
 does not contain any root resource classes.
         at 
 org.mortbay.jetty.servlet.ServletHolder.makeUnavailable(ServletHolder.java: 
 415)
         at 
 org.mortbay.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:458)
         at 
 org.mortbay.jetty.servlet.ServletHolder.doStart(ServletHolder.java:263)
         at 
 org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
         at 
 org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:685 )
         at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
         at 
 org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1250

 Or sometimes this additionnal error, depending on the content of my web.xml 
 file:

 com.sun.jersey.server.impl.application.RootResourceUriRules init: The 
 ResourceConfig instance does not contain any root resource classes.

 Did sommeone successfully manage to deploy a Jersey 1.8 app on GAE ? If so, 
 could you please provide the related web.xml secttion,
 code snippet or whatever could help ?

 Thanks a lot.

 Yann

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Transactions and ancestor in a different namespace

[Hugo Visser]

Correct, I'm still on 2.2.3 (it's great, thanks!). Would I run into
big problems switching to 3.0 because of this? In only one or two
instances I ran into the last minute key conversion issue, but I've
worked around that with a ObjectifyWrapper which is actually not bad
at all. In the spirit of if it ain't broke... I haven't switched yet
to 3.0.

The thing is that we have data in a main namespace with historical
data in different namespaces. Previously this was all in one namespace
and because I want to update that data in a transaction it was in the
same entity group.
Now that it is crossing namespaces it looks like that can't be done
anymore, which is a bit of a pain but not a huge problem for now.

Hugo

On 23 jul, 21:39, Jeff Schnitzer j...@infohazard.org wrote:
 If you are using Objectify 3.0, the Key? is a simple wrapper for the
 native datastore Key - with all the same namespace consequences.  In
 previous versions, the Key? - Key conversion was done last-minute so it
 would inherit the namespace then.

 Jeff







 On Sat, Jul 23, 2011 at 5:51 AM, Hugo Visser botte...@gmail.com wrote:
  Well it appears that the entire key path for an entity group is bound
  to one namespace. I am using Objectify in my app which doesn't have
  the namespace in the keys but when checking out the low level Java api
  that became apparent.

  On 22 jul, 12:48, Hugo Visser botte...@gmail.com wrote:
   Is it anywhere explicitly documented that the ancestor of an entity
   must have it's key in the same namespace as the entity itself? I'm
   using namespaces a lot in our app and in production I get an error if
   I try to do a transactional update of an entity with parent x in a
   different namespace than the entity that I'm updating.

   Is this intentional? If it is, I guess the dev server should also
   check this, my unit tests pass :)

   Hugo

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: For deployment only does the sdk version matter?

[Hugo Visser]

Thanks Ikai, keeping it somewhat up to date will be fine for CI
purposes.

On 22 jul, 16:27, Ikai Lan (Google) ika...@google.com wrote:
 Only your version matters. However, it is a good idea to stay somewhat up to
 date, as some APIs (experimental ones) get deprecated.

 One more thing: I tried running a really old application a while ago that
 was developed using an SDK version that was 1.1.x, and it wouldn't work with
 the newest SDK.

 Ikai Lan
 Developer Programs Engineer, Google App Engine
 Blog:http://googleappengine.blogspot.com
 Twitter:http://twitter.com/app_engine
 Reddit:http://www.reddit.com/r/appengine

 On Mon, Jul 18, 2011 at 3:49 PM, Ernesto Oltra ernestoka...@gmail.comwrote:







  As far as my knowledge goes, you upload the source code (allowing downloads
  of it later), and its compiled, precompiled or whatever (Python/Java/Go...)
  in their machines; so the only problem here is you might be using a feature
  that has change their behaviour since the version you locally tested it.

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/3atHYio-ZoEJ.

  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Transactions and ancestor in a different namespace

[Hugo Visser]

Well it appears that the entire key path for an entity group is bound
to one namespace. I am using Objectify in my app which doesn't have
the namespace in the keys but when checking out the low level Java api
that became apparent.

On 22 jul, 12:48, Hugo Visser botte...@gmail.com wrote:
 Is it anywhere explicitly documented that the ancestor of an entity
 must have it's key in the same namespace as the entity itself? I'm
 using namespaces a lot in our app and in production I get an error if
 I try to do a transactional update of an entity with parent x in a
 different namespace than the entity that I'm updating.

 Is this intentional? If it is, I guess the dev server should also
 check this, my unit tests pass :)

 Hugo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Transactions and ancestor in a different namespace

[Hugo Visser]

Is it anywhere explicitly documented that the ancestor of an entity
must have it's key in the same namespace as the entity itself? I'm
using namespaces a lot in our app and in production I get an error if
I try to do a transactional update of an entity with parent x in a
different namespace than the entity that I'm updating.

Is this intentional? If it is, I guess the dev server should also
check this, my unit tests pass :)

Hugo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Billing - Alerts when nearing maximum quota?

[Hugo Visser]

Thanks for recommending that, I'm the developer of that app :) Feel
free to contact me if you need a feature or things like that. I
haven't touched it in a while because it does what I need.

Hugo

On 21 jul, 10:05, emerix rafa...@gmail.com wrote:
 I recommend the free app Engine 
 Watchhttps://market.android.com/details?id=com.neenbedankt.enginewatchfea...on
  android. You can set limit on quota to be notified when they are
 reached.
 But it would be great to have something similar directly from Google.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Warm up request a magnitude slower?

[Hugo Visser]

I'm using GAE/J and I'm notice that since a day or two, maybe longer, 
instance startup is way slower than it was. My typical startup request was 
around 4-8 seconds, now I'm seeing 12-20 seconds without any significant 
code changes. Do others experience the same?

Hugo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/Nt6CBPQMZVEJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Warm up request a magnitude slower?

[Hugo Visser]

Hi Johan,

I'm seeing this on app id talentosvs. I haven't checked my other apps
yet but this one is low-traffic so it's the most apparent one.

Hugo

On Thu, Jul 21, 2011 at 1:07 PM, Johan Euphrosine pro...@google.com wrote:
 Hi Hugo,

 What is your appid ?

 Feel free to open a Type=Production issue if this is affecting your
 application operation:
 http://code.google.com/p/googleappengine/issues/entry?template=Production%20issue

 On Thu, Jul 21, 2011 at 1:01 PM, Hugo Visser botte...@gmail.com wrote:
 I'm using GAE/J and I'm notice that since a day or two, maybe longer,
 instance startup is way slower than it was. My typical startup request was
 around 4-8 seconds, now I'm seeing 12-20 seconds without any significant
 code changes. Do others experience the same?
 Hugo

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/Nt6CBPQMZVEJ.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




 --
 Johan Euphrosine (proppy)
 Developer Programs Engineer
 Google Developer Relations

 --
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Billing - Alerts when nearing maximum quota?

[Hugo Visser]

If the question is if I'd open source the app, unsure, maybe I should :) But 
I'm open for suggestions.

Hugo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/nZ_t5M3gEjQJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] For deployment only does the sdk version matter?

[Hugo Visser]

Hi,

When deploying an app, does the sdk version you deploy it with (not
compiled with in case of Java) matter? Is it OK to deploy an app using
the 1.5.x API's with an older SDK such as 1.4.x?

I'm asking because I'm setting up an automated deploy and some day I'm
bound to forget to update to the latest sdk on the CI machine :)

Thanks,

Hugo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Allocating id's after import to another app

[Hugo Visser]

Any suggestions? Would iterating the current entities and call the
allocate function with those keys help? I'm getting errors on migrated
data in production which is pretty bad.

On Oct 6, 4:40 pm, Hugo Visser botte...@gmail.com wrote:
 I've imported data from one app id to another and now I'm getting
 spurious messages of duplicate id's, apparently because the id's are
 not automatically allocated.

 What would be the best strategy to allocate these id's again so that
 sequence generation will not attempt to use the existing id's?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Allocating id's after import to another app

[Hugo Visser]

I've imported data from one app id to another and now I'm getting
spurious messages of duplicate id's, apparently because the id's are
not automatically allocated.

What would be the best strategy to allocate these id's again so that
sequence generation will not attempt to use the existing id's?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: List of namespaces in the datastore

[Hugo Visser]

That's what I'm doing of course, but it is an accident waiting to
happen. If I lose the list in some way, I'm screwed as it stands now
and my customer data is inaccessible.

Oh there doesn't seem a good way to backup the entire datastore at
this point if your data contains namespaces. The only option appears
to be per kind, per namespace. The statistic that the bulk loader uses
for downloading all entities is only available in the default
namespace, so that won't fly if you have multiple namespaces, maybe
with different entities in a namespace.

I've filed an issue for listing the namespaces here:
http://code.google.com/p/googleappengine/issues/detail?id=3813 and for
downloading all data here 
http://code.google.com/p/googleappengine/issues/detail?id=3814.
Please star these issues.

Hugo

On Oct 2, 6:55 pm, Robert Kluin robert.kl...@gmail.com wrote:
 Yes. That is correct.

 Maybe you should keep a list of all the used namespaces so you can
 clean up if needed.

 Robert



 On Sat, Oct 2, 2010 at 06:16, Hugo Visser botte...@gmail.com wrote:
  Currently there doesn't seem to be a way to list all namespaces in the
  datastore, is this correct?

  This could potentially lead to the situation that data is lost
  because the namespace name is lost, especially if an app uses
  generated namespace names.

  Hugo

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] List of namespaces in the datastore

[Hugo Visser]

Currently there doesn't seem to be a way to list all namespaces in the
datastore, is this correct?

This could potentially lead to the situation that data is lost
because the namespace name is lost, especially if an app uses
generated namespace names.

Hugo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Total Stored Data at 91%

[Hugo Visser]

Another me too again. This happens once in a while, and it is
mentioned in a couple of threads. I never got a clear explanation on
what causes this. The amount of data that my app stores is constant,
but there are a lot of updates to that data. I also use memcache, but
the things I put in memcache have an expire time set so that should
also be kinda constant.

The quota page says: The total amount of data stored in datastore
entities and corresponding indexes, in the memcache, and in the
Blobstore.
I've flushed the memcache again for my app, let's see if it helps...


On Sep 24, 11:07 pm, chrisfarms chrisfa...@gmail.com wrote:
 I see similar.

 But didn't think too much of it since I have lots of data in the
 blobstore which the quota registers as 0, I assumed my blobstore
 storage was getting lumped into the datastore pile.

 Also see:http://code.google.com/p/googleappengine/issues/detail?id=2479

 On Sep 23, 5:09 pm, David wordsmithg...@gmail.com wrote:



  The Total Stored Data quota seems to be very incorrect.  The Datastore
  statistics only show 71MB for the size of all entities.  And I only
  have one index in the Datastore indexes section which is an index on a
  string and an integer.  The string is typically 30 characters long or
  so.  I do have the app periodically delete entities when it is done
  with them, but they are typically short lived entities where they may
  only be in there for a number of days.  I saw the Total Stored Data
  drop once when I added the auto pruning of data, but now I'm getting
  close the 1GB limit and I don't want my application to stop working.
  Does anybody know what might be going on?  How do it get it to reduce
  the number to where it should be?

  Thanks,
  David

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Serious problem error code 204

[Hugo Visser]

I'm getting consistent error 204 in my logs and the request is
aborted. What causes these errors and how can I debug this?

Hugo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Serious problem error code 204

[Hugo Visser]

I think I figured it out, I was doing a batch fetch of some entities
with blob fields. I think that this error is produced if you hit the 1
mb limit while fetching the entities. I changed my code to fetch less
entities in batch and now it works in production.

On Sep 3, 10:03 pm, Hugo Visser botte...@gmail.com wrote:
 I'm getting consistent error 204 in my logs and the request is
 aborted. What causes these errors and how can I debug this?

 Hugo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to add email alias to app engine?

[Hugo Visser]

*nutch* I still cannot get this to work. I've tried several things.

My domain abc.com is a google apps domain. That domain has a domain
alias of xyz.com. My account is m...@abc.com. Now I like to send an
email from my app as i...@xyz.com.

Adding an alias to my account for info works in the sense that I can
receive mail. But I can't add this alias as an administrator for the
app, hence I cannot send email on behalf of i...@xyz.com.

The only option I see is to create a separate google app account for
xyz.com, add a info user and add that user as an administrator for the
app. Can anybody confirm if this is the only option?

Hugo


On Jul 12, 2:40 pm, Hugo Visser botte...@gmail.com wrote:
 Hi,

 I have an app that is set to apps domain only mode for authentication.
 Now I want to send someemailfrom that app. It is working when I use
 one of the administrator accounts, but I like to set up anemailalias
 and I'm failing to get that working.

 To complicate things a little bit more, the apps account has multiple
 domains associated and I'd prefer to use one of the aliases to send
 the mail from.

 I tried to create Google accounts for the aliasses but I guess since
 the app is set to an app domain that won't work. Signing up with 
 analiasdoesn't work either and I rather not create an extra user if
 it's not really needed.

 Any suggestions how to set this up? To summerize, my domain is a.com
 and that also analiasb.com. I'd like to send from emailal...@b.com.

 Hugo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: New index creation in queued state for a long time

[Hugo Visser]

Index building is extremely slow the last few days. I hardly have any
data in my app but building the indexes takes up to 8 hours to
complete, or they are in a queued state very long. This really slows
down development :(

Any word from the app engine team on this?

Hugo

On Aug 24, 9:45 pm, Robert Kluin robert.kl...@gmail.com wrote:
 On Mon, Aug 23, 2010 at 17:06, baus cmb...@gmail.com wrote:
  2) Is it best to create the indexes ahead of app changes which depend
  on those indexes? If so, is there a best practices guide that
  describes doing this?

 Yes.  You can also upload the app to a different version, then wait to
 make the new version default until the required indexes are built.





  The app id is yieldcurvegraph

  Thanks,

  Chris

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Images API: resize png, transparency gone?

[Hugo Visser]

Hi,

I ran into an issue using the Images API. When I resize or crop a
transparent PNG image, the transparency is lost. This is in
production, not the dev server. I was wondering if this is by design
or if this a bug? I'm using the Java SDK and haven't tried the python
sdk yet...

Thanks,

Hugo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] How to add email alias to app engine?

[Hugo Visser]

Hi,

I have an app that is set to apps domain only mode for authentication.
Now I want to send some email from that app. It is working when I use
one of the administrator accounts, but I like to set up an email alias
and I'm failing to get that working.

To complicate things a little bit more, the apps account has multiple
domains associated and I'd prefer to use one of the aliases to send
the mail from.

I tried to create Google accounts for the aliasses but I guess since
the app is set to an app domain that won't work. Signing up with an
alias doesn't work either and I rather not create an extra user if
it's not really needed.

Any suggestions how to set this up? To summerize, my domain is a.com
and that also an alias b.com. I'd like to send from emailal...@b.com.

Hugo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Can't enable billing

[Hugo Visser]

At the moment, according to the billing page my app is operating in
free mode. When I look at the quotas on the dashboard however it is
using the quotas that I've set on the billing page, while again the
billing status is displayed as free.

On May 29, 2:22 am, Ikai L (Google) ika...@google.com wrote:
 It looks like billing is a bit slow. The UI may say that you are not in the
 queue, but this is mislabeled. When billing processes, your app will
 automatically be updated to the correct status.

 On Fri, May 28, 2010 at 1:44 PM, Ikai L (Google) ika...@google.com wrote:



  I'm seeing the same issue. Let me follow up on this.

  On Fri, May 28, 2010 at 12:04 PM, G badbackj...@googlemail.com wrote:

  I'm trying to enable billing, and it seems to be working - I check out
  and then the status says it's updating for 15 or so minutes, but after
  that it goes back to free with the Enable Billing button visible.

  Is there some extra step I should be doing?

  Thanks

  G

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
  .
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

  --
  Ikai Lan
  Developer Programs Engineer, Google App Engine
  Blog:http://googleappengine.blogspot.com
  Twitter:http://twitter.com/app_engine
  Reddit:http://www.reddit.com/r/appengine

 --
 Ikai Lan
 Developer Programs Engineer, Google App Engine
 Blog:http://googleappengine.blogspot.com
 Twitter:http://twitter.com/app_engine
 Reddit:http://www.reddit.com/r/appengine

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Billing missing?

[Hugo Visser]

Another me too...
My app was billing enabled, but now it shows that billing is disabled.
I tried to enable billing again, but after a while it flips back to
free mode again.

Hugo


On May 27, 5:42 pm, Millisecond millisec...@gmail.com wrote:
 We're seeing (and wondering) the same things

 On May 26, 3:26 pm, jay kyburz@gmail.com wrote:

  Hey all, I just noticed that thebillinginfo for my app is missing
  and it'sbillingstatus is Free.

  Is this just some compensation for the downtime or is something wrong
  with my app?

  Jay.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Total stored data creeping up again

[Hugo Visser]

Hi,

Last few days I suddenly see the quota for Total Stored Data going up
for no apparent reason. My app stores little data in the data store
and none of the properties are indexed (I only retrieve an entity by
key).
So I can't really tell where the sudden rise of stored data is coming
from, the app has been running for a while like this and data store
use is normally something like 1%.

My app does use memcache a lot, is that also counted in the total
stored data? I flushed memcache to be sure, but the quota display does
not reflect any change.

Thanks,

Hugo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Total stored data creeping up again

[Hugo Visser]

Rewrites I do a lot. The dataset in the datastore gets updated
frequently. I guess that would explain the low percentage I see
usually. But now it's jumping over 50% which seems a lot.
I just like some confirmation if it's something that I'm doing wrong,
or if it is an app engine thing that I shouldn't worry about.

On Mar 14, 3:57 pm, Alkis Evlogimenos ('Αλκης Ευλογημένος)
evlogime...@gmail.com wrote:
 Did you do any schema changes or rewrites of your data? From past experience
 the quota counter/garbage collector seem to be periodic jobs that take more
 than a day to update. For example I did a schema upgrade recently: disk
 usage doubled. After 2 days it went back to normal.

 - alkis

 On Sun, Mar 14, 2010 at 2:20 PM, Hugo Visser botte...@gmail.com wrote:
  Hi,

  Last few days I suddenly see the quota for Total Stored Data going up
  for no apparent reason. My app stores little data in the data store
  and none of the properties are indexed (I only retrieve an entity by
  key).
  So I can't really tell where the sudden rise of stored data is coming
  from, the app has been running for a while like this and data store
  use is normally something like 1%.

  My app does use memcache a lot, is that also counted in the total
  stored data? I flushed memcache to be sure, but the quota display does
  not reflect any change.

  Thanks,

  Hugo

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
  .
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Android app to monitor appengine quotas

[Hugo Visser]

At the moment, no, but I'm considering it for a future version once I
have a bit more time on my hands to do a proper open source release vs
a code dump.

Hugo

On Feb 10, 5:18 am, kang areyouloo...@gmail.com wrote:
 thanks for sharing. will your app be opensourced?



 On Wed, Feb 10, 2010 at 5:05 AM, Hugo Visser botte...@gmail.com wrote:
  Hi,

  I've updated Engine Watch for Android. New:
  - Now uses the build-in system Google Accounts on Android 2.0 and up
  (no more entering passwords)
  - Add shortcuts to a specific app on the home screen
  - Display of billing stats

  Full details on my blog athttp://code.neenbedankt.com

  Hugo

  On Feb 1, 5:05 pm, Hugo Visser botte...@gmail.com wrote:
   Hi,

   I've released a little android app to monitor your app engine quotas.
   It's called Engine Watch and is available from the android market. See
   my blog athttp://code.neenbedankt.comformore details. I hope it's
   useful to some of you too :)

   Hugo

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
  .
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

 --
 Stay hungry,Stay foolish.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Android app to monitor appengine quotas

[Hugo Visser]

Hi,

I've updated Engine Watch for Android. New:
- Now uses the build-in system Google Accounts on Android 2.0 and up
(no more entering passwords)
- Add shortcuts to a specific app on the home screen
- Display of billing stats

Full details on my blog at http://code.neenbedankt.com

Hugo

On Feb 1, 5:05 pm, Hugo Visser botte...@gmail.com wrote:
 Hi,

 I've released a little android app to monitor your app engine quotas.
 It's called Engine Watch and is available from the android market. See
 my blog athttp://code.neenbedankt.comfor more details. I hope it's
 useful to some of you too :)

 Hugo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: why my datastore nearly be full?

[Hugo Visser]

Well I guess what I'm seeing is not normal. I have 129 entities that
take 1 MB in total according to the statistics. I have 48 MB in
memcache and that's it.
Still my total stored data quota is at 59% and didn't get reset, even
after flushing the memcache and deleting all of my entities from the
datastore.

Besides memcache and datastore I also use task queues, anybody else
using those and seeing a quota rise?

This started somewhere around the weekend so I really think this is a
bug of somekind...I hope Google will look into it soon.
This post seems to be related:
http://groups.google.com/group/google-appengine/browse_thread/thread/e619865eaf584706

Hugo

On Feb 1, 8:51 pm, Robert Kluin robert.kl...@gmail.com wrote:
 We see about 10x overhead, I assume from indexes, in our apps.  One
 app is using 30M for storing entities and metadata, the indexes take
 another 300M.  We see similar usage multipliers in all of our apps.

 Robert

 2010/2/1 Hugo Visser botte...@gmail.com:

  I'm seeing the same thing. I got hardly anything stored in the
  datastore, but my quota is at 50% right now. I did have some stuff in
  the memcache however.
  This morning I flushed memcache and deleted all entities in the
  datastore but the quota usage is still going up. I'm waiting to see
  what happens after the daily reset.

  Hugo

  On Feb 1, 5:27 pm, Wooble geoffsp...@gmail.com wrote:
  The stored data quota in the dashboard isn't updated in realtime, so
  you wouldn't see the change immediately even if that is the problem.
  I'd suspect some buggy indexes or something, though, if that really
  was your only entity.  For one thing, you can't have a single entity
  larger than 1MB, and it would be tough to use 949 MB in datastore
  overhead even if you wanted to.

  On Jan 31, 3:05 am, saintthor saintt...@gmail.com wrote:

   i deleted the only entity of ltWords. the Total Stored Data is still
   95%!

   who can help me?

   On 1月30日, 下午8时42分, Sylvain sylvain.viv...@gmail.com wrote:

maybe this bug ?

   http://code.google.com/p/googleappengine/issues/detail?id=631

On Jan 30, 9:27 am, saintthor saintt...@gmail.com wrote:

 in quota details, i notic my Total Stored Data is 95%. it shouldn' t
 be so large.

 in Datastore Statistics, Size of all entities is only 9MB.

 all that happened from about 4% to 95% is i updated an entity for
 about 200 times. the class is:

 class ltWords( db.Model ):
 WordsDText = db.TextProperty()

 after this, there is only one entity of ltWords and the WordsDText is
 no longer than 1MB. why does it take more than 90% of my datastore
 quota and how to release it?

 for  saintthor.appspot.com

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How Google calculate datastore usage?

[Hugo Visser]

I'm seeing the same, and others are too, see this thread:
http://groups.google.com/group/google-appengine/browse_thread/thread/76b71174fd94300f/e4d64b5d6618cd76#e4d64b5d6618cd76

When did you notice the quota increasing?

On Feb 2, 8:55 am, lookon areyouloo...@gmail.com wrote:
 Here is my app:http://reader2twitter.appspot.com

 In the Dashboard, I see Total Stored Data : 0.70 of 1.00 GBytes.

 But in the Datastore Statistics, I see Size of all entities : 159
 MBytes.

 I am confused. Can you give me some help? Thanks.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: why my datastore nearly be full?

[Hugo Visser]

Any suggestions from Google folks such as Nick maybe?

On Feb 2, 9:56 am, Hugo Visser botte...@gmail.com wrote:
 Well I guess what I'm seeing is not normal. I have 129 entities that
 take 1 MB in total according to the statistics. I have 48 MB in
 memcache and that's it.
 Still my total stored data quota is at 59% and didn't get reset, even
 after flushing the memcache and deleting all of my entities from the
 datastore.

 Besides memcache and datastore I also use task queues, anybody else
 using those and seeing a quota rise?

 This started somewhere around the weekend so I really think this is a
 bug of somekind...I hope Google will look into it soon.
 This post seems to be 
 related:http://groups.google.com/group/google-appengine/browse_thread/thread/...

 Hugo

 On Feb 1, 8:51 pm, Robert Kluin robert.kl...@gmail.com wrote:

  We see about 10x overhead, I assume from indexes, in our apps.  One
  app is using 30M for storing entities and metadata, the indexes take
  another 300M.  We see similar usage multipliers in all of our apps.

  Robert

  2010/2/1 Hugo Visser botte...@gmail.com:

   I'm seeing the same thing. I got hardly anything stored in the
   datastore, but my quota is at 50% right now. I did have some stuff in
   the memcache however.
   This morning I flushed memcache and deleted all entities in the
   datastore but the quota usage is still going up. I'm waiting to see
   what happens after the daily reset.

   Hugo

   On Feb 1, 5:27 pm, Wooble geoffsp...@gmail.com wrote:
   The stored data quota in the dashboard isn't updated in realtime, so
   you wouldn't see the change immediately even if that is the problem.
   I'd suspect some buggy indexes or something, though, if that really
   was your only entity.  For one thing, you can't have a single entity
   larger than 1MB, and it would be tough to use 949 MB in datastore
   overhead even if you wanted to.

   On Jan 31, 3:05 am, saintthor saintt...@gmail.com wrote:

i deleted the only entity of ltWords. the Total Stored Data is still
95%!

who can help me?

On 1月30日, 下午8时42分, Sylvain sylvain.viv...@gmail.com wrote:

 maybe this bug ?

http://code.google.com/p/googleappengine/issues/detail?id=631

 On Jan 30, 9:27 am, saintthor saintt...@gmail.com wrote:

  in quota details, i notic my Total Stored Data is 95%. it shouldn' 
  t
  be so large.

  in Datastore Statistics, Size of all entities is only 9MB.

  all that happened from about 4% to 95% is i updated an entity for
  about 200 times. the class is:

  class ltWords( db.Model ):
  WordsDText = db.TextProperty()

  after this, there is only one entity of ltWords and the WordsDText 
  is
  no longer than 1MB. why does it take more than 90% of my datastore
  quota and how to release it?

  for  saintthor.appspot.com

   --
   You received this message because you are subscribed to the Google Groups 
   Google App Engine group.
   To post to this group, send email to google-appeng...@googlegroups.com.
   To unsubscribe from this group, send email to 
   google-appengine+unsubscr...@googlegroups.com.
   For more options, visit this group 
   athttp://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: why my datastore nearly be full?

[Hugo Visser]

One other observation is that in the billing summary all seems fine:

Stored Data:
$0.005/GByte-day0.001.000.00$0.00

Maybe a dashboard/statistics bug?

On Feb 2, 12:58 pm, Hugo Visser botte...@gmail.com wrote:
 Any suggestions from Google folks such as Nick maybe?

 On Feb 2, 9:56 am, Hugo Visser botte...@gmail.com wrote:

  Well I guess what I'm seeing is not normal. I have 129 entities that
  take 1 MB in total according to the statistics. I have 48 MB in
  memcache and that's it.
  Still my total stored data quota is at 59% and didn't get reset, even
  after flushing the memcache and deleting all of my entities from the
  datastore.

  Besides memcache and datastore I also use task queues, anybody else
  using those and seeing a quota rise?

  This started somewhere around the weekend so I really think this is a
  bug of somekind...I hope Google will look into it soon.
  This post seems to be 
  related:http://groups.google.com/group/google-appengine/browse_thread/thread/...

  Hugo

  On Feb 1, 8:51 pm, Robert Kluin robert.kl...@gmail.com wrote:

   We see about 10x overhead, I assume from indexes, in our apps.  One
   app is using 30M for storing entities and metadata, the indexes take
   another 300M.  We see similar usage multipliers in all of our apps.

   Robert

   2010/2/1 Hugo Visser botte...@gmail.com:

I'm seeing the same thing. I got hardly anything stored in the
datastore, but my quota is at 50% right now. I did have some stuff in
the memcache however.
This morning I flushed memcache and deleted all entities in the
datastore but the quota usage is still going up. I'm waiting to see
what happens after the daily reset.

Hugo

On Feb 1, 5:27 pm, Wooble geoffsp...@gmail.com wrote:
The stored data quota in the dashboard isn't updated in realtime, so
you wouldn't see the change immediately even if that is the problem.
I'd suspect some buggy indexes or something, though, if that really
was your only entity.  For one thing, you can't have a single entity
larger than 1MB, and it would be tough to use 949 MB in datastore
overhead even if you wanted to.

On Jan 31, 3:05 am, saintthor saintt...@gmail.com wrote:

 i deleted the only entity of ltWords. the Total Stored Data is still
 95%!

 who can help me?

 On 1月30日, 下午8时42分, Sylvain sylvain.viv...@gmail.com wrote:

  maybe this bug ?

 http://code.google.com/p/googleappengine/issues/detail?id=631

  On Jan 30, 9:27 am, saintthor saintt...@gmail.com wrote:

   in quota details, i notic my Total Stored Data is 95%. it 
   shouldn' t
   be so large.

   in Datastore Statistics, Size of all entities is only 9MB.

   all that happened from about 4% to 95% is i updated an entity for
   about 200 times. the class is:

   class ltWords( db.Model ):
   WordsDText = db.TextProperty()

   after this, there is only one entity of ltWords and the 
   WordsDText is
   no longer than 1MB. why does it take more than 90% of my 
   datastore
   quota and how to release it?

   for  saintthor.appspot.com

--
You received this message because you are subscribed to the Google 
Groups Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group 
athttp://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Sitewide 500 errors

[Hugo Visser]

All sites mentioned work for me here, hope that helps...

On Feb 2, 2:12 pm, johntray john.tur...@gmail.com wrote:
 My site (http://gmailnotes.appspot.com) is down too, returning the
 same error. System status page indicates no issues. What's going on?

 On Feb 2, 8:00 am, Jesse Grosjean je...@hogbaysoftware.com wrote:

  I'm seeing the following error on all of my app engine sites
  including:

 https://appengine.google.com/http://www.hogbaysoftware.com/http://www...
  ...

  -
  Error: Server Error

  The server encountered an error and could not complete your request.
  If the problem persists, please report your problem and mention this
  error message and the query that caused it.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: why my datastore nearly be full?

[Hugo Visser]

OK, searched through the groups again and all of the threads
mentioning this are referring to the size of the index. Initially I
thought I was imune to this issue because I have no indexes defined :)
But turned out that I every property of my entity was indexed, while
they weren't supposed to. I changed my code, deleted the old data and
the quota is down now.

Hope this helps others finding this thread.

On Feb 2, 1:04 pm, Hugo Visser botte...@gmail.com wrote:
 One other observation is that in the billing summary all seems fine:

 Stored Data:
 $0.005/GByte-day0.001.000.00$0.00

 Maybe a dashboard/statistics bug?

 On Feb 2, 12:58 pm, Hugo Visser botte...@gmail.com wrote:

  Any suggestions from Google folks such as Nick maybe?

  On Feb 2, 9:56 am, Hugo Visser botte...@gmail.com wrote:

   Well I guess what I'm seeing is not normal. I have 129 entities that
   take 1 MB in total according to the statistics. I have 48 MB in
   memcache and that's it.
   Still my total stored data quota is at 59% and didn't get reset, even
   after flushing the memcache and deleting all of my entities from the
   datastore.

   Besides memcache and datastore I also use task queues, anybody else
   using those and seeing a quota rise?

   This started somewhere around the weekend so I really think this is a
   bug of somekind...I hope Google will look into it soon.
   This post seems to be 
   related:http://groups.google.com/group/google-appengine/browse_thread/thread/...

   Hugo

   On Feb 1, 8:51 pm, Robert Kluin robert.kl...@gmail.com wrote:

We see about 10x overhead, I assume from indexes, in our apps.  One
app is using 30M for storing entities and metadata, the indexes take
another 300M.  We see similar usage multipliers in all of our apps.

Robert

2010/2/1 Hugo Visser botte...@gmail.com:

 I'm seeing the same thing. I got hardly anything stored in the
 datastore, but my quota is at 50% right now. I did have some stuff in
 the memcache however.
 This morning I flushed memcache and deleted all entities in the
 datastore but the quota usage is still going up. I'm waiting to see
 what happens after the daily reset.

 Hugo

 On Feb 1, 5:27 pm, Wooble geoffsp...@gmail.com wrote:
 The stored data quota in the dashboard isn't updated in realtime, so
 you wouldn't see the change immediately even if that is the problem.
 I'd suspect some buggy indexes or something, though, if that really
 was your only entity.  For one thing, you can't have a single entity
 larger than 1MB, and it would be tough to use 949 MB in datastore
 overhead even if you wanted to.

 On Jan 31, 3:05 am, saintthor saintt...@gmail.com wrote:

  i deleted the only entity of ltWords. the Total Stored Data is 
  still
  95%!

  who can help me?

  On 1月30日, 下午8时42分, Sylvain sylvain.viv...@gmail.com wrote:

   maybe this bug ?

  http://code.google.com/p/googleappengine/issues/detail?id=631

   On Jan 30, 9:27 am, saintthor saintt...@gmail.com wrote:

in quota details, i notic my Total Stored Data is 95%. it 
shouldn' t
be so large.

in Datastore Statistics, Size of all entities is only 9MB.

all that happened from about 4% to 95% is i updated an entity 
for
about 200 times. the class is:

class ltWords( db.Model ):
WordsDText = db.TextProperty()

after this, there is only one entity of ltWords and the 
WordsDText is
no longer than 1MB. why does it take more than 90% of my 
datastore
quota and how to release it?

for  saintthor.appspot.com

 --
 You received this message because you are subscribed to the Google 
 Groups Google App Engine group.
 To post to this group, send email to 
 google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group 
 athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Android app to monitor appengine quotas

[Hugo Visser]

Hi,

I've released a little android app to monitor your app engine quotas.
It's called Engine Watch and is available from the android market. See
my blog at http://code.neenbedankt.com for more details. I hope it's
useful to some of you too :)

Hugo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: why my datastore nearly be full?

[Hugo Visser]

I'm seeing the same thing. I got hardly anything stored in the
datastore, but my quota is at 50% right now. I did have some stuff in
the memcache however.
This morning I flushed memcache and deleted all entities in the
datastore but the quota usage is still going up. I'm waiting to see
what happens after the daily reset.

Hugo

On Feb 1, 5:27 pm, Wooble geoffsp...@gmail.com wrote:
 The stored data quota in the dashboard isn't updated in realtime, so
 you wouldn't see the change immediately even if that is the problem.
 I'd suspect some buggy indexes or something, though, if that really
 was your only entity.  For one thing, you can't have a single entity
 larger than 1MB, and it would be tough to use 949 MB in datastore
 overhead even if you wanted to.

 On Jan 31, 3:05 am, saintthor saintt...@gmail.com wrote:

  i deleted the only entity of ltWords. the Total Stored Data is still
  95%!

  who can help me?

  On 1月30日, 下午8时42分, Sylvain sylvain.viv...@gmail.com wrote:

   maybe this bug ?

  http://code.google.com/p/googleappengine/issues/detail?id=631

   On Jan 30, 9:27 am, saintthor saintt...@gmail.com wrote:

in quota details, i notic my Total Stored Data is 95%. it shouldn' t
be so large.

in Datastore Statistics, Size of all entities is only 9MB.

all that happened from about 4% to 95% is i updated an entity for
about 200 times. the class is:

class ltWords( db.Model ):
WordsDText = db.TextProperty()

after this, there is only one entity of ltWords and the WordsDText is
no longer than 1MB. why does it take more than 90% of my datastore
quota and how to release it?

for  saintthor.appspot.com

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Android app to monitor appengine quotas

[Hugo Visser]

Currently my appengine instance is running in free mode so I don't
know if I can test that. If it's in the admin console it shouldn't be
too hard to add.

On Feb 2, 4:07 am, Andrei gml...@gmail.com wrote:
 can u find way to get orders for paid Android apps from goog checkout?

 On Feb 1, 11:05 am, Hugo Visser botte...@gmail.com wrote:

  Hi,

  I've released a little android app to monitor your app engine quotas.
  It's called Engine Watch and is available from the android market. See
  my blog athttp://code.neenbedankt.comformore details. I hope it's
  useful to some of you too :)

  Hugo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.