Re: [google-appengine] Re: Creating Unique Entities

[Francis Stephens]

Maybe this clarifies my question. The paragraph at bottom is a description
of a transaction from Google. It refers to checking the 'last update time'
of an entity group and then checking whether 'it has changed since our
initial check'.

While I don't expect this short paragraph to describe all of
the complexities of App Engine transactions it does lay out a simplistic
implementation which does not cover the 'create if it doesn't exist'
transaction given on the same page. I have read a number of other
descriptions of App Engine transactions and none of them indicate that this
case is covered. My expectation is that it is, in fact, covered and these
transactions work. But I do need to be sure.

*"When a transaction starts, App Engine uses **optimistic concurrency
control  by
checking the last update time for the entity groups used in the
transaction. Upon commiting a transaction for the entity groups, App Engine
again checks the last update time for the entity groups used in the
transaction. If it has changed since our initial check, an error is
returned."*

https://cloud.google.com/appengine/docs/go/datastore/transactions

Francis

On 15 May 2015 at 22:45, Francis Stephens  wrote:

> I appreciate that the question is difficult, and I'm sure it could be
> worded differently. However, the reason I have phrased it as I did was
> because I really need to clarify this one case. It's very important to me.
>
> Perhaps it would be clearer if I try to address your description of an app
> engine transaction.
>
> *"When you start a transaction, each entity group (defined by the root
> key, whether or not an entity exists at that key) touched is enlisted in
> the transaction."*
>
> In the case I described the transaction is entered simultaneously by two
> instances.
>
> In this scenario both Get(...) calls returns an ErrNoSuchEntity we know
> (or I believe) at that point that the key does not exist anywhere in the
> datastore, attached to an entity or not. So I would like to know how the
> entity group is enlisted in this case.
>
> *"When you commit a transaction, if any of those entity groups have been
> changed by anyone anywhere, your transaction rolls back"*
>
> When the two Put(...) calls are executed one of them must fail with an
> ErrConcurrentTransaction (or ConcurrentModificationException in Java). So
> the question I have is, where the information which would allow for one of
> the Put(...) calls to fail?
>
> I have two reasons for asking for clarification on this point.
>
> 1: Data races are often very subtle, and I would like to clarify my
> understanding in this instance.
> 2: In the case given above a low probability data-race where the first
> calls might be over-written would be acceptable for most counters. That
> would be a very small counting loss and the chance of such a race, if it is
> possible at all, would have negligible impact. We don't have the same
> luxury, so I want be absolutely sure.
>
> Thanks for your response.
>
> Francis
>
> On 15 May 2015 at 01:34, Jeff Schnitzer  wrote:
>
>> I'm honestly having a hard time understanding your questions.
>>
>> GAE transactions are pretty simple. When you start a transaction, each
>> entity group (defined by the root key, whether or not an entity exists at
>> that key) touched is enlisted in the transaction. When you commit a
>> transaction, if any of those entity groups have been changed by anyone
>> anywhere, your transaction rolls back (in Java you get
>> ConcurrentModificationException).
>>
>> Your questions would be easier to understand without bringing up
>> implementation details like timestamps. The implementation is not
>> particularly relevant to the behavior.
>>
>> Jeff
>>
>>
>> On Thu, May 14, 2015 at 5:49 AM, Francis Stephens  wrote:
>>
>>> It's unclear to me where I should be posting this question. I've posted
>>> it to stack overflow in case that is more appropriate, however the nature
>>> of the question goes against the SO guidelines.
>>>
>>>
>>> http://stackoverflow.com/questions/30233093/app-engine-mechanics-of-creating-a-unique-entity-if-it-doesnt-exist
>>>
>>>
>>> On Wednesday, 13 May 2015 13:59:57 UTC+1, Francis Stephens wrote:

 We have an issue where we want to lazily create an entity if it does
 not exist. There is some discussion going on about how to do this and I
 would like to clarify some things around app engine transactions. I will
 limit my query to single entity group transactions.

 I am using Go in my examples, but I hope the code is clear enough for
 non-Go programmers.

 My understanding is that a transaction, on a single entity group, will
 succeed only if the entity group is not modified externally during the
 transaction. The 'entity group timestamp' indicating when an entity group
 was changed is stored in the root entity of the entity group. So during a
 transaction the current 'entity group t

Re: [google-appengine] Re: Creating Unique Entities

[Francis Stephens]

I appreciate that the question is difficult, and I'm sure it could be
worded differently. However, the reason I have phrased it as I did was
because I really need to clarify this one case. It's very important to me.

Perhaps it would be clearer if I try to address your description of an app
engine transaction.

*"When you start a transaction, each entity group (defined by the root key,
whether or not an entity exists at that key) touched is enlisted in the
transaction."*

In the case I described the transaction is entered simultaneously by two
instances.

In this scenario both Get(...) calls returns an ErrNoSuchEntity we know (or
I believe) at that point that the key does not exist anywhere in the
datastore, attached to an entity or not. So I would like to know how the
entity group is enlisted in this case.

*"When you commit a transaction, if any of those entity groups have been
changed by anyone anywhere, your transaction rolls back"*

When the two Put(...) calls are executed one of them must fail with an
ErrConcurrentTransaction (or ConcurrentModificationException in Java). So
the question I have is, where the information which would allow for one of
the Put(...) calls to fail?

I have two reasons for asking for clarification on this point.

1: Data races are often very subtle, and I would like to clarify my
understanding in this instance.
2: In the case given above a low probability data-race where the first
calls might be over-written would be acceptable for most counters. That
would be a very small counting loss and the chance of such a race, if it is
possible at all, would have negligible impact. We don't have the same
luxury, so I want be absolutely sure.

Thanks for your response.

Francis

On 15 May 2015 at 01:34, Jeff Schnitzer  wrote:

> I'm honestly having a hard time understanding your questions.
>
> GAE transactions are pretty simple. When you start a transaction, each
> entity group (defined by the root key, whether or not an entity exists at
> that key) touched is enlisted in the transaction. When you commit a
> transaction, if any of those entity groups have been changed by anyone
> anywhere, your transaction rolls back (in Java you get
> ConcurrentModificationException).
>
> Your questions would be easier to understand without bringing up
> implementation details like timestamps. The implementation is not
> particularly relevant to the behavior.
>
> Jeff
>
>
> On Thu, May 14, 2015 at 5:49 AM, Francis Stephens  wrote:
>
>> It's unclear to me where I should be posting this question. I've posted
>> it to stack overflow in case that is more appropriate, however the nature
>> of the question goes against the SO guidelines.
>>
>>
>> http://stackoverflow.com/questions/30233093/app-engine-mechanics-of-creating-a-unique-entity-if-it-doesnt-exist
>>
>>
>> On Wednesday, 13 May 2015 13:59:57 UTC+1, Francis Stephens wrote:
>>>
>>> We have an issue where we want to lazily create an entity if it does not
>>> exist. There is some discussion going on about how to do this and I would
>>> like to clarify some things around app engine transactions. I will limit my
>>> query to single entity group transactions.
>>>
>>> I am using Go in my examples, but I hope the code is clear enough for
>>> non-Go programmers.
>>>
>>> My understanding is that a transaction, on a single entity group, will
>>> succeed only if the entity group is not modified externally during the
>>> transaction. The 'entity group timestamp' indicating when an entity group
>>> was changed is stored in the root entity of the entity group. So during a
>>> transaction the current 'entity group timestamp' is read and the
>>> transaction can only succeed if it hasn't changed by the end of the
>>> transaction.
>>>
>>>  key := datastore.NewKey(c, "Counter", "mycounter", 0, nil)
>>>  count := new(Counter)
>>>  err := datastore.RunInTransaction(c, func(c appengine.Context) error {
>>>err := datastore.Get(c, key, count)
>>>if err != nil && err != datastore.ErrNoSuchEntity {
>>>  return err
>>>}
>>>count.Count++
>>>_, err = datastore.Put(c, key, count)
>>>return err
>>>  }, nil)
>>>
>>>
>>> In the example above (taken from
>>> https://cloud.google.com/appengine/docs/go/datastore/transactions)
>>> there are two non-error cases, I can see.
>>>
>>> 1: The Get succeeds and the 'entity group timestamp' on the counter can
>>> be used to ensure no other transactions update the counter during this
>>> transaction.
>>> 2: The Get fails with ErrNoSuchEntity and the Put is used to store the
>>> counter for the first time.
>>>
>>> In the second case it is possible that another identical transaction is
>>> running. If both transactions' Get return ErrNoSuchEntity how does the
>>> datastore ensure that only one put succeeds? I would expect there to be no
>>> 'entity group timestamp' in the datastore to test against?
>>>
>>> Does the transaction know that it needs to test for the non-existence of
>>> the counter in order for the Put and the

[google-appengine] Re: official informations website

[Patrice]

Bonjour Philippe!

Okay, so basically, Here is a breakdown of the information and how we 
spread it :

Latest version and the APIs related should be seen in our public site, here 
, and follow our SDK releases here 
. 

Google Groups should be used to ask open-ended questions, kinda like 
"discussions" about the best practices and ideas for the platform.

Stack Overflow should be used for specific programming questions about the 
platform, whereas Serverfault should be used for Google Compute Engine 
questions

There is also the Issue Tracker  
you can use to 
notify us (or look up) Defect Reports and Feature Requests about the 
platform.

Is there anything else I can help with? :).

Cheers



On Friday, May 15, 2015 at 3:06:43 AM UTC-4, philippe pithon wrote:
>
> hi,
>
> what the official appengine website for the latest information: new 
> version, new apis, etc ...
>
> - Google forum?
> - Stack Overflow forum? (why there are 2 forums ???)
> - Others?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/c293ce9f-1415-4234-899a-d3543ef8c325%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Including my own file as part of a golang GAE deployment

[Karl MacMillan]

  Can I suggest that it is not really safer to store the encrypted data on the 
server if the key is there as well. If your main concern is not storing the 
sensitive information in the source code repository you might as well make the 
decryption part of the build or a custom deployment script.


Karl



> On May 15, 2015, at 2:07 PM, Mobile Web Dev  wrote:
> 
> 
> Hello, I want to include a binary file the contains come sensitive AES 
> encrypted info that I do not want included in the go source.  The go server 
> will read it and decrypt it.
> 
> How Can I deploy this app and what would the file spec look like to read it 
> in my go server app?  I would assume something ./myfile.dat
> 
> 
> 
> 
> Thanks!
> 
> 
> 
> 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to google-appengine+unsubscr...@googlegroups.com
> .
> To post to this group, send email to google-appengine@googlegroups.com
> .
> Visit this group at http://groups.google.com/group/google-appengine
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/google-appengine/b706de27-da84-454e-ad56-ee2f42dd9a68%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/FA5ECB2C-01E7-4DD0-AD21-107D5BD8D3AB%40rakkoon.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Including my own file as part of a golang GAE deployment

[Mobile Web Dev]

Hello, I want to include a binary file the contains come sensitive AES 
encrypted info that I do not want included in the go source.  The go server 
will read it and decrypt it.

How Can I deploy this app and what would the file spec look like to read it 
in my go server app?  I would assume something ./myfile.dat


Thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/b706de27-da84-454e-ad56-ee2f42dd9a68%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: How to alleviate 200gb/day - 20$+ daily outgoing bandwidth image serving costs

[Rafael]

We had the same problem very early on and no money in the bank.

The cost per GB on appengine image serving is too high. Maybe it tries to
include the image resizing service fee or be a balance for many apps that
don't have social traffic. Social networks are expensive.

We migrated to maxcdn which I thought wasn't top tier. Then we migrated to
akamai and their admin tools were super outdated, for example no support
for spdy or http2. So, we came back to maxcdn and are happy with the
decision.

We paid $3k for 150TB of maxcdn traffic. You can see that's probably 10x
cheaper than blobstore serving.
On May 15, 2015 4:34 AM, "Kaan Soral"  wrote:

> After the traffic increased and costs jumped to $80's, I ended up using an
> existing CloudFlare integration I had, just for the images, saved 90gb's
> just in the first hour + didn't notice any issues at all
> So for now, cloudflare saves the day, don't know why I didn't think of
> using the existing integration before ... :)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at http://groups.google.com/group/google-appengine.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/google-appengine/ae373a09-987e-40fa-8180-a7828002f099%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/CALdMK1wwk_S5pMutQpNc5p1j3WN7SrU_m6QWPbKcCmqWyUvvOw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: How to alleviate 200gb/day - 20$+ daily outgoing bandwidth image serving costs

[Kaan Soral]

After the traffic increased and costs jumped to $80's, I ended up using an 
existing CloudFlare integration I had, just for the images, saved 90gb's 
just in the first hour + didn't notice any issues at all
So for now, cloudflare saves the day, don't know why I didn't think of 
using the existing integration before ... :)

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/ae373a09-987e-40fa-8180-a7828002f099%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Frontend Instances Class adjust/configure way changed

[ARay Chen]

Self update.

Found some tips and info related to this topic here:
https://cloud.google.com/appengine/docs/java/modules/converting

With these info, now I know the reason regarding to what I described and 
saw.

ARay Chen於 2015年5月15日星期五 UTC+8下午2時58分34秒寫道：
>
>
> Hi friends,
> Need some ideas from you.
>
> Not sure if I miss something but I just got lost today when I tried to 
> configure frontend instance class from F4 to F2 for one of my applications 
> (let's called project-A).
> The way to set frontend instance class in app engine admin console changes 
> as below: 
>
> 
>
>
>
> The other applications looks normal as before, there are some 
> configuration inputs like Frontend Instance Class, Idle Instances, Pending 
> Latency:
>
>
> 
>
>
> Now I'm not sure what should I do to set frontend instance class for 
> project-A?
> The most confused point is, why the other applications keep without any 
> changes but only project-A?
>
> If you know the reason, it would be much appreciated to share me know.
> Aray
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/ac4d5eeb-7ea4-41dd-9d14-71ef8b2dc5bf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] AppEngine/General Cold/Bulk Email Advice

[Kaan Soral]

Hi everyone

Over the years, I collected emails from facebook app users, I intended to 
use email as a part of app logic, but I never did, so I've never emailed 
anyone

Now I'm wondering whether it would be technically possible and generally 
appropriate to email users announcing a new product

My first concern is the general appropriateness, I will explain in the 
email that this is a one time thing and the source of the email
My second concern is spam filters, I'm concerned that after a couple 
thousand emails, all of the emails could be marked as spam by filters

I would really appreciate some advice
(I'm either going to use appengine/aws's email service, ease email sending, 
increase the volume over time, or use a service like sendgrid, the success 
is the main concern)

Thanks,
Kaan

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/4c5bc3af-074c-4104-8ab7-2b924f91e7bb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] official informations website

[philippe pithon]

hi,

what the official appengine website for the latest information: new 
version, new apis, etc ...

- Google forum?
- Stack Overflow forum? (why there are 2 forums ???)
- Others?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/52c19554-c929-4209-85d4-9297ce062d5b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.