[google-appengine] Re: Model datastore application

[Susan Lin]

Thank you very much for the detail response ! One question as I am not famiar 
with the parent type entities. If I need to list all the contacts would this 
result in 1 read operation of 200 read operations (considering 200 friends)?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/0c79a9dd-ead1-45f7-af1b-7a8b5b952c66%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] GAE instance prices?

[Susan Lin]

Hard to just see why then the regular GAE instances are still so expensive.

On Sunday, April 10, 2016 at 1:01:56 PM UTC-7, George-Cristian Bîrzan wrote:
>
> I doubt however that they will go up 8 fold, which seems to be the gap 
> currently 
> On 10 Apr 2016 10:07 pm, "Susan Lin" > 
> wrote:
>
>> Hi Jeff, yes you are correct they currently use these resources, but 
>> according to Google from https://cloud.google.com/appengine/pricing 
>> these prices will increase (which I assume is to the standard GAE prices) 
>> once flex environment becomes GA.
>>
>> "Instances within the flexible environment are charged the cost of the 
>> underlying Google Compute Engine Virtual Machines 
>> <https://cloud.google.com/compute/pricing>. This is Beta-only pricing 
>> and will increase once the flexible environment is GA."
>>
>> So these lower prices seem to be only temporary.
>>
>> On Saturday, April 9, 2016 at 2:25:28 PM UTC-7, Jeff Schnitzer wrote:
>>>
>>> “nearly exact” I meant to say. EB and Flexible runtimes are both mapped 
>>> to a whole hypervisor VM.
>>>
>>> Jeff
>>>
>>> On Sat, Apr 9, 2016 at 2:24 PM, Jeff Schnitzer  
>>> wrote:
>>>
>>>> If you want a nearly close comparison to ElasticBeanstalk, use a 
>>>> Managed VM (or “Flexible Runtime” I think they’re being called now). They 
>>>> are currently billed at the same rate as the underlying Compute Engine 
>>>> instance.
>>>>
>>>> Jeff
>>>>
>>>> On Fri, Apr 8, 2016 at 11:00 AM, Susan Lin  wrote:
>>>>
>>>>> I tested out Google App Engine 4 years ago and am coming back now to 
>>>>> see if it is a possibility for another project I will be working on. I 
>>>>> see 
>>>>> the pricing for instances are still the same as they were 4 years ago:
>>>>>
>>>>> This table summarizes the CPU, memory, and hourly billing rate of the 
>>>>> various instance classes.
>>>>> Instance ClassMemory LimitCPU LimitCost per Hour per Instance
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> F1 128 MB 600 Mhz $0.05
>>>>> F2 256 MB 1.2 Ghz $0.10
>>>>> F4 512 MB 2.4 Ghz $0.20
>>>>> F4_1G 1024 MB 2.4 Ghz $0.30
>>>>> When compared to Amazon's PaaS ElasticBeanstalk a 1 cpu 1GB instance 
>>>>> costs $0.013 an hour which is almost 30 times cheaper than Google App 
>>>>> Engine front end instances. I understand they offer different services, 
>>>>> but 
>>>>> as far as front end instances it is hard to see why in 2016 (Moore's Law) 
>>>>> a 
>>>>> 1gb GAE instance is still so expensive. Does Google have any plans to 
>>>>> lower 
>>>>> the costs of these instances in the very near future?
>>>>>
>>>>> -- 
>>>>> You received this message because you are subscribed to the Google 
>>>>> Groups "Google App Engine" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send 
>>>>> an email to google-appengi...@googlegroups.com.
>>>>> To post to this group, send email to google-a...@googlegroups.com.
>>>>> Visit this group at https://groups.google.com/group/google-appengine.
>>>>> To view this discussion on the web visit 
>>>>> https://groups.google.com/d/msgid/google-appengine/9020a9f4-886b-4841-9de0-314c14c0ffee%40googlegroups.com
>>>>>  
>>>>> <https://groups.google.com/d/msgid/google-appengine/9020a9f4-886b-4841-9de0-314c14c0ffee%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>
>>>>
>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Google App Engine" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to google-appengi...@googlegroups.com .
>> To post to this group, send email to google-a...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/google-appengine.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/google-appengine/4a50cbef-7fc0-49f2-a275-67feebdcd990%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/google-appengine/4a50cbef-7fc0-49f2-a275-67feebdcd990%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/3abcabf0-71f8-4b03-a305-9194e688a066%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Model datastore application

[Susan Lin]

down votefavorite 


I am looking how to create an efficient model which will satisfy the 
requirements I put below. I have tried using gcloud-node but have noticed 
it has limitations with read consistencies, references, etc. I would prefer 
to write this is nodejs, but would be open to writing in java or python as 
long as it would improve my model. I am building around the new pricing 
model which will come July 1st.

My application consists of a closed email system. In essence what happens 
is users register to the site. These user's can make friends. Then they can 
send emails to each other.

*Components of the app:*

Users - Unlimited amount of users can join.

Friends - A User can have 200 confirmed friends and 100 pending friend 
requests. When a friendlist is retrieved it should show the name of the 
friend. (I will also need to receive the id of the friends so I can use it 
on my client side to create emails).

Emails - Users can send emails to their friends and they can receive emails 
from their friends. The user can then view all their sent emails 
independently(sentbox) and all their received emails independently(inbox). 
They can also view the the emails sent between themselves and a friend 
order by newest. The emails should show the senders and receivers names. 
Once an email is read it needs to be marked as read.

My model looks something like this, but as you can see their are 
inefficiencies.

*Datastore Kinds:*

USER
-email (id) //The email doesn't need to be the id, but I need to be able to 
retrieve users by their email
-hash_password
-name
-account_status
-created_date

FRIEND
-id (auto-generated)
-friend1
-friend2
-status

EMAIL
-id (auto-generated)
-from
-to
-mutual_id
-message
-created_date
-has_seen

*Procedures of the application:*

*Register* - Get operation to see if a user with this email exists. If does 
not insert key.

*Login* - Get operation to get user based on email. If exists retrieve the 
hash_password from the entity and compare to user's input.

*Send friend request* - Friend data will be written twice for every 
relationship. Then using the index on friend1 and index on status I will 
query all the friends for a user and filter only those which are 'pending'. 
I will then count these friends and see if they are over X. Again I will do 
this for the other user. If they are both not over the pending limit, I 
will insert the friend request. This needs to run in a transaction.

*Accept a friend request* - Friend data will be written twice for every 
relationship. Then using the index on friend1 and index on status I will 
query all the friends for a user and filter only those which are pending. I 
will then count these friends and see if they are over X. Again I will do 
this for the other user. If they are both not over the pending limit, I 
will change both entities's status to accepted as a transaction.

*Show confirmed friends* - Friend data will be written twice for every 
relationship. Then using the index on friend1 and index on status I will 
query all the friends for a user and filter only those which are accepted. 
Not sure how I will show the friend's names (e.g what happens if a user 
changed their name this needs to be reflected in all friend relationships 
and emails!).

*Show pending friends* - Friend data will be written twice for every 
relationship. Then using the index on friend1 and index on status I will 
query all the friends for a user and filter only those which are pending. 
Not sure how I will show the friend's names (e.g what happens if a user 
changed their name this needs to be reflected in all friend relationships 
and emails!).

*View sent emails* - Using the index on the from property I would query to 
get all the sent emails from a user 5 at a time ordered by created_date 
(newest first). (e.g what happens if a user changed their name this needs 
to be reflected in all friend relationships and emails!).

*View received emails* - Using the index on the to property I would query 
to get all the received emails to a user 5 at a time ordered by 
created_date (newest first). When a emails is seen it will update that 
entities has_seen property to true. (e.g what happens if a user changed 
their name this needs to be reflected in all friend relationships and 
emails!).

*View emails between 2 users* - Using the index on mutual_id which is based 
on [lower_lexicographic_email]:[higher_lexicographic_email] to query the 
mutual emails. Ordered by newest, 5 at a time. (e.g what happens if a user 
changed their name this needs to be reflected in all friend relationships 
and emails!).

*Create email* - Using the friend1 and status index I will confirm the 
user's are friends. If they are friends, I will insert an email.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiv

[google-appengine] Google Datastore Pricing

[Susan Lin]

Is it possible to request for a new project that the prices which will be 
effective July 1st be effective now for the new project?

Thanks,
Susan

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/1fea7c71-506b-414b-a725-ba4838c59e2f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Google datastore pricing

[Susan Lin]

is it possible to request to have the prices which will be effective 
July1st on Google Datastore to be effective now for a new project?

Thanks,
Victor

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/da99f35b-a917-442e-b89f-64f78b68c5e8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] GAE instance prices?

[Susan Lin]

Hi Jeff, yes you are correct they currently use these resources, but 
according to Google from https://cloud.google.com/appengine/pricing these 
prices will increase (which I assume is to the standard GAE prices) once 
flex environment becomes GA.

"Instances within the flexible environment are charged the cost of the 
underlying Google Compute Engine Virtual Machines 
<https://cloud.google.com/compute/pricing>. This is Beta-only pricing and 
will increase once the flexible environment is GA."

So these lower prices seem to be only temporary.

On Saturday, April 9, 2016 at 2:25:28 PM UTC-7, Jeff Schnitzer wrote:
>
> “nearly exact” I meant to say. EB and Flexible runtimes are both mapped to 
> a whole hypervisor VM.
>
> Jeff
>
> On Sat, Apr 9, 2016 at 2:24 PM, Jeff Schnitzer  > wrote:
>
>> If you want a nearly close comparison to ElasticBeanstalk, use a Managed 
>> VM (or “Flexible Runtime” I think they’re being called now). They are 
>> currently billed at the same rate as the underlying Compute Engine instance.
>>
>> Jeff
>>
>> On Fri, Apr 8, 2016 at 11:00 AM, Susan Lin > > wrote:
>>
>>> I tested out Google App Engine 4 years ago and am coming back now to see 
>>> if it is a possibility for another project I will be working on. I see the 
>>> pricing for instances are still the same as they were 4 years ago:
>>>
>>> This table summarizes the CPU, memory, and hourly billing rate of the 
>>> various instance classes.
>>> Instance ClassMemory LimitCPU LimitCost per Hour per Instance
>>>
>>>
>>>
>>>
>>> F1 128 MB 600 Mhz $0.05
>>> F2 256 MB 1.2 Ghz $0.10
>>> F4 512 MB 2.4 Ghz $0.20
>>> F4_1G 1024 MB 2.4 Ghz $0.30
>>> When compared to Amazon's PaaS ElasticBeanstalk a 1 cpu 1GB instance 
>>> costs $0.013 an hour which is almost 30 times cheaper than Google App 
>>> Engine front end instances. I understand they offer different services, but 
>>> as far as front end instances it is hard to see why in 2016 (Moore's Law) a 
>>> 1gb GAE instance is still so expensive. Does Google have any plans to lower 
>>> the costs of these instances in the very near future?
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Google App Engine" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to google-appengi...@googlegroups.com .
>>> To post to this group, send email to google-a...@googlegroups.com 
>>> .
>>> Visit this group at https://groups.google.com/group/google-appengine.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/google-appengine/9020a9f4-886b-4841-9de0-314c14c0ffee%40googlegroups.com
>>>  
>>> <https://groups.google.com/d/msgid/google-appengine/9020a9f4-886b-4841-9de0-314c14c0ffee%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/4a50cbef-7fc0-49f2-a275-67feebdcd990%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] GAE instance prices?

[Susan Lin]

I tested out Google App Engine 4 years ago and am coming back now to see if 
it is a possibility for another project I will be working on. I see the 
pricing for instances are still the same as they were 4 years ago:

This table summarizes the CPU, memory, and hourly billing rate of the 
various instance classes.
Instance ClassMemory LimitCPU LimitCost per Hour per Instance




F1 128 MB 600 Mhz $0.05
F2 256 MB 1.2 Ghz $0.10
F4 512 MB 2.4 Ghz $0.20
F4_1G 1024 MB 2.4 Ghz $0.30
When compared to Amazon's PaaS ElasticBeanstalk a 1 cpu 1GB instance costs 
$0.013 an hour which is almost 30 times cheaper than Google App Engine 
front end instances. I understand they offer different services, but as far 
as front end instances it is hard to see why in 2016 (Moore's Law) a 1gb 
GAE instance is still so expensive. Does Google have any plans to lower the 
costs of these instances in the very near future?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/9020a9f4-886b-4841-9de0-314c14c0ffee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] New pricing question datastore

[Susan Lin]

I am trying to confirm the details of the new pricing effective July 1st. I 
think an example would work best.

Say I have an entity with 10 attributes. I put an index on all 10 
attributes. 

1) Now I want to create a new entity, what would be the cost?
2) I want to do a query and it will return 100 entities, what would be the 
cost ? 
3) If I do a projection query(all 10 indexed attributes returned) and again 
100 entities returned, what will be the cost?
4) I want to update all 10 indexed attributes of this entity what would be 
the cost?

Thanks

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/b69e163f-153c-4d3e-911c-2b43f4d4abfb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Cost of projection query?

[Susan Lin]

I would like to confirm the cost of a projection query. According 
to https://cloud.google.com/appengine/pricing the cost of a project query 
is 1 read operation while a a normal query getting the whole entity is 1 
read + 1 read per entity retrieved. 

If I have entities in this structure:

MyEntity
-ID
-Name
-Birthday

If I select do a projection query to get retrieve the ID and Name of 
everyone who's birthday is greater than a some date lets say Jan 1st 2000. 
If there are 100 entities which match does the return only count as 1 read 
to get all 100 entities back? If this is the case then say I also want to 
retrieve the birthday property, but don't wan't to select all the 
properties to avoid a query which gets the whole entity then why not then 
add a dummy property that has no use to the entities so then you can still 
select all the properties I need (ID,Name, and Birthday) and still only use 
1 read operation. 

If I did a query to select all the properties of a user who's birthday is 
greater than a some date lets say Jan 1st 2000. If there are 100 entities 
which match does the return will costs be 1 + 100 read operations? 

Cheers

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/6f6b6f39-e936-4f62-a960-90bbabed1907%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Data Modeling help for simple messaging

[Susan Lin]

To update an entity the costs is 1 write + 4 writes per modified index 
property.

On Tuesday, December 29, 2015 at 4:30:14 PM UTC-8, timh wrote:
>
> From your description "When the message has been seen for the first time I 
> would go to the message (1 get) and update the entity (1 write + (4 writes 
> x 3) = 13 writes to update the entity)."  I am having trouble working out 
> where you get the "(4 writes x 3)" when updating a single message as 
> "hasRead"
>
> T
>
> On Wednesday, December 30, 2015 at 1:31:31 AM UTC+8, Susan Lin wrote:
>>
>> Overall, read/write costs in Google Datastore are completely 
>> deterministic. I don't believe it would be very efficient for me to 
>> implement something which I already know would be too expensive. I have 
>> just outlined some ideas which could meet the requirement, but they are too 
>> expensive for me.
>>
>> Yes, users need to be able to read old messages (in batches of 10 at a 
>> time). This I think is my biggest issue. How to efficiently model the 
>> messages to be able to be retrieved and modified (updated to the hasRead 
>> attribute) efficiently.
>>
>> On Monday, December 28, 2015 at 10:04:32 PM UTC-8, timh wrote:
>>>
>>> To be honest.
>>>
>>> I would try a niave implementation and deal with how you will 
>>> fetch/send/receive information and how you plan to implement your workflows.
>>> Then profile the application and then start looking at how you might 
>>> optimise things.
>>>
>>> Doing things like "I then thought of creating an entity to store all the 
>>> sent/received messages in a string for a user inside of a single entity" 
>>> means you can't scale the number of messages without dealing with 
>>> overflows, and will people read old messages.  If not then retrieving that 
>>> single growing entity may cost you more than the writes (in terms of time, 
>>> cache hit ratios etc)).  Trying to optimise things before you have working 
>>> code is possibly not the best way to approach development.
>>>
>>>
>>> T
>>>
>>> On Tuesday, December 29, 2015 at 12:29:48 PM UTC+8, Susan Lin wrote:
>>>>
>>>> I have a very simple requirement. I have an application where users log 
>>>> in, add/remove/view friends, and can send/receive (also view the message 
>>>> you sent and received) a message (message is more like an email, not a 
>>>> real 
>>>> time chat) from a friend. When they receive the message, the application 
>>>> needs to mark the message as 'read'. The messages must also come ordered 
>>>> by 
>>>> newest to oldest.
>>>>
>>>> My thoughts:
>>>>
>>>> KINDS:
>>>>
>>>>
>>>> USER
>>>> username  (ID)
>>>> password 
>>>> friends  (will contain a max of 
>>>> 100 friends).
>>>>
>>>> MESSAGE
>>>> msg_id  (ID)
>>>> from  (indexed)
>>>> to  (indexed)
>>>> timestamp (indexed)
>>>> msg
>>>> hasRead
>>>>
>>>>
>>>> 1) User would login using their username and password
>>>> 2) User could get friends by getting their USER entity based on their 
>>>> username
>>>> 3) User could add/remove friends by getting the entity of user1 and 
>>>> user2 and either adding or removing friend via transaction to make sure 
>>>> they are consistent,
>>>> 4) User could get all the message they have sent by using indexing the 
>>>> the 'from' attribute (limit of 10 message per request). The same could be 
>>>> done to view all the messages they have received by using the 'to' 
>>>> attribute. When the message has been seen for the first time I would go to 
>>>> the message (1 get) and update the entity (1 write + (4 writes x 3) = 13 
>>>> writes to update the entity).
>>>>  
>>>> My major concern - If a user gets 10 messages, this will require 10 get 
>>>> requests plus if all 10 messages are new I will need to update each entity 
>>>> and if all of them are new that is (10 x 14) 140 writes. Then if I get 
>>>> another 10 message for this user the same process and this could all add 
>>>> up 
>>>> very quickly.
>>>>
>>>>
>>>> I then thought of creating an entity to store all the sent/received 
>>>&

[google-appengine] Re: Data Modeling help for simple messaging

[Susan Lin]

Overall, read/write costs in Google Datastore are completely deterministic. 
I don't believe it would be very efficient for me to implement something 
which I already know would be too expensive. I have just outlined some 
ideas which could meet the requirement, but they are too expensive for me.

Yes, users need to be able to read old messages (in batches of 10 at a 
time). This I think is my biggest issue. How to efficiently model the 
messages to be able to be retrieved and modified (updated to the hasRead 
attribute) efficiently.

On Monday, December 28, 2015 at 10:04:32 PM UTC-8, timh wrote:
>
> To be honest.
>
> I would try a niave implementation and deal with how you will 
> fetch/send/receive information and how you plan to implement your workflows.
> Then profile the application and then start looking at how you might 
> optimise things.
>
> Doing things like "I then thought of creating an entity to store all the 
> sent/received messages in a string for a user inside of a single entity" 
> means you can't scale the number of messages without dealing with 
> overflows, and will people read old messages.  If not then retrieving that 
> single growing entity may cost you more than the writes (in terms of time, 
> cache hit ratios etc)).  Trying to optimise things before you have working 
> code is possibly not the best way to approach development.
>
>
> T
>
> On Tuesday, December 29, 2015 at 12:29:48 PM UTC+8, Susan Lin wrote:
>>
>> I have a very simple requirement. I have an application where users log 
>> in, add/remove/view friends, and can send/receive (also view the message 
>> you sent and received) a message (message is more like an email, not a real 
>> time chat) from a friend. When they receive the message, the application 
>> needs to mark the message as 'read'. The messages must also come ordered by 
>> newest to oldest.
>>
>> My thoughts:
>>
>> KINDS:
>>
>>
>> USER
>> username  (ID)
>> password 
>> friends  (will contain a max of 100 
>> friends).
>>
>> MESSAGE
>> msg_id  (ID)
>> from  (indexed)
>> to  (indexed)
>> timestamp (indexed)
>> msg
>> hasRead
>>
>>
>> 1) User would login using their username and password
>> 2) User could get friends by getting their USER entity based on their 
>> username
>> 3) User could add/remove friends by getting the entity of user1 and user2 
>> and either adding or removing friend via transaction to make sure they are 
>> consistent,
>> 4) User could get all the message they have sent by using indexing the 
>> the 'from' attribute (limit of 10 message per request). The same could be 
>> done to view all the messages they have received by using the 'to' 
>> attribute. When the message has been seen for the first time I would go to 
>> the message (1 get) and update the entity (1 write + (4 writes x 3) = 13 
>> writes to update the entity).
>>  
>> My major concern - If a user gets 10 messages, this will require 10 get 
>> requests plus if all 10 messages are new I will need to update each entity 
>> and if all of them are new that is (10 x 14) 140 writes. Then if I get 
>> another 10 message for this user the same process and this could all add up 
>> very quickly.
>>
>>
>> I then thought of creating an entity to store all the sent/received 
>> messages in a string for a user inside of a single entity:
>>
>> MESSAGE
>> user_entitynum1  (ID)
>> messages > msg2_touser8_read_timestamp, msg3_touser5_unread_timestamp, etc>
>>
>> This way I could store all the message (under 1mb) inside of this one 
>> entity, but I would have to keep track of which entity each user is at (if 
>> the user exceeds the 1mb limit I will have to create a new entity).  This 
>> proved to also not be too efficient because if I send a message to perform 
>> 2 gets to see which message entity I am currently at and which message 
>> entity they are currently at. From there I must now use another 2 reads to 
>> get those entities and another 4 writes to update them both (considering I 
>> do not need to create another entity if it is full). 
>>
>>
>> I was looking for any ideas to accomplish what I need in a more efficient 
>> way. I have nothing implemented yet so I  am open to ANY ideas. My main 
>> concern is for this to be efficient,
>>
>> Cheers!
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/7edf29f7-f8b6-4911-98ac-6410d66c8915%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Data Modeling help for simple messaging

[Susan Lin]

I have a very simple requirement. I have an application where users log in, 
add/remove/view friends, and can send/receive (also view the message you 
sent and received) a message (message is more like an email, not a real 
time chat) from a friend. When they receive the message, the application 
needs to mark the message as 'read'. The messages must also come ordered by 
newest to oldest.

My thoughts:

KINDS:


USER
username  (ID)
password 
friends  (will contain a max of 100 
friends).

MESSAGE
msg_id  (ID)
from  (indexed)
to  (indexed)
timestamp (indexed)
msg
hasRead


1) User would login using their username and password
2) User could get friends by getting their USER entity based on their 
username
3) User could add/remove friends by getting the entity of user1 and user2 
and either adding or removing friend via transaction to make sure they are 
consistent,
4) User could get all the message they have sent by using indexing the the 
'from' attribute (limit of 10 message per request). The same could be done 
to view all the messages they have received by using the 'to' attribute. 
When the message has been seen for the first time I would go to the message 
(1 get) and update the entity (1 write + (4 writes x 3) = 13 writes to 
update the entity).
 
My major concern - If a user gets 10 messages, this will require 10 get 
requests plus if all 10 messages are new I will need to update each entity 
and if all of them are new that is (10 x 14) 140 writes. Then if I get 
another 10 message for this user the same process and this could all add up 
very quickly.


I then thought of creating an entity to store all the sent/received 
messages in a string for a user inside of a single entity:

MESSAGE
user_entitynum1  (ID)
messages 

This way I could store all the message (under 1mb) inside of this one 
entity, but I would have to keep track of which entity each user is at (if 
the user exceeds the 1mb limit I will have to create a new entity).  This 
proved to also not be too efficient because if I send a message to perform 
2 gets to see which message entity I am currently at and which message 
entity they are currently at. From there I must now use another 2 reads to 
get those entities and another 4 writes to update them both (considering I 
do not need to create another entity if it is full). 


I was looking for any ideas to accomplish what I need in a more efficient 
way. I have nothing implemented yet so I  am open to ANY ideas. My main 
concern is for this to be efficient,

Cheers!


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/980c6d49-83dc-4fac-aaea-1521cf14b52e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Async calls from Java to datastore

[Susan Lin]

I do not want to block threads in my application and so I am wondering are 
calls to the the Google Datastore async? For example the docs show 
something like this to retrieve an entity:

// Key employeeKey = ...;LookupRequest request = 
LookupRequest.newBuilder().addKey(employeeKey).build();LookupResponse response 
= datastore.lookup(request);if (response.getMissingCount() == 1) {
  throw new RuntimeException("entity not found");}Entity employee = 
response.getFound(0).getEntity();

This does not look like an async call to me, so it is possible to make 
aysnc calls to the database in Java? I noticed App engine has some 
libraries for async calls in its Java API, but I am not using appengine, I 
will be calling the datastore from my own instances. As well, if there is 
an async library can I test it on my local server (for example app engine's 
async library I could not find a way to set it up to use my local server 
for example I this library can't get my environment variables).

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/b9fac7d0-2d49-479b-8e8c-ea1f2d35f196%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Is there network cost for Google Datastore to Google compute engine?

[Susan Lin]

I will be calling Google Datastore from a Google compute engine instance. 
Some of the entities will be quite large (almost 1mb). I was wondering if 
there are network costs associated with this? Looking at the docs 
(https://cloud.google.com/datastore/docs/pricing) I could not find anything 
to do with network pricing. So is there no cost to the datastore if it is 
egressing data?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/26aca542-f5f0-4304-a368-2bc67690cfb4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.