[google-appengine] Are Weekly Discounted Instance Hours still a thing?
Looking at my billing it looks like my reserved weekly discounted instance hours aren't being figured in any way. Is this because all instance hours are now the same price and there's no need for reserved hours? Why is there still a setting for reserving hours? Am I crazy or has all documentation of this stuff vanished too? I dropped my reserved hours down to 0 since there's no price difference, but I'd like to know if there's still some reason to use them. Thanks -n8 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Re: Outgoing bandwidth explosion
No, nothing like that. The app is a backend for an iPhone game that
doesn't use blobstore or allow uploads at all.
Thanks,
-Nathan
On Jan 6, 4:38 am, Andrin von Rechenberg and...@miumeet.com wrote:
Maybe you have user uploaded hot-linked images that are served from
blobstore?
Do you have any UGC? For an evil guy there is nothing more interesting than
hosting
porn images on the blazing fast GAE infrastructure with you paying for it.
I really wish one could disabling hot linking of blobstore images.
Cheers,
-Andrin
On Fri, Jan 6, 2012 at 8:40 AM, n8gray n8g...@gmail.com wrote:
Hi,
I've got an app that I've had running for a long time now and its
resource usage is highly predictable. The app consistently uses about
60-70MB outgoing bandwidth per day. But today the dashboard tells me
it's used over 2GB! When I look at the bytes sent/sec graph for the
last 7 days there's no difference between today and any other day. In
fact, all of my plots look typical. I tried searching my logs for
bytes:[1-9]\d{5,10} and got nothing -- there were no huge requests.
What's going on here??
Thanks,
-Nathan
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Outgoing bandwidth explosion
Hi,
I've got an app that I've had running for a long time now and its
resource usage is highly predictable. The app consistently uses about
60-70MB outgoing bandwidth per day. But today the dashboard tells me
it's used over 2GB! When I look at the bytes sent/sec graph for the
last 7 days there's no difference between today and any other day. In
fact, all of my plots look typical. I tried searching my logs for
bytes:[1-9]\d{5,10} and got nothing -- there were no huge requests.
What's going on here??
Thanks,
-Nathan
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: email sent by GAE would be regarded as spam for some email services
Ikai, I totally understand that you guys want to focus on the interesting parts of the problem, but you could make one really simple change that would probably *solve* the problem for most of your users: allow us to set the DKIM header on outgoing mail. All it would require is allowing write access to one more header. See this bug for details: http://code.google.com/p/googleappengine/issues/detail?id=3161 Cheers, -n8 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Please help! App Engine mail marked as spam.
Thanks from me as well. I updated my record, but I'm pretty sure my messages were already getting a pass on SPF. I'll see if it helps with Yahoo. Cheers, -n8 On Jul 31, 4:59 am, Greg g.fawc...@gmail.com wrote: Thanks Onestone - I've updated my SPF records and my messages now get an SPF pass instead of neutral. It'll be interesting to see how much this improves email delivery. On Jul 30, 12:49 am, Onestone onest...@gmail.com wrote: Look here: http://www.kyle-jensen.com/increase-email-deliverability-on-appengine-by -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Static images not being served
Hi all, My static images aren't being served. I've got this in my app.yaml file: handlers: # Static - url: /html static_dir: html - url: /css static_dir: css - url: /images static_dir: images ... and so on. My static html and css files are served without problems, but my image files are not. They do appear to be uploaded, since if I change nothing but a single image then redeploy, one file gets uploaded. For an example of a broken image on a page, see: http://playhexalex.appspot.com/actions/test Thanks for any help, and please CC me with replies. -n8 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Static images not being served
On Jun 21, 12:51 pm, n8gray n8g...@gmail.com wrote: Hi all, My static images aren't being served. ... Thanks for any help, and please CC me with replies. Ok, sorry for replying to myself, but just after posting this I tried renaming my image file from HexaLex-logo.png to logo.png, and that solved the problem. I'm guessing the '-' is the problem. That's gotta be a bug in GAE, right? Thanks, -n8 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Globally monotonic counter
Hi Nick, Wow, that's impressive! That's a very useful bit of information. Are memcache writes guaranteed ordered wrt datastore writes as well, or is it possible for another part of the system to see them in different orders? Cheers, -n8 On Jul 22, 9:56 am, Nick Johnson (Google) nick.john...@google.com wrote: Hi n8gray, The Datastore is strongly consistent. As such, all processes will see changes happen in the same order, even across entity groups - and at any one point in time, all processes will have a consistent view of the datastore. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Globally monotonic counter
On Jul 23, 3:18 pm, Nick Johnson (Google) nick.john...@google.com wrote: Memcache is also strongly consistent. Since both APIs are synchronous, by the time your API call returns, the changes are visible everywhere - so a datastore write followed by a memcache write will be seen in that order everywhere. There's no guarantee of atomicity across the two, though, so you need to assume that either or both operations could fail independently. Ah, I had no idea those APIs were synchronous! In fact, I assumed that writes would be asynchronous. That makes things much easier. It would be good to mention that in the documentation for those subsystems. Cheers, -n8 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Globally monotonic counter
On Jul 16, 10:35 pm, Andy Freeman ana...@earthlink.net wrote: I'm starting to think that the GAE takes care of the messy details of distributed systems programming claim is a bit overstated... Global clock consistency requires very expensive clocks accessible from every server with known latency (and even that's a bit dodgy). AFAIK, GAE doesn't provide that, but who does? GAE doesn't do the impossible, but also doesn't say that it does. WRT the latter, would you really prefer otherwise? But that's just it -- in many places it's claimed that GAE makes it all a cakewalk. From the datastore docs: Storing data in a scalable web application can be tricky. A user could be interacting with any of dozens of web servers at a given time, and the user's next request could go to a different web server than the one that handled the previous request. All web servers need to be interacting with data that is also spread out across dozens of machines, possibly in different locations around the world. Thanks to Google App Engine, you don't have to worry about any of that. App Engine's infrastructure takes care of all of the distribution, replication and load balancing of data behind a simple API—and you get a powerful query engine and transactions as well. You could argue that that's not claiming to do the impossible, but you don't have to worry about any of that is certainly not true. Nowhere in the documentation is there a discussion of the kinds of subtle gotchas that you need to be aware of when programming for this kind of system. It's all just golly isn't this so gosh-darn easy! You have to go digging to find the article on transaction isolation where you find out that your queries can return results that, um, don't match your queries. And AFAICT you *do* have to worry about subsequent requests being handled by different servers, since there doesn't seem to be any guarantee that the datastore writes made in one request will be seen in the next. Memcache doesn't have transactions, so it seems like guaranteeing coherence with the datastore is tricky. I worked in a distributed systems group for many years, so I know that many of these problems are simply inherent to distributed systems. It doesn't disturb me that they exist. What bothers me is the way these issues are broadly *ignored* by GAE's documentation. If I wasn't a bit savvy about distributed systems I probably wouldn't have realized that clock skew could cause problems, and nothing I read in GAE's docs would have helped me figure it out. So no, I don't want GAE to claim to do the impossible, I want them to *stop* claiming to do the impossible. I would love to see some articles about the pitfalls of the system and how to avoid them or mitigate them. The transaction isolation article is great in that respect -- I hope people at Google are planning more along those lines. Cheers, -n8 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Globally monotonic counter
On Jul 17, 8:29 am, Andy Freeman ana...@earthlink.net wrote:
You're mis-parsing the sentence. Note that they even tell you what
they mean by take care of messy details.
Let's look at another example. MS's Visual {whatever} documentation
claims that it makes programming easy. Do you think that that claim
implies that using said product will let anyone produce an O(N)
solution to an NP-complete problem?
I'm sorry, but you don't have to worry about any of that right after
a description of a massively distributed database is a much stronger,
more specific statement than your strawman, and it's also demonstrably
untrue.
I worked in a distributed systems group for many years, so I know that
many of these problems are simply inherent to distributed systems. It
doesn't disturb me that they exist.
You're complaining that GAE doesn't solve them.
Where, specifically? You seem to be reacting to this statement:
I'm starting to think that the GAE takes care of the messy details of
distributed systems programming claim is a bit overstated...
You may notice that I'm criticizing GAE's advertising, not its
implementation.
Cheers,
-n8
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Globally monotonic counter
Thanks for the advice, Nick. I'd still like to know more about the consistency model though. For example, I wonder if there's any guarantee that two transactions on different entity groups executed by one process in a given order will be observed in the same order. I suspect the answer is no. I'm starting to think that the GAE takes care of the messy details of distributed systems programming claim is a bit overstated... Cheers, -n8 On Jul 14, 2:27 am, Nick Johnson (Google) nick.john...@google.com wrote: Hi Nathan, Your best options are either to keep track of one event stream per game, or to use system time, and 'rewind' the timestamp a bit to capture any missed events, as you suggest. Global monotonic counters aren't very practical in large distributed systems. -Nick Johnson --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Globally monotonic counter
Hi Albert, Thanks for the suggestion. I think I can live without uniqueness as long as timestamps don't go backwards. But I think my problem still exists with a sharded counter. Having a counter is certainly better than using datetime.utcnow() and lets me assign an order to all events that have been generated, but that's not really the problem. The tricky part is deciding, on the client end, which events to request based on the events you've received. When the client asks for all events after time T it gets some last event with a new timestamp S. But I don't think you can trust this S because there might be some other event in some corner of GAE with an earlier timestamp that hasn't yet been observed by the server that answered the client's request. I guess the root of the problem is that I know that transactions on entity groups give me the ACID properties but when it comes to operations outside of transactions I have no idea what the consistency model is. Has this been described somewhere? Thanks, -n8 On Jul 13, 7:06 pm, Albert albertpa...@gmail.com wrote: Hi! This is a quick suggestion. How about using a global counter (just like your title suggests). You can use a sharded global counter to facilitate your counting. And use that counter as a timestamp / bookmark. On every event, you read from the global counter, use that value as your timestamp, and then increment the global counter. I'm not sure of it's implications, though. I'm not also sure if it actually guarantees uniqueness of timestamps when two events happen almost at the same time. Perhaps you can get an idea from this. Enjoy! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Finding out if a key is in the datastore
Hi folks, Is there a way to find out if a key refers to an entity in the datastore without retrieving it? Something like myKey.exists() would be nice, but I don't see it in the docs. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: DB design best practices
On Jun 10, 5:21 pm, Tony fatd...@gmail.com wrote: The biggest question I wished I'd asked myself before designing my app is: what sorts of models will I want to update together in transactions? Because in Datastore you can only run transactions on entities in the same entity group (meaning each entity is an ancestor/ child of the others). So if you need certain operations to maintain consistency, for instance add new Turn entity, update Game's current_player you will need to plan that ahead of time (perhaps assigning each Turn as a child of the Game it takes place in). Right, I came to the same conclusion. The game's state is the only super-entity that needs to be updated atomically, so I decided to group it together. Other than that, just remember that writing to Datastore is very expensive (as is fetching large numbers of entities from a query) - you will want to figure out ways to retrieve entities by key as much as possible. I'll keep that in mind. It would be nice to have some rule of thumb advice on when and how to use memcache. Thanks, -n8 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: DB design best practices
Hi Nick, On Jun 11, 4:19 am, Nick Johnson (Google) nick.john...@google.com wrote: Hi n8gray, Excellent question! Given the amount of information you're likely to store against Players2Games (GamePlayers, perhaps?), and the number of games any one player may have, and the likely access patterns, I would suggest sticking with a separate model for it. Interesting -- I came to the opposite conclusion but maybe my reasoning isn't sound. Here's what I've got right now (untested, so it may contain syntax errors and such): class GameState(VersionedModel): # Parent should be a GameMeta scores = db.ListProperty(int) other gritty details of the current game state class GameTurn(VersionedModel): # Parent should be a GameMeta creationDate = db.DateTimeProperty(auto_now_add=True) player = db.ReferenceProperty(User) turn_score = db.IntegerProperty() more gritty details about this turn This is the stuff you only care about if you're currently playing the game. The scores may be better placed in GameMeta -- I haven't decided yet. class GameMeta(VersionedModel): name = db.StringProperty(required=True) password = db.StringProperty(indexed=False) creationDate = db.DateTimeProperty(auto_now_add=True) isActive = db.BooleanProperty(default=True) # In case of tie, there can be more than one winner winners = db.ListProperty(db.Key, default=None) playerCount = db.IntegerProperty(required=True) currentPlayer = db.ReferenceProperty(User, required=True) currentPlayerNumber = db.IntegerProperty(default=0) gameState = db.ReferenceProperty(GameState) players = db.ListProperty(db.Key) GameMeta holds all the metadata of the game. I moved the players list in here (despite watching Brett Slatkin's I/O talk on list properties) because I reasoned that a) the serialization/deserialization overhead for 4 elements wouldn't be too bad, and b) You're going to want the player list every time you retrieve the game anyway. If you think this is unwise, however, I'm interested to hear why. The main lesson for using the datastore instead of a relational database is simply to denormalize and precalculate. In this case, that likely means storing running totals (number of turns, score, etc) against the Players2Games entity, instead of calculating them when needed as you might in a relational database. Yeah, I was planning to do a fair bit of denormalization. Tony's point about entity groups is an excellent one. Based on the sort of updates you're likely to want to do, and the access patterns in an app like this, I would suggest making the Players2Games entities child entities of the related Games entity, and making the Turns entities likewise child entities of their Games entity. This way, each game has its own entity group, so you can make atomic (transactional) updates across the whole game with ease. At least I got that part right! Thanks, -n8 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] DB design best practices
Hi everybody, I'm about to start working on an AppEngine backend for an iPhone game I'm developing. It's a simple board game, with 2-4 players who each take turns making plays. Originally I had planned to set up a LAMP server for my project, but AE has changed my plans (for the better!). I've never written database code before but I've read up on the basics of database design, and I came to the conclusion that I would need a DB schema with tables something like this: Players: username, email, userid, ... Games: gameid, time_started, current_player, is_finished, ... Players2Games: userid, gameid, score, ... Turns: userid, gameid, timestamp, turn_number, play, turn_score, ... It seems clear, however, that Datastore is not a traditional database and perhaps my schema needs to be revisited. Is it still necessary or advisable to use a table like Players2Games in order to represent many- to-many relationships? What should my roots and parent/child relationships be? Typical queries will be (unsurprising) things like: get all games for player x get all players for game x get the scores of all players in game x get any turns in game x that have occurred since time t Any advice, or pointers to articles/posts/documentation are appreciated! Thanks, -n8 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
