Re: [google-appengine] Re: Datastore: how to design for huge time-series data
i've got some code for this from a recent project . hit me up . log individual events, then run map reduce to aggregate into time slices also by field values to create preagrregrated counts . querying is not as nimble as say mongo, so this works, but a few extra steps -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/443ef0ca-b83a-470f-bb05-d4d235afd62e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Re: App Engine 1.7.0 Release is Out!
It surprises me at how the cost of disk is getting applied everywhere.. Disk is fuckin cheap!! I'd rather pay where paying counts -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/GqkS49vAD90J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] MightyClient
This is awesome.. I like how you kept the code succinct and terse -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/GkD9-qMeFyYJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Calculation of frontend hours seems off by almost factor 2.
Not directly answering the question, I'm posing another, why the eff are you running instances in such a manner for such little qps Also, I can also suggest that at that low of qps, you could stand to substantially optimize the work you're doing reading data from the data store, which will also help minimize the frontend instance workload -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/uvm50fhNdsYJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] URGENT main url path: /, not serving.. but getting 200 response
*Richard Watson's *post was money.. ackkk.. someone unwittingly let the cache-control header slip into a base request handler.. thnx everyone.. On Sunday, April 29, 2012 12:53:15 AM UTC-7, Richard Watson wrote: What if Gregory put (e.g.) Cloudflare in front of the app and used it to fetch app requests and serve them as www (I assume that's possible). This way, people continue to see www and can add www links. Can try it as a short-term solution while you tinker with the setup and/or wait for the edge cache to sort itself out. It's quite possible that Google's edge cache caches for a maximum period shorter than a year due to demand/bandwidth priorities, so maybe this'll be solved in a week or month. On Saturday, April 28, 2012 10:13:26 AM UTC+2, alex wrote: I think you can actually do it without manual intervention from Google internally: Let's say your app (with cached blank page) is currently accessible as www.example.org 1. make sure you remove cache-control: public ... headers from the response 2. add a mapping that will make your app accessible as app.example.org 3. like Joshua suggested, change www CNAME into www A DNS record pointing somewhere else, where you do a simple (maybe even permanent) redirect to app.example.org Even though it seems like your blank page is cached for a long time now, www DNS record will expire sooner (normally 24 ours if you haven't changed TTL) Hope that helps. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/llofFYky_t8J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] URGENT main url path: /, not serving.. but getting 200 response
yep.. bingo.. what are my options from here? On Fri, Apr 27, 2012 at 12:29 PM, Joshua Smith joshuaesm...@charter.netwrote: You could test this theory by hitting /?foo On Apr 27, 2012, at 3:17 PM, Matthew Blain wrote: Typically a 204 with nothing run is simply a signal that your content has been cached elsewhere in Google's network. If you responded with cache-control: public with a max-age of something, it may be cached until that expires. Without futher info (such as an appid or the domain), it's hard to tell what's going on. On Apr 27, 11:34 am, Gregory Nicholas faction.greg...@gmail.com wrote: really need help.. i'm at a place where i can't really fix this on my own.. it's 11:00am.. been trouble shooting this since 9pm last night (thank god for adderall) *Preface (kinda):* - we've been working with appengine for a couple years now (haven't seen everything, but been through a lot of troubleshooting) - python 2.7, webapp 2.3, django 1.2, HRD - absolutely NO issues with local dev environ, production issue only *Issue / Symptoms:* - only the main url path: / for my app domain is not serving the request to my app - essentially breaks down to only the home page not working on http://www.domainname.com - the main url path for the appspot domain: http://domainname.appspot.com serves - request is routed properly from the app.yaml to my request handler - on http://domainname.com, any request to a subpath STILL WORKS! - ex: http:// domainname .com/user/login serves, works - the request IS hitting my application: - attached is a screen shot of my app logs, where the main path serves - not hitting my app.yaml file, and not routed to any request handlers - appears as though my code is never being executed for the request *What I've tried:* - i telnet'ed and tracerouted the domain name to rule out it being a DNS issue - see attached terminal screenshot - i've redeployed my app several times - i reverted the default version to a previous version - i've shutdown any instances several times - i switched from automatic instance allocation to manual, switching from F1 to F2 classes PLEASE HELP!!! Screen Shot 2012-04-27 at 11.14.15 AM.png 193KViewDownload Screen Shot 2012-04-27 at 11.31.23 AM.png 156KViewDownload -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] URGENT main url path: /, not serving.. but getting 200 response
by bingo i meant that was the request with the /? worked indeed.. dare i ask on options to cache bust? On Fri, Apr 27, 2012 at 12:53 PM, Gregory Nicholas nicholas.g.greg...@gmail.com wrote: yep.. bingo.. what are my options from here? On Fri, Apr 27, 2012 at 12:29 PM, Joshua Smith joshuaesm...@charter.netwrote: You could test this theory by hitting /?foo On Apr 27, 2012, at 3:17 PM, Matthew Blain wrote: Typically a 204 with nothing run is simply a signal that your content has been cached elsewhere in Google's network. If you responded with cache-control: public with a max-age of something, it may be cached until that expires. Without futher info (such as an appid or the domain), it's hard to tell what's going on. On Apr 27, 11:34 am, Gregory Nicholas faction.greg...@gmail.com wrote: really need help.. i'm at a place where i can't really fix this on my own.. it's 11:00am.. been trouble shooting this since 9pm last night (thank god for adderall) *Preface (kinda):* - we've been working with appengine for a couple years now (haven't seen everything, but been through a lot of troubleshooting) - python 2.7, webapp 2.3, django 1.2, HRD - absolutely NO issues with local dev environ, production issue only *Issue / Symptoms:* - only the main url path: / for my app domain is not serving the request to my app - essentially breaks down to only the home page not working on http://www.domainname.com - the main url path for the appspot domain: http://domainname.appspot.com serves - request is routed properly from the app.yaml to my request handler - on http://domainname.com, any request to a subpath STILL WORKS! - ex: http:// domainname .com/user/login serves, works - the request IS hitting my application: - attached is a screen shot of my app logs, where the main path serves - not hitting my app.yaml file, and not routed to any request handlers - appears as though my code is never being executed for the request *What I've tried:* - i telnet'ed and tracerouted the domain name to rule out it being a DNS issue - see attached terminal screenshot - i've redeployed my app several times - i reverted the default version to a previous version - i've shutdown any instances several times - i switched from automatic instance allocation to manual, switching from F1 to F2 classes PLEASE HELP!!! Screen Shot 2012-04-27 at 11.14.15 AM.png 193KViewDownload Screen Shot 2012-04-27 at 11.31.23 AM.png 156KViewDownload -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] deploy only static files
is there currently a way to only deploy certain, or only static files? this is what i'm currently facing, and hoping to use a selective deploy feature for: - my app is currently managing a couple dozen automated system emails - created them as django templates to reduce maintenance overhead - some emails have 2 or 3 different versions - allow us to test and compare responses between variations ideally, i want to provide a fast path to updating the email templates, without redeploying the entire application. is this possible or should i look to not use static files / django if they need to be updated more often than i deploy? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: fastest way to serialize db.Model(s) to json?
if it's as bad as 6 to 8 seconds, have you looked at creating a list
of string for the property names? i know its more maintenance
overhead, but might be worth the tradeoff..
ex:
class SerializableModel(object):
serializable = []
def to_json(self):
result = []
for x in self.serializable:
result.append(self.__dict__[x])
## append values from self.properties().items()
class A(db.Model, SerializableModel):
serializable = ['user_keys']
_user_keys = db.StringListProperty(default=[], indexed=False)
@property
def user_keys(self):
return [db.Key(x) for x in self._user_keys]
@property
def user_keys_count(self):
not in serializable list
return len(self._user_keys)
On Jan 10, 9:05 am, Andreas a.schmi...@gmail.com wrote:
hi,
im using more or less this
(http://www.awebcoder.com/post/91001/extended-jsonify-function-for-app...)
jsonify function to serialize whatever object i pass into the jsonify()
function and i have to say it does all i need and works pretty good.
the only downside i see right now is the speed. if i try to serialize 50
db.Model objects with around 10 properties each in production it takes 6-8
seconds which is not exactly what i would expect.
is there a way to make this part of code more performant?
for property in dir(obj):
if property[0] != '_':
try:
value = obj.__getattribute__(property)
valueClass = str(value.__class__)
if not(('function' in valueClass) or ('built' in valueClass)
or ('method' in valueClass)):
value = dumps(value)
if value != None:
properties[property] = value
else:
properties[property] =
except: continue
if len(properties) == 0:
return str(obj)
else:
return properties
i know i could use obj.properties().items() to loop over the properties but i
would not be able to serialize properties which are @property functions
because those don't get listed in the obj.properties() list.
right now im caching the results so im gaining speed because i get the result
from memcache but when i flush the cache or the memcached result is not
available it takes too much until the result is ready.
thx
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
