[google-appengine] Re: http://carpoolfinder.appspot.com/ won't work now and svn update fails

[thebrianschott]

Using logging.info() I have determined that the appspot is always
return None for "user=users.get_current_user()" and that is why the
app is working differently at appspot that locally. I have tried to
make this a single "transaction" to allow the value of user to be
determined in a timely way, but either I am doing it wrong, or it does
not work. Please advise.

def make_new_map(place_id,self):
  user = users.get_current_user()
  logging.info("User %s " % user)
  if user:#send to add_place.html
  logging.info("User if %s " % user)
  place= Group(key_name=place_id)
  place.place = place_id
  place.user = user
  place.put()
  key = db.Key.from_path("Group", place_id)
  path = os.path.join(os.path.dirname(__file__),
'add_place.html')
  template_values = dict(place=place.place, user=place.user)
  self.response.out.write(template.render(path,
template_values))
  else: #send back to choose_place.html
  logging.info("User else %s " % user)
  path = os.path.join(os.path.dirname(__file__),
'choose_place.html')
  template_values = dict()
  self.response.out.write(template.render(path,
template_values))


db.run_in_transaction(make_new_map,place_id,self)

Below is a copy of my request log from the appspot; notice the user is
None.

01-07 11:50PM 57.128 /?place=Tester 200 15ms 42ms-cpu 1kb
68.223.88.73 - - [07/Jan/2009:23:50:57 -0800] "GET /?place=Tester HTTP/
1.1" 200 1466 "http://carpoolfinder.appspot.com/"; -
I 01-07 11:50PM 57.140
User None
I 01-07 11:50PM 57.140
User else None

Thanks,

Brian in Atlanta
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Reminder: App Engine chat time, tonight 7-8PM PST

[David Symonds]

On Thu, Jan 8, 2009 at 6:54 PM, cz  wrote:

> I've been unable to make the chat times, is there a transcript
> archived somewhere?

They're posted to this list not long after each chat time. Check the
list archives for the previous ones.


Dave.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: cpu usage

[reyjexter]

try using a profiler.

in my case, i'm able to determine that web service & using stringio
use lots of cpu. you may need to remove some things like for example
not using both web service call and saving of information on a session
backed memcache.

On Jan 8, 1:49 pm, Ray Malone  wrote:
> Hi I get a message on my script that my average cpu is over 1300.
> However, when I review the details I can't see any cpu time used or
> any High CPU requests used.  Second of all my script is not all that
> complicated and only makes 8 datastore api calls per request.  I'm not
> sure where to look to tune it.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: AuthToken for Appengine

[Magnolia 8575]

Thanks Jeff

I have added this in the issues list

http://code.google.com/p/googleappengine/issues/detail?id=974


On Tue, Jan 6, 2009 at 3:57 PM, Jeff S  wrote:

>
> At the moment, there isn't really an option that avoids asking the
> user for their Google Account password. The Google Calendar API which
> you mentioned has a few options for web based authorization which
> would provide an auth token including OAuth and AuthSub
>
> http://code.google.com/apis/accounts/docs/OAuth.html
> http://code.google.com/apis/accounts/docs/AuthSub.html
>
> I think adding these auth mechanisms to App Engine would be a good
> feature request. Would you mind addind it to our issues list?
>
> http://code.google.com/p/googleappengine/issues
>
> Thank you,
>
> Jeff
>
> On Jan 4, 11:52 am, "magnolia8...@gmail.com" 
> wrote:
> > Is it possible in anyway for a web application to get an AuthToken for
> > appengine like any other service like google calendar etc?
> >
> > The idea is to mange end users appengine apps from a web application
> > using google authetication with out using user's google id and
> > password.
> >
> > If one can get hold the AuthToken   for appengine then the appcfg.py
> > can be tweaked to use that Authtoken to get the things done.
> >
> > Any information in this regard will be greatly appreciated.
> >
> > Many thanks
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: 1 application, multiple datastores

[hawkett]

Hi Joseph,

   I've previously made this point here -
http://groups.google.com/group/google-appengine/browse_thread/thread/6319dceae6ec73e7/4d4d464c25537bda?lnk=gst&q=#4d4d464c25537bda
- so Google - people really do recommend against GAE because your
rodmap is so impenetrable.  Joseph, you may recognise a little of Bob
in yourself :)

   I am a C -> C++ -> Java -> J2EE guy over many years, so python is
not my language of choice - I only started using python with app
engine.  I'm not here because of comfort with a technology I know, I'm
here because of  business requirements.  I thik you'll find as the
next few years progress that IT houses everywhere will be pushing back
against massive in house platforms.  Amaszon manages the hardware side
of that pushback, Google manages hardware *and* sofware. The killer
feature with GAE is the platform offering - with Amazon you have to
roll your own software platform, which is time consuming and
maintenance intensive.  Yes Amazon provides a hardware virtualisation
service that lets you do that, and I use it for a number if things.
But for a lean architecture, which leads to agility, which makes
business happy, you want as much as possible in the not-my-problem
basket - i.e. in GAE.

   If you've spent much time in the world of relational databases,
then BigTable is an absolute killer feature.  It collapses
Scalability, Performance, Fault Tolerance, and ORM into a simple
offering with sensible mechanisms for transaction management.  All of
this stuff ends up as not-my-problem.  With Amazon, most of it is my
problem to some degree.

   To a degree, I think the argument that if it isn't on the
(laughably undefined) roadmap then you have to go elsewhere is a bit
hasty for a beta software platform.  Granted, google is always beta,
but GAE is in early beta by their standard - you can't even pay for it
yet.  This is an ideal time to try and influence the architectural
direction of the platform.  That said, I think google should realise
that if they don't publish a proper roadmap (especially for must-have
platform features - e.g. SLA, asynch etc.), then people are going to
recommend dumping GAE in favour of something else.

   The only feature request (that I am proposing) here is customer
data partitioning.  It is not a massive big deal, especially
considering that this is already in place, you just have to deploy one
app per customer, which is a bit tedious.  It doesn't seem like a
sufficient reason to jump ship and roll my own everything at Amazon.
It seems to me like an opportunity to say 'Hey, could you make this
aspect, which already exists, a little easier?'

   It seems to be like a ludicrously obvious step for Google to set up
an application market place, like the iphone app store, and open that
market to all of their google apps customers.  It is a license for
Google to print money - more so than the iphone, because we are
talking business customers, not joe-public.  If it isn't on their
(secret) roadmap, then my name is Fred.  This is the direction I have
suggested for the customer based data partitioning in
http://code.google.com/p/googleappengine/issues/detail?can=2&q=945

   Finally, it is obvious that Andy and I are coming at this from
different (strongly held) positions.  That is usually a good thing, as
long as both parties are honest in their attempts to understand the
other.

Colin
(not Fred)

On Jan 7, 2:17 pm, "bowman.jos...@gmail.com" 
wrote:
> Guys, I think you need to take a step back and look at this from a
> higher level.
>
> Appengine supplies you with an instance in a cloud that includes a
> customized python set, and a BigTable backend. It does not support
> multiple BigTable backends and design wise I doubt it ever will. There
> comes a time when you have to look at your application and determine
> what is the right environment for it to be built in to meet your
> business requirements. In this case, it does not sound like appengine
> in and of itself is going to meet those requirements.
>
> Generally business requirements dictate the speed at which your
> product must become available for use. Google has a published roadmap
> for appengine, and support for multiple BigTable instances per
> application is not on it, and they have not even implied it's
> something they have any interest in implementing.
>
> So, at this point, I'd suggest you look at other alternatives in order
> to meet your business requirements.
>
>  - Separate it by table within BigTable as has been suggested.
>  - Pull everything back inhouse and build server(s) capable of
> supporting your application with the requirements you have. Such as
> with MySQL running a different database for each of your users.
>  - Examine other cloud db storage options to see if they can meet your
> requirements, such as the offering from Amazon. Though, while you
> could use appengine combined with that solution, I would question how
> quick you'd hit the urlfetch quota limits.
>  - Examine all

[google-appengine] Re: http://carpoolfinder.appspot.com/ won't work now and svn update fails

[Rodrigo Moraes]

On Thu, Jan 8, 2009 at 6:05 AM, thebrianschott wrote:
> Using logging.info() I have determined that the appspot is always
> return None for "user=users.get_current_user()" and that is why the
> app is working differently at appspot that locally. I have tried to
> make this a single "transaction" to allow the value of user to be
> determined in a timely way, but either I am doing it wrong, or it does
> not work. Please advise.

you don't need a transaction for that. actually, it seems that you got
completely wrong what transactions are for.

what you need is to add a link so that users can login and then, after
login, you will be able to get the current user inside your app.

in your app:

template_values = {}
template_values['login_url'] = users.get_login_url()

in your template:

login here, buddy!

that's it, simple as that.

-- rodrigo

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Add Domain (New Domain Depoloyment)

[PatHaugen]

I searched groups and help, but still couldn't figure something out
that may be misleading...

http://appengine.google.com/deployment/newdomain?app_id=XXX

So it informs me:
"Note: You must sign up for Google Apps to register this domain or
prove that you already own it."

1. Sign up for Google Apps to register this domain
2. Prove that you already own it

I would like to use option #2, to merely prove I own the domain.

Not sure how or where you do this, but my domain is already setup. I'm
sure many others as well, got your Windows PDC, logins, intranet,
everything said and done, Exchange in place, everyone using the
domain... but you got some worthless webhost in BFE displaying your
little external webpage.

Okay in other terms: No need at all for Google Apps, just want to have
the public side of the domain for the website point to a shiny new
Google App Engine application.

How do you do this? I'd assume the 'proof' would be in pointing
current domain registrar www to Google's server for your app or
something like that?

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] email with Django and appenginepatch fails (1,'Uknown Error')

[Dave]

I am having a problem that I hope has an easy answer although I've
looked high and low for an answer. I am using app engine with
appenginepatch, including Django 1.x. When I attempt to send an email
I get an 'herror" Exception with Exception Value "(1, uknown host)".
>From the error output, line 111 is what seems to be raising the
Exception is below:


 104. possibly existing aliases. In case no FQDN is available,
hostname
 105. from gethostname() is returned.
 106. """
 107. name = name.strip()
 108. if not name or name == '0.0.0.0':
 109. name = gethostname()
 110. try:

 111. hostname, aliases, ipaddrs = gethostbyaddr(name) ...

 112. except error:
 113. pass
 114. else:
 115. aliases.insert(0, hostname)
 116. for name in aliases:
 117. if '.' in name:

▼ Local vars
VariableValue
name
'imac.local'

The syntax for my mail call is:

profile is an object in my models where I have defined:

def email_user(self, subject, message, from_email=None):
"Sends an e-mail to this User."
from django.core.mail import send_mail
send_mail(subject, message, from_email, [self.email])

and then in my view I call it:

profile.email_user('Congrats','you are now live',
'd...@blindrebel.com'django )

I have also tried to change this over to use app engines mail api,
using from google.appengine.api import mail and then modifying the
code appropriately but end up with the same Exception.

Any help is GREATLY appreciated!

Thanks,

Dave

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Python web service on Google app engine

[Puru]

Hi all
I have installed google app engine sdk 1.1.7 on windows xp with python
2.5.4. I am a fresh starter on python and what i need to do is to
create a web service in python that communicates with google
datastore. Though i am reading the "Getting Started" tutorial at
http://code.google.com/appengine/docs/gettingstarted/
but does not get any idea of how a web service will be created? What
steps I need to develop it locally on my computer [inside development
environment]?
Can anybody elaborate on this?
Thanks in advance

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] OSX Leopard "NotImplementedError: Unable to find the Python PIL library."

[PatHaugen]

Installed the Google App Engine SDK: 1.1.7 11/21/08
http://code.google.com/appengine/downloads.html

Worked perfect.

Wanted to go into images, and test locally:
http://code.google.com/appengine/docs/images/installingPIL.html#mac

Installed this:
http://pythonmac.org/packages/py25-fat/dmg/PIL-1.1.6-py2.5-macosx10.4-2007-05-18.dmg

Error on install: requires system python 2.5

Installed what was requested (although python is bundled with Leopard
and things were already working):
http://pythonmac.org/packages/py25-fat/dmg/python-2.5-macosx.dmg

Installed Python, then ran the PIL install, worked this time.

Things seemed good, but trying to do localhost image manipulation
gives:
"NotImplementedError: Unable to find the Python PIL library.  Please
view the SDK documentation for details about installing PIL on your
system."

Odd, since I followed the instructions...

Checked Google App Engine Preferences and saw this gem:
"Python Path: __
The path to the Python installation. If left blank, the Launcer will
attempt to find it. If you have installed Python Imaging Library (PIL)
from pythonmac.org, try setting this to /usr/local/bin/python2.5"

Searched HDD, found this path was there, so set it to:
/usr/local/bin/python2.5

Same error on test.

Anyone get this to cleanly work as documented by Google?

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: App Engine Gadget doesn't authenticate to Google Apps account

[Marzia Niccolai]

Hi Ryan,

The behavior you've observed is by design.  Google Accounts users must
explicitly sign in to any application wishing to have access to their
email.  Google Apps does not have the same login policy.

-Marzia

On Tue, Jan 6, 2009 at 7:17 PM, ryan  wrote:

>
> I've written a basic App Engine app that authenticates with my Google
> Apps account fine.  That is, if I'm logged in to my Google Apps mail,
> I can jump over to my App Engine app and I'll already be logged in.
> The opposite also works fine.
>
> However, I've now written a small Google Gadget that offers a sliver
> of functionality of the main app, but when I place that on the start
> page for the Google Apps account, it doesn't respect the login status
> of the user.  That is, the user can be logged in on the start page,
> but the Gadget doesn't know.  Similarly I can log in to the App Engine
> app, but then go to the start page and click sign out.  The rest of
> the page shows the logged out status, but the gadget remains logged
> in.
>
> It's almost like my gadget is using a separate auth mechanism than the
> main app pages, but they aren't.  They're both using the Users API
> (http://code.google.com/appengine/docs/users/).
>
> Does it have something to do with the fact that my gadget is using URL
> content-type instead of HTML?
>
> Thanks,
> Ryan
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: OSX Leopard "NotImplementedError: Unable to find the Python PIL library."

[Rodrigo Moraes]

On Thu, Jan 8, 2009 at 7:11 AM, PatHaugen wrote:
> Anyone get this to cleanly work as documented by Google?

not me. this is how i got it working:

http://appengine-cookbook.appspot.com/recipe/install-pil-in-mac-os-x

-- rodrigo

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: 500 server error despite proper testing on dev_appserver.py and upload

[Marzia Niccolai]

Hi,

Usually if you are getting a 500 server error with no logs like you have
described, this means there is some typo in the app.yaml file that was
uploaded.  We've seen this issue come up because of improper spacing, line
endings or a typo in the script name.

-Marzia

On Tue, Jan 6, 2009 at 6:18 PM, pgctesting001 wrote:

>
> On Jan 3, 7:34 am, pgctesting001  wrote:
> > I used the google app engin sdk to develop a simple app for testing on
> > upload to my standard test site. the sdk and gdata installed properly,
> > the tests ran ok, my html file showed in the development server on my
> > platfor, I created the app engine with cell phone call back procedure
> > and uploaded the app to my test site and it is active and showing
> > activity from the control panel. when I try to invoke it, I get a 500
> > Server Error from google, but nothing in the app's error logs.
> >
> > the app:
> >
> > print "Content-type: text/html\n\n";
> > print ''
> > print 'Hello, world!'
> > print ''
> > print 'This is a test of the Google loader'
> >
> > print ''
> > print 'Hello, Thomas. How do you like me now?'
> >
> > print ''
> > print 'Hello, Tony. See what your help has produced?'
> >
> > print ''
> > print 'From San Francisco, January 2, 2009.'
> >
> > the yaml file:
> >
> > application: pgctesting0102
> > version: 1
> > runtime: python
> > api_version: 1
> >
> > handlers:
> > - url: /.*
> >   script: pgctesting0102.py
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: URL Fetch API --- Does Google log IP addresses to prevent abuses?

[Marzia Niccolai]

Hi,

Logs are kept around for awhile in the admin console, and you can access
different app version logs by changing the version number in your
application's dashboard (the number on in the left hand corner).  You can
download your logs by using appcfg.py:

http://code.google.com/appengine/docs/appcfgpy.html

-Marzia

On Wed, Jan 7, 2009 at 9:32 AM, KillIEbrowser wrote:

>
> ...??
>
> The other side of the question is:
> Can I don't make any log, delegating the checks to the GAE
> administrators Team?
>
> What is the responsability of a free developer about its free
> application?
>
> (Thanks for the patience)
>
>
> On Jan 6, 2:15 am, KillIEbrowser  wrote:
> > Hi,
> > the log will be deleted when I will update a new version of my
> > application.
> > - Can I see the old logs?
> > - Can I export a log?
> >
> > I'm thinking to make a Table log...
> >
> > On Jan 5, 7:38 pm, Marzia Niccolai  wrote:
> >
> > > Hi,
> >
> > > The ip address of requests are also available in the logs.
> >
> > > -Marzia
> >
> > > On Sat, Jan 3, 2009 at 1:06 PM, Barry Hunter <
> barrybhun...@googlemail.com>wrote:
> >
> > > > Do it yourself:
> > > >http://code.google.com/appengine/docs/python/logging.html
> >
> > > > 2009/1/3 KillIEbrowser :
> >
> > > > > Now I'm building a new free web App using HTTPS:
> >
> > > > > "Private URL Fetching"
> >
> > > > > and I think is useful to see a "URL Fetch Log" into the App Engine
> > > > > Admin.
> > > > > Can you make this feature, please?
> >
> > > > > On Dec 30 2008, 9:32 pm, "Barry Hunter" <
> barrybhun...@googlemail.com>
> > > > > wrote:
> > > > >> IP addresses of what?
> >
> > > > >> 2008/12/30 KillIEbrowser :
> >
> > > > >> > Hi All,
> > > > >> > Does Google log IP addresses to prevent abuses?
> >
> > > > >> --
> > > > >> Barry
> >
> > > > >> -www.nearby.org.uk-www.geograph.org.uk-
> >
> > > > --
> > > > Barry
> >
> > > > -www.nearby.org.uk-www.geograph.org.uk-
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Yahoo Domain redirect

[Marzia Niccolai]

Hi,

Currently the 302 redirect to 'www' is the only method available for using
the naked domain.

-Marzia

On Wed, Jan 7, 2009 at 1:11 PM, Pankaj Vishwani wrote:

>
> I already tried all that but yahoo doesn't give any such thing for
> redirecting...
> can google give some other way to do it like IPs???
>
>
> On Jan 6, 4:37 pm, Marzia Niccolai  wrote:
> > Hi,
> >
> > A Record configurations for naked domains will no longer work with App
> > Engine.  Please see:
> http://code.google.com/appengine/kb/commontasks.html#naked_domain
> >
> > You will need to configure a 302 redirect from the naked domain with your
> > domain registrar to the 'www' domain.
> >
> > -Marzia
> >
> > On Tue, Jan 6, 2009 at 1:45 PM, Pankaj Vishwani <
> pankajvishw...@gmail.com>wrote:
> >
> >
> >
> > > Hi,
> >
> > > I saw a lot of posts about redirecting from domain.com
> towww.domain.com
> > > but unfortunately none of them were helpful in my case.
> > > So here's what is happening:
> >
> > > I bought a domain from yahoo and there I added
> > > * A Record 'hullabu.com' to destination '216.239.38.21' for
> > > hullabu.com rediecting towww.hullabu.com
> > > * CNAME Record '*.hullabu.com' to destination 'ghs.google.com' for
> > > rediectingwww.hullabu.comto app engine.
> >
> > > Now,www.hullabu.comworks fine but not hullabu.com.
> >
> > > Tried Yahoo support and they say both hullabu.com andwww.hullabu.com
> > > go to same IP, so it's google app engine's problem.
> >
> > > Any help is greatly appreciated.
> >
> > > Regards,
> > > Pankaj Vishwani
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Add Domain (New Domain Depoloyment)

[Marzia Niccolai]

Hi,

In order to serve an App Engine app off your domain it must be a Google Apps
domain.  (2) alone is not sufficient.

-Marzia

On Thu, Jan 8, 2009 at 7:59 AM, PatHaugen  wrote:

>
> I searched groups and help, but still couldn't figure something out
> that may be misleading...
>
> http://appengine.google.com/deployment/newdomain?app_id=XXX
>
> So it informs me:
> "Note: You must sign up for Google Apps to register this domain or
> prove that you already own it."
>
> 1. Sign up for Google Apps to register this domain
> 2. Prove that you already own it
>
> I would like to use option #2, to merely prove I own the domain.
>
> Not sure how or where you do this, but my domain is already setup. I'm
> sure many others as well, got your Windows PDC, logins, intranet,
> everything said and done, Exchange in place, everyone using the
> domain... but you got some worthless webhost in BFE displaying your
> little external webpage.
>
> Okay in other terms: No need at all for Google Apps, just want to have
> the public side of the domain for the website point to a shiny new
> Google App Engine application.
>
> How do you do this? I'd assume the 'proof' would be in pointing
> current domain registrar www to Google's server for your app or
> something like that?
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Zip Importer High CPU every request

[Guido van Rossum]

One question -- are you defining a main() function? If not, your
entire app may be reloaded each time. Try putting logging statements
(or prints to sys.stderr) at the top-level of your module, and check
in the logs for these -- they should only be logged for the first
request. If they are logged for each request, then somehow your main
is being reloaded each time.

On Jan 7, 4:39 pm, Anthony  wrote:
> It looks like the info in the logs could be down to the whole page
> processing, but very simple pages (no db) consistently taking 500ms+
> Ill give it a try without the zipimport.
>
> I'm just using the loader cache from the cookbook (removed the
> bytecode cache), jinja works great, although I did need to force
> caching off in dev or the internal caches do not update after template
> changes...
>
> import config
> import os
> import sys
> from google.appengine.api import memcache
>
> sys.path.insert(0, 'lib/jinja2.zip')
> from jinja2 import Environment, FunctionLoader , TemplateNotFound
>
> def jinja2_template_loader(templatename):
>         templatepath = os.path.abspath(os.curdir+'/templates/'+templatename)
>         template = memcache.get(templatepath+config.VERSION)
>         if template is None:
>                 try:
>                         template = file(templatepath).read()
>                         if config.LIVE:
>                                 
> memcache.set(templatepath+config.VERSION,template)
>                 except:
>                         template = None
>         return template
>
> if config.LIVE:
>         jinja2_environment = Environment(loader = FunctionLoader
> (jinja2_template_loader),cache_size=50)
> else:
>         jinja2_environment = Environment(loader = FunctionLoader
> (jinja2_template_loader),cache_size=0)
>
> def render(template_name,context):
>         template = jinja2_environment.get_template(template_name)
>         context["stats"]="LIVE:" +str(config.LIVE) + " DEBUG:" +str
> (config.DEBUG)
>         return template.render(context)
>
> On Jan 7, 7:39 pm, "Rodrigo Moraes"  wrote:
>
> > On Wed, Jan 7, 2009 at 2:16 PM, Anthony wrote:
> > > Is it normal for the zipimporter to take 500-1000ms every request -
> > > should it be caching internally?
>
> > > I'm importing like so:
>
> > > # templates.py
> > > sys.path.insert(0, 'lib/jinja2.zip')
> > > from jinja2 import Environment, FunctionLoader , TemplateNotFound ,
> > > MemcachedBytecodeCache
>
> > > Will it still be caching if im not importing the zip dircetly in the
> > > main.py?
>
> > Not an answer to your question (which I'd like to know :), just a
> > warning: you won't be able to use MemcachedBytecodeCache from Jinja2,
> > because bytecode usage is not allowed in App Engine.
>
> > The example using MemcachedBytecodeCache in the App Engine Cookbook
> > only works in the dev server, not in production. :-/
>
> > Apart from this, Jinja2 implementation is very smooth. :)
>
> > -- rodrigo
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Announcing: System Status Dashboard, Quota Details Page, and a Preview of Billing

[mclovin]

Am i missing something somewhere? I cant seem to find any pricing for
commericial applications even though google advertises app engine as a
solution for commericial deployment.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Trouble with NeedIndexError when calling count()

[Ryan W]

All of a sudden this morning, I began getting this error in my app,
when doing a search inside of an internal process (searching for
duplicates) and then taking a count() of the results.  It was working
fine before.  I have this same query on a search page as well, and
it's still fine.  The search page does not call count() though.
Perhaps it has something to do with the number of results being
returned, I'll try to instrument to get a apples to apples search
string comparison.   Any ideas on this though?

This is the stack trace:

File "/base/data/home/apps/...", line 196, in approvePost
if qPosts.count() > 0:
  File "/base/python_lib/versions/1/google/appengine/ext/db/
__init__.py", line 1353, in count
return self._get_query().Count(limit=limit)
  File "/base/python_lib/versions/1/google/appengine/api/
datastore.py", line 960, in Count
raise _ToDatastoreError(err)
  File "/base/python_lib/versions/1/google/appengine/api/
datastore.py", line 1637, in _ToDatastoreError
raise errors[err.application_error](err.error_detail)
NeedIndexError: The built-in indices are not efficient enough for this
query and your data. Please add a composite index for this query.


This is the offending code:

qPosts = datamodel.Post.all().search(search_string).filter("approved =
", True)
if qPosts.count() > 0:


This is the model:

class Post(search.SearchableModel):
user = db.ReferenceProperty(User, collection_name='posts')
status_id = db.IntegerProperty(required=True)
entry = db.TextProperty(required=True)
entry_title = db.StringProperty(required=True,multiline=True)
entry_link = db.LinkProperty(required=True)
approved = db.BooleanProperty(required=True, default=False)
reject_reason = db.StringListProperty(default=None)
retweet_status_id = db.IntegerProperty(required=False)
in_reply_to_status_id = db.IntegerProperty(required=False)
short_url = db.LinkProperty(required=False)
hash = db.StringProperty(required=False)
source = db.StringProperty(required=False)
view_count = db.IntegerProperty(required=True, default=0)
reply_count = db.IntegerProperty(required=True, default=0)
like_count = db.IntegerProperty(required=True, default=0)
published = db.DateTimeProperty()
created = db.DateTimeProperty(auto_now_add=True)


I tried adding this index, but was still getting the error:

- kind: Post
  properties:
  - name: __searchable_text_index
  - name: approved


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Autogenerated indexes being cached?

[Ryan W]

I'm working to tune my indexes.  I noticed a lot of unused indexes, so
I removed everything from my index.yaml and then added back one by one
while running the application locally with the "dev_appserver.py --
require_indexes" command.  Everything was working good, so I went
ahead and ran the index vacuum for the production site.  All was good
there too.

But, now when I run the app locally with the GoogleAppEngineLauncher
app, not using the --require_indexes flag, it keeps adding back some
autogenerated indexes that aren't used.  How do I know they aren't
used?  Because the app will run without them with the --
require_indexes flag, and because the counts of the "has been used x
times" are not being incremented.

I've tried restarting the app, removing the app altogether from
GoogleAppEngineLauncher, then adding back.  It still keeps adding
these indexes though.  Is there a cache somewhere that can reset?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: http://carpoolfinder.appspot.com/ won't work now and svn update fails

[thebrianschott]

Rodrigo,

Thank you very much. That does look simple.

If found this link which describes using 
http://code.google.com/appengine/docs/thedevwebserver.html
but it describes only what should be done on the development system. I
cannot find a page which describes your approach on the appspot
system.

I don't understand how I combine your suggestion with the template
information and  I already have in my "template" (which is
the .html file that creates the google map for the user, right?). Can
you say more please?

Or are you saying that I must create a separate .html page with the
information you are describing?

Oh, and do you mean literally "'login_url'" and "'login_url'" and if
not, what?

To be more specific, I already have the following in my "app".

  path = os.path.join(os.path.dirname(__file__),
'add_place.html')
  template_values = dict(place=place.place, user=place.user)
  self.response.out.write(template.render(path,
template_values))

And in my "template" I have the following code (I have snipped some).

  var place_name = '{{place|addslashes}}';
  var user_name = '{{user}}';


  var iwform = 'Enter your group\'s information:'
+ ''
+ 'Group Name or Initials: '
+ ''
+ '' + place_name + ''
+ ''
+ 'People'
+ '  '
+ ''
[snip]
+ '<\/form>';

  function process(form) {
// == populate map data
map.setCenter(lastcenter, map.getZoom());
var people = form.people.value;
var contact = form.contact.value;

[snip]

var url = "/add_place?place="+place_name
+"¢er_lat="+center_lat+"¢er_lng="+center_lng+"&zoom="+zoom
+"&people="+people+"&contact="+contact+"&moreinfo="+moreinfo
+"&marker_lat="+marker_lat+"&marker_lng="+marker_lng;

// = send the data to the server
GDownloadUrl(url, function(doc) {
});

Thank you, again,


On Jan 8, 9:17 am, "Rodrigo Moraes"  wrote:
> you don't need a transaction for that. actually, it seems that you got
> completely wrong what transactions are for.
>
> what you need is to add a link so that users can login and then, after
> login, you will be able to get the current user inside your app.
>
> in your app:
>
>     template_values = {}
>     template_values['login_url'] = users.get_login_url()
>
> in your template:
>
>     login here, buddy!
>
> that's it, simple as that.
>
> -- rodrigo

Brian in Atlanta
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Is the Datastore just too slow for 100+ returned records?

[Tzakie]

I've spend alot of time profiling and optimizing my app.
I've watched all the google io videos and done a ton of reading...

But there is one query I can't optimize away:

SELECT * FROM USER_INVENTORY WHERE TUSER_ID=637 AND CATEGORY_ID=2752
ORDER BY ITEM_NUMBER ASC, ITEM_NAME, SUFFIX_NAME

Yes there is a custom index for it

It looks like a simple enough query, but it's taking the datastore
2.170703-1.863699 seconds to load and get about 500 records. The app
engine translates 2 real seconds into 20 'processor seconds' and of
course various quotas all go crazy. It's a simple list of a web stores
items in a category any more than 50 or so and there is a noticeable
delay.

Why it it taking the datastore so long to return something like this?

Entities seem to take forever to retrieve.

Questions:
I assume it's the number of returned entities here that causes the
delay? E.G. The big penalty is for returning an entity not the query.

Once you have a custom index the trailing ORDER BY items ITEM_NAME,
SUFFIX_NAME don't matter much for speed since the key just gets a
little longer.

Do the number of properties on the entity matter much for query time?

Is there some way to return JUST the properties I want? Whould that
help for speed?

Is there some way to get the just keys back from a query and memcache
the objects?

Though the queries are limited at 1,000 which is reasonable. The data
store is so slow you can't get more than about 50 records on the
screen at once. It seems like there is a fundamental limit, E.G. If
you want a list of 500 items app engine just can't do it not matter
what the technique.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Request/Response objects and WebOb

[Alex Popescu]

Is there any particular reason for having the Request class extend the
WebOb Request, while the Response is just extending object? As far as
I checked the Request class is the single dependency on WebOb
framework and I find it quite curious.

./alex
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Running multiple Dev Web Servers on same machine

[Alex Popescu]

I am trying to run 2 dev web servers on my local machine in order to
test some fetch functionality. Anyways, even if both apps are
configured to use different ports and they are running in 2 different
Python processes, this doesn't seem to work. Is this a known
limitation? Is there any workaround?

tia,

./alex
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Running multiple Dev Web Servers on same machine

[Bill]

Alex,

I've had no problem running 2 dev web servers on my MacBook Pro
(Leopard OS X).  What OS are you using and are you launching
dev_appserver.py with the port options?  You have to give more
detailed information.

Best,
Bill

On Jan 8, 2:39 pm, Alex Popescu 
wrote:
> I am trying to run 2 dev web servers on my local machine in order to
> test some fetch functionality. Anyways, even if both apps are
> configured to use different ports and they are running in 2 different
> Python processes, this doesn't seem to work. Is this a known
> limitation? Is there any workaround?
>
> tia,
>
> ./alex
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Running multiple Dev Web Servers on same machine

[boson]

These dev_appserver.py params seem important for running multiple
instances:

--address=ADDRESS, -a ADDRESS
Address to which this server should bind. (Default
localhost).

--port=PORT, -p PORT
Port for the server to run on. (Default 8080)

--datastore_path=PATH
Path to use for storing Datastore file stub data.
(Default /var/folders/PP/PP05HWgpEyadqSwuIRPRSU+++TI/-Tmp-/
dev_appserver.datastore)

--history_path=PATH
Path to use for storing Datastore history.
(Default /var/folders/PP/PP05HWgpEyadqSwuIRPRSU+++TI/-Tmp-/
dev_appserver.datastore.history)

I would expect multiple processes might butt heads when accessing the
same Datastore.  And of course you need to put the servers on
different address/port combinations.


On Jan 8, 11:39 am, Alex Popescu 
wrote:
> I am trying to run 2 dev web servers on my local machine in order to
> test some fetch functionality. Anyways, even if both apps are
> configured to use different ports and they are running in 2 different
> Python processes, this doesn't seem to work. Is this a known
> limitation? Is there any workaround?
>
> tia,
>
> ./alex
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Running multiple Dev Web Servers on same machine

[bowman.jos...@gmail.com]

When writing the Cron application for gaeutilities I ran into the need
to ran two instances, as the dev_appserver can't make a request to
itself. All I did was run the second instance on a different port,
didn't have to mess with the address. Mind you, it didn't make a lot
of requests to the datastore for the purposes I was using it for, but
I had no problems with both instances using the same datastore.

On Jan 8, 3:12 pm, boson  wrote:
> These dev_appserver.py params seem important for running multiple
> instances:
>
> --address=ADDRESS, -a ADDRESS
> Address to which this server should bind. (Default
> localhost).
>
> --port=PORT, -p PORT
> Port for the server to run on. (Default 8080)
>
> --datastore_path=PATH
> Path to use for storing Datastore file stub data.
> (Default /var/folders/PP/PP05HWgpEyadqSwuIRPRSU+++TI/-Tmp-/
> dev_appserver.datastore)
>
> --history_path=PATH
> Path to use for storing Datastore history.
> (Default /var/folders/PP/PP05HWgpEyadqSwuIRPRSU+++TI/-Tmp-/
> dev_appserver.datastore.history)
>
> I would expect multiple processes might butt heads when accessing the
> same Datastore.  And of course you need to put the servers on
> different address/port combinations.
>
> On Jan 8, 11:39 am, Alex Popescu 
> wrote:
>
> > I am trying to run 2 dev web servers on my local machine in order to
> > test some fetch functionality. Anyways, even if both apps are
> > configured to use different ports and they are running in 2 different
> > Python processes, this doesn't seem to work. Is this a known
> > limitation? Is there any workaround?
>
> > tia,
>
> > ./alex
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Is the Datastore just too slow for 100+ returned records?

[boson]

Tzakie,

A few things:

1. What is the shape of your data?  i.e. how big are your entities and
what do they contain?

2. How are you using entity groups?  Is all your data in a single
entity group (shared ancestor)?

3. Also try to experiment with permutations of your index and query to
try to isolate the bottleneck.  i.e. if you just fetch(500) on "SELECT
* FROM USER_INVENTORY", does it still take as long?  (and if so, then
rephrase the question to eliminate the index aspects)



On Jan 8, 10:48 am, Tzakie  wrote:
> I've spend alot of time profiling and optimizing my app.
> I've watched all the google io videos and done a ton of reading...
>
> But there is one query I can't optimize away:
>
> SELECT * FROM USER_INVENTORY WHERE TUSER_ID=637 AND CATEGORY_ID=2752
> ORDER BY ITEM_NUMBER ASC, ITEM_NAME, SUFFIX_NAME
>
> Yes there is a custom index for it
>
> It looks like a simple enough query, but it's taking the datastore
> 2.170703-1.863699 seconds to load and get about 500 records. The app
> engine translates 2 real seconds into 20 'processor seconds' and of
> course various quotas all go crazy. It's a simple list of a web stores
> items in a category any more than 50 or so and there is a noticeable
> delay.
>
> Why it it taking the datastore so long to return something like this?
>
> Entities seem to take forever to retrieve.
>
> Questions:
> I assume it's the number of returned entities here that causes the
> delay? E.G. The big penalty is for returning an entity not the query.
>
> Once you have a custom index the trailing ORDER BY items ITEM_NAME,
> SUFFIX_NAME don't matter much for speed since the key just gets a
> little longer.
>
> Do the number of properties on the entity matter much for query time?
>
> Is there some way to return JUST the properties I want? Whould that
> help for speed?
>
> Is there some way to get the just keys back from a query and memcache
> the objects?
>
> Though the queries are limited at 1,000 which is reasonable. The data
> store is so slow you can't get more than about 50 records on the
> screen at once. It seems like there is a fundamental limit, E.G. If
> you want a list of 500 items app engine just can't do it not matter
> what the technique.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] How to determine top 10 scoring users from last 24 hours?

[Amir Michail]

Hi,

How can this be done efficiently with the current version of the GAE?

I suppose you could keep track of all events affecting scoring for the
last 24 hours.  Whenever you handle a request -- even one that does
not affect scoring -- you could prune out events older than 24 hours
and add a new one if the player has received more points as a result
of the current request.

As you take out old events and add in new ones, you also update the
top 10 ranking for the last 24 hours accordingly.

The problem here is that taking out all the old events can be quite
expensive.  As an extreme example, if you don't get any requests for
over 24 hours, then on the next request, you will need to clear out at
a day's worth of scoring events.

Amir


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: http://carpoolfinder.appspot.com/ won't work now and svn update fails

[boson]

Brian,

It's hard to tell what you're actually stuck on, because you're
throwing out a lot of information about SVN, transactions, logging,
production vs. development, etc.

I suggest you start a new thread with a simple and distilled version
of your question, and only include short and relevant snippets of
code.  You will likely receive more helpful attention that way.

On Jan 8, 10:37 am, thebrianschott  wrote:
> Rodrigo,
>
> Thank you very much. That does look simple.
>
> If found this link which describes 
> usinghttp://code.google.com/appengine/docs/thedevwebserver.html
> but it describes only what should be done on the development system. I
> cannot find a page which describes your approach on the appspot
> system.
>
> I don't understand how I combine your suggestion with the template
> information and  I already have in my "template" (which is
> the .html file that creates the google map for the user, right?). Can
> you say more please?
>
> Or are you saying that I must create a separate .html page with the
> information you are describing?
>
> Oh, and do you mean literally "'login_url'" and "'login_url'" and if
> not, what?
>
> To be more specific, I already have the following in my "app".
>
>           path = os.path.join(os.path.dirname(__file__),
> 'add_place.html')
>           template_values = dict(place=place.place, user=place.user)
>           self.response.out.write(template.render(path,
> template_values))
>
> And in my "template" I have the following code (I have snipped some).
>
>       var place_name = '{{place|addslashes}}';
>       var user_name = '{{user}}';
>
>       var iwform = 'Enter your group\'s information:'
>         + ''
>         + 'Group Name or Initials: '
>         + ''
>         + '' + place_name + ''
>         + ''
>         + 'People'
>         + '   size="31" maxlength="31"/>'
>         + ''
> [snip]
>         + '<\/form>';
>
>       function process(form) {
>         // == populate map data
>         map.setCenter(lastcenter, map.getZoom());
>         var people = form.people.value;
>         var contact = form.contact.value;
>
> [snip]
>
>         var url = "/add_place?place="+place_name
> +"¢er_lat="+center_lat+"¢er_lng="+center_lng+"&zoom="+zoom
> +"&people="+people+"&contact="+contact+"&moreinfo="+moreinfo
> +"&marker_lat="+marker_lat+"&marker_lng="+marker_lng;
>
>         // = send the data to the server
>         GDownloadUrl(url, function(doc) {
>             });
>
> Thank you, again,
>
> On Jan 8, 9:17 am, "Rodrigo Moraes"  wrote:
>
>
>
> > you don't need a transaction for that. actually, it seems that you got
> > completely wrong what transactions are for.
>
> > what you need is to add a link so that users can login and then, after
> > login, you will be able to get the current user inside your app.
>
> > in your app:
>
> >     template_values = {}
> >     template_values['login_url'] = users.get_login_url()
>
> > in your template:
>
> >     login here, buddy!
>
> > that's it, simple as that.
>
> > -- rodrigo
>
> Brian in Atlanta
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Is the Datastore just too slow for 100+ returned records?

[Tzakie]

> 1. What is the shape of your data?  
> i.e. how big are your entities and what do they contain?

class USER_INVENTORY(db.Model):
TUSER_ID = db.IntegerProperty()
ITEM_TYPE = db.StringProperty()
ITEM_ID = db.IntegerProperty()
SUFFIX_ID = db.IntegerProperty()
HAVE_INVENTORY = db.IntegerProperty()
SELL_INVENTORY = db.IntegerProperty()
SELL_RESERVED = db.IntegerProperty()
SELL_AVAILABLE = db.IntegerProperty()
SELL_PRICE = db.FloatProperty()
SELL_IDEAL_INVENTORY = db.IntegerProperty()
BUY_INVENTORY = db.IntegerProperty()
BUY_RESERVED = db.IntegerProperty()
BUY_AVAILABLE = db.IntegerProperty()
BUY_PRICE = db.FloatProperty()
SHIPPING_VALUE = db.FloatProperty()
SELL_RANKING = db.IntegerProperty()
VERSION = db.IntegerProperty()
CATEGORY_ID = db.IntegerProperty() #
L0_CATEGORY_ID = db.IntegerProperty()
L1_CATEGORY_ID = db.IntegerProperty()
L2_CATEGORY_ID = db.IntegerProperty()
L3_CATEGORY_ID = db.IntegerProperty()
L4_CATEGORY_ID = db.IntegerProperty()
L5_CATEGORY_ID = db.IntegerProperty()
L6_CATEGORY_ID = db.IntegerProperty()
L7_CATEGORY_ID = db.IntegerProperty()
L8_CATEGORY_ID = db.IntegerProperty()
L9_CATEGORY_ID = db.IntegerProperty()
INVENTORY_TYPE = db.StringProperty() # G-Global C-Custom A-Auction
P-Pointer
INVENTORY_NAME = db.StringProperty()
ITEM_NUMBER = db.StringProperty()
ITEM_NAME = db.StringProperty()
SUFFIX_NAME = db.StringProperty()
ATTR_Set = db.StringProperty()
ATTR_Game = db.StringProperty()
ATTR_Rarity = db.StringProperty()
ATTR_Language = db.StringProperty()
ATTR_Style = db.StringProperty()
ATTR_Condition = db.StringProperty()
ATTR_Edition = db.StringProperty()

This is really the minimum you need to describe an inventory object.
If the number of properties is
the problem can I some how just return a subset of the properties I
need in the query?

> 2. How are you using entity groups?  Is all your data in a single
> entity group (shared ancestor)?
Not at all.

> 3. Also try to experiment with permutations of your index and query to
> try to isolate the bottleneck.  i.e. if you just fetch(500) on "SELECT
> * FROM USER_INVENTORY", does it still take as long?  (and if so, then
> rephrase the question to eliminate the index aspects)

Changing to SELECT * FROM USER_INVENTORY doesn't help
it's the loop where the entities are being fetched. Is there some way
to
not return all the properties? Can I just get the keys that match the
query?

I remember some one in the goolge io videos saying not to sort or
filter
in memory. So I was trying to stick with making indexes do the work.
After looking at this setup I think sorting in memory with memcached
objects might be the only way to get anything done. That's rewriting
a database in python and using bigtable as nothing but a file store.

Also I can't see logs anymore:
HTTP response was too large: 1808121. The limit is: 1048576.
Clear logs please.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: How to determine top 10 scoring users from last 24 hours?

[boson]

I've been planning something like this for my app, and so far this is
the best I've come up with:

First of all, you don't want to be querying to sum up scores for
individual users, so you need to keep a tally in one place.  This is
BigTable business as usual.

In each relevant entity where you are tallying, track a day and a
score for that day:
  user = "Kobe"
  weight = "180"
  ...
  lifetime_score = 14382  # total score over eternity
  daily_day = "20090108"  # the last day a score was recorded
  daily_score = 153   # the score for daily_day

When you update the score, do a transaction as follows:
  if today's date matches entity's daily_day:
lifetime_score += new_score
daily_score += new_score
  else:
lifetime_score += new_score
daily_day = today
daily_score = new score

Then your final query is:
  filter("day =", today_as("MMDD")).order("-score")

That requires a fairly simple (but custom) index.

The query ignores people who haven't scored today (the first filter
drops them), and sorts in descending on score.

And of course you should memcache the query results.

Hope this helps!

On Jan 8, 12:33 pm, Amir  Michail  wrote:
> Hi,
>
> How can this be done efficiently with the current version of the GAE?
>
> I suppose you could keep track of all events affecting scoring for the
> last 24 hours.  Whenever you handle a request -- even one that does
> not affect scoring -- you could prune out events older than 24 hours
> and add a new one if the player has received more points as a result
> of the current request.
>
> As you take out old events and add in new ones, you also update the
> top 10 ranking for the last 24 hours accordingly.
>
> The problem here is that taking out all the old events can be quite
> expensive.  As an extreme example, if you don't get any requests for
> over 24 hours, then on the next request, you will need to clear out at
> a day's worth of scoring events.
>
> Amir
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: How to determine top 10 scoring users from last 24 hours?

[Amir Michail]

On Thu, Jan 8, 2009 at 3:53 PM, boson  wrote:
>
> I've been planning something like this for my app, and so far this is
> the best I've come up with:

But how would you do this with a 24 hour rolling window?

Amir

>
> First of all, you don't want to be querying to sum up scores for
> individual users, so you need to keep a tally in one place.  This is
> BigTable business as usual.
>
> In each relevant entity where you are tallying, track a day and a
> score for that day:
>  user = "Kobe"
>  weight = "180"
>  ...
>  lifetime_score = 14382  # total score over eternity
>  daily_day = "20090108"  # the last day a score was recorded
>  daily_score = 153   # the score for daily_day
>
> When you update the score, do a transaction as follows:
>  if today's date matches entity's daily_day:
>lifetime_score += new_score
>daily_score += new_score
>  else:
>lifetime_score += new_score
>daily_day = today
>daily_score = new score
>
> Then your final query is:
>  filter("day =", today_as("MMDD")).order("-score")
>
> That requires a fairly simple (but custom) index.
>
> The query ignores people who haven't scored today (the first filter
> drops them), and sorts in descending on score.
>
> And of course you should memcache the query results.
>
> Hope this helps!
>
> On Jan 8, 12:33 pm, Amir  Michail  wrote:
>> Hi,
>>
>> How can this be done efficiently with the current version of the GAE?
>>
>> I suppose you could keep track of all events affecting scoring for the
>> last 24 hours.  Whenever you handle a request -- even one that does
>> not affect scoring -- you could prune out events older than 24 hours
>> and add a new one if the player has received more points as a result
>> of the current request.
>>
>> As you take out old events and add in new ones, you also update the
>> top 10 ranking for the last 24 hours accordingly.
>>
>> The problem here is that taking out all the old events can be quite
>> expensive.  As an extreme example, if you don't get any requests for
>> over 24 hours, then on the next request, you will need to clear out at
>> a day's worth of scoring events.
>>
>> Amir
> >
>



-- 
http://readmytweets.com
http://chatbotgame.com
http://numbrosia.com
http://twitter.com/amichail

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Is the Datastore just too slow for 100+ returned records?

[Tzakie]

Looking at it more deeply every 20th one takes a long time. I assume
that's the data fetch.

Still need a way to optimize...
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Is the Datastore just too slow for 100+ returned records?

[Dan Sanderson]

If you only need some of the properties for the query that needs 100+
results, you'll need to create a separate set of entities with just those
properties, and query those.  Similarly, if you want the query to return
just the keys, you'll need entities containing the properties that are the
subjects of query filters and the keys for the full entities.

100+ entities in a single request is a lot, especially with 40 properties on
each entity.  Smaller entities will get() faster, but you might also
consider avoiding needing so many results at once. If you're hitting request
timeouts and really need that much data, you could spread the requests
across multiple requests using JavaScript.  This won't reduce the total user
time for the complete result set, but you could reduce perceived latency by
displaying the first 20 results immediately while the remaining 80 are
fetched.

If you're trying to deliver the results all at once like in a downloadable
spreadsheet, you'll have to get clever, maybe use memcache as a workspace
and build it over multiple requests.

-- Dan

On Thu, Jan 8, 2009 at 1:07 PM, Tzakie  wrote:

>
> Looking at it more deeply every 20th one takes a long time. I assume
> that's the data fetch.
>
> Still need a way to optimize...
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: http://carpoolfinder.appspot.com/ won't work now and svn update fails

[thebrianschott]

Boson,

Thanks, somehow I had misread the link in the docs to say the
information there was only for the developer's local system. But it
was for the productions system. So I read more carefully and our app
is primitive, but it works now. So I may start another thread, as you
suggest, but not till I hit another snag.

Thank you to all for your great help.

Brian in Atlanta

On Jan 8, 3:40 pm, boson  wrote:
> Brian,
>
> It's hard to tell what you're actually stuck on, because you're
> throwing out a lot of information about SVN, transactions, logging,
> production vs. development, etc.
>
> I suggest you start a new thread with a simple and distilled version
> of your question, and only include short and relevant snippets of
> code.  You will likely receive more helpful attention that way.
>
> On Jan 8, 10:37 am, thebrianschott  wrote:
>
> > Rodrigo,
>
> > If found this link which describes using users
> > http://code.google.com/appengine/docs/thedevwebserver.html
> > but it describes only what should be done on the development system. I
> > cannot find a page which describes your approach on the appspot
> > system.
> >[blah blah blah from Brian]

>

Brian in Atlanta
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: http://carpoolfinder.appspot.com/ won't work now and svn update fails

[thebrianschott]

Actually the link that helped me is this one.

http://code.google.com/appengine/docs/users/loginurls.html

On Jan 8, 4:32 pm, thebrianschott  wrote:
> > >http://code.google.com/appengine/docs/thedevwebserver.html

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Is the Datastore just too slow for 100+ returned records?

[Tzakie]

>Similarly, if you want the query to return
>just the keys, you'll need entities containing the properties that are the
>subjects of query filters and the keys for the full entities.

so there is no way to

QueryString="SELECT * FROM USER_INVENTORY WHERE TUSER_ID=432 AND
CATEGORY_ID=23423"
CategoryRows = db.GqlQuery(QueryString)

KeyNames=CategoryRows.getKeyNames()

Without creating separate entities?

That's crazy. There must be a way to dig the identifiers out of the
db.GqlQuery() return.

Unless you can get the keys out and memcache I'm beginning to think
beyond trivial
applications app engine is just unusable. I've put so much time into
this some expert
person help me out here please.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Announcing: System Status Dashboard, Quota Details Page, and a Preview of Billing

[Marzia Niccolai]

Hi,

First, regarding Google Checkout - this will be the only method of payment
available when billing launches, however, we are still committed to
supporting our developers who live in the few countries where Google
Checkout is not available. For developers located in those countries, quotas
for particular applications may be raised.  Decisions will be made strictly
on a case-by-case basis and may take up to two weeks

The pricing upon launch of billing can be found here:
http://googleappengine.blogspot.com/2008/05/announcing-open-signups-expected.html

-Marzia

On Thu, Jan 8, 2009 at 9:44 AM, mclovin  wrote:

>
> Am i missing something somewhere? I cant seem to find any pricing for
> commericial applications even though google advertises app engine as a
> solution for commericial deployment.
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Transcript: App Engine chat time January 7

[Amir Michail]

Hi,

About that probabilistic stuff I was talking about, it turned out to
be unnecessary for my app.

A very simple solution not involving probability:

* associate a ratio with each user taking into account the score and
the number of times that user has been selected

* when selecting a user, use this query:

SELECT * FROM User ORDER BY ratio DESC

In this way, users with higher scores will be selected more times.

Amir


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: How to determine top 10 scoring users from last 24 hours?

[boson]

On Jan 8, 1:00 pm, "Amir Michail"  wrote:
> But how would you do this with a 24 hour rolling window?

My solution was for a simple fixed 24-hour chunks.

But you could adapt it to store 24 fixed hourly chunks in each entity
and a timestamp to identify the window.  Then roll the data back by
the proper # of hours on each transactional write.

  hourly_scores = [13, 0, 0, 18, 28, ...]   # always 24 items
  hourly_score = 182# sum of the 24 items
  hourly_hour = "2009010814"# MMDDHH

The update logic involves shifting and pushing and some array
shuffling, and summing the 24 values into hourly_score.  But it's all
still relatively straightforward.

I believe this would accomplish the goal (approximating a rolling 24
hour window, with 1 hour granularity), and in a way that still only
requires a single transaction for updates and a simple indexed query
for fetching the results.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Zip Importer High CPU every request

[Anthony]

Yes I have a main(), but the zipimporter is not in the .py file
containing the main() it is in an imported file.

The zipimporter is writing out its own info into the log.. for every
request - should this happen if it is cached?

On Jan 8, 5:34 pm, Guido van Rossum  wrote:
> One question -- are you defining a main() function? If not, your
> entire app may be reloaded each time. Try putting logging statements
> (or prints to sys.stderr) at the top-level of your module, and check
> in the logs for these -- they should only be logged for the first
> request. If they are logged for each request, then somehow your main
> is being reloaded each time.
>
> On Jan 7, 4:39 pm, Anthony  wrote:
>
> > It looks like the info in the logs could be down to the whole page
> > processing, but very simple pages (no db) consistently taking 500ms+
> > Ill give it a try without the zipimport.
>
> > I'm just using the loader cache from the cookbook (removed the
> > bytecode cache), jinja works great, although I did need to force
> > caching off in dev or the internal caches do not update after template
> > changes...
>
> > import config
> > import os
> > import sys
> > from google.appengine.api import memcache
>
> > sys.path.insert(0, 'lib/jinja2.zip')
> > from jinja2 import Environment, FunctionLoader , TemplateNotFound
>
> > def jinja2_template_loader(templatename):
> >         templatepath = os.path.abspath(os.curdir+'/templates/'+templatename)
> >         template = memcache.get(templatepath+config.VERSION)
> >         if template is None:
> >                 try:
> >                         template = file(templatepath).read()
> >                         if config.LIVE:
> >                                 
> > memcache.set(templatepath+config.VERSION,template)
> >                 except:
> >                         template = None
> >         return template
>
> > if config.LIVE:
> >         jinja2_environment = Environment(loader = FunctionLoader
> > (jinja2_template_loader),cache_size=50)
> > else:
> >         jinja2_environment = Environment(loader = FunctionLoader
> > (jinja2_template_loader),cache_size=0)
>
> > def render(template_name,context):
> >         template = jinja2_environment.get_template(template_name)
> >         context["stats"]="LIVE:" +str(config.LIVE) + " DEBUG:" +str
> > (config.DEBUG)
> >         return template.render(context)
>
> > On Jan 7, 7:39 pm, "Rodrigo Moraes"  wrote:
>
> > > On Wed, Jan 7, 2009 at 2:16 PM, Anthony wrote:
> > > > Is it normal for the zipimporter to take 500-1000ms every request -
> > > > should it be caching internally?
>
> > > > I'm importing like so:
>
> > > > # templates.py
> > > > sys.path.insert(0, 'lib/jinja2.zip')
> > > > from jinja2 import Environment, FunctionLoader , TemplateNotFound ,
> > > > MemcachedBytecodeCache
>
> > > > Will it still be caching if im not importing the zip dircetly in the
> > > > main.py?
>
> > > Not an answer to your question (which I'd like to know :), just a
> > > warning: you won't be able to use MemcachedBytecodeCache from Jinja2,
> > > because bytecode usage is not allowed in App Engine.
>
> > > The example using MemcachedBytecodeCache in the App Engine Cookbook
> > > only works in the dev server, not in production. :-/
>
> > > Apart from this, Jinja2 implementation is very smooth. :)
>
> > > -- rodrigo
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Is the Datastore just too slow for 100+ returned records?

[boson]

BigTable is an object store, so you can't ask it for specific fields
like you can in a traditional database.  In the future we may get new
tools from Google that operate in cloud data and return a result
(MapReduce, etc.), but not yet.

You might try (like Dan said) breaking your entities up into smaller
parts, and using a common key_name format to identify parts.  I think
if you loaded a few hundred entities with just (e.g.) USER_INVENTORY,
TUSER_ID, ITEM_NAME, and ITEM_NUMBER, it would go much quicker.

Sorry this isn't what you wanted to hear.  Maybe somebody else has a
better answer.

On Jan 8, 2:00 pm, Tzakie  wrote:
> >Similarly, if you want the query to return
> >just the keys, you'll need entities containing the properties that are the
> >subjects of query filters and the keys for the full entities.
>
> so there is no way to
>
> QueryString="SELECT * FROM USER_INVENTORY WHERE TUSER_ID=432 AND
> CATEGORY_ID=23423"
> CategoryRows = db.GqlQuery(QueryString)
>
> KeyNames=CategoryRows.getKeyNames()
>
> Without creating separate entities?
>
> That's crazy. There must be a way to dig the identifiers out of the
> db.GqlQuery() return.
>
> Unless you can get the keys out and memcache I'm beginning to think
> beyond trivial
> applications app engine is just unusable. I've put so much time into
> this some expert
> person help me out here please.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Static File Questions

[MajorProgamming]

Just a few questions on static files:

1. Does serving a static file cost CPU usage?

2. Does having a large number of static files (let's say like 700)
have _any_ effect on performance of page serving [similarly, does
having a lot of mappings affect performance]?

3. When a python script reads a file off the app's filesystem (like a
template html file), is that any faster than a get_by_key on the
datastore? [i.e. filesystem vs. bigtable performance]?

4. Will the paid version of Google App Engine allow over 1000 files
and/or larger than 1MB files (or entities)?

Thanks,
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Autogenerated indexes being cached?

[Alexander Kojevnikov]

Try removing the database access history file, then restarting the
development server. The file is called dev_appserver.datastore.history
and located in the tmp directory.

The development server auto-generates the indices based on the history
file, even if you don't run the corresponding queries after cleaning
up index.yaml

Hope this helps.

Cheers,
Alex
---
www.muspy.com

On Jan 9, 4:57 am, Ryan W  wrote:
> I'm working to tune my indexes.  I noticed a lot of unused indexes, so
> I removed everything from my index.yaml and then added back one by one
> while running the application locally with the "dev_appserver.py --
> require_indexes" command.  Everything was working good, so I went
> ahead and ran the index vacuum for the production site.  All was good
> there too.
>
> But, now when I run the app locally with the GoogleAppEngineLauncher
> app, not using the --require_indexes flag, it keeps adding back some
> autogenerated indexes that aren't used.  How do I know they aren't
> used?  Because the app will run without them with the --
> require_indexes flag, and because the counts of the "has been used x
> times" are not being incremented.
>
> I've tried restarting the app, removing the app altogether from
> GoogleAppEngineLauncher, then adding back.  It still keeps adding
> these indexes though.  Is there a cache somewhere that can reset?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Is the Datastore just too slow for 100+ returned records?

[Dan Sanderson]

There is currently no way to retrieve only parts of entities, nor just keys,
from the datastore in response to queries.  There's no way to dig out the
keys either: the datastore returns the full entities in response to queries,
there is no intermediate app-side step that fetches entities for keys.
The datastore is designed with the expectation that the complete entity is
needed as the result of a query.  If you're storing many facts about a
notional thing (such as a product in a catalog), and you only need some of
those facts back in response to a query, you may want to represent the thing
using multiple entities.  Depending on the nature of the queries, the
entities for a thing may store duplicate information.  You can manage this
duplication using the Python class API and transactions.

For what it's worth, I was once an engineer in the catalog department of a
major online retailer, and we used a similar technique for "fast" product
data, which showed up in search and browser results and the like, and "slow"
product data which was only needed on the product page.

-- Dan

On Thu, Jan 8, 2009 at 2:00 PM, Tzakie  wrote:

>
> >Similarly, if you want the query to return
> >just the keys, you'll need entities containing the properties that are the
> >subjects of query filters and the keys for the full entities.
>
> so there is no way to
>
> QueryString="SELECT * FROM USER_INVENTORY WHERE TUSER_ID=432 AND
> CATEGORY_ID=23423"
> CategoryRows = db.GqlQuery(QueryString)
>
> KeyNames=CategoryRows.getKeyNames()
>
> Without creating separate entities?
>
> That's crazy. There must be a way to dig the identifiers out of the
> db.GqlQuery() return.
>
> Unless you can get the keys out and memcache I'm beginning to think
> beyond trivial
> applications app engine is just unusable. I've put so much time into
> this some expert
> person help me out here please.
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Is the Datastore just too slow for 100+ returned records?

[Alexander Kojevnikov]

> Looking at it more deeply every 20th one takes a long time. I assume
> that's the data fetch.
>
I guess you are iterating over a Query or GqlQuery object to get the
entities? This explains explain why every 20th iteration. From
http://code.google.com/appengine/docs/datastore/queryclass.html :

  The iterator retrieves results from the datastore in small batches,
allowing for the app to stop iterating on results to avoid fetching
more than is needed.

If you know beforehand how many entities you need, try using the fetch
() method and iterate over an array of entities it returns.

This may be a bit faster, but still, your problem is the size of your
entities and their number. Try reducing either or both of them.

Dan and boson already covered the size part, regarding the number, do
you really need 500 entities at once? Unless you are going to
calculate some aggregate values, your users won't be able to digest so
much information on a page.

Cheers,
Alex
--
www.muspy.com
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Beginner Question - About using App Engine

[mscwd01]

I have taken a look around the App Engine documentation but havent
been able to find a definitive answer to my question, so I hoped i'd
get an answer by posting here ;)

I am developing an Android application which makes substantial use of
Google APIs; I had planned to set up my own dedicated server for tasks
such as user authentication and storing certain data relating to
members using the application. It would be great, having learnt of the
inbuilt API support offered by App Engine, if I could make use of the
App Engine service for my needs, without the need to setup and run my
own server.

My question however, is, am I able to use App Engine as a "web
service" instead of a usual "web application"?

Basically, what I had planned to do was use App Engine, via http calls
from my Android App, as a mechanism to authenticate users and return
security "priviliges" of their account (what content they can/cant
access) and also a method of storing a "facebook style wall" for each
user.

In short, i'd like to use App Engine to return xml responses to my
Android App - instead of using App Engine for the usual browser based
web application.

Is this allowed?

Thanks

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Is the Datastore just too slow for 100+ returned records?

[Tzakie]

Hey Dan and Boson,

Thanks very much for your time guys.

When you call:
CategoryRows = db.GqlQuery(QueryString)
It's fast.

It's the loop where you try and access the returned entities that
is slow. Doesn't it follow that the CategoryRows has the keys in it
and the next() in the python loop is fetching them? every 20th
next() call is also takes extra time. Like it's fetching 20 at a
time from some place.

I don't think the problem is the aspects of BigTable. The thing
is just 10 times too slow. It needs to be able to load about
1,000 records at max too make the returned record count a
non-issue. You need about 50 properties to make the property
count a non-issue. The thing needs to return 1,000 entities or
50,000 properties in 0.5 seconds then the problem goes away.

You can cache them for the next user. We just need it to go
off once.

Right now it takes 3 seconds upgraded to '30 processor seconds'
and the thing flips out.

To the smart guys at google. Putting a couple hundred inventory
lines in front of a user seems like something the app engine should
do.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: How to determine top 10 scoring users from last 24 hours?

[Amir Michail]

On Thu, Jan 8, 2009 at 5:34 PM, boson  wrote:
>
> On Jan 8, 1:00 pm, "Amir Michail"  wrote:
>> But how would you do this with a 24 hour rolling window?
>
> My solution was for a simple fixed 24-hour chunks.
>
> But you could adapt it to store 24 fixed hourly chunks in each entity
> and a timestamp to identify the window.  Then roll the data back by
> the proper # of hours on each transactional write.
>
>  hourly_scores = [13, 0, 0, 18, 28, ...]   # always 24 items
>  hourly_score = 182# sum of the 24 items
>  hourly_hour = "2009010814"# MMDDHH

What would your query be?  How would you ensure that you include not
only those who scored points in the last hour but also everyone who
scored points in the last 24 hours?

Amir

>
> The update logic involves shifting and pushing and some array
> shuffling, and summing the 24 values into hourly_score.  But it's all
> still relatively straightforward.
>
> I believe this would accomplish the goal (approximating a rolling 24
> hour window, with 1 hour granularity), and in a way that still only
> requires a single transaction for updates and a simple indexed query
> for fetching the results.
> >
>



-- 
http://readmytweets.com
http://chatbotgame.com
http://numbrosia.com
http://twitter.com/amichail

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Is the Datastore just too slow for 100+ returned records?

[Dan Sanderson]

On Thu, Jan 8, 2009 at 3:35 PM, Tzakie  wrote:

>
> Thanks very much for your time guys.
>

No problem, happy to help.


> When you call:
> CategoryRows = db.GqlQuery(QueryString)
> It's fast.
>
> It's the loop where you try and access the returned entities that
> is slow. Doesn't it follow that the CategoryRows has the keys in it
> and the next() in the python loop is fetching them? every 20th
> next() call is also takes extra time. Like it's fetching 20 at a
> time from some place.


Actually, db.GqlQuery() (and db.Query() and Model.all() and Model.gql())
only sets up the query, it does not execute it.  The query is executed when
you retrieve the results using q.fetch(), q.get(), or the iterator interface
(for result in q: ...).

We definitely hear your concerns about the desired scale of single
operations, and we're always working to improve these numbers.  But it's
worth considering that App Engine is a scalable system intended to offer the
same performance characteristics whether your data set has thousands of
items or millions of items.  To do that, it has to behave differently than a
single-server relational database.

Quite frankly, I can't think of a Google web app that displays 100 of
anything all at once...

-- Dan

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Beginner Question - About using App Engine

[Dan Sanderson]

Yup, you can definitely use App Engine to create web services like you
describe.  A networked backend for an Android app is a great use of App
Engine.
-- Dan

On Thu, Jan 8, 2009 at 2:56 PM, mscwd01  wrote:

>
> I have taken a look around the App Engine documentation but havent
> been able to find a definitive answer to my question, so I hoped i'd
> get an answer by posting here ;)
>
> I am developing an Android application which makes substantial use of
> Google APIs; I had planned to set up my own dedicated server for tasks
> such as user authentication and storing certain data relating to
> members using the application. It would be great, having learnt of the
> inbuilt API support offered by App Engine, if I could make use of the
> App Engine service for my needs, without the need to setup and run my
> own server.
>
> My question however, is, am I able to use App Engine as a "web
> service" instead of a usual "web application"?
>
> Basically, what I had planned to do was use App Engine, via http calls
> from my Android App, as a mechanism to authenticate users and return
> security "priviliges" of their account (what content they can/cant
> access) and also a method of storing a "facebook style wall" for each
> user.
>
> In short, i'd like to use App Engine to return xml responses to my
> Android App - instead of using App Engine for the usual browser based
> web application.
>
> Is this allowed?
>
> Thanks
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Beginner Question - About using App Engine

[mscwd01]

Thanks for your quick reply Dan, I shall be using App Engine in the
not-to-distant future it seems.

... I shall need to learn Python now :)

On Jan 8, 11:57 pm, Dan Sanderson  wrote:
> Yup, you can definitely use App Engine to create web services like you
> describe.  A networked backend for an Android app is a great use of App
> Engine.
> -- Dan
>
> On Thu, Jan 8, 2009 at 2:56 PM, mscwd01  wrote:
>
> > I have taken a look around the App Engine documentation but havent
> > been able to find a definitive answer to my question, so I hoped i'd
> > get an answer by posting here ;)
>
> > I am developing an Android application which makes substantial use of
> > Google APIs; I had planned to set up my own dedicated server for tasks
> > such as user authentication and storing certain data relating to
> > members using the application. It would be great, having learnt of the
> > inbuilt API support offered by App Engine, if I could make use of the
> > App Engine service for my needs, without the need to setup and run my
> > own server.
>
> > My question however, is, am I able to use App Engine as a "web
> > service" instead of a usual "web application"?
>
> > Basically, what I had planned to do was use App Engine, via http calls
> > from my Android App, as a mechanism to authenticate users and return
> > security "priviliges" of their account (what content they can/cant
> > access) and also a method of storing a "facebook style wall" for each
> > user.
>
> > In short, i'd like to use App Engine to return xml responses to my
> > Android App - instead of using App Engine for the usual browser based
> > web application.
>
> > Is this allowed?
>
> > Thanks
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Beginner Question - About using App Engine

[Faber Fedor]

On Thu, Jan 8, 2009 at 5:56 PM, mscwd01  wrote:

>
>
> In short, i'd like to use App Engine to return xml responses to my
> Android App - instead of using App Engine for the usual browser based
> web application.
>
> Is this allowed?


I hope so!  I'm planning on doing the same thing! (Except with JSON :-)



-- 

Faber Fedor
Cloud Computing New Jersey
http://cloudcomputingnj.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] ImportError: cannot import name webapp

[Kwame]

I have a problem that I am unable to understand. One minute my
appengine app was running fine, the next it was giving this error:
ImportError: cannot import name webapp
Older projects that I built on app engine seem to be working fine.

The full stack trace is:

ERROR2009-01-09 00:19:27,800 dev_appserver.py] Exception
encountered handling request
Traceback (most recent call last):
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/dev_appserver.py", line 2443, in _HandleRequest
base_env_dict=env_dict)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/dev_appserver.py", line 353, in Dispatch
base_env_dict=base_env_dict)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/dev_appserver.py", line 1869, in Dispatch
self._module_dict)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/dev_appserver.py", line 1787, in ExecuteCGI
reset_modules = exec_script(handler_path, cgi_path, hook)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/dev_appserver.py", line 1683, in
ExecuteOrImportScript
exec module_code in script_module.__dict__
  File "/Users/iwegbue/Documents/workspace/com.healthax.gae/index.py",
line 11, in 
from google.appengine.ext import webapp
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/dev_appserver.py", line 847, in decorate
return func(self, *args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/dev_appserver.py", line 1443, in load_module
return self.FindAndLoadModule(submodule, fullname, search_path)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/dev_appserver.py", line 847, in decorate
return func(self, *args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/dev_appserver.py", line 1351, in
FindAndLoadModule
description)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/dev_appserver.py", line 847, in decorate
return func(self, *args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/dev_appserver.py", line 1301, in
LoadModuleRestricted
description)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/ext/webapp/__init__.py", line 68, in 
import webob
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/dev_appserver.py", line 847, in decorate
return func(self, *args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/dev_appserver.py", line 1443, in load_module
return self.FindAndLoadModule(submodule, fullname, search_path)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/dev_appserver.py", line 847, in decorate
return func(self, *args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/dev_appserver.py", line 1351, in
FindAndLoadModule
description)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/dev_appserver.py", line 847, in decorate
return func(self, *args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/dev_appserver.py", line 1301, in
LoadModuleRestricted
description)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_a

[google-appengine] Re: How to determine top 10 scoring users from last 24 hours?

[boson]

On Jan 8, 3:44 pm, "Amir Michail"  wrote:
> On Thu, Jan 8, 2009 at 5:34 PM, boson  wrote:
>
> > On Jan 8, 1:00 pm, "Amir Michail"  wrote:
> >> But how would you do this with a 24 hour rolling window?
>
> > My solution was for a simple fixed 24-hour chunks.
>
> > But you could adapt it to store 24 fixed hourly chunks in each entity
>
> What would your query be?  How would you ensure that you include not
> only those who scored points in the last hour but also everyone who
> scored points in the last 24 hours?
>

Good question.  I don't think the model I proposed works for *rolling*
windows... Just fixed windows like "day" or "month".

Please do update this thread if/when you come up with something!


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Is the Datastore just too slow for 100+ returned records?

[Tzakie]

> Quite frankly, I can't think of a Google web app that displays 100 of
> anything all at once...

I'm getting the impression that people think what I'm asking for is
ridiculous
and off the radar. I sent you an e-mail with the urls of the current
app and
what I am working on. When you see it in context I think it looks
pretty
reasonable.

E commerce apps particularly need long lists for a lot of things. On
the
e-commerce apps I do paging kills customers. They don't "next" they
just leave.

Can't you guys make something that just returns the keys from a query?
That seems consistent with how I think big table works.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Running multiple Dev Web Servers on same machine

[Alex Popescu]

I am running the latest GAppEngine SDK on Leopard with Python 2.5.4.
As mentioned in the initial post, I have configured each of the web
servers to run on specific ports (and both are configured to use their
own data storage).
I am noticing the issue when one of the web servers if responding to a
request from the 2nd one. The exact scenario is the following:

1. web server 1 is posting some data to web server 2
2. web server 2 is requiring authentication and it returns a 401
3. web server 1 receives back a 200

I have debugged the web server 2 request cycle and the response status
was correctly set up to the latest execution steps of the cycle. But,
it finally returns a 200 (my impression is that the framework is using
sys.stdout redirection and at some point the 2 processes are stepping
on each other toes).

I'll try to create a very simple test case to reproduce the problem,
but if you have a simple environment just try to emulate the above
scenario.

tia,

./alex

On Jan 8, 10:03 pm, Bill  wrote:
> Alex,
>
> I've had no problem running 2 dev web servers on my MacBook Pro
> (Leopard OS X).  What OS are you using and are you launching
> dev_appserver.py with the port options?  You have to give more
> detailed information.
>
> Best,
> Bill
>
> On Jan 8, 2:39 pm, Alex Popescu 
> wrote:
>
> > I am trying to run 2 dev web servers on my local machine in order to
> > test some fetch functionality. Anyways, even if both apps are
> > configured to use different ports and they are running in 2 different
> > Python processes, this doesn't seem to work. Is this a known
> > limitation? Is there any workaround?
>
> > tia,
>
> > ./alex
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Is the Datastore just too slow for 100+ returned records?

[Alex Popescu]

On Jan 9, 4:15 am, Tzakie  wrote:
> [...]
>
> Can't you guys make something that just returns the keys from a query?
> That seems consistent with how I think big table works.

I'm pretty sure there should be a solution for this as BigTable is
basically a distributed hashmap, so fetching only the key set should
not be a real problem. Now, I don't know the details of the
implementation, but there might be another implementation where the
entities are just serialized and the engine is using some surrogate
keys (that are not visible to the app).

./alex
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Is the Datastore just too slow for 100+ returned records?

[Alex Popescu]

On Jan 9, 1:22 am, Alexander Kojevnikov 
wrote:
> > Looking at it more deeply every 20th one takes a long time. I assume
> > that's the data fetch.
>
> I guess you are iterating over a Query or GqlQuery object to get the
> entities? This explains explain why every 20th iteration. 
> Fromhttp://code.google.com/appengine/docs/datastore/queryclass.html:
>
>   The iterator retrieves results from the datastore in small batches,
> allowing for the app to stop iterating on results to avoid fetching
> more than is needed.
>

That's a very interesting bit that I've missed while reading the docs,
but it is quite consistent with the statistics I've noticed. Anyways,
from my tests any result set that goes beyond 20-30 results is quickly
degrading the performance.

./alex
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Admin interface that let's me enter dummy data

[adelevie]

I really like GAE's admin interface save for one thing: it doesnt let
me enter dummy data--it says I must do it programmatically. While I'm
sure there is some legitimate reason for this, I still would like an
automated set of forms generated from my existing models. Is anyone
else in the same boat as me? Right now I'm stuck manually creating a
form class and url for each models--quite messy. Are there any clever
workarounds?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Is the Datastore just too slow for 100+ returned records?

[Dan Sanderson]

Repeating a couple of things I mentioned to Greg in email, for the list
discussion:
A single-server SQL database with low load and tables with rows numbering in
the thousands can do lots of cool things with that data in a reasonable
amount of time.  But many of the features of a SQL query engine have
difficulty scaling to hundreds of machines, millions of records, and 50+
qps.  The App Engine datastore is designed to scale, and inevitably lacks
features of single-server SQL databases that don't scale.  The datastore is
designed to maintain its performance characteristics over very large data
sets and heavy traffic, so performance with small data sets and low traffic
may not always compare to a single-server SQL database under similar
conditions. in the same sense that it won't compare to a RAM cache of the
same data.
I'll be the first to admit that this means some applications are not
well-suited to App Engine.  However, I wouldn't say that a table of 300
items of product data is necessarily impractical, even if it isn't as simple
as rendering the results of a SQL query.  One simple option in this case is
to store an entity per item as you're doing now, but use range queries and
multiple requests driven by JavaScript to build the page.  (Watch how Google
Reader renders lists of hundreds of items; you can barely tell the data
wasn't all there on the first request.)

Going to the primary database to retrieve 300 items of 40 fields each is
asking for 12,000 instantly dynamic fields for a single page.  That's a
*lot* to ask if the app expects to scale.  Part of scaling data is
determining how quickly updates need to be available, so you can
pre-calculate and cache results to optimize queries, i.e. make data that
doesn't need to be instantly dynamic less dynamic.

In the case of a product catalog, most of the catalog data can be largely
static.  You can pre-build and store the product lists, then store dynamic
data (like product availability) on separate entities queried at render
time.  The upcoming cron service will make this kind of pre-building and
cache warm-up easier to do, and in some cases you can just update lists and
caches when you update individual fields in primary storage.  You have the
right idea with wanting to cache this information, but you'll probably need
to do something other than performing the 12,000-field query during a user
request with the intent to cache.

Incidentally, returning just keys for a query has been on our radar for a
while, we just haven't gotten to it.  I can't promise anything, but it's on
our list.  Feel free to file a feature request in the issue tracker to
promote it.

-- Dan

On Thu, Jan 8, 2009 at 6:15 PM, Tzakie  wrote:

>
>
> > Quite frankly, I can't think of a Google web app that displays 100 of
> > anything all at once...
>
> I'm getting the impression that people think what I'm asking for is
> ridiculous
> and off the radar. I sent you an e-mail with the urls of the current
> app and
> what I am working on. When you see it in context I think it looks
> pretty
> reasonable.
>
> E commerce apps particularly need long lists for a lot of things. On
> the
> e-commerce apps I do paging kills customers. They don't "next" they
> just leave.
>
> Can't you guys make something that just returns the keys from a query?
> That seems consistent with how I think big table works.
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Autogenerated indexes being cached?

[Ryan W]

That's it!  Thanks Alex.

I had a little trouble tracking down the file, wish this were better
documented.  For anybody who discovers this later, I found it here

$ sudo find / -name "dev_appserver.datastore.history" -print
/private/var/folders/Mq/Mq6HCC1-F8enD-Rr+rKv2k+++TI/-Tmp-/
dev_appserver.datastore.history


On Jan 8, 3:02 pm, Alexander Kojevnikov 
wrote:
> Try removing the database access history file, then restarting the
> development server. The file is called dev_appserver.datastore.history
> and located in the tmp directory.
>
> The development server auto-generates the indices based on the history
> file, even if you don't run the corresponding queries after cleaning
> up index.yaml
>
> Hope this helps.
>
> Cheers,
> Alex
> ---www.muspy.com
>
> On Jan 9, 4:57 am, Ryan W  wrote:
>
> > I'm working to tune my indexes.  I noticed a lot of unused indexes, so
> > I removed everything from my index.yaml and then added back one by one
> > while running the application locally with the "dev_appserver.py --
> > require_indexes" command.  Everything was working good, so I went
> > ahead and ran the index vacuum for the production site.  All was good
> > there too.
>
> > But, now when I run the app locally with the GoogleAppEngineLauncher
> > app, not using the --require_indexes flag, it keeps adding back some
> > autogenerated indexes that aren't used.  How do I know they aren't
> > used?  Because the app will run without them with the --
> > require_indexes flag, and because the counts of the "has been used x
> > times" are not being incremented.
>
> > I've tried restarting the app, removing the app altogether from
> > GoogleAppEngineLauncher, then adding back.  It still keeps adding
> > these indexes though.  Is there a cache somewhere that can reset?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] user login URLs in html/javascript

[thebrianschott]

http://code.google.com/appengine/docs/users/loginurls.html

At the link above it explains how to send a user off to sign-in or
register in a python script. The code below does what I want, but I
don't know how to do the same thing as an html hyperlink on my
applications "home" page. Not all users need to login to proceed
further into my application, only those who want to create their own
map.

greeting = ("Sign in or register." %
  users.create_login_url("/"))

self.response.out.write("%s" % greeting)

Thanks,

Brian in Atlanta

Btw if you want to play with my app, it is at
http://carpoolfinder.appspot.com
There is a sort of a sandbox that does not require a sign-in if you
enter "Tester" (without quotes) in the inital text field.



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Proper use of BASE HTML tag

[MajorProgamming]

Is this treated well (and accepted as proper usage) by all standards
and browser?


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: 500 server error despite proper testing on dev_appserver.py and upload

[Alexander Konovalenko]

On Thu, Jan 8, 2009, Marzia Niccolai  wrote:
>
> Usually if you are getting a 500 server error with no logs like you have
> described, this means there is some typo in the app.yaml file that was
> uploaded.  We've seen this issue come up because of improper spacing, line
> endings or a typo in the script name.

It would be nice if troubleshooting a typo in app.yaml was easier.
What about showing a noticeable error message somewhere when the user
uploads a broken app.yaml file?

See this suggestion in the issue tracker:
http://code.google.com/p/googleappengine/issues/detail?id=976

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Your client does not have permission to get URL /admin/ from this server.

[DocDay]

I'm using the "$PYTHON_LIB/google/appengine/ext/admin" script in
several apps exactly as shown in the example here:

http://code.google.com/appengine/docs/configuringanapp.html#Referring_to_the_Python_Library_Directory

All apps authenticate as expected, except for one. It returns:

"Your client does not have permission to get URL /admin/ from this
server."

I added a second administrator account to the problem app, but cannot
login with that either. How can I get this to work? Interestingly,
this is the only app in the bunch that does not use Google Accounts
for sign in. Is that relevant?

Doc

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Admin interface that let's me enter dummy data

[Alexander Kojevnikov]

You can include the 'shell' application from the official appengine
samples with your app, and add/edit/delete the entities from Python
shell.

Source: http://code.google.com/p/google-app-engine-samples/downloads/list
Demo: http://shell.appspot.com/

Don't forget to restrict access to shell, e.g. only to the app admins.

Cheers,
Alex
--
www.muspy.com

On Jan 9, 3:54 pm, adelevie  wrote:
> I really like GAE's admin interface save for one thing: it doesnt let
> me enter dummy data--it says I must do it programmatically. While I'm
> sure there is some legitimate reason for this, I still would like an
> automated set of forms generated from my existing models. Is anyone
> else in the same boat as me? Right now I'm stuck manually creating a
> form class and url for each models--quite messy. Are there any clever
> workarounds?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Admin interface that let's me enter dummy data

[adelevie]

Thanks for the response. I just dont see a reason why Google can't
allow dummy data to be entered via the built in admin interface. If it
can create forms when there is data, why cant it create forms without
data?

On Jan 9, 2:44 am, Alexander Kojevnikov 
wrote:
> You can include the 'shell' application from the official appengine
> samples with your app, and add/edit/delete the entities from Python
> shell.
>
> Source:http://code.google.com/p/google-app-engine-samples/downloads/list
> Demo:http://shell.appspot.com/
>
> Don't forget to restrict access to shell, e.g. only to the app admins.
>
> Cheers,
> Alex
> --www.muspy.com
>
> On Jan 9, 3:54 pm, adelevie  wrote:
>
> > I really like GAE's admin interface save for one thing: it doesnt let
> > me enter dummy data--it says I must do it programmatically. While I'm
> > sure there is some legitimate reason for this, I still would like an
> > automated set of forms generated from my existing models. Is anyone
> > else in the same boat as me? Right now I'm stuck manually creating a
> > form class and url for each models--quite messy. Are there any clever
> > workarounds?
>
>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---