[web2py] Re: GAE: insert using key_name
ok, understood. This is really the only thing I need: def _pre_process(self, table, **fields): Takes a w2p table and a dictionary with values and processes the field input and add defaults, computes, etc using the web2py table. fields = table._defaults(fields) return dict((f.name,table._db._adapter.represent(v,f.type)) for f,v in table._listify(fields)) At this moment this works for me and I don't need to subclass the adapter. But: In the other thread you spoke about people possibly relying on _something functions (for debugging/logging etc.) I guest my question is, can users safely rely on _something methods? What is w2p's policy regarding underscore-prefixed functions. Are they considered private? So can I use the above function as it is? Oh and, Sorry; however, you could temporarily implement a similar method by subclassing GoogleDatastoreAdapter so it pre-processes data without applying changes. What do you mean by temporarily? Could the feature be implemented in the future with a different name? Quint On Saturday, January 11, 2014 2:07:56 PM UTC+1, Alan Etkin wrote: Sorry for the confussion.. But it works fine. I will definitally use this. (I posted something stupid and removed the stupid post but left some traces. ;-)) Well there are good reasons not to add the feature, at least with the current name: https://groups.google.com/d/msg/web2py-developers/bN0WS9_skzw/NJq4bs5M8KIJ Sorry; however, you could temporarily implement a similar method by subclassing GoogleDatastoreAdapter so it pre-processes data without applying changes. http://www.web2py.com/books/default/chapter/29/06/the-database-abstraction-layer#Note-on-new-DAL-and-adapters -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: GAE: insert using key_name
I gues my question is, can users safely rely on _something methods? The change revert is due this concern about api reliability, so adapters behave as specified in http://www.web2py.com/books/default/chapter/29/06/the-database-abstraction-layer?search=_insert#Raw-SQL What is w2p's policy regarding underscore-prefixed functions. Are they considered private? Underscore is required in some cases for example to avoid name collisions. I suppose that the policy about the use of underscore follows the pep8 style guide except for special cases described in the book. For example (chapter 4): Functions that take arguments or start with a double underscore are not publicly exposed and can only be called by other functions. So can I use the above function as it is? I think so What do you mean by temporarily? Could the feature be implemented in the future with a different name? It could, If there's agreement about adding it to the adapter. -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: GAE: insert using key_name
Sorry for the confussion.. But it works fine. I will definitally use this. (I posted something stupid and removed the stupid post but left some traces. ;-)) Well there are good reasons not to add the feature, at least with the current name: https://groups.google.com/d/msg/web2py-developers/bN0WS9_skzw/NJq4bs5M8KIJ Sorry; however, you could temporarily implement a similar method by subclassing GoogleDatastoreAdapter so it pre-processes data without applying changes. http://www.web2py.com/books/default/chapter/29/06/the-database-abstraction-layer#Note-on-new-DAL-and-adapters -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: GAE: insert using key_name
How about adding support in dal.py for the following:
# processes the field input and add defaults, computes, etc. (does not
make actual insertion).
values = db._insert(spam=alot, ...)
{spam: alot, ...}
I made a pr about supporting _insert for processing values without applying
changes to the database as proposed in the previous post
https://github.com/web2py/web2py/pull/341
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: GAE: insert using key_name
Excellent!
I already included it to try it out but how do i use it?
When I use your example I get:
AttributeError: 'DAL' object has no attribute '_insert'
(obviously because there is no _select() in DAL)
When i try:
db._adapter._insert(props)
I get:
File C:\Users\Quint*\gluon\dal.py, line 5302, in genexpr
return dict((f.name,self.represent(v,f.type)) for f,v in fields)
ValueError: too many values to unpack
What do I need to supply to _select()?
It looks like I need to supply a collection of Fields.
I'm missing something...
On Thursday, January 9, 2014 2:00:08 PM UTC+1, Alan Etkin wrote:
How about adding support in dal.py for the following:
# processes the field input and add defaults, computes, etc. (does not
make actual insertion).
values = db._insert(spam=alot, ...)
{spam: alot, ...}
I made a pr about supporting _insert for processing values without
applying changes to the database as proposed in the previous post
https://github.com/web2py/web2py/pull/341
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: GAE: insert using key_name
Please Ignore that.
I got it.
Works great!
On Thursday, January 9, 2014 7:21:45 PM UTC+1, Quint wrote:
Excellent!
I already included it to try it out but how do i use it?
When I use your example I get:
AttributeError: 'DAL' object has no attribute '_insert'
(obviously because there is no _select() in DAL)
When i try:
db._adapter._insert(props)
I get:
File C:\Users\Quint*\gluon\dal.py, line 5302, in genexpr
return dict((f.name,self.represent(v,f.type)) for f,v in fields)
ValueError: too many values to unpack
What do I need to supply to _select()?
It looks like I need to supply a collection of Fields.
I'm missing something...
On Thursday, January 9, 2014 2:00:08 PM UTC+1, Alan Etkin wrote:
How about adding support in dal.py for the following:
# processes the field input and add defaults, computes, etc. (does not
make actual insertion).
values = db._insert(spam=alot, ...)
{spam: alot, ...}
I made a pr about supporting _insert for processing values without
applying changes to the database as proposed in the previous post
https://github.com/web2py/web2py/pull/341
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: GAE: insert using key_name
Works great!
On Thursday, January 9, 2014 7:21:45 PM UTC+1, Quint wrote:
Excellent!
I already included it to try it out but how do i use it?
Your message is confusing, does it work or it does not? It worked for my
environment (recent release of gae sdk and development server)
Specifically, what worked was:
myvalues = db._insert(k=v, ...)
{k: processed value}
Where db is a google app engine connection (DAL instance)
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: GAE: insert using key_name
Sorry for the confussion.. But it works fine. I will definitally use this. (I posted something stupid and removed the stupid post but left some traces. ;-)) -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: GAE: insert using key_name
But it works fine. I will definitally use this. Ok. If you can, it would be useful if you could share your tests about supported features for records lacking id values and any issue you find with setters and getters using key names. Note that the Rows class supports defining objects using dictionaries to construct them. Finally, I belive a plugin to extended DAL class so it provides retrieving and setting records by key name would be also useful. -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: GAE: insert using key_name
I would like to be able to do this using web2py. I could also choose to use the GAE API directly to do this but then a could not make use thing like calculated Fields etc in my web2py tables. One issue about forcing key names is that there's no guarantee that the new key will provide a valid ID integer when retrieved, and the DAL api requires record insertions to provide that kind of value. Note that calls to Key.id() can return None, which is used by the insert method to create the db reference object. IIRC, the exception of that rule is the case of keyed tables, but I'm not if it is supported for gae nosql. In case it is supported, the patch could instead use keyed tables for storing entity key names. From the datastore Python api: ... The identifier may be either a *key name* string, assigned explicitly by the application when the instance is created, or an integer *numeric ID,*assigned automatically by App Engine when the instance is written ( puthttps://developers.google.com/appengine/docs/python/datastore/modelclass#Model_put) to the Datastore ... I have not tested the behavior but according to the above I think that the datastore api would return None instead of valid integer identifiers for the case of entities stored with manual key names. -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: GAE: insert using key_name
Yes,
Your post reminds me, I forgot something. I thought that when a key_name is
used, it could simply return that as an id.
But if you say that DAL requires that it returns an integer id, than yes,
that would break it.
When you use a key_name, the records do not have an numeric id.
But does that mean you should not allow it to be inserted that way?
Nothing restricts users from operating (using web2py api) on rows inserted
with GAE api
dfields=dict((f.name,self.represent(v,f.type)) for f,v in fields)
# table._db['_lastsql'] = self._insert(table,fields)
#quint
keyname = None
if 'gae_key_name' in dfields:
keyname = dfields['gae_key_name']
if self.use_ndb:
dfields['id'] = dfields.pop('gae_key_name')
else:
dfields['key_name'] = dfields.pop('gae_key_name')
tmp = table._tableobj(**dfields)
tmp.put()
*if keyname:return** keyname*
key = tmp.key if self.use_ndb else tmp.key()
rid = Reference(key.id() if self.use_ndb else key.id_or_name())
(rid._table, rid._record, rid._gaekey) = (table, None, key)
return rid
On Wednesday, January 8, 2014 2:18:53 PM UTC+1, Alan Etkin wrote:
I would like to be able to do this using web2py. I could also choose to
use the GAE API directly to do this but then a could not make use thing
like calculated Fields etc in my web2py tables.
One issue about forcing key names is that there's no guarantee that the
new key will provide a valid ID integer when retrieved, and the DAL api
requires record insertions to provide that kind of value. Note that calls
to Key.id() can return None, which is used by the insert method to create
the db reference object. IIRC, the exception of that rule is the case of
keyed tables, but I'm not if it is supported for gae nosql. In case it is
supported, the patch could instead use keyed tables for storing entity key
names.
From the datastore Python api:
... The identifier may be either a *key name* string, assigned
explicitly by the application when the instance is created, or an integer
*numeric
ID,* assigned automatically by App Engine when the instance is written (
puthttps://developers.google.com/appengine/docs/python/datastore/modelclass#Model_put)
to the Datastore ...
I have not tested the behavior but according to the above I think that the
datastore api would return None instead of valid integer identifiers for
the case of entities stored with manual key names.
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: GAE: insert using key_name
But does that mean you should not allow it to be inserted that way? Nothing restricts users from operating (using web2py api) on rows inserted with GAE api There's no restriction on the use of the datastore, since it is not a service reserved for web2py apps, although I belive storing records without id assignment would be restrictive for apps (and I doubt it will be actually compatible at all, with the exception of the keyed tables mentioned before, in case it is supported). -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: GAE: insert using key_name
Sorry, I don't understand this line. although I belive storing records without id assignment would be restrictive for apps Do you mean web2py apps or GAE apps? What did you mean by restrictive (and I doubt it will be actually compatible at all Compatible with what? Sure, when you would insert using a key_name, and without an numeric id, the will be web2py API functionalities that wont work anymore. (the ones that expect a numeric id returned) But you would know that you don't need those functionalities if you would decide to use a key_name. On Wednesday, January 8, 2014 3:52:18 PM UTC+1, Alan Etkin wrote: But does that mean you should not allow it to be inserted that way? Nothing restricts users from operating (using web2py api) on rows inserted with GAE api There's no restriction on the use of the datastore, since it is not a service reserved for web2py apps, although I belive storing records without id assignment would be restrictive for apps (and I doubt it will be actually compatible at all, with the exception of the keyed tables mentioned before, in case it is supported). -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: GAE: insert using key_name
Compatible with what? With any web2py feature expecting entity keys (web2py records) with integer ids. But you would know that you don't need those functionalities if you would decide to use a key_name I think this is usually called a corner case. I would't add the feature for the purpose for storing entities by name since it does not require much coding to implement it per-application using the gae api. -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: GAE: insert using key_name
How about adding support in dal.py for the following:
# processes the field input and add defaults, computes, etc. (does not make
actual insertion).
values = db._insert(spam=alot, ...)
{spam: alot, ...}
So one can use the output for storing named keys (with app specific logic
or maybe plugin for extended datastore functions).
AFAIK, the patch is trivial (it would need adding a _insert adapter method
like the case of mongodb or imap)
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
