[sqlalchemy] error reporting wishlist -- not sure if this is possible

2014-05-19 Thread Jonathan Vanasco
occasionally, i run into this problem with Postgres; though it would likely happen on other systems too. my database might miss a migration, and we end up with this situation SqlAlchemy Model - Column = varchar(1000) PostgreSQL - Field = varchar(255) Data = 500 characters in length When I try

Re: [sqlalchemy] error reporting wishlist -- not sure if this is possible

2014-05-19 Thread Jonathan Vanasco
i think it is from the DBAPI. i was (sadly) expecting you to say that. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To

Re: [sqlalchemy] How do you limit/specify the columns that are loaded via a relationship()?

2014-05-15 Thread Jonathan Vanasco
Michael- Quick question for clarity... I have a table with a few deferred columns. If I want to eagerly load them during a query, I should pass in the undefer option right ? -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from

Re: [sqlalchemy] How do you limit/specify the columns that are loaded via a relationship()?

2014-05-15 Thread Jonathan Vanasco
that's not too bad. at least for me; i only defer a few HSTORE columns. On Thursday, May 15, 2014 2:24:51 PM UTC-4, Michael Bayer wrote: yeah there’s some bad history with the API here, in that you can’t easily pass them all at once, but right now it’s q.options(undefer(‘*’)) for

[sqlalchemy] eagerloading on an already loaded object ?

2014-04-28 Thread Jonathan Vanasco
I'm not sure if there is a trick to do this or not, but it's worth asking... I have an object that has been loaded into the Session, as per whatever eager-loading requirements : foo = dbSession.query( Foo ).filter(...).options(...).one() I'm now in a position where I need to access

Re: [sqlalchemy] eagerloading on an already loaded object ?

2014-04-28 Thread Jonathan Vanasco
thanks for the writeup. i'll tinker with some ideas in my downtime. for now, i'll stick with the secondary , non-Foo, query I have that pulls that bar/baz stuff. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and

Re: [sqlalchemy] Logging query in a single line

2014-04-23 Thread Jonathan Vanasco
the easiest trick i came up with for this, was to create a logging table in sqlalchemy/postgres and directly log the query into it. my table looks roughly like this: id , timestamp, request_id ( if under pyramid ), query_id (if available, guid assigned to different queries ), query, vars,

Re: [sqlalchemy] celery and race conditions

2014-04-15 Thread Jonathan Vanasco
if i have any time after shipping , i'll probably build in transaction support for celery and pyramid. I keep away from tossing ORM objects around the system. GETS are pretty cheap. my task arguments are generally: int = primary key of ORM object dict = instructions payload of what

Re: [sqlalchemy] defining a relationship with IS NOT

2014-04-15 Thread Jonathan Vanasco
i'll try this later. i ended up 'monkeypatching' all these relationships onto the classes at the end of my models.py file , using the non-string column syntax. would have preferred to keep the entire class definition, together.. but I documented everything. -- You received this message

[sqlalchemy] celery and race conditions

2014-04-14 Thread Jonathan Vanasco
I just ran into an issue where it looks like I could have race conditions using SqlAlchemy and Celery. Wondering if anyone here has some ideas. Here's the scenario: A1 Process A - Pyramid - Creates SQLalchemy session. A2 Process A - Pyramid - Creates data A3 Process A - Pyramid - flushes data

Re: [sqlalchemy] celery and race conditions

2014-04-14 Thread Jonathan Vanasco
i've got that now as a stopgap; i was hoping someone has better ideas. i don't like the idea of a post-commit hook, because i fear requesting the celery task request will create an error. I really don't want to build `transaction` support for celery, but i might need to. -- You received

[sqlalchemy] defining a relationship with IS NOT

2014-04-14 Thread Jonathan Vanasco
I'm on postgres and have a boolean column that allows NULL values. I need to create a relationship between 2 ORM classes , where there is a filter that states IS NOT TRUE. The ORM likes these 2 commands : photo = sa.orm.relationship(Photo, primaryjoin=and_( Useraccount.photo_id==Photo.id

[sqlalchemy] Re: database design question

2014-04-08 Thread Jonathan Vanasco
I had a similar situation years ago. We had software that helped automate online promotions for music releases. Everyone insisted on keeping their data separate; forcing different databases was required by contract and we were strong-armed into it. Today, things would be different. I ended

[sqlalchemy] is it possible to joinedload/subqueryload a single column ?

2014-04-04 Thread Jonathan Vanasco
I have these 2 tables in the database: class TableA(base): id field_a field_b items = sa.orm.relationship(TableA_Items, primaryjoin=TableA.id== TableA_Items.table_a_id, backref=table_a) class TableA_Items(base): id table_a_id

Re: [sqlalchemy] is it possible to joinedload/subqueryload a single column ?

2014-04-04 Thread Jonathan Vanasco
thank you !! -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to

[sqlalchemy] is it possible to load two relationships during a subquery or similar ?

2014-04-04 Thread Jonathan Vanasco
sorry for overload the list with random questions. i'm trying to get a release out the door and dealing with performance bottlenecks... given this setup: class Items2Attributes(base): id item_id condition_id attribute_id class Items(base): id

Re: [sqlalchemy] is it possible to load two relationships during a subquery or similar ?

2014-04-04 Thread Jonathan Vanasco
On Friday, April 4, 2014 8:11:11 PM UTC-4, Michael Bayer wrote: yeah, make one relationship and just filter them with a @property for each of “a” and “b”. if the object is usually used such that both collections are needed, that’s the approach. brilliant. thanks! -- You received

[sqlalchemy] is it possible to eagerload a specific collection on a loaded object ?

2014-04-02 Thread Jonathan Vanasco
I'm calling this eagerloading , but it's not the right term... I have an object that is already loaded... user = dbSession.query( model.User ).filter_by( id = 1 ).first() over time it's seen a handful of relationships grow... user = dbSession.query( model.User ).filter_by( id = 1 )\

Re: [sqlalchemy] is it possible to eagerload a specific collection on a loaded object ?

2014-04-02 Thread Jonathan Vanasco
I'd like to avoid that, if at all possible. that was actually a solution to an earlier thing i had a few weeks ago! I'm fine with doing suqueryloads instead of joinedloads or hitting the database multiple times. with my current design... building another query where i need to specify loads

[sqlalchemy] Re: Session execute mapping

2014-04-01 Thread Jonathan Vanasco
If you have: class MyTable(base): pass You could do session.query( MyTable ).from_statement() * http://docs.sqlalchemy.org/en/rel_0_9/orm/query.html#sqlalchemy.orm.query.Query.from_statement See also: http://docs.sqlalchemy.org/en/rel_0_9/core/tutorial.html#using-text

[sqlalchemy] Re: Mapping lots of similar tables / database design

2014-04-01 Thread Jonathan Vanasco
ah! i didn't catch this before... backref_name = timeseries_%s % name ## might be better as an the id of the location location = relationship('Location', backref=backref(backref_name, lazy='dynamic')) I don't think he can do one table. originally i did, but I looked into some

[sqlalchemy] Re: Bitwise Flag Type

2014-03-31 Thread Jonathan Vanasco
I'm interested in what you find. I know TypeDecorator is the right solution, I was looking at that for this exact same situation a few weeks ago ( https://groups.google.com/forum/#!searchin/sqlalchemy/vanasco%7Csort:date/sqlalchemy/sQtOYxSUiqI/5ns2vWMFaGAJ ) I have a similar situation. I

[sqlalchemy] Is it possible to change the Select of a query?

2014-03-30 Thread Jonathan Vanasco
I have a handful of queries that are fairly complex. For these, I need to support both SELECT object.* ( for full records ) SELECT object.id ( for caching of collection ids + COUNT operations ) My current approach is really dirty. three functions wrap a core generator, which

Re: [sqlalchemy] Is it possible to change the Select of a query?

2014-03-30 Thread Jonathan Vanasco
No. I'm want to edit the target of the 'select' from an existing query. For many select operations, I need three variations - the full Objects - just the ObjectIds - a count of the objects ( which is sometimes faster as an explicit select obj.id than a select obj , so i can influence the pg

Re: [sqlalchemy] Best sql platform to use with sqlalchemy on a limited shared network

2014-03-28 Thread Jonathan Vanasco
If you can only use FTP, then your options is probably just to write a program that downloads the sqlite via ftp, does the queries, then uploads. that's going to be messy with race conditions though, so you'd have to create some sort of system where the file is renamed/hidden, downloaded,

[sqlalchemy] Re: Definition for scalar relationship?

2014-03-25 Thread Jonathan Vanasco
collection = one to many / many to many. scalar relationship = one-to-one or many-to-one relationship. when defining a relationship, you pass in 'uselist=False' to create define the relationship as scalar -- You received this message because you are subscribed to the Google Groups

[sqlalchemy] Re: Bulk Inserts and Unique Constraints

2014-03-24 Thread Jonathan Vanasco
The data comes in unordered and sometimes contains duplicates, so there's a UniqueConstraint on Entry on sub, division, created. Have you tried pre-processing the list first ? I've had similar situations, when dealing with browser , user and app analytics. I normally do a first pass to

[sqlalchemy] Re: What is the rationale of having to manually set up a relationship between two tables?

2014-03-24 Thread Jonathan Vanasco
That's a really simple use case -- you want the primary key to be a 1:1 relationship, and you don't have potentially competing fields. But this gets complicated when: - You want User.addresses to be a single item ( User.address ) - You don't want User.addresses to just be all the addresses --

[sqlalchemy] Re: Bulk Inserts and Unique Constraints

2014-03-24 Thread Jonathan Vanasco
Since you're using Postgres... have you considered using python to generate a COPY file ? Sqlalchemy doesn't seem to support it natively... maybe via 'text', but your underlying psycopg2 driver does. it's way way way faster. i've found it significantly faster than dropping fkeys and using

[sqlalchemy] troubleshooting mapper error

2014-03-17 Thread Jonathan Vanasco
i'm updating my db code, and this popped up: InvalidRequestError: One or more mappers failed to initialize - can't proceed with initialization of other mappers. Original exception was: id() takes exactly one argument is there any way to tell where/how this got triggered? There's nothing

Re: [sqlalchemy] troubleshooting mapper error

2014-03-17 Thread Jonathan Vanasco
thanks. i finally found it. i was querying TableA TableA mapped somewhere to TableB Table B had a backref on an 'id' field. that field got removed. the query on TableA triggered the error , even though I wasn't loading the 'id' field. because of how the backref was constructed, it was

Re: [sqlalchemy] sql expression performance is bad ?

2014-03-14 Thread Jonathan Vanasco
You would probably do better with a pattern where you have a Session for every request , and just use that session. That is how most people implement SqlAlchemy for web. * request start * create a sqlalchemy session, either scoped or explicit * do things with your session : read ,

Re: [sqlalchemy] sql expression performance is bad ?

2014-03-13 Thread Jonathan Vanasco
Before writing stored procedures, you might want to test speeds like : 1. raw speed within the mysql client for these queries. 2. speed of pushing these queries through your python dbapi driver ( ie, mysqldb ) those should give you a better idea on where the potential speed problems, if any

Re: [sqlalchemy] tracking history across a 'flush()'

2014-03-10 Thread Jonathan Vanasco
ok. i guess i'll experiment with listeners to see if I can grab and store the original object into a dict on load, and then compare against that. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving

[sqlalchemy] Re: databases and asynchronous programming

2014-03-09 Thread Jonathan Vanasco
I'm using twisted and looking to move raw SQL ( via twisted's `runInteraction` ) into my SqlAlchemy model. I've only played with it a bit. A few notes: Under Twisted you can't have threads spawned threads. Your computer will want to die. I did and crashed the server several times,

[sqlalchemy] tracking history across a 'flush()'

2014-03-09 Thread Jonathan Vanasco
I have some lightweight revision tracking on some models. I generate a diff based on the history of the object (via inspector). a limitation I just realized, is that this history only dates back to the most recent flush() -- it doesn't date back to the initial load. are there any existing

Re: [sqlalchemy] tying multiple sessions together with one transaction

2014-03-03 Thread Jonathan Vanasco
On Monday, March 3, 2014 5:26:29 AM UTC-5, Simon King wrote: Partially innocent question: If one of the flushes you describe above fails (say with an integrity error), then the transactions would be left in need of a rollback on all engines and definitely no data would be

[sqlalchemy] possible bug in .get()

2014-02-28 Thread Jonathan Vanasco
If this behavior is a bug , I can build out a testcase. FWIW , I think it's a bug. class Foo : id = int primary key bar = relationship(bar) baz = relationship(baz) # hits the db a = session.query( Foo ).get(1) # doesn't hit the db b =

Re: [sqlalchemy] possible bug in .get()

2014-02-28 Thread Jonathan Vanasco
On Friday, February 28, 2014 12:41:46 PM UTC-5, Michael Bayer wrote: nope. get() means, “the SELECT may not happen”. eager loads are always secondary to the fact that the row is being loaded. if you want to definitely load the row, say filter_by(id…).first(). Poking at the source

Re: [sqlalchemy] possible bug in .get()

2014-02-28 Thread Jonathan Vanasco
On Friday, February 28, 2014 2:33:11 PM UTC-5, Michael Bayer wrote: OK well it has to emit the SELECT for the primary row, but yes populate_existing will overwrite, so really, just any equivalent of query(User).filter_by(primary key).first() is really all you need here…we used to have a

Re: [sqlalchemy] Update a parameter based on input in sqlalchemy

2014-02-25 Thread Jonathan Vanasco
On Tuesday, February 25, 2014 5:07:11 AM UTC-5, Simon King wrote: However, Python has a handy shortcut for passing arbitrary keyword arguments to a function: status_args = {trans_status: 1} status = Status(**status_args) You could also do that in one line, but i'm sure it violates

Re: [sqlalchemy] tying multiple sessions together with one transaction

2014-02-25 Thread Jonathan Vanasco
If you do need to use zope.transaction ( and not just the SqlAlchemy option that Simon posted ) , there already exists Zope/SqlAlchemy integration -- https://pypi.python.org/pypi/zope.sqlalchemy -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To

Re: [sqlalchemy] Could anyone please help explain expired object in plain language?

2014-02-24 Thread Jonathan Vanasco
To expand and clarify Simon's answer into bulletpoints : Expired Object * The database Session the object belongs to has been closed ( or committed unless you tweaked the config ) * SqlAlchemy considers it 'expired' because it is not reasonable to expect the object to reflect the current state

[sqlalchemy] ORM question - when are changes to joined collections enacted ?

2014-02-24 Thread Jonathan Vanasco
quick question on the definitive behavior for the orm I'm have some code that reassigns objects from one owner to another. instead of objects being assigned directly to an owner, they're assigned into an intermediary Library, which is then assigned to an Owner i also keep the owner_Id on the

Re: [sqlalchemy] ORM question - when are changes to joined collections enacted ?

2014-02-24 Thread Jonathan Vanasco
Wonderful. Thanks a ton. Seriously. Thanks for answering all these super-simple questions. I'm doing some code-audits and last-minute prep for release. There have been a handful of simple things that just seemed to work ( this, the dogpile cache, etc ) , and I want to confirm that

Re: [sqlalchemy] Possible bug in MySQLDialect._check_unicode_returns() (missing optional parameter) (0.9.3 )

2014-02-21 Thread Jonathan Vanasco
I'd suggest recompiling mysqldb I've had weird/odd issues happen when either Python or MySQL/PostgreSQL were updated and the driver wasn't. Even for minor updates ( Python 2.7.4 - 2.7.5 ; Postgres 9.3 - 9.4, etc ), the compiled c-extension will often cause random errors if anything that

[sqlalchemy] recommendation sought - implementing custom column type

2014-02-21 Thread Jonathan Vanasco
i currently have a table that looks like this: class MyTable(DeclaredTable): id = sa.Column(sa.Integer, primary_key=True) features_enabled = sa.Column(sa.Integer, nullable=False, default=0, ) _feature_manager = None @property def

Re: [sqlalchemy] filter vs get question

2014-02-20 Thread Jonathan Vanasco
that's fine. this is for a webapp where we have in a single request : begin; user = .filter().first() DO LOTS OF STUFF, all over the place DO EVEN MORE STUFF , in more places user = .get(user_id) commit; if this behavior is intended, then we can just rely on it for now.

Re: [sqlalchemy] Re: SQLAlchemy 0.9.2 UnicodeEncodeErrors with PG 9.3.2 on Amazon RDS

2014-02-19 Thread Jonathan Vanasco
Wow! Thanks. This is by far, the best written and more informative stuff I've read on this. I've been considering switching to AWS sooner-than-later for the RDS ( we bootstrapped on Linode with a plan to redeploy on AWS in the future ) , but it looks like we shouldn't pursue that yet! On

[sqlalchemy] Re: SQLAlchemy 0.9.2 UnicodeEncodeErrors with PG 9.3.2 on Amazon RDS

2014-02-18 Thread Jonathan Vanasco
I've had problems with this in the past. I thought it was SqlAlchemy, and it ended up being issues with my data in Pyramid (the Pylons successor). A few items had the wrong character encoding -- I was expecting it to be UTF8/ASCII, but it was using a different charset. It could be the

[sqlalchemy] Re: Twisted + SQLAlchemy

2014-02-14 Thread Jonathan Vanasco
I don't know if any of this will help - I'm too tired to read though all your code: 1. Twisted isn't threadsafe, so anything happening in deferToThread needs to be entirely encapsulated. This has totally screwed me up many times. You should avoid a scoped_session , as it has to do with

Re: [sqlalchemy] override relationship in subclass

2014-02-12 Thread Jonathan Vanasco
Couldn't it be handled with a mixin? {{{ class _LoadCore(Base): whatever you want for both classes here pass class Load(_LoadCore): __tablename__ = 'load' __mapper_args__ = { 'polymorphic_identity':'load', 'polymorphic_on':'polymorphic_type', } id =

Re: [sqlalchemy] prepared statements

2014-02-06 Thread Jonathan Vanasco
just wondering -- would it be possible to mimic this behavior using a custom view for this select ( in postgresql ) and then querying that ? i think the query planner might only run on the creation. -- You received this message because you are subscribed to the Google Groups sqlalchemy

Re: [sqlalchemy] prepared statements

2014-02-06 Thread Jonathan Vanasco
On Thursday, February 6, 2014 2:18:51 PM UTC-5, Klauss wrote: I think views don't cache the plan, they're handled as rules. What you'd need is a function (pg's version of stored procedures). I had time to look it up; this generally seems correct. Looking at some explain syntax, it seems

Re: [sqlalchemy] MySQL's sql_mode (ORM)

2014-02-05 Thread Jonathan Vanasco
you can pass custom connect arguments to the DBAPI in `create_engine`. http://docs.sqlalchemy.org/en/rel_0_9/faq.html?#how-do-i-pass-custom-connect-arguments-to-my-database-api depending on which driver you use, it will be different. you shouldn't use `text`, because sqlalchemy

Re: [sqlalchemy] MySQL's sql_mode (ORM)

2014-02-05 Thread Jonathan Vanasco
just to clarify... mysqldb - `connect` accepts a `sql_mode` string: http://mysql-python.sourceforge.net/MySQLdb.html pymysql accepts it too https://github.com/PyMySQL/PyMySQL/blob/3576863f9cd0b66ce6c8b32ab3448ab68f55f489/pymysql/connections.py oursql: i couldn't find it

[sqlalchemy] Re: instance is not bound to a session

2014-02-05 Thread Jonathan Vanasco
When you called 'commit', the 'account' object got expired. There are a few ways in the FAQ or Wiki to deal with this. IIRC, you can `merge` objects back into the session or `refresh` the object. One or both might hit the database. It would probably be better to change your code a bit, so

[sqlalchemy] Re: Using SQ with Django models

2014-02-01 Thread Jonathan Vanasco
Check out this project: https://pypi.python.org/pypi/sqlacodegen it's a replacement to sqlautocode the packages are designed to inspect your database and create sqlalchemy models ( in python code) for you. you'll have to edit/audit the generated code -- but it will save A LOT of time. --

[sqlalchemy] Re: group_by and lazy=False relationship problem

2014-01-31 Thread Jonathan Vanasco
i believe you need to be explicit on your group_by : group_by(P.id) or group_by(P.attrs) or group_by(P.id,P.attrs) -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it,

[sqlalchemy] Re: Sqlalchemy ORM operates on database not created from sqlalchemy

2014-01-23 Thread Jonathan Vanasco
Just to rephrase everyone else's It doesn't matter response: Many ORMs out there REQUIRE the database be built by the ORM or designed by the ORM. This is because the ORM stores and accesses data in a very particular manner -- so tables and columns must adhere to certain naming conventions,

[sqlalchemy] Re: Relationship with complicated join conditions

2014-01-22 Thread Jonathan Vanasco
Would an association proxy help ? http://docs.sqlalchemy.org/en/rel_0_9/orm/extensions/associationproxy.html -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to

[sqlalchemy] was there ever any work done on minifying queries ?

2014-01-20 Thread Jonathan Vanasco
i'm just wondering for performance over-the-wire from the bandwidth alone... I see great, DBA toubleshooting friendly stuff like: SELECT table_one.column_one AS table_one_column_one FROM table_one ; but that could be... SELECT t1.column_1 AS t1_column_1 FROM table_one t1; or even

Re: [sqlalchemy] was there ever any work done on minifying queries ?

2014-01-20 Thread Jonathan Vanasco
so wow. that's really neat. amazingly neat. and yeah, i'm just looking for ways to get some more performance out of a few boxes , so we don't have to add another box. trying to cut the fat here and there -- and noticed some very verbose sql. wondering if losing it will get 1-2% more out of

Re: [sqlalchemy] was there ever any work done on minifying queries ?

2014-01-20 Thread Jonathan Vanasco
sorry , should have been more clear. i'm trying to get some more juice out of of the database server. it is streaming sql nonstop. the webservers are doing fine, and are simple to cluster out. -- You received this message because you are subscribed to the Google Groups sqlalchemy group.

Re: [sqlalchemy] was there ever any work done on minifying queries ?

2014-01-20 Thread Jonathan Vanasco
i couldn't find anything on trac earlier for `coerce_from_config` i'd be happy to whip up a patch for the .8 branch that updates coerce_from_config to support all the create_engine kwargs. i have no idea how to do that for the .9 branch though. my sql is pretty optimized as is, with

Re: [sqlalchemy] Custom logic for query execution

2014-01-13 Thread Jonathan Vanasco
if the pessimistic listener doesn't work, maybe he could do something like this: * create an event listener to grab the sql executes * wrap the execute in try/except with a savepoint savepoint() while True : try: execute() break except: rollback() fails

[sqlalchemy] docs - contextmanager

2014-01-13 Thread Jonathan Vanasco
i needed to do some savepoint work in sqlalchemy, and remembered mike directed me to a FAQ or example with a contextmanager syntax: with session.begin_nested(): pass i jumped on the sqlalchemy docs, and couldn't find that at all. the orm docs (

[sqlalchemy] Re: SQLAlchemy 0.9.1 released

2014-01-06 Thread Jonathan Vanasco
automap sounds neat! thanks! -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to

[sqlalchemy] Re: Read-only session?

2013-12-16 Thread Jonathan Vanasco
You can also lock down the database to read-only -- * create Read-Only database users/privileges if you have an authenticated database * if you're on something like sqlite, you can run your process as different users -- so one can access the DB for read and other for write. -- You received

Re: [sqlalchemy] Anybody have twophase/zope.sqlalchemy/MySQL working?

2013-12-16 Thread Jonathan Vanasco
On Saturday, December 14, 2013 9:58:14 PM UTC-5, Jeff Dairiki wrote: Oh my god. I'm just going to close that page, and try to forget I ever saw that... On quick look though, none of those modes seem to have anything to do with transactions (thankfully). Yeah, it's pretty evil. I've

Re: [sqlalchemy] Anybody have twophase/zope.sqlalchemy/MySQL working?

2013-12-13 Thread Jonathan Vanasco
just a thought-- two things: 1- The thing is, for me, if the session has only been used for read operation, self.tx seems to be None. So the datamanager never commits anything. Check out Tres Seaver's reply here: https://groups.google.com/d/msg/pylons-discuss/R4S-UwHV6ww/ekD7M9UEvp8J

Re: [sqlalchemy] multiple databases?

2013-12-11 Thread Jonathan Vanasco
richard - how are you handle the scoping and management of sessions ? i'm wanting (badly) to convert some twisted code that is using raw sql through their db api onto the sqlalchemy model for my core app. -- You received this message because you are subscribed to the Google Groups sqlalchemy

[sqlalchemy] Re: Parallel connection attempts in QueuePool

2013-12-06 Thread Jonathan Vanasco
On Friday, December 6, 2013 10:16:21 AM UTC-5, Sunil Adapa wrote: Before making a connection attempt the overflow counter lock is obtained and it is being released only after the connection either succeeds or fails. In my case, a connection remained hung possibly because of a surge in

[sqlalchemy] Re: Many2many, referential integrity and introspection

2013-12-06 Thread Jonathan Vanasco
can you share your existing schema for these relations ? -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this

[sqlalchemy] Re: Functions sometimes return decimal or float

2013-12-06 Thread Jonathan Vanasco
what about using custom compiler functions ( http://docs.sqlalchemy.org/en/rel_0_9/core/compiler.html#utc-timestamp-function ) ? you could make a custom function... sum_integer() and just call that instead of the default sqlalchemy option -- You received this message because you are

Re: [sqlalchemy] Re: Functions sometimes return decimal or float

2013-12-06 Thread Jonathan Vanasco
I just learned all about the Events model. You could potentially write a listener that can filter the data for you. I'm not very familiar with the events though, and it could be a bit difficult to do -- as some items would be functions, others column operations, and others from 'text'

[sqlalchemy] Re: postgres schema per model

2013-12-05 Thread Jonathan Vanasco
this is a really great idea! thanks for asking this question. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to

[sqlalchemy] question about race conditions

2013-12-05 Thread Jonathan Vanasco
i'm looking at moving some raw sql in twisted to SqlAlchemy and have a question. I have a multi-threaded twisted daemon that tends to generate a lot of race conditions on a few tables that are frequently hit. I get integrity errors from something like this : domain = SELECT * FROM

Re: [sqlalchemy] question about race conditions

2013-12-05 Thread Jonathan Vanasco
oh that's great - I didn't expect SqlAlchemy to aggregate/support the different driver errors like that! thanks so much, Michael! -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it,

Re: [sqlalchemy] full outer join?

2013-12-02 Thread Jonathan Vanasco
On Monday, December 2, 2013 2:46:20 PM UTC-5, Michael Bayer wrote: well the alternative to full outer join is a union of two outer joins - more tedious.“full outer join” isn’t in the library but you can just subclass Join and do a @compiles to get that exact syntax. you could also

Re: [sqlalchemy] Can sqlalchemy help me solve this contraint issue?

2013-11-27 Thread Jonathan Vanasco
I don't like this schema. 1. It makes more sense to have an 'contact_id' as a column on Company Account , than vice versa. A contact is a representative for those entities, not the other way around. I think you're running into issues, because you seem to be trying to adapt this real-world

Re: [sqlalchemy] Filter options

2013-11-25 Thread Jonathan Vanasco
On Sunday, November 24, 2013 5:56:29 PM UTC-5, Joseph Casale wrote: On Sunday, November 24, 2013 3:44:32 PM UTC-7, Michael Bayer wrote conditonals? Yeah, the queries are just so long and with all the combinations of possible criteria it would get out of hand as I have about 6 optional

[sqlalchemy] Re: sqlalchemy.util.queue.Empty

2013-11-25 Thread Jonathan Vanasco
for what it's worth... usually I see errors in psycopg and other c-extended python libraries when the python and/or postgres versions got upgraded. it can get tricky when you're using a virtualenv, because you need to rebuild the entire virtualenv - and sometimes people forget that. ( i know

Re: [sqlalchemy] [wishlist] ORM/engine Feature Request - comment()

2013-11-19 Thread Jonathan Vanasco
On Monday, November 18, 2013 8:40:13 PM UTC-5, Michael Bayer wrote: there’s a recipe for doing this with events: http://www.sqlalchemy.org/trac/wiki/UsageRecipes/SessionModifiedSQL Oh wow. This is great. Playing with it now. I still like my idea for syntax though , it's simpler and

Re: [sqlalchemy] [wishlist] ORM/engine Feature Request - comment()

2013-11-19 Thread Jonathan Vanasco
Thats a great idea. I'll tackle that next. I'm having a bit of trouble adapting the recipe to my existing [pyramid] application. I'm not seeing a .info property on my sessions in `_connection_for_session`. I am using scoped sessions, and the recipe does work fine on this machine /

Re: [sqlalchemy] [wishlist] ORM/engine Feature Request - comment()

2013-11-19 Thread Jonathan Vanasco
how stable is 0.9 ? is it fairly safe to switch to now ? -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this

Re: [sqlalchemy] [wishlist] ORM/engine Feature Request - comment()

2013-11-19 Thread Jonathan Vanasco
screw it. updated to 0.9. everything works as-needed within 5 minutes. happy. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to

Re: [sqlalchemy] HSTORE Column Update

2013-11-18 Thread Jonathan Vanasco
is the HSTORE column inheriting from MutableDIct ? -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group,

Re: [sqlalchemy] HSTORE Column Update

2013-11-18 Thread Jonathan Vanasco
forgot to add -- this creates the following SQL: UPDATE user_data SET prefs=(user_data.prefs || hstore(ARRAY[%(param_1)s, %(param_2)s])) WHERE user_data.prefs ? %(prefs_1)s {'prefs_1': 'inboundBusStop', 'param_1': 'transitAvaiable', 'param_2': 'true'} Note that it's using one of the htsore

Re: [sqlalchemy] HSTORE Column Update

2013-11-18 Thread Jonathan Vanasco
i remember getting annoyed with this a while back. I fought with SqlAlchemy and Postgresql to get the right SQL generated ( Postgres had a few bad lines in the docs that confused things ). I ended up NOT being able to get this to work using the ORM. However I found a workaround using the

Re: [sqlalchemy] HSTORE Column Update

2013-11-18 Thread Jonathan Vanasco
On Monday, November 18, 2013 1:46:18 PM UTC-5, Michael Bayer wrote: s.query(A).update({A.data: A.data + {foo: bat, hoho: lala}}, synchronize_session=fetch) ... the UPDATE looks like: UPDATE a SET data=(a.data || %(data_1)s) {'data_1': {'foo': 'bat', 'hoho': 'lala’}} that's a whole

[sqlalchemy] anyone have strategies for limiting queries of related objects to specific column(s) ?

2013-11-18 Thread Jonathan Vanasco
this is just wild thinking -- and might not be possible. I'm wondering if anyone has some ideas on creating a relationship where only certain columns are queried during relationship loading. I have a few relationships like this ( pseudocode ) class Tag2Posting: tag_id INT

[sqlalchemy] [wishlist] ORM/engine Feature Request - comment()

2013-11-18 Thread Jonathan Vanasco
Something that I realized would be very useful, is to enable comments on query compilation. That would make it SO much easier to understand logs. Yes this sounds slightly silly. Yes I am 100% serious. I've been working all day on optimizing a web page. It originally had 200 queries,

Re: [sqlalchemy] [wishlist] ORM/engine Feature Request - comment()

2013-11-18 Thread Jonathan Vanasco
Wouldn't selecting the sql logger and logging the comment provide you the same information? You'd only be able to push a comment into Python's log, but not into Postgres/Mysql/Oracle/etc. placing a comment into the sql allows one to grep the database logs, or use the database profiling.

Re: [sqlalchemy] using Column.in_(a_large_list) is very slow.

2013-11-16 Thread Jonathan Vanasco
Well the queries are totally different... SELECT * FROM test WHERE id IN (SELECT test.id FROM test LIMIT 10) That's 100% in your database, and a single executable. SELECT * FROM test WHERE id IN ( 1...10 ) You have SQL Alchemy generating that query all the bind params, then

[sqlalchemy] Re: How can I join to a query/subquery column?

2013-11-06 Thread Jonathan Vanasco
You can do something like: q1 = session.query(\ ContactInfo.account_id.label(q1_account_id ), ContactInfo.other_id.label(q1_otherid ) )\ .filter(..) q2 = session.quiery(Site.account_id.label(q2_account_id)).filter(..) _q3 =

[sqlalchemy] Re: How can I join to a query/subquery column?

2013-11-06 Thread Jonathan Vanasco
sidenote: there are typos. that's just to illustrate. i mostly copied that from production code. the core concepts are: 1. `label()` the columns 2. use the `engine` (not orm) to join the queries 3. `alias` the union to search against them 4. the labeled columns in the earlier queries are now

[sqlalchemy] Re: Speed up bulk inserts

2013-11-06 Thread Jonathan Vanasco
In my experience, you need to write a script that you can configure as-needed. Depending on the types of inserts, I've had different performances. * I always set up batches to be configurable ; I start at 100, then try to go up down * I set toggles/thresholds on when I `flush`. After every

[sqlalchemy] Re: How can I join to a query/subquery column?

2013-11-06 Thread Jonathan Vanasco
i think the call to alias ( making it a subquery ) dropped the support for all in my example; though that was written in .8.1 -- might be working since .8.3 or 9 it took me a bit of trial and error to get stuff working. awesome that it's working for you so quickly! -- You received this

[sqlalchemy] Re: Immediate access to value of a new object's Sequence primary key column?

2013-11-06 Thread Jonathan Vanasco
You missed adding the object to the session, before the flush. this should work: new_foo = Foo('red') dbSession.add(new_foo) dbSession.flush() print new_foo.id -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from

<    4   5   6   7   8   9   10   11   >