Re: [sqlalchemy] style question: correct practice for creating relationship ?
thanks, michael. i'll look into that! -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
Re: [sqlalchemy] style question: correct practice for creating relationship ?
On Wed, Nov 29, 2017 at 3:32 PM, Jonathan Vanascowrote: > I have a potential relationship between Foo and Bar that is defined as > follows, and has usually been read-only from legacy data. > > All the code works, I just want to make sure I'm handling this case in a > clear and concise way. > > I've (over)simplified the case down to this: > > class Foo(Base): > __tablename__ = 'foo' > id = Column(Integer, primary_key=True) > name = Column(Unicode(6), nullable=False) > > bar = relationship("Bar", >primaryjoin="Foo.id==Bar.id__foo", >uselist=False, >back_populates='foo', >) > > class Bar(Base): > __tablename__ = 'bar' > id = Column(Integer, primary_key=True) > name = Column(Unicode(6), nullable=False) > id__foo = Column(Integer, ForeignKey("foo.id"), nullable=True, ) > > foo = relationship("Foo", >primaryjoin="Bar.id__foo==Foo.id", >uselist=False, >back_populates='bar', >) > > > In a current feature ticket, I'm in a situation where I need to upgrade an > instance of Foo with an instance Bar and dealing with some get/create > functions that do their own flushes. That block of code basically looks > like this: > > def getcreate_bar(bar_name, foo=None): > b = s.query(Bar).filter(Bar.name==bar_name).first() > if not b: > b = Bar() > b.name = bar_name > b.id__foo = foo.id if foo else None > s.add(b) > s.flush(objects=[b, ]) > return b > > foo = s.query(Foo).get(1) > if not foo.bar: > bar = getcreate_bar(bar_name, foo) > foo.bar = bar # attribute needed for additional work > s.flush(objects=[foo, ]) # flush only needed if bar isn't new > > > my question covers where I create the `bar` and make it instantly available > as a foo attribute. Is there a better way to handle this or make it more > readable? One way would be to make a @property or hybrid that generates it, but maybe you want to try this new event: http://docs.sqlalchemy.org/en/latest/orm/events.html?highlight=init_scalar#sqlalchemy.orm.events.AttributeEvents.init_scalar it's probably only tested for scalar values but in theory should work for objects too. But you'd be testing that that actually works b.c. I'm not sure. If I create a new foo, the flush of `foo` doesn't emit SQL after > sqlalchemy decides it doesn't have to. I do need that flush if I already > had bar, but it wasn't associated to foo. > > > > -- > SQLAlchemy - > The Python SQL Toolkit and Object Relational Mapper > > http://www.sqlalchemy.org/ > > To post example code, please provide an MCVE: Minimal, Complete, and > Verifiable Example. See http://stackoverflow.com/help/mcve for a full > description. > --- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sqlalchemy+unsubscr...@googlegroups.com. > To post to this group, send email to sqlalchemy@googlegroups.com. > Visit this group at https://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/d/optout. -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
[sqlalchemy] style question: correct practice for creating relationship ?
I have a potential relationship between Foo and Bar that is defined as follows, and has usually been read-only from legacy data. All the code works, I just want to make sure I'm handling this case in a clear and concise way. I've (over)simplified the case down to this: class Foo(Base): __tablename__ = 'foo' id = Column(Integer, primary_key=True) name = Column(Unicode(6), nullable=False) bar = relationship("Bar", primaryjoin="Foo.id==Bar.id__foo", uselist=False, back_populates='foo', ) class Bar(Base): __tablename__ = 'bar' id = Column(Integer, primary_key=True) name = Column(Unicode(6), nullable=False) id__foo = Column(Integer, ForeignKey("foo.id"), nullable=True, ) foo = relationship("Foo", primaryjoin="Bar.id__foo==Foo.id", uselist=False, back_populates='bar', ) In a current feature ticket, I'm in a situation where I need to upgrade an instance of Foo with an instance Bar and dealing with some get/create functions that do their own flushes. That block of code basically looks like this: def getcreate_bar(bar_name, foo=None): b = s.query(Bar).filter(Bar.name==bar_name).first() if not b: b = Bar() b.name = bar_name b.id__foo = foo.id if foo else None s.add(b) s.flush(objects=[b, ]) return b foo = s.query(Foo).get(1) if not foo.bar: bar = getcreate_bar(bar_name, foo) foo.bar = bar # attribute needed for additional work s.flush(objects=[foo, ]) # flush only needed if bar isn't new my question covers where I create the `bar` and make it instantly available as a foo attribute. Is there a better way to handle this or make it more readable? If I create a new foo, the flush of `foo` doesn't emit SQL after sqlalchemy decides it doesn't have to. I do need that flush if I already had bar, but it wasn't associated to foo. -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
Re: [sqlalchemy] Dictionaries with mapped objects as keys and integers as values
On Wed, Nov 29, 2017 at 11:45 AM, Svenwrote: > Hello everybody, > > Is it possible to map dictionaries whose keys are objects and the values > simple integers? > > I have the following case : > > In the program, there is one instance of "Options" which contains a > dictionary. This dictionary has players as keys and integers as values. > These integers represents the options of the player used as a key. > > from sqlalchemy.ext.declarative import declarative_base > from sqlalchemy import Column, Integer > > Base = declarative_base() > > class Options(Base): > > __tablename__ = "options" > id = Column(Integer, primary_key=True) > > def __init__(self): > self.options = {} # Player -> Integer > > def set_options(self, player, value): > self.options[player] = value > > class Player(Base): > > __tablename__ = "players" > id = Column(Integer, primary_key=True) > > > opt = Options() > > john = Player() > jack = Player() > > opt.set_options(john, 2) > opt.set_options(jack, 5) > > print(opt.options) > > > Display : > > {<__main__.Player object at 0x05611908>: 5, <__main__.Player object > at 0x05611860>: 2} > > Of course, in this particular case, it doesn't make a lot of sense and it > could be designed in another way. It is just an example. I have a lot of > dictionaries with objects as keys in my project and I have no idea how I > should map these... and curiously, I am not able to find any example on > Internet. > > I found in the SQLAlchemy documentation explanations related to > mapped_collection and it sounds to be a bit what I'm looking for. > >> sqlalchemy.orm.collections.mapped_collection(keyfunc) >> "A dictionary-based collection type with arbitrary keying." > > > http://docs.sqlalchemy.org/en/latest/orm/collections.html > > The Composite Association example seems also to be a good base to do what I > want : > > http://docs.sqlalchemy.org/en/latest/orm/extensions/associationproxy.html#composite-association-proxy that example might even be too complicated, but sure you need to use the association proxy whenever you want a collection of scalar (non-object) values. It looks like you have an association table implied so here is a demo of what you request: class Options(Base): __tablename__ = "options" id = Column(Integer, primary_key=True) def __init__(self): self.options = {} def set_options(self, player, value): self.options[player] = value player_assoc = relationship( "PlayerOptAssoc", collection_class=attribute_mapped_collection("player")) options = association_proxy( "player_assoc", "int_value", creator=lambda key, value: PlayerOptAssoc(player=key, int_value=value)) class PlayerOptAssoc(Base): __tablename__ = "player_opt_assoc" player_id = Column(ForeignKey('players.id'), primary_key=True) option_id = Column(ForeignKey('options.id'), primary_key=True) player = relationship("Player") int_value = Column(Integer) class Player(Base): __tablename__ = "players" id = Column(Integer, primary_key=True) opt = Options() john = Player() jack = Player() opt.set_options(john, 2) opt.set_options(jack, 5) print(opt.options) does the output: {<__main__.Player object at 0x7fbcb361f090>: 2, <__main__.Player object at 0x7fbcb361fe10>: 5} > > Do you have suggestions ? Is it even possible ? What would be the good > method to map that ? > > Thank you :-) > > -- > SQLAlchemy - > The Python SQL Toolkit and Object Relational Mapper > > http://www.sqlalchemy.org/ > > To post example code, please provide an MCVE: Minimal, Complete, and > Verifiable Example. See http://stackoverflow.com/help/mcve for a full > description. > --- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sqlalchemy+unsubscr...@googlegroups.com. > To post to this group, send email to sqlalchemy@googlegroups.com. > Visit this group at https://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/d/optout. -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
[sqlalchemy] Dictionaries with mapped objects as keys and integers as values
Hello everybody, Is it possible to map dictionaries whose keys are objects and the values simple integers? I have the following case : *In the program, there is one instance of "Options" which contains a dictionary. This dictionary has players as keys and integers as values. These integers represents the options of the player used as a key.* from sqlalchemy.ext.declarative import declarative_base from sqlalchemy import Column, Integer Base = declarative_base() class Options(Base): __tablename__ = "options" id = Column(Integer, primary_key=True) def __init__(self): self.options = {} # Player -> Integer def set_options(self, player, value): self.options[player] = value class Player(Base): __tablename__ = "players" id = Column(Integer, primary_key=True) opt = Options() john = Player() jack = Player() opt.set_options(john, 2) opt.set_options(jack, 5) print(opt.options) Display : >>> {<__main__.Player object at 0x05611908>: 5, <__main__.Player object at 0x05611860>: 2} >>> Of course, in this particular case, it doesn't make a lot of sense and it could be designed in another way. It is just an example. I have a lot of dictionaries with objects as keys in my project and I have no idea how I should map these... and curiously, I am not able to find any example on Internet. I found in the SQLAlchemy documentation explanations related to mapped_collection and it sounds to be a bit what I'm looking for. sqlalchemy.orm.collections.mapped_collection(keyfunc) > *"A dictionary-based collection type with arbitrary keying."* http://docs.sqlalchemy.org/en/latest/orm/collections.html The Composite Association example seems also to be a good base to do what I want : http://docs.sqlalchemy.org/en/latest/orm/extensions/associationproxy.html#composite-association-proxy Do you have suggestions ? Is it even possible ? What would be the good method to map that ? Thank you :-) -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
Re: [sqlalchemy] How about: Disabling relationship cascades on a per-object basis.
On Wed, Nov 29, 2017 at 11:14 AM, Michael Elsdörferwrote: > So, I've been a very happy user of SQLAlchemy over many years, though maybe > not very deeply. I certainly don't really feel qualified to contribute any > thoughts on it's design, but I did want to bring this up, since it's > something I have run into again and again. > > I'd say that in the majority of projects that involve SQLAlchemy, sooner or > later I find myself in a situation where I want to work with database > objects that I don't want to save to the session. It's just so damn > convenient to pass them around. > > Here are two examples off the top of my head: > > - A billing tool was supposed to generate a bunch of objects (line items and > so on); in preview mode, I want to use the same functions to calculate and > generate the objects, and serialize them to the browser, but I don't want to > save them. > > - In a system to check if the user has the permission to "add an object", I > really wanted to construct the object temporarily, unsaved, and send it to > the permission-layer, but without adding it to the session. > > Now often, what I do to solve this is what I think SQLAlchemy wants me to > do, if I understand correctly: Configure the cascades on the relationships > accordingly. But this has some problems: > > 1) It's *really* hard. Each time I find myself going back to the > documentation, and trying to figure out where stuff needs to be changed > (what does cascade_backrefs on a backref() mean again?) > > 2) It's error prone. It's easy to later pull in an object through some other > relationship; it's also hard to later figure out exactly why the cascades > where configured in the way that they are, and which code paths depend on > that particular cascade setting. Changing any cascade may easily cause side > effects that are not predictable. > > So thinking about it, changing the cascade settings on the relationship is > not really what I want to do. What I really want: The cascade to work in > most cases, *except this one time*. I find myself searching for a version of > `session.no_autoflush`. For example: > > with session.no_cascade: >order = Order() >order.account = current_account > > Since current_account is in the session, the order would ordinarily be > pulled into it, too. But the decorator could prevent that. you mean the context manager, sure. this is actually not a bad idea, at least in some contexts. I'm not sure folks would really understand this API very often, but you do and I know what you want, sure. I'd have no_cascade be more of a function so that it allows some configurabilty: with session.no_cascade("save-update", Order)with session.no_cascade("all"), something like that. you'd want to be able to limit it to the type of cascade and the type of object for it to be generally useful. it's *mostly* feasible sure, because the Session is the thing that actually invokes the cascade of things which occur. However some cascades don't occur until a flush later on, most notably "delete" and "delete-orphan" cascade, so im not sure this context manager is that useful for that kind of thing unless you were using it for a broader block of operations (hence the arguments I proposed). If you were to block off current_account from being added in the first place, this would be all you need. I know that save-update is the one you really want, and it's also the easiest to accomplish, so some thought would have to be put into how best to present such a feature. Perhaps some session.dont_cascade_adds or something more limited like that. not sure. > > Or maybe: > > from sqlalchemy import taint_no_cascade > order = Order() > taint_no_cascade(order) > order.account = current_account > > The whole thing is probably much more complicated than that, but I desire > this so frequently, I wanted to ask if it's feasable, or has been discussed > before. > > Thanks, > > Michael > > -- > SQLAlchemy - > The Python SQL Toolkit and Object Relational Mapper > > http://www.sqlalchemy.org/ > > To post example code, please provide an MCVE: Minimal, Complete, and > Verifiable Example. See http://stackoverflow.com/help/mcve for a full > description. > --- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sqlalchemy+unsubscr...@googlegroups.com. > To post to this group, send email to sqlalchemy@googlegroups.com. > Visit this group at https://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/d/optout. -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google
[sqlalchemy] How about: Disabling relationship cascades on a per-object basis.
So, I've been a very happy user of SQLAlchemy over many years, though maybe not very deeply. I certainly don't really feel qualified to contribute any thoughts on it's design, but I did want to bring this up, since it's something I have run into again and again. I'd say that in the majority of projects that involve SQLAlchemy, sooner or later I find myself in a situation where I want to work with database objects that I don't want to save to the session. It's just so damn convenient to pass them around. Here are two examples off the top of my head: - A billing tool was supposed to generate a bunch of objects (line items and so on); in preview mode, I want to use the same functions to calculate and generate the objects, and serialize them to the browser, but I don't want to save them. - In a system to check if the user has the permission to "add an object", I really wanted to construct the object temporarily, unsaved, and send it to the permission-layer, but without adding it to the session. Now often, what I do to solve this is what I think SQLAlchemy wants me to do, if I understand correctly: Configure the cascades on the relationships accordingly. But this has some problems: 1) It's *really* hard. Each time I find myself going back to the documentation, and trying to figure out where stuff needs to be changed (what does cascade_backrefs on a backref() mean again?) 2) It's error prone. It's easy to later pull in an object through some other relationship; it's also hard to later figure out exactly why the cascades where configured in the way that they are, and which code paths depend on that particular cascade setting. Changing any cascade may easily cause side effects that are not predictable. So thinking about it, changing the cascade settings on the relationship is not really what I want to do. What I really want: The cascade to work in most cases, *except this one time*. I find myself searching for a version of `session.no_autoflush`. For example: with session.no_cascade: order = Order() order.account = current_account Since current_account is in the session, the order would ordinarily be pulled into it, too. But the decorator could prevent that. Or maybe: from sqlalchemy import taint_no_cascade order = Order() taint_no_cascade(order) order.account = current_account The whole thing is probably much more complicated than that, but I desire this so frequently, I wanted to ask if it's feasable, or has been discussed before. Thanks, Michael -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
Re: [sqlalchemy] Possible Exceptions
On Wed, Nov 29, 2017 at 3:37 AM,wrote: > Hello everybody, > > I am writing some piece of code, which should not exit unforseen in the best > case, at least it should write to a log file what went wrong (e.g. Database > not reachable, etc). So I tried figuring out, which Exceptions would be > possibly thrown by SQLAlchemy: the exception classes are documented at: http://docs.sqlalchemy.org/en/latest/core/exceptions.html http://docs.sqlalchemy.org/en/latest/orm/exceptions.html i will note that while the ORM exceptions are linked from the front page of the docs, the Core ones are not, you need to go into the full table of contents to see it (http://docs.sqlalchemy.org/en/latest/contents.html). as to the exact codepath that can raise these, it depends on the exception. The docstrings for the exceptions themselves can of course have some additional detail as to what kinds of operations might raise them.Though generic ones like "InvalidRequestError" or "ArgumentError" are thrown in dozens of places wherever the library is asked to do something that doesn't make sense or function arguments that don't make sense are detected. > > The first thing I did was googling for how to find out which Exceptions > could be thrown in Python. The answers I found on Stack Overflow etc. were > like "You simply don't", "This is Python, dumbass, you can't predict which > are thrown so don't even try" or "Just write tests until you found them all" > and "Just catch the general Exception class." > So I tried looking at the SQLAlchemy Documentation to see if there is > something written about when something goes wrong, but still no luck. > Before I started digging into the code I thought I'd ask here first > > So is there any hint to know which Exceptions could be thrown by SQLAlchemy? > The error cases I could think of were mostly wrrors while connecting to the > database or having errors in queries. I would totally be willing to help > with documenting at a certain point but even for this I need to know if I > just did't find any documentation for this and if you consider this as > neccessary. I feel that it is neccessary for me not just to kill the process > with maybe a stack trace on stdout. > > > Cheers > > Lars Liedtke > > -- > SQLAlchemy - > The Python SQL Toolkit and Object Relational Mapper > > http://www.sqlalchemy.org/ > > To post example code, please provide an MCVE: Minimal, Complete, and > Verifiable Example. See http://stackoverflow.com/help/mcve for a full > description. > --- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sqlalchemy+unsubscr...@googlegroups.com. > To post to this group, send email to sqlalchemy@googlegroups.com. > Visit this group at https://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/d/optout. -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
[sqlalchemy] Possible Exceptions
Hello everybody, I am writing some piece of code, which should not exit unforseen in the best case, at least it should write to a log file what went wrong (e.g. Database not reachable, etc). So I tried figuring out, which Exceptions would be possibly thrown by SQLAlchemy: - The first thing I did was googling for how to find out which Exceptions could be thrown in Python. The answers I found on Stack Overflow etc. were like "You simply don't", "This is Python, dumbass, you can't predict which are thrown so don't even try" or "Just write tests until you found them all" and "Just catch the general Exception class." - So I tried looking at the SQLAlchemy Documentation to see if there is something written about when something goes wrong, but still no luck. - Before I started digging into the code I thought I'd ask here first So is there any hint to know which Exceptions could be thrown by SQLAlchemy? The error cases I could think of were mostly wrrors while connecting to the database or having errors in queries. I would totally be willing to help with documenting at a certain point but even for this I need to know if I just did't find any documentation for this and if you consider this as neccessary. I feel that it is neccessary for me not just to kill the process with maybe a stack trace on stdout. Cheers Lars Liedtke -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.