Re: [sqlalchemy] Checking approaches around parallel data import for records
On Wed, Apr 24, 2019 at 12:19 PM Markus Elfring wrote: > > > Why not report these problems to the cochinelle tool > > which you are trying to integrate ? > > I suggest to take another look at corresponding information sources. > https://systeme.lip6.fr/pipermail/cocci/2019-April/thread.html > https://github.com/coccinelle/coccinelle/issues wow, so I got off light then! good luck > > Regards, > Markus -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
Re: [sqlalchemy] Checking approaches around parallel data import for records
> Why not report these problems to the cochinelle tool > which you are trying to integrate ? I suggest to take another look at corresponding information sources. https://systeme.lip6.fr/pipermail/cocci/2019-April/thread.html https://github.com/coccinelle/coccinelle/issues Regards, Markus -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
Re: [sqlalchemy] Checking approaches around parallel data import for records
On Wed, Apr 24, 2019, 1:50 AM Markus Elfring wrote: > >> Other software architectures can support parallelisation better, > can't they? > > > > Can you clarify what you hope to achieve when you continue to make > statements > > of this form? I don't find them to be very constructive. > > I dared to point out that I stumbled on another software limitation. > Now I am also looking again for possible adjustments and extensions. > Why not report these problems to the cochinelle tool which you are trying to integrate ? > > > There are many software architectures available for your use > > I check a few of them once more. > > > > and you should choose the ones which work best for you. > > I hope to achieve more helpful evolution for affected software areas. > > Regards, > Markus > -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
[sqlalchemy] Re: What is the standard/best way to merge an object graph into the session?
Okay let me answer my own question. The problem is that my parent-child relationship does not have the delete-orphan cascade. So when I set the new children, the old child_2 loses its parent (as is expected, because it's no longer a child) and then there's an error because the DB has a not null constraint on the parent_pk coming from nullable=False. I guess the moral of the story is that parent_pk being non-nullable essentially requires delete-orphan. -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
[sqlalchemy] Re: What is the standard/best way to merge an object graph into the session?
Oooo the problem is not what I thought. The problem is that in my 'new data' there is no new_child_2. This is an expected case, as sometimes children disappear, so will update the post. -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
[sqlalchemy] What is the standard/best way to merge an object graph into the session?
I have a parent child relationship which I construct from a data feed. At
the time of constructing the object graph I don't have access to the
primary keys of the entities, so I build up the object graph by using the
relationship attributes. My understanding was that I could perform a
session.merge to get the new state of the whole object graph into the
database, but when I try do this I get an exception.
Sample code that reproduces the problem I encounter:
from sqlalchemy import Column, Integer, ForeignKey, create_engine, String
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import scoped_session, sessionmaker, relationship
Base = declarative_base()
PARENT_PK = 5
CHILD_1_PK = 6
CHILD_2_PK = 7
class Parent(Base):
__tablename__ = 'parent'
pk = Column(Integer, primary_key=True)
data = Column(String)
children = relationship(
'Child',
back_populates='parent'
)
class Child(Base):
__tablename__ = 'child'
pk = Column(Integer, primary_key=True)
parent_pk = Column(Integer, ForeignKey('parent.pk'), nullable=False)
data = Column(String)
parent = relationship(
'Parent',
back_populates='children'
)
engine = create_engine('sqlite:///temp.db')
session_factory = sessionmaker(bind=engine, autoflush=False)
Session = scoped_session(session_factory)
session = Session()
Base.metadata.drop_all(engine)
Base.metadata.create_all(engine)
# Put some data in the database from some previous feed update
parent = Parent(pk=PARENT_PK, data='First')
child_1 = Child(pk=CHILD_1_PK, data='First child')
child_2 = Child(pk=CHILD_2_PK)
parent.children = [child_1, child_2]
session.add(parent)
session.commit()
# New data in the new feed update
new_parent = Parent(pk=PARENT_PK, data='Second')
new_child_1 = Child(pk=CHILD_1_PK, data='Second child')
new_parent.children = [new_child_1]
session.merge(new_parent)
session.commit()
Exception:
sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) NOT NULL constraint
failed: child.parent_pk
[SQL: UPDATE child SET parent_pk=? WHERE child.pk = ?]
[parameters: (None, 7)]
(Background on this error at: http://sqlalche.me/e/gkpj)
Manually setting new_child_1.parent_pk before the merge doesn't do anything
as the relationship takes precedence. To avoid the exception I need to do
something like:
new_child_1.parent_pk = PARENT_PK
del new_parent.children
del new_child_1.parent
Is there an easier way to use session.merge for a graph - or a more
standard method? Or do I always have to do some 'post processing' to strip
out the relationships before using it?
--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
http://www.sqlalchemy.org/
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable
Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
Re: [sqlalchemy] Checking approaches around parallel data import for records
>> Other software architectures can support parallelisation better, can't >> they? > > Can you clarify what you hope to achieve when you continue to make statements > of this form? I don't find them to be very constructive. I dared to point out that I stumbled on another software limitation. Now I am also looking again for possible adjustments and extensions. > There are many software architectures available for your use I check a few of them once more. > and you should choose the ones which work best for you. I hope to achieve more helpful evolution for affected software areas. Regards, Markus -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
Re: [sqlalchemy] How to reload data from table/all tables using sqlalchemy ORM?
Hi, many thanks for advice. I did research and found some materials about this topic, so will study little bit. Jano On Tue, Apr 23, 2019 at 12:18 PM Simon King wrote: > I think you need to consider the transactional behaviour of your > application. Remember that if your application crashes (or more > specifically, if a DB connection is unexpectedly closed), anything > that has not been committed will be lost. > > It is possible for one connection to see data from another connection > before it has been committed, by changing the transaction isolation > level of the connections. See your DB's documentation for details. > > The answers to these questions tend to be application-specific. I > don't think it's possible to give advice that will be correct in every > situation. > > It's possible that you are looking for something more like a message > queue. Databases can be used as message queues, but you need to > understand the implications if you are going to do that. > > Simon > > On Tue, Apr 23, 2019 at 10:53 AM Jan Sakalos wrote: > > > > Hi, > > > > API will add rows to table and main loop will have to read them. > > > > Meanwhile i found out following > > > > ses.commit() will have this effect, but maybe there is more > straightforward command. > > > > As suggested in documentation, session should not live for so long so > maybe it is more suitable to use new session every run (there will be min > 10s sleep). > > > > But not sure if any of this is good approach, also i am in doubt if API > writing directly to db is correct approach, but now it is not critical for > so i will do some more searching on this topic later. > > > > Whats your opinion on this? > > > > Thanks, > > Jano > > > > > > On Tue, Apr 23, 2019 at 10:17 AM Simon King > wrote: > >> > >> On Mon, Apr 22, 2019 at 5:09 PM Jan Sakalos wrote: > >> > > >> > Hello, > >> > > >> > How to reload data from table/all tables using sqlalchemy ORM? > >> > > >> > Thanks > >> > Jano > >> > > >> > >> Hi, > >> > >> Can you explain what you mean by "reload data"? > >> > >> Thanks, > >> > >> Simon > >> > >> -- > >> SQLAlchemy - > >> The Python SQL Toolkit and Object Relational Mapper > >> > >> http://www.sqlalchemy.org/ > >> > >> To post example code, please provide an MCVE: Minimal, Complete, and > Verifiable Example. See http://stackoverflow.com/help/mcve for a full > description. > >> --- > >> You received this message because you are subscribed to the Google > Groups "sqlalchemy" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an email to sqlalchemy+unsubscr...@googlegroups.com. > >> To post to this group, send email to sqlalchemy@googlegroups.com. > >> Visit this group at https://groups.google.com/group/sqlalchemy. > >> For more options, visit https://groups.google.com/d/optout. > > > > -- > > SQLAlchemy - > > The Python SQL Toolkit and Object Relational Mapper > > > > http://www.sqlalchemy.org/ > > > > To post example code, please provide an MCVE: Minimal, Complete, and > Verifiable Example. See http://stackoverflow.com/help/mcve for a full > description. > > --- > > You received this message because you are subscribed to the Google > Groups "sqlalchemy" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to sqlalchemy+unsubscr...@googlegroups.com. > > To post to this group, send email to sqlalchemy@googlegroups.com. > > Visit this group at https://groups.google.com/group/sqlalchemy. > > For more options, visit https://groups.google.com/d/optout. > > -- > SQLAlchemy - > The Python SQL Toolkit and Object Relational Mapper > > http://www.sqlalchemy.org/ > > To post example code, please provide an MCVE: Minimal, Complete, and > Verifiable Example. See http://stackoverflow.com/help/mcve for a full > description. > --- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sqlalchemy+unsubscr...@googlegroups.com. > To post to this group, send email to sqlalchemy@googlegroups.com. > Visit this group at https://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/d/optout. > -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
