Re: [sqlalchemy] unregister Tables from Metadata

[Mike Bayer]

On 10/1/15 5:56 AM, Pavel S wrote:

Hi,

we use application-level partitioning (but no real partitioning in 
mysqld), where new tables are created daily with the same structure, 
but different name (suffix).


  * mysqld is shared for these components:

  * daemon written in C++
  o creates new tables every day and fills them with records

  * other daemon written in Python:

  o periodically scans /information_schema.tables/ for new tables
and creates Table objects, which haven't been initialized before
  o the same daemon then processes records from all tables and
computes some statistics on top of them

  * daily cronjob drops tables older than _n_ days


_The problem:_


Pythonic daemon has to somehow notice that some tables were dropped 
and unload them from Metadata object. Otherwise, it will fail on 
ProgrammingError: table does not exit


removal of individual tables is not a supported API feature of the 
MetaData object because the problem of resolving foreign key references 
to these tables is complicated and error prone.


However, you can just whack an individual Table that is guaranteed to 
not be referenced by any other Table objects by removing it from the 
dictionary (which requires special tricks as we've had to add checks in 
place to prevent people from doing this):


dict.__del__(my_metadata.tables, "mytable")


Also, if you are using any ORM functionality here, ORM classes that 
refer to these tables would also have to be dealt with (e.g. class 
should be dereferenced and there cannot be any other mappings which 
refer to these classes using relationship()).



A better option is likely to use a separate MetaData object for *each* 
table.There's no requirement that tables are all present in just one 
MetaData collection.




We don't want to establish any messaging between daemons, just to keep 
the setup as it is.


Daemon should compare what's in /information_schema/ and what's in 
Metadata and remove Table objects. How to do that?


--
You received this message because you are subscribed to the Google 
Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to sqlalchemy+unsubscr...@googlegroups.com 
.
To post to this group, send email to sqlalchemy@googlegroups.com 
.

Visit this group at http://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at http://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

[sqlalchemy] unregister Tables from Metadata

[Pavel S]

Hi,

we use application-level partitioning (but no real partitioning in mysqld), 
where new tables are created daily with the same structure, but different 
name (suffix).

   - mysqld is shared for these components:
   

   - daemon written in C++
  - creates new tables every day and fills them with records
   

   - other daemon written in Python:


   - periodically scans *information_schema.tables* for new tables and 
  creates Table objects, which haven't been initialized before
  - the same daemon then processes records from all tables and computes 
  some statistics on top of them
   

   - daily cronjob drops tables older than *n* days


*The problem:*


Pythonic daemon has to somehow notice that some tables were dropped and 
unload them from Metadata object. Otherwise, it will fail on 
ProgrammingError: table does not exit

We don't want to establish any messaging between daemons, just to keep the 
setup as it is.

Daemon should compare what's in *information_schema* and what's in Metadata 
and remove Table objects. How to do that?

-- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at http://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.