Mike, Thanks for merging my change on Alembic!
On Thu, 2020-02-27 at 15:38 -0500, Mike Bayer wrote: > > > On Thu, Feb 27, 2020, at 2:49 PM, Ke Zhu - k...@us.ibm.com wrote: > > On Wed, 2020-02-26 at 11:07 -0500, Mike Bayer wrote: > > > oh, that issue is talking about rowcount. Alembic does not need > > > rowcount to function correctly. I see that Alembic is doing > > > this now, however there is a dialect-level flag called > > > "supports_sane_rowcount", if this were False, Alembic should be > > > checking this and skipping that particular check. > > > > I see. it would be great if Alembic will respect the > > "support_sane_rowcount" specified in a SQLAlchemy dialect when > > updating/deleting revision. If this is a suggested design, I can > > patch it via a PR. > > yes. > > > > According to specific dialect, it just need to use this option > > instead of returning "-1". I can document this in the issue > > https://github.com/dropbox/PyHive/issues/315 > > the pyhive dialect advertises the attribute correctly right? That's the fix I proposed in that issue. > > > > is the "rowcount" check the only issue here? this part can be > > > changed on the Alembic side. > > > > Besides the above issue, I want to get input on > > https://github.com/dropbox/PyHive/issues/314 as well. How could I > > use SQLAlchemy API to create table like what this sql does: > > CREATE TABLE hello_acid (key int, value int) > > STORED AS ORC TBLPROPERTIES ('transactional'='true'); > > I've no idea how to make `STORED AS and TBLPROPERTIES` part for a > > Hive table. Now I just use SQLAlchemy compiler to append them for > > CreateTable. > > There's a few paths to make this work depending on where and how far > you want to go, or at least how this works. > > > Assuming these are *optional* arguments for the tables that you > create with the pyhive dialect, the official way is that the dialect > can be made to support these (I thought we had a generic "suffixes" > section right now, but we don't). The way they work can be seen in > the Postgresql dialect. start with the keywords you want, here I > might call it pyhive_stored_as="ORC" > pyhive_tblproperties="('transactional'='true'). then they'd be set > up as "stored_as" and "tblproperties" in the dialect's > construct_arguments like Postgresql does here: > > https://github.com/sqlalchemy/sqlalchemy/blob/master/lib/sqlalchemy/dialects/postgresql/base.py#L2407 > > then you consume these in post_create_table: > > https://github.com/sqlalchemy/sqlalchemy/blob/master/lib/sqlalchemy/dialects/postgresql/base.py#L2037 > > > if you need Alembic's alembic_version to have these keywords, im not > sure we have a keyword for that right now however you can use > metadata events to intercept when alembic_version is built up, or you > can intercept the compiler as you're doing now. > > if alternatively all pyhive tables need these keywords > unconditionally, then you'd just implement post_create_table in the > compiler to add them in. > Thanks for the pointers! It's very useful. I will try these paths since not all Hive tables need such keywords so far. It depends on how data is ingested in Hive/hadoop. > > > > > > > > > -- > > You received this message because you are subscribed to the Google > > Groups "sqlalchemy-alembic" group. > > To unsubscribe from this group and stop receiving emails from it, > > send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/sqlalchemy-alembic/ab42854af49a98c83fd3693c92dbd9ad5c59b9df.camel%40us.ibm.com > > . > > -- You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy-alembic/c1186fc5514dda850a9ae08e61624b765cc8d686.camel%40us.ibm.com.
