Mike,
Thanks for merging my change on Alembic!
On Thu, 2020-02-27 at 15:38 -0500, Mike Bayer wrote:
>
>
> On Thu, Feb 27, 2020, at 2:49 PM, Ke Zhu - k...@us.ibm.com wrote:
> > On Wed, 2020-02-26 at 11:07 -0500, Mike Bayer wrote:
> > > oh, that issue is talking about rowcount. Alembic does not need
> > > rowcount to function correctly.I see that Alembic is doing
> > > this now, however there is a dialect-level flag called
> > > "supports_sane_rowcount", if this were False, Alembic should be
> > > checking this and skipping that particular check.
> >
> > I see. it would be great if Alembic will respect the
> > "support_sane_rowcount" specified in a SQLAlchemy dialect when
> > updating/deleting revision. If this is a suggested design, I can
> > patch it via a PR.
>
> yes.
>
>
> > According to specific dialect, it just need to use this option
> > instead of returning "-1". I can document this in the issue
> > https://github.com/dropbox/PyHive/issues/315
>
> the pyhive dialect advertises the attribute correctly right?
That's the fix I proposed in that issue.
>
> > > is the "rowcount" check the only issue here? this part can be
> > > changed on the Alembic side.
> >
> > Besides the above issue, I want to get input on
> > https://github.com/dropbox/PyHive/issues/314 as well. How could I
> > use SQLAlchemy API to create table like what this sql does:
> > CREATE TABLE hello_acid (key int, value int)
> > STORED AS ORC TBLPROPERTIES ('transactional'='true');
> > I've no idea how to make `STORED AS and TBLPROPERTIES` part for a
> > Hive table. Now I just use SQLAlchemy compiler to append them for
> > CreateTable.
>
> There's a few paths to make this work depending on where and how far
> you want to go, or at least how this works.
>
>
> Assuming these are *optional* arguments for the tables that you
> create with the pyhive dialect, the official way is that the dialect
> can be made to support these (I thought we had a generic "suffixes"
> section right now, but we don't). The way they work can be seen in
> the Postgresql dialect. start with the keywords you want, here I
> might call it pyhive_stored_as="ORC"
> pyhive_tblproperties="('transactional'='true'). then they'd be set
> up as "stored_as" and "tblproperties" in the dialect's
> construct_arguments like Postgresql does here:
>
> https://github.com/sqlalchemy/sqlalchemy/blob/master/lib/sqlalchemy/dialects/postgresql/base.py#L2407
>
> then you consume these in post_create_table:
>
> https://github.com/sqlalchemy/sqlalchemy/blob/master/lib/sqlalchemy/dialects/postgresql/base.py#L2037
>
>
> if you need Alembic's alembic_version to have these keywords, im not
> sure we have a keyword for that right now however you can use
> metadata events to intercept when alembic_version is built up, or you
> can intercept the compiler as you're doing now.
>
> if alternatively all pyhive tables need these keywords
> unconditionally, then you'd just implement post_create_table in the
> compiler to add them in.
>
Thanks for the pointers! It's very useful. I will try these paths since
not all Hive tables need such keywords so far. It depends on how data
is ingested in Hive/hadoop.
>
> >
> >
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "sqlalchemy-alembic" group.
> > To unsubscribe from this group and stop receiving emails from it,
> > send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com.
> > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/sqlalchemy-alembic/ab42854af49a98c83fd3693c92dbd9ad5c59b9df.camel%40us.ibm.com
> > .
>
>
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy-alembic" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy-alembic+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/sqlalchemy-alembic/c1186fc5514dda850a9ae08e61624b765cc8d686.camel%40us.ibm.com.
