Is it possible to extend Beam's SchemaRegistry to do this? On Mon, Aug 13, 2018 at 4:06 PM Anton Kedin <[email protected]> wrote:
> Hi, > > I am planning to work on implementing a support for external schema > providers for Beam SQL and wanted to share a high level idea how I think > this can work. > > *Short Version* > Implement CREATE FOREIGN SCHEMA statement: > > CREATE FOREIGN SCHEMA > > TYPE 'bigquery' > > LOCATION 'dataset_example' > > AS bq; > > CREATE FOREIGN SCHEMA > > TYPE 'hcatalog' > > LOCATION 'hive-server:2341' > > AS hive; > > SELECT * > > FROM > > bq.table_example_bq AS bq_table1 > > JOIN > > hive.table_example_hive AS hive_table1 > > ON > bq_table1.some_field = hive_table1.some_other_field; > > *A Bit Longer Version: * > https://docs.google.com/document/d/1Ilk3OpDxrp3bHNlcnYDoj29tt9bd1E0EXt8i0WytNmQ > > Thoughts, ideas? > > Regards, > Anton >
