https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/table/hive/hive_catalog.html#hivecatalog
Can I use the hive catalog to store view definitions in HDFS? I am assuming
the metastore can be anything or does it have to be have MySQL?
On Thu, Feb 27, 2020 at 4:46 AM kant kodali <kanth...@gmail.com> wrote:
> Hi,
>
> 1) Where does the hive catalog persist view definitions? in mysql? or HDFS?
>
> 2) If the views are not persisted what happens if the application crashes
> and restarted? will it create the view again and safely read the data from
> where it left off?
>
> Thanks!
>
> On Wed, Feb 26, 2020 at 6:47 AM godfrey he <godfre...@gmail.com> wrote:
>
>> Hi Kant, if you want the store the catalog data in Local Filesystem/HDFS,
>> you can implement a user defined catalog (just need to implement Catalog
>> interface)
>>
>> Bests,
>> Godfrey
>>
>> kant kodali <kanth...@gmail.com> 于2020年2月26日周三 下午12:28写道：
>>
>>> Hi Jingsong,
>>>
>>> Can I store it in Local Filesystem/HDFS?
>>>
>>> Thanks!
>>>
>>> On Mon, Jan 20, 2020 at 6:59 PM Jingsong Li <jingsongl...@gmail.com>
>>> wrote:
>>>
>>>> Hi Kant,
>>>>
>>>> If you want your view persisted, you must to dock a catalog like hive
>>>> catalog, it stores views in the metastore with mysql.
>>>> - In 1.10, you can store views in catalog through
>>>> "Catalog.createTable", you can create a "CatalogViewImpl". This is an
>>>> internal API, which is not easy to use.
>>>> - In 1.11, we will introduce create view DDL for "TableEnv.sqlUpdate"
>>>> and "TableEnv.createView". It will be easy to use.
>>>>
>>>> Best,
>>>> Jingsong Lee
>>>>
>>>> On Tue, Jan 21, 2020 at 10:03 AM Jark Wu <imj...@gmail.com> wrote:
>>>>
>>>>> Hi Kant,
>>>>>
>>>>> The TableEnv#createTemporaryView and CREATE VIEW in SQL Cli both
>>>>> creates temporary views which is not persisted and will lost after session
>>>>> close.
>>>>> I think the persisted views will be supported in 1.11.
>>>>>
>>>>> Best,
>>>>> Jark
>>>>>
>>>>> 2020年1月20日 18:46，kant kodali <kanth...@gmail.com> 写道：
>>>>>
>>>>> Hi Jingsong,
>>>>>
>>>>> Thanks a lot, I think I can live with
>>>>> TableEnvironment.createTemporaryView in Flink 1.10 (which I am expecting
>>>>> to
>>>>> be released this month) but are these views persisted somewhere? for
>>>>> example across sessions? or say I stop my application and start again will
>>>>> it work as expected?
>>>>>
>>>>> Thanks!
>>>>>
>>>>>
>>>>> On Mon, Jan 20, 2020 at 1:12 AM Jingsong Li <jingsongl...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Kant,
>>>>>>
>>>>>> Sorry, 1.10 not support "CREATE VIEW" in raw SQL too. Workaround is:
>>>>>> - Using TableEnvironment.createTemporaryView...
>>>>>> - Or using "create view" and "drop view" in the sql-client.
>>>>>> - Or using hive catalog, in 1.10, we support query catalog views.
>>>>>>
>>>>>> FLIP-71 will be finished in 1.11 soon.
>>>>>>
>>>>>> Best,
>>>>>> Jingsong Lee
>>>>>>
>>>>>> On Sun, Jan 19, 2020 at 4:10 PM kant kodali <kanth...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I tried the following.
>>>>>>>
>>>>>>> bsTableEnv.sqlUpdate("CREATE VIEW my_view AS SELECT * FROM sample1 FULL
>>>>>>> OUTER JOIN sample2 on sample1.f0=sample2.f0");
>>>>>>>
>>>>>>> Table result = bsTableEnv.sqlQuery("select * from my_view");
>>>>>>>
>>>>>>> It looks like
>>>>>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71+-+E2E+View+support+in+FLINK+SQL
>>>>>>> Views
>>>>>>> are not supported. Can I expect them to be supported in Flink 1.10?
>>>>>>>
>>>>>>> Currently, with Spark SQL when the query gets big I break it down
>>>>>>> into views and this is one of the most important features my application
>>>>>>> relies on. is there any workaround for this at the moment?
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>> On Sat, Jan 18, 2020 at 6:24 PM kant kodali <kanth...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> Does Flink 1.9 support create or replace views syntax in raw SQL?
>>>>>>>> like spark streaming does?
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best, Jingsong Lee
>>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> Best, Jingsong Lee
>>>>
>>>
