Hi,
1) Where does the hive catalog persist view definitions? in mysql? or HDFS?
2) If the views are not persisted what happens if the application crashes
and restarted? will it create the view again and safely read the data from
where it left off?
Thanks!
On Wed, Feb 26, 2020 at 6:47 AM godfrey he <godfre...@gmail.com> wrote:
> Hi Kant, if you want the store the catalog data in Local Filesystem/HDFS,
> you can implement a user defined catalog (just need to implement Catalog
> interface)
>
> Bests,
> Godfrey
>
> kant kodali <kanth...@gmail.com> 于2020年2月26日周三 下午12:28写道：
>
>> Hi Jingsong,
>>
>> Can I store it in Local Filesystem/HDFS?
>>
>> Thanks!
>>
>> On Mon, Jan 20, 2020 at 6:59 PM Jingsong Li <jingsongl...@gmail.com>
>> wrote:
>>
>>> Hi Kant,
>>>
>>> If you want your view persisted, you must to dock a catalog like hive
>>> catalog, it stores views in the metastore with mysql.
>>> - In 1.10, you can store views in catalog through "Catalog.createTable",
>>> you can create a "CatalogViewImpl". This is an internal API, which is not
>>> easy to use.
>>> - In 1.11, we will introduce create view DDL for "TableEnv.sqlUpdate"
>>> and "TableEnv.createView". It will be easy to use.
>>>
>>> Best,
>>> Jingsong Lee
>>>
>>> On Tue, Jan 21, 2020 at 10:03 AM Jark Wu <imj...@gmail.com> wrote:
>>>
>>>> Hi Kant,
>>>>
>>>> The TableEnv#createTemporaryView and CREATE VIEW in SQL Cli both
>>>> creates temporary views which is not persisted and will lost after session
>>>> close.
>>>> I think the persisted views will be supported in 1.11.
>>>>
>>>> Best,
>>>> Jark
>>>>
>>>> 2020年1月20日 18:46，kant kodali <kanth...@gmail.com> 写道：
>>>>
>>>> Hi Jingsong,
>>>>
>>>> Thanks a lot, I think I can live with
>>>> TableEnvironment.createTemporaryView in Flink 1.10 (which I am expecting to
>>>> be released this month) but are these views persisted somewhere? for
>>>> example across sessions? or say I stop my application and start again will
>>>> it work as expected?
>>>>
>>>> Thanks!
>>>>
>>>>
>>>> On Mon, Jan 20, 2020 at 1:12 AM Jingsong Li <jingsongl...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Kant,
>>>>>
>>>>> Sorry, 1.10 not support "CREATE VIEW" in raw SQL too. Workaround is:
>>>>> - Using TableEnvironment.createTemporaryView...
>>>>> - Or using "create view" and "drop view" in the sql-client.
>>>>> - Or using hive catalog, in 1.10, we support query catalog views.
>>>>>
>>>>> FLIP-71 will be finished in 1.11 soon.
>>>>>
>>>>> Best,
>>>>> Jingsong Lee
>>>>>
>>>>> On Sun, Jan 19, 2020 at 4:10 PM kant kodali <kanth...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I tried the following.
>>>>>>
>>>>>> bsTableEnv.sqlUpdate("CREATE VIEW my_view AS SELECT * FROM sample1 FULL
>>>>>> OUTER JOIN sample2 on sample1.f0=sample2.f0");
>>>>>>
>>>>>> Table result = bsTableEnv.sqlQuery("select * from my_view");
>>>>>>
>>>>>> It looks like
>>>>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-71+-+E2E+View+support+in+FLINK+SQL
>>>>>> Views
>>>>>> are not supported. Can I expect them to be supported in Flink 1.10?
>>>>>>
>>>>>> Currently, with Spark SQL when the query gets big I break it down
>>>>>> into views and this is one of the most important features my application
>>>>>> relies on. is there any workaround for this at the moment?
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> On Sat, Jan 18, 2020 at 6:24 PM kant kodali <kanth...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> Does Flink 1.9 support create or replace views syntax in raw SQL?
>>>>>>> like spark streaming does?
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Best, Jingsong Lee
>>>>>
>>>>
>>>>
>>>
>>> --
>>> Best, Jingsong Lee
>>>
>>
