Re: SQL Client Limitations

[Timo Walther]

Hi Eron,

yes, FLINK-9172 covers how the SQL Client will discover ExternalCatalog 
similar to how it discovers connectors and formats today. The exact 
design has to be fleshed out but the SQL Client's environment file will 
declare catalogs and their properties. The SQL Client's gateway will 
then perform the communication to external systems.


Regards,
Timo

Am 26.08.18 um 04:54 schrieb Eron Wright:
I'd like to better understand how catalogs will work in SQL Client.  
 I assume we'll be able to reference catalog classes from the 
environment file (e.g. FLINK-9172).


Thanks

On Tue, Aug 21, 2018 at 4:56 AM Fabian Hueske > wrote:


Hi Dominik,

The SQL Client supports the same subset of SQL that you get with
Java /
Scala embedded queries.
The documentation [1] covers all supported operations.

There are some limitations because certain operators require
special time
attributes (row time or processing time attributes) which are
monotonically
increasing.
Some operators such as a regular join (in contrast to a
time-windowed join)
remove the monotonicity property of time attributes such that
time-based
operations cannot be applied anymore.

Best,
Fabian

[1] https://ci.apache.org/projects/flink/flink-docs-
release-1.6/dev/table/sql.html



2018-08-21 13:27 GMT+02:00 Till Rohrmann mailto:trohrm...@apache.org>>:

> Hi Dominik,
>
> I think such a list would be really helpful. I've pulled Timo
and Fabian
> into this conversation because they probably know more.
>
> Cheers,
> Till
>
> On Mon, Aug 20, 2018 at 12:43 PM Dominik Wosiński
mailto:wos...@gmail.com>>
> wrote:
>
>> Hey,
>>
>> Do we have any list of current limitations of SQL Client available
>> somewhere or the only way is to go through JIRA issues?
>>
>> For example:
>> I tried to make Group By Tumble Window and Inner Join in one
query and it
>> seems that it is not possible currently and I was wondering
whether it's
>> and issue with my query or known limitation.
>>
>> Thanks,
>> Best Regards,
>> Dominik.
>>
>

Re: [DISCUSS] Remove the slides under "Community & Project Info"

[vino yang]

+1
The reason is the same as Hequn, because we have given a link to SlideShare
under the "Flink Forward" section.

Thanks, vino.

Hequn Cheng  于2018年8月27日周一 上午9:31写道：

> Hi Stephan,
>
> Thanks for bringing up this discussion.
> I think we can just remove it, because slides have already be provided
> from the `Flink Forward` section in this page. Adding another slides
> section not only brings redundancy but also introduces problems you listed
> above.
>
> Best, Hequn
>
> On Sun, Aug 26, 2018 at 11:13 PM Stephan Ewen  wrote:
>
>> Hi all!
>>
>> In the past, we collected slide sets under the "Community & Project Info"
>> side.
>>
>> I would like to see what the community thinks about removing them. There
>> are currently several issues:
>>
>>   - The list is not well maintained. Tthere are for example no 2018 slides
>> at all.
>>   - Many slide sets are outdated. For example 2014 Flink internals are not
>> really relevant any more.
>>   - When searching for certain terms, the Youtube videos and slideshare
>> uploads appear first anyways
>>   - The list may create a wrong impression about having definitive
>> resources.
>>
>> Best,
>> Stephan
>>
>

Re: [DISCUSS] Remove the slides under "Community & Project Info"

[Hequn Cheng]

Hi Stephan,

Thanks for bringing up this discussion.
I think we can just remove it, because slides have already be provided from
the `Flink Forward` section in this page. Adding another slides section not
only brings redundancy but also introduces problems you listed above.

Best, Hequn

On Sun, Aug 26, 2018 at 11:13 PM Stephan Ewen  wrote:

> Hi all!
>
> In the past, we collected slide sets under the "Community & Project Info"
> side.
>
> I would like to see what the community thinks about removing them. There
> are currently several issues:
>
>   - The list is not well maintained. Tthere are for example no 2018 slides
> at all.
>   - Many slide sets are outdated. For example 2014 Flink internals are not
> really relevant any more.
>   - When searching for certain terms, the Youtube videos and slideshare
> uploads appear first anyways
>   - The list may create a wrong impression about having definitive
> resources.
>
> Best,
> Stephan
>

[DISCUSS] Remove the slides under "Community & Project Info"

[Stephan Ewen]

Hi all!

In the past, we collected slide sets under the "Community & Project Info"
side.

I would like to see what the community thinks about removing them. There
are currently several issues:

  - The list is not well maintained. Tthere are for example no 2018 slides
at all.
  - Many slide sets are outdated. For example 2014 Flink internals are not
really relevant any more.
  - When searching for certain terms, the Youtube videos and slideshare
uploads appear first anyways
  - The list may create a wrong impression about having definitive
resources.

Best,
Stephan

[jira] [Created] (FLINK-10219) remove the IRC mention from the website

[Nicos Maris (JIRA)]

Nicos Maris created FLINK-10219:
---

 Summary: remove the IRC mention from the website
 Key: FLINK-10219
 URL: https://issues.apache.org/jira/browse/FLINK-10219
 Project: Flink
  Issue Type: Task
  Components: Project Website
Reporter: Nicos Maris


This is the outcome of the following ticket along with a PR.

https://issues.apache.org/jira/browse/FLINK-10217



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (FLINK-10218) Allow write DataSet without path parameter

[Paul Lin (JIRA)]

Paul Lin created FLINK-10218:


 Summary: Allow write DataSet without path parameter
 Key: FLINK-10218
 URL: https://issues.apache.org/jira/browse/FLINK-10218
 Project: Flink
  Issue Type: Improvement
  Components: DataSet API
Affects Versions: 1.6.0
Reporter: Paul Lin


Currently, DataSet API has two overloaded write methods for using 
FileOutputFormat as output format, and both require a path parameter, but the 
output path could already be set in the FileOutputFormat object. What's more, 
the subclasses of FileOutputFormat mostly don't have default constructors and 
required a path parameter too, so users have to set output path twice in the 
code, like:
{code:java}
  String output = "hdfs:///tmp/";
  dataset.write(new TextOutputFormat<>(new Path(output)), output);
{code}
So I propose to add another write method that requires no path parameter. May 
someone assign this issue to me?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

??????Flink tableApi/sql calendar

[xueyu]

hi, Sergey,


I tried on my local mysql query  `select timestampadd(YEAR,-1000,'2017-11-29 
22:58:58.998');` it looks that the result of mysql is "1017-11-29 
22:58:58.998000". I don't know why the results of us are different...





Maybe I could not answer your question, however there are some other info about 
time in flink you could refer. One is that flink should support time zone [1] 
and it seems that flink has julian process function [2]


Thanks




[1] 
https://issues.apache.org/jira/browse/FLINK-8353?jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20creator%20in%20(twalthr)%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC


[2] 
https://github.com/apache/flink/blob/master/flink-libraries/flink-table/src/main/java/org/apache/calcite/avatica/util/DateTimeUtils.java#L365




--  --
??: "Sergey Nuyanzin";
: 2018??8??26??(??) 7:19
??: "dev";

: Flink tableApi/sql calendar



Hello,

I have just faced with a question: what calendar is used by Flink
tableAPI/sql?
For instance there was a TIMESTAMPADD related issue [1] with references to
MySql documentation. However there are some contradictions like:
from one side there is a testTimestampAdd[2] in Flink which expects
1017-11-29 after adding -1000 years. From the other side there is MySql
documentation about their calendar [3] and result of the same query
1017-12-05T22:58:58.998 vs expected by Flink's test 2017-11-29 22:58:58.998

Should it be treated as an issue or just another calendar usage (without
Gregorian cutover e.g.)?

[1] https://issues.apache.org/jira/browse/FLINK-6811
[2]

https://github.com/apache/flink/blob/505dca174128ebb3bf765778ee36d58f680d6a1e/flink-libraries/flink-table/src/test/scala/org/apache/flink/table/expressions/ScalarFunctionsTest.scala#L2166


[3] https://dev.mysql.com/doc/refman/5.5/en/mysql-calendar.html
Best regards,
Sergey