[jira] [Commented] (FLINK-3991) Remove deprecated configuration keys from ConfigConstants
[ https://issues.apache.org/jira/browse/FLINK-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812161#comment-16812161 ] LiuJi commented on FLINK-3991: -- Hi [~Zentol] It seems now in ConfigConstants, some deprecated keys have no usages. Do you think it is a proper time to remove them? If so, I would like to work on it. > Remove deprecated configuration keys from ConfigConstants > - > > Key: FLINK-3991 > URL: https://issues.apache.org/jira/browse/FLINK-3991 > Project: Flink > Issue Type: Sub-task > Components: Build System >Reporter: Robert Metzger >Priority: Major > Fix For: 2.0.0 > > > In > https://github.com/apache/flink/commit/b0acd97935cd21843bac3b9b5afa3662b52bb95d#diff-40616c4678c3fbfe07c0701505ce0567 > I deprecated some configuration keys. > They are unused and need to be removed with the 2.0 release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-7916) Remove NetworkStackThroughputITCase
[
https://issues.apache.org/jira/browse/FLINK-7916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812155#comment-16812155
]
LiuJi commented on FLINK-7916:
--
Agree, I think move these tests to _org.apache.flink.test.manual_ under
_flink-tests_ is a good idea.
> Remove NetworkStackThroughputITCase
> ---
>
> Key: FLINK-7916
> URL: https://issues.apache.org/jira/browse/FLINK-7916
> Project: Flink
> Issue Type: Task
> Components: Runtime / Network, Tests
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Priority: Major
>
> Flink's code base contains the {{NetworkStackThroughputITCase}} which is not
> really a test. Moreover it is marked as {{Ignored}}. I propose to remove this
> test because it is more of a benchmark. We could think about creating a
> benchmark project where we move these kind of "tests".
> In general I think we should remove ignored tests if they won't be fixed
> immediately. The danger is far too high that we forget about them and then we
> only keep the maintenance burden of it. This is especially true for the above
> mentioned test case.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-9787) Change ExecutionConfig#getGlobalJobParameters to return an instance of GlobalJobParameters instead of null if no custom globalJobParameters are set yet
[ https://issues.apache.org/jira/browse/FLINK-9787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812145#comment-16812145 ] LiuJi commented on FLINK-9787: -- Agree, it's a good idea to avoid NPE when invoke getGlobalJobParameters.toMap(). Is there anyone work on this issue？ > Change ExecutionConfig#getGlobalJobParameters to return an instance of > GlobalJobParameters instead of null if no custom globalJobParameters are set > yet > --- > > Key: FLINK-9787 > URL: https://issues.apache.org/jira/browse/FLINK-9787 > Project: Flink > Issue Type: Improvement > Components: API / DataStream >Reporter: Florian Schmidt >Priority: Minor > Fix For: 2.0.0 > > > Currently when accessing ExecutionConfig#getGlobalJobParameters this will > return `null` if no globalJobParameters are set. This can easily lead to > NullPointerExceptions when used with getGlobalJobParameters.toMap() > An easy improvement for this would be to just return a new instance of > GlobalJobParameters if none is set with the empty map as the parameters > This would be a breaking change since we expose this via > RuntimeContext#getExecutionConfig -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10437) Some of keys under withDeprecatedKeys aren't marked as @depreacted
[
https://issues.apache.org/jira/browse/FLINK-10437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812138#comment-16812138
]
LiuJi commented on FLINK-10437:
---
You mean we should check all classes which use _withDeprecatedKeys_，if the key
is not marked as deprecated, then it should be replaced with
_withFallbackKeys_, right? [~Tison]
> Some of keys under withDeprecatedKeys aren't marked as @depreacted
> --
>
> Key: FLINK-10437
> URL: https://issues.apache.org/jira/browse/FLINK-10437
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Web Frontend
>Affects Versions: 1.7.0
>Reporter: TisonKun
>Priority: Major
>
> as title. For example {{RestOptions#BIND_ADDRESS}} is
> {{withDeprecatedKeys(WebOptions.ADDRESS.key())}}, but {{WebOptions.ADDRESS}}
> isn't marked as deprecated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10049) Unify the processing logic for NULL arguments in SQL built-in functions
[ https://issues.apache.org/jira/browse/FLINK-10049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811903#comment-16811903 ] LiuJi commented on FLINK-10049: --- It seems string functions in different SQL engines may have different behaviors with NULL value. for example, as you mentioned above, LOG10(NULL) throws NPE, but in other SQL engine it will return null. The same applies to ABS, it may throw Exception with NULL param. I'm not sure if there is a grammatical standard in string functions? [~xccui] > Unify the processing logic for NULL arguments in SQL built-in functions > --- > > Key: FLINK-10049 > URL: https://issues.apache.org/jira/browse/FLINK-10049 > Project: Flink > Issue Type: Improvement > Components: Table SQL / API >Reporter: Xingcan Cui >Priority: Major > > Currently, the built-in functions treat NULL arguments in different ways. > E.g., ABS(NULL) returns NULL, while LOG10(NULL) throws an NPE. The general > SQL-way of handling NULL values should be that if one argument is NULL the > result is NULL. We should unify the processing logic for that. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (FLINK-10049) Unify the processing logic for NULL arguments in SQL built-in functions
[ https://issues.apache.org/jira/browse/FLINK-10049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811903#comment-16811903 ] LiuJi edited comment on FLINK-10049 at 4/7/19 2:53 PM: --- It seems string functions in different SQL engines may have different behaviors with NULL value. for example, as you mentioned above, LOG10(NULL) throws NPE, but in other SQL engine it could return null. The same applies to ABS, it may throw Exception with NULL param. I'm not sure if there is a grammatical standard in string functions? [~xccui] was (Author: tianchen92): It seems string functions in different SQL engines may have different behaviors with NULL value. for example, as you mentioned above, LOG10(NULL) throws NPE, but in other SQL engine it will return null. The same applies to ABS, it may throw Exception with NULL param. I'm not sure if there is a grammatical standard in string functions? [~xccui] > Unify the processing logic for NULL arguments in SQL built-in functions > --- > > Key: FLINK-10049 > URL: https://issues.apache.org/jira/browse/FLINK-10049 > Project: Flink > Issue Type: Improvement > Components: Table SQL / API >Reporter: Xingcan Cui >Priority: Major > > Currently, the built-in functions treat NULL arguments in different ways. > E.g., ABS(NULL) returns NULL, while LOG10(NULL) throws an NPE. The general > SQL-way of handling NULL values should be that if one argument is NULL the > result is NULL. We should unify the processing logic for that. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-10684) Improve the CSV reading process
[
https://issues.apache.org/jira/browse/FLINK-10684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811901#comment-16811901
]
LiuJi commented on FLINK-10684:
---
Hi，[~xccui] Is there any update on this issue? As you said，this is an umbrella
issue for CSV reading progress，do you have plan to open some sub-tasks? And if
you don't mind, I would like to take on part of the work.
> Improve the CSV reading process
> ---
>
> Key: FLINK-10684
> URL: https://issues.apache.org/jira/browse/FLINK-10684
> Project: Flink
> Issue Type: Improvement
> Components: API / DataSet
>Reporter: Xingcan Cui
>Priority: Major
>
> CSV is one of the most commonly used file formats in data wrangling. To load
> records from CSV files, Flink has provided the basic {{CsvInputFormat}}, as
> well as some variants (e.g., {{RowCsvInputFormat}} and
> {{PojoCsvInputFormat}}). However, it seems that the reading process can be
> improved. For example, we could add a built-in util to automatically infer
> schemas from CSV headers and samples of data. Also, the current bad record
> handling method can be improved by somehow keeping the invalid lines (and
> even the reasons for failed parsing), instead of logging the total number
> only.
> This is an umbrella issue for all the improvements and bug fixes for the CSV
> reading process.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10437) Some of keys under withDeprecatedKeys aren't marked as @depreacted
[
https://issues.apache.org/jira/browse/FLINK-10437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811899#comment-16811899
]
LiuJi commented on FLINK-10437:
---
Hi [~Tison] Does the problem still exist? if so, I would like to fix this.
> Some of keys under withDeprecatedKeys aren't marked as @depreacted
> --
>
> Key: FLINK-10437
> URL: https://issues.apache.org/jira/browse/FLINK-10437
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Web Frontend
>Affects Versions: 1.7.0
>Reporter: TisonKun
>Priority: Major
>
> as title. For example {{RestOptions#BIND_ADDRESS}} is
> {{withDeprecatedKeys(WebOptions.ADDRESS.key())}}, but {{WebOptions.ADDRESS}}
> isn't marked as deprecated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (FLINK-11425) Support of “Hash Teams” in Hybrid Hash Join
[ https://issues.apache.org/jira/browse/FLINK-11425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LiuJi reassigned FLINK-11425: - Assignee: LiuJi > Support of “Hash Teams” in Hybrid Hash Join > --- > > Key: FLINK-11425 > URL: https://issues.apache.org/jira/browse/FLINK-11425 > Project: Flink > Issue Type: New Feature > Components: Runtime / Operators, SQL / Planner >Reporter: LiuJi >Assignee: LiuJi >Priority: Major > > Hybrid Hash Join is already supported in current version. The join starts > operating in memory and gradually starts spilling contents to disk, when the > memory is not sufficient. > > Current hash join only support two inputs, so when a job contains multiple > hash joins which have the same join keys, it will consume some unnecessary > resources (I/O, memory, etc) because some upstream output data may useless > for downstream hash join. > > According to the above observations, we want to provide a HashTeamManager to > implement multiway inputs hash join by combining several two way hash join > which have same join keys. HashTeamManager manage the relations of multiple > HashTables and improve efficiency in memory use and lower I/O operations by > joining multiple relations at one time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-11425) Support of “Hash Teams” in Hybrid Hash Join
[ https://issues.apache.org/jira/browse/FLINK-11425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754964#comment-16754964 ] LiuJi commented on FLINK-11425: --- I am very glad to hear that, thanks very much! [~ykt836] > Support of “Hash Teams” in Hybrid Hash Join > --- > > Key: FLINK-11425 > URL: https://issues.apache.org/jira/browse/FLINK-11425 > Project: Flink > Issue Type: New Feature > Components: Core, Optimizer >Reporter: LiuJi >Priority: Major > > Hybrid Hash Join is already supported in current version. The join starts > operating in memory and gradually starts spilling contents to disk, when the > memory is not sufficient. > > Current hash join only support two inputs, so when a job contains multiple > hash joins which have the same join keys, it will consume some unnecessary > resources (I/O, memory, etc) because some upstream output data may useless > for downstream hash join. > > According to the above observations, we want to provide a HashTeamManager to > implement multiway inputs hash join by combining several two way hash join > which have same join keys. HashTeamManager manage the relations of multiple > HashTables and improve efficiency in memory use and lower I/O operations by > joining multiple relations at one time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-11425) Support of “Hash Teams” in Hybrid Hash Join
[ https://issues.apache.org/jira/browse/FLINK-11425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752037#comment-16752037 ] LiuJi commented on FLINK-11425: --- Thanks for the quick response [~StephanEwen]. I would like to work on this if it is feasible to introduce multiway hash join after the blink merge job. > Support of “Hash Teams” in Hybrid Hash Join > --- > > Key: FLINK-11425 > URL: https://issues.apache.org/jira/browse/FLINK-11425 > Project: Flink > Issue Type: New Feature > Components: Core, Optimizer >Reporter: LiuJi >Priority: Major > > Hybrid Hash Join is already supported in current version. The join starts > operating in memory and gradually starts spilling contents to disk, when the > memory is not sufficient. > > Current hash join only support two inputs, so when a job contains multiple > hash joins which have the same join keys, it will consume some unnecessary > resources (I/O, memory, etc) because some upstream output data may useless > for downstream hash join. > > According to the above observations, we want to provide a HashTeamManager to > implement multiway inputs hash join by combining several two way hash join > which have same join keys. HashTeamManager manage the relations of multiple > HashTables and improve efficiency in memory use and lower I/O operations by > joining multiple relations at one time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (FLINK-11425) Support of “Hash Teams” in Hybrid Hash Join
LiuJi created FLINK-11425: - Summary: Support of “Hash Teams” in Hybrid Hash Join Key: FLINK-11425 URL: https://issues.apache.org/jira/browse/FLINK-11425 Project: Flink Issue Type: New Feature Components: Core, Optimizer Reporter: LiuJi Hybrid Hash Join is already supported in current version. The join starts operating in memory and gradually starts spilling contents to disk, when the memory is not sufficient. Current hash join only support two inputs, so when a job contains multiple hash joins which have the same join keys, it will consume some unnecessary resources (I/O, memory, etc) because some upstream output data may useless for downstream hash join. According to the above observations, we want to provide a HashTeamManager to implement multiway inputs hash join by combining several two way hash join which have same join keys. HashTeamManager manage the relations of multiple HashTables and improve efficiency in memory use and lower I/O operations by joining multiple relations at one time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
