[
https://issues.apache.org/jira/browse/FLINK-17789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110051#comment-17110051
]
Chesnay Schepler edited comment on FLINK-17789 at 5/18/20, 8:51 AM:
--------------------------------------------------------------------
Alright hold on, this is getting really confusing.
Let's clarify the behaviors a bit.
For a DelegatingConfiguration with prefix P:
1) get(X) reads P.X from the backing configuration
1) set(X) writes P.X to the backing configuration
2) addAllToProperties() writes all options starting with P into the properties,
such that properties.get(X) works.
3) toMap() writes all options of the backing configuration, with an added
prefix P. map.get(X) will not work, but map.get(P.X) will.
It is not a problem that toMap() behaves differently to addAllToProperties() in
general, as they actually serve different purposes.
I can see that the behavior could be, but currently it is necessary and we
would break code if we were to change anything.
Change addAllToProperties() and the metric system breaks (reporters look up
options by suffix, since the prefix contains the reporter name and is hence
dynamic), change toMap() and the configuration of formats is broken (because
suddenly the options are no longer fully qualified.
However, one problem I do have with 3) is the following:
{code}
Configuration conf = new Configuration();
conf.setString("k0", "v0");
conf.setString("prefix.k1", "v1");
DelegatingConfiguration dc = new DelegatingConfiguration(conf, "prefix.");
System.out.println(dc.getString("k1", null)); //v1
System.out.println(dc.getString("k0", null)); //null
Map<String, String> map = dc.toMap();
System.out.println(map.get("prefix.k1")); //v1
System.out.println(map.get("prefix.k0")); //v0
{code}
Basically, toMap() conjures settings out of thin air, where it should be an
accurate representation of the configuration.
was (Author: zentol):
Alright hold on, this is getting really confusing.
Let's clarify the behaviors a bit.
For a DelegatingConfiguration with prefix P:
1) get(X) reads P.X from the backing configuration
1) set(X) writes P.X to the backing configuration
2) addAllToProperties() writes all options starting with P into the properties,
such that properties.get(X) works.
- this behavior is required by the metric system, where reporters use
suffixes for specifying options, since the prefix is dynamic (because includes
the reporter name)
3) toMap() writes all options of the backing configuration, with an added
prefix P. map.get(X) will not work, but map.get(P.X) will.
It is not a problem that toMap() behaves differently to addAllToProperties() in
general, as they actually serve different purposes. I can see that the
behavior could be, but currently it is necessary and we would break code if we
were to change anything. Change addAllToProperties() and the metric system is
broken, change toMap() and the configuration of formats is broken.
However, one problem I do have with 3) is the following:
{code}
Configuration conf = new Configuration();
conf.setString("k0", "v0");
conf.setString("prefix.k1", "v1");
DelegatingConfiguration dc = new DelegatingConfiguration(conf, "prefix.");
System.out.println(dc.getString("k1", null)); //v1
System.out.println(dc.getString("k0", null)); //null
Map<String, String> map = dc.toMap();
System.out.println(map.get("prefix.k1")); //v1
System.out.println(map.get("prefix.k0")); //v0
{code}
Basically, toMap() conjures settings out of thin air, where it should be an
accurate representation of the configuration.
> DelegatingConfiguration should remove prefix instead of add prefix in toMap
> ---------------------------------------------------------------------------
>
> Key: FLINK-17789
> URL: https://issues.apache.org/jira/browse/FLINK-17789
> Project: Flink
> Issue Type: Bug
> Components: API / Core
> Reporter: Jingsong Lee
> Priority: Major
> Fix For: 1.11.0
>
>
> {code:java}
> Configuration conf = new Configuration();
> conf.setString("k0", "v0");
> conf.setString("prefix.k1", "v1");
> DelegatingConfiguration dc = new DelegatingConfiguration(conf, "prefix.");
> System.out.println(dc.getString("k0", "empty")); // empty
> System.out.println(dc.getString("k1", "empty")); // v1
> System.out.println(dc.toMap().get("k1")); // should be v1, but null
> System.out.println(dc.toMap().get("prefix.prefix.k1")); // should be null,
> but v1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
