[jira] [Commented] (FLINK-10294) Split MiniClusterResource to be usable in runtime/streaming-java/clients
[
https://issues.apache.org/jira/browse/FLINK-10294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17334091#comment-17334091
]
Flink Jira Bot commented on FLINK-10294:
This issue was marked "stale-assigned" and has not received an update in 7
days. It is now automatically unassigned. If you are still working on it, you
can assign it to yourself again. Please also give an update about the status of
the work.
> Split MiniClusterResource to be usable in runtime/streaming-java/clients
>
>
> Key: FLINK-10294
> URL: https://issues.apache.org/jira/browse/FLINK-10294
> Project: Flink
> Issue Type: Improvement
> Components: API / DataStream, Tests
>Affects Versions: 1.7.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Labels: stale-assigned
>
> h5. Problem
> The {{MiniClusterResource}} is a utility class to create and manage flink
> clusters for testing purposes. It is incredibly convenient, but unfortunately
> resides in {{flink-test-utils}} which depends on flink-runtime,
> flink-streaming-java and flink-clients, making the class not usable in these
> modules.
> The current version does require these dependencies, but only for specific,
> *optional*, parts. {{streaming-java}} is only required for accessing
> {{TestStreamEnvironment}} and {{flink-clients}} only for tests that want to
> work against a {{ClusterClient}}.
> h5. Proposal
> Split the {{MiniClusterResource}} as follows:
> h5. 1)
> Remove client/streaming-java dependent parts and move the class to
> flink-runtime.
> h5. 2)
> Add a new class {{StreamingMiniClusterResourceExtension}} that accepts a
> {{MiniClusterResource}} as an argument and contains the streaming parts.
> Usage would look like this:
> {code}
> private final MiniClusterResource cluster = ...
> private final StreamingMiniClusterResourceExtension ext = new
> StreamingMiniClusterResourceExtension(cluster);
> @Rule
> public RuleChain chain= RuleChain
> .outerRule(cluster)
> .around(ext),
> {code}
> h5. 3)
> Add a new class {{ClientMiniClusterResourceExtension}} that accepts a
> {{MiniClusterResource}} as an argument and contains the client parts.
> Usage would look like this:
> {code}
> private final MiniClusterResource cluster = ...
> private final ClientMiniClusterResourceExtensionext = new
> ClientMiniClusterResourceExtension(cluster);
> @Rule
> public RuleChain chain= RuleChain
> .outerRule(cluster)
> .around(ext),
> {code}
> [~till.rohrmann] WDYT?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-10294) Split MiniClusterResource to be usable in runtime/streaming-java/clients
[
https://issues.apache.org/jira/browse/FLINK-10294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17323465#comment-17323465
]
Flink Jira Bot commented on FLINK-10294:
This issue is assigned but has not received an update in 7 days so it has been
labeled "stale-assigned". If you are still working on the issue, please give an
update and remove the label. If you are no longer working on the issue, please
unassign so someone else may work on it. In 7 days the issue will be
automatically unassigned.
> Split MiniClusterResource to be usable in runtime/streaming-java/clients
>
>
> Key: FLINK-10294
> URL: https://issues.apache.org/jira/browse/FLINK-10294
> Project: Flink
> Issue Type: Improvement
> Components: API / DataStream, Tests
>Affects Versions: 1.7.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Labels: stale-assigned
>
> h5. Problem
> The {{MiniClusterResource}} is a utility class to create and manage flink
> clusters for testing purposes. It is incredibly convenient, but unfortunately
> resides in {{flink-test-utils}} which depends on flink-runtime,
> flink-streaming-java and flink-clients, making the class not usable in these
> modules.
> The current version does require these dependencies, but only for specific,
> *optional*, parts. {{streaming-java}} is only required for accessing
> {{TestStreamEnvironment}} and {{flink-clients}} only for tests that want to
> work against a {{ClusterClient}}.
> h5. Proposal
> Split the {{MiniClusterResource}} as follows:
> h5. 1)
> Remove client/streaming-java dependent parts and move the class to
> flink-runtime.
> h5. 2)
> Add a new class {{StreamingMiniClusterResourceExtension}} that accepts a
> {{MiniClusterResource}} as an argument and contains the streaming parts.
> Usage would look like this:
> {code}
> private final MiniClusterResource cluster = ...
> private final StreamingMiniClusterResourceExtension ext = new
> StreamingMiniClusterResourceExtension(cluster);
> @Rule
> public RuleChain chain= RuleChain
> .outerRule(cluster)
> .around(ext),
> {code}
> h5. 3)
> Add a new class {{ClientMiniClusterResourceExtension}} that accepts a
> {{MiniClusterResource}} as an argument and contains the client parts.
> Usage would look like this:
> {code}
> private final MiniClusterResource cluster = ...
> private final ClientMiniClusterResourceExtensionext = new
> ClientMiniClusterResourceExtension(cluster);
> @Rule
> public RuleChain chain= RuleChain
> .outerRule(cluster)
> .around(ext),
> {code}
> [~till.rohrmann] WDYT?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-10294) Split MiniClusterResource to be usable in runtime/streaming-java/clients
[
https://issues.apache.org/jira/browse/FLINK-10294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16633949#comment-16633949
]
Till Rohrmann commented on FLINK-10294:
---
Alright, this makes sense. Maybe we could offer for the most common use case
convenience classes or alternatively document how to use them with the
{{RuleChain}}.
> Split MiniClusterResource to be usable in runtime/streaming-java/clients
>
>
> Key: FLINK-10294
> URL: https://issues.apache.org/jira/browse/FLINK-10294
> Project: Flink
> Issue Type: Improvement
> Components: Tests
>Affects Versions: 1.7.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Fix For: 1.7.0
>
>
> h5. Problem
> The {{MiniClusterResource}} is a utility class to create and manage flink
> clusters for testing purposes. It is incredibly convenient, but unfortunately
> resides in {{flink-test-utils}} which depends on flink-runtime,
> flink-streaming-java and flink-clients, making the class not usable in these
> modules.
> The current version does require these dependencies, but only for specific,
> *optional*, parts. {{streaming-java}} is only required for accessing
> {{TestStreamEnvironment}} and {{flink-clients}} only for tests that want to
> work against a {{ClusterClient}}.
> h5. Proposal
> Split the {{MiniClusterResource}} as follows:
> h5. 1)
> Remove client/streaming-java dependent parts and move the class to
> flink-runtime.
> h5. 2)
> Add a new class {{StreamingMiniClusterResourceExtension}} that accepts a
> {{MiniClusterResource}} as an argument and contains the streaming parts.
> Usage would look like this:
> {code}
> private final MiniClusterResource cluster = ...
> private final StreamingMiniClusterResourceExtension ext = new
> StreamingMiniClusterResourceExtension(cluster);
> @Rule
> public RuleChain chain= RuleChain
> .outerRule(cluster)
> .around(ext),
> {code}
> h5. 3)
> Add a new class {{ClientMiniClusterResourceExtension}} that accepts a
> {{MiniClusterResource}} as an argument and contains the client parts.
> Usage would look like this:
> {code}
> private final MiniClusterResource cluster = ...
> private final ClientMiniClusterResourceExtensionext = new
> ClientMiniClusterResourceExtension(cluster);
> @Rule
> public RuleChain chain= RuleChain
> .outerRule(cluster)
> .around(ext),
> {code}
> [~till.rohrmann] WDYT?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10294) Split MiniClusterResource to be usable in runtime/streaming-java/clients
[
https://issues.apache.org/jira/browse/FLINK-10294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16625802#comment-16625802
]
Chesnay Schepler commented on FLINK-10294:
--
Alright I remembered why i initially went with the RuleChain approach: It
easily allows composition of different extensions. In your approach it would
not be possible to create a MiniClusterResource that has both the streaming and
client parts (without creating a separate class obviously).
> Split MiniClusterResource to be usable in runtime/streaming-java/clients
>
>
> Key: FLINK-10294
> URL: https://issues.apache.org/jira/browse/FLINK-10294
> Project: Flink
> Issue Type: Improvement
> Components: Tests
>Affects Versions: 1.7.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Fix For: 1.7.0
>
>
> h5. Problem
> The {{MiniClusterResource}} is a utility class to create and manage flink
> clusters for testing purposes. It is incredibly convenient, but unfortunately
> resides in {{flink-test-utils}} which depends on flink-runtime,
> flink-streaming-java and flink-clients, making the class not usable in these
> modules.
> The current version does require these dependencies, but only for specific,
> *optional*, parts. {{streaming-java}} is only required for accessing
> {{TestStreamEnvironment}} and {{flink-clients}} only for tests that want to
> work against a {{ClusterClient}}.
> h5. Proposal
> Split the {{MiniClusterResource}} as follows:
> h5. 1)
> Remove client/streaming-java dependent parts and move the class to
> flink-runtime.
> h5. 2)
> Add a new class {{StreamingMiniClusterResourceExtension}} that accepts a
> {{MiniClusterResource}} as an argument and contains the streaming parts.
> Usage would look like this:
> {code}
> private final MiniClusterResource cluster = ...
> private final StreamingMiniClusterResourceExtension ext = new
> StreamingMiniClusterResourceExtension(cluster);
> @Rule
> public RuleChain chain= RuleChain
> .outerRule(cluster)
> .around(ext),
> {code}
> h5. 3)
> Add a new class {{ClientMiniClusterResourceExtension}} that accepts a
> {{MiniClusterResource}} as an argument and contains the client parts.
> Usage would look like this:
> {code}
> private final MiniClusterResource cluster = ...
> private final ClientMiniClusterResourceExtensionext = new
> ClientMiniClusterResourceExtension(cluster);
> @Rule
> public RuleChain chain= RuleChain
> .outerRule(cluster)
> .around(ext),
> {code}
> [~till.rohrmann] WDYT?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10294) Split MiniClusterResource to be usable in runtime/streaming-java/clients
[
https://issues.apache.org/jira/browse/FLINK-10294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16625587#comment-16625587
]
Chesnay Schepler commented on FLINK-10294:
--
Note that with the RuleChain we wouldn't have to deal with the Rule life-cycle
internally.
> Split MiniClusterResource to be usable in runtime/streaming-java/clients
>
>
> Key: FLINK-10294
> URL: https://issues.apache.org/jira/browse/FLINK-10294
> Project: Flink
> Issue Type: Improvement
> Components: Tests
>Affects Versions: 1.7.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Fix For: 1.7.0
>
>
> h5. Problem
> The {{MiniClusterResource}} is a utility class to create and manage flink
> clusters for testing purposes. It is incredibly convenient, but unfortunately
> resides in {{flink-test-utils}} which depends on flink-runtime,
> flink-streaming-java and flink-clients, making the class not usable in these
> modules.
> The current version does require these dependencies, but only for specific,
> *optional*, parts. {{streaming-java}} is only required for accessing
> {{TestStreamEnvironment}} and {{flink-clients}} only for tests that want to
> work against a {{ClusterClient}}.
> h5. Proposal
> Split the {{MiniClusterResource}} as follows:
> h5. 1)
> Remove client/streaming-java dependent parts and move the class to
> flink-runtime.
> h5. 2)
> Add a new class {{StreamingMiniClusterResourceExtension}} that accepts a
> {{MiniClusterResource}} as an argument and contains the streaming parts.
> Usage would look like this:
> {code}
> private final MiniClusterResource cluster = ...
> private final StreamingMiniClusterResourceExtension ext = new
> StreamingMiniClusterResourceExtension(cluster);
> @Rule
> public RuleChain chain= RuleChain
> .outerRule(cluster)
> .around(ext),
> {code}
> h5. 3)
> Add a new class {{ClientMiniClusterResourceExtension}} that accepts a
> {{MiniClusterResource}} as an argument and contains the client parts.
> Usage would look like this:
> {code}
> private final MiniClusterResource cluster = ...
> private final ClientMiniClusterResourceExtensionext = new
> ClientMiniClusterResourceExtension(cluster);
> @Rule
> public RuleChain chain= RuleChain
> .outerRule(cluster)
> .around(ext),
> {code}
> [~till.rohrmann] WDYT?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10294) Split MiniClusterResource to be usable in runtime/streaming-java/clients
[
https://issues.apache.org/jira/browse/FLINK-10294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16625556#comment-16625556
]
Chesnay Schepler commented on FLINK-10294:
--
Yeah I guess that would make things simpler, the cluster resource could be
exposed with a simple getter.
> Split MiniClusterResource to be usable in runtime/streaming-java/clients
>
>
> Key: FLINK-10294
> URL: https://issues.apache.org/jira/browse/FLINK-10294
> Project: Flink
> Issue Type: Improvement
> Components: Tests
>Affects Versions: 1.7.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Fix For: 1.7.0
>
>
> h5. Problem
> The {{MiniClusterResource}} is a utility class to create and manage flink
> clusters for testing purposes. It is incredibly convenient, but unfortunately
> resides in {{flink-test-utils}} which depends on flink-runtime,
> flink-streaming-java and flink-clients, making the class not usable in these
> modules.
> The current version does require these dependencies, but only for specific,
> *optional*, parts. {{streaming-java}} is only required for accessing
> {{TestStreamEnvironment}} and {{flink-clients}} only for tests that want to
> work against a {{ClusterClient}}.
> h5. Proposal
> Split the {{MiniClusterResource}} as follows:
> h5. 1)
> Remove client/streaming-java dependent parts and move the class to
> flink-runtime.
> h5. 2)
> Add a new class {{StreamingMiniClusterResourceExtension}} that accepts a
> {{MiniClusterResource}} as an argument and contains the streaming parts.
> Usage would look like this:
> {code}
> private final MiniClusterResource cluster = ...
> private final StreamingMiniClusterResourceExtension ext = new
> StreamingMiniClusterResourceExtension(cluster);
> @Rule
> public RuleChain chain= RuleChain
> .outerRule(cluster)
> .around(ext),
> {code}
> h5. 3)
> Add a new class {{ClientMiniClusterResourceExtension}} that accepts a
> {{MiniClusterResource}} as an argument and contains the client parts.
> Usage would look like this:
> {code}
> private final MiniClusterResource cluster = ...
> private final ClientMiniClusterResourceExtensionext = new
> ClientMiniClusterResourceExtension(cluster);
> @Rule
> public RuleChain chain= RuleChain
> .outerRule(cluster)
> .around(ext),
> {code}
> [~till.rohrmann] WDYT?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-10294) Split MiniClusterResource to be usable in runtime/streaming-java/clients
[
https://issues.apache.org/jira/browse/FLINK-10294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16615198#comment-16615198
]
Till Rohrmann commented on FLINK-10294:
---
I like the idea :-)
Maybe one thought concerning the chaining of rules. It might be simpler to let
the {{StreamingMiniClusterResourceExtension}} and
{{ClientMiniClusterResourceExtension}} instantiate a {{MiniClusterResource}}
internally. Then it would be simply:
{code}
@Rule
public final ClientMiniClusterResource miniCluster = new
ClientMiniClusterResource()
{code}
> Split MiniClusterResource to be usable in runtime/streaming-java/clients
>
>
> Key: FLINK-10294
> URL: https://issues.apache.org/jira/browse/FLINK-10294
> Project: Flink
> Issue Type: Improvement
> Components: Tests
>Affects Versions: 1.7.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Fix For: 1.7.0
>
>
> h5. Problem
> The {{MiniClusterResource}} is a utility class to create and manage flink
> clusters for testing purposes. It is incredibly convenient, but unfortunately
> resides in {{flink-test-utils}} which depends on flink-runtime,
> flink-streaming-java and flink-clients, making the class not usable in these
> modules.
> The current version does require these dependencies, but only for specific,
> *optional*, parts. {{streaming-java}} is only required for accessing
> {{TestStreamEnvironment}} and {{flink-clients}} only for tests that want to
> work against a {{ClusterClient}}.
> h5. Proposal
> Split the {{MiniClusterResource}} as follows:
> h5. 1)
> Remove client/streaming-java dependent parts and move the class to
> flink-runtime.
> h5. 2)
> Add a new class {{StreamingMiniClusterResourceExtension}} that accepts a
> {{MiniClusterResource}} as an argument and contains the streaming parts.
> Usage would look like this:
> {code}
> private final MiniClusterResource cluster = ...
> private final StreamingMiniClusterResourceExtension ext = new
> StreamingMiniClusterResourceExtension(cluster);
> @Rule
> public RuleChain chain= RuleChain
> .outerRule(cluster)
> .around(ext),
> {code}
> h5. 3)
> Add a new class {{ClientMiniClusterResourceExtension}} that accepts a
> {{MiniClusterResource}} as an argument and contains the client parts.
> Usage would look like this:
> {code}
> private final MiniClusterResource cluster = ...
> private final ClientMiniClusterResourceExtensionext = new
> ClientMiniClusterResourceExtension(cluster);
> @Rule
> public RuleChain chain= RuleChain
> .outerRule(cluster)
> .around(ext),
> {code}
> [~till.rohrmann] WDYT?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
