Ufuk Celebi created FLINK-3172:
--
Summary: Specify jobmanager port in HA mode
Key: FLINK-3172
URL: https://issues.apache.org/jira/browse/FLINK-3172
Project: Flink
Issue Type: Bug
Compon
Stephan Ewen created FLINK-3171:
---
Summary: Consolidate zoo of wrapper classes for
input/output-stream to data-input/output-view
Key: FLINK-3171
URL: https://issues.apache.org/jira/browse/FLINK-3171
Proj
Yes, that would be the way to go.
We could follow Cask CDAP hydrator plugin repository [1] that support
different plugins to run in their main CDAP hydrator [2] product
- Henry
[1] https://github.com/caskdata/hydrator-plugins
[2] https://github.com/caskdata/cdap
On Mon, Dec 14, 2015 at 1:49 AM
Hi!
The Netty memory usually goes much lower than 2*network memory (that is
theoretical).
Netty needs memory at the size two buffers on the sender and receiver side,
per TCP connection.
Since Flink usually multiplexes many Channels (that need network buffers)
through the same TCP connection, the
Just to confirm, building with maven 3.2.5 appears to work as expected.
Thanks a lot for the work-around.
On Thu, Dec 10, 2015 at 10:23 AM, Robert Metzger
wrote:
> You are right. I'll post the link with my next message on the maven user
> list.
> Here is the link to the maven discussion:
>
> htt
Maximilian Michels created FLINK-3170:
-
Summary: Expose task manager metrics via JMX
Key: FLINK-3170
URL: https://issues.apache.org/jira/browse/FLINK-3170
Project: Flink
Issue Type: New F
Ping squirrels! Any thoughts/opinions on this?
On 9 December 2015 at 20:40, Vasiliki Kalavri
wrote:
> Hello squirrels,
>
> I have been discussing with the Apache Tinkerpop [1] community regarding
> an integration with Flink/Gelly.
> You can read our discussion in [2].
>
> Tinkerpop has a graph t
Stephan Ewen created FLINK-3169:
---
Summary: Drop {{Key}} type from Record Data Model
Key: FLINK-3169
URL: https://issues.apache.org/jira/browse/FLINK-3169
Project: Flink
Issue Type: Improvement
If I’m not mistaken copying is still performed in the streaming API by default.
> On 14 Dec 2015, at 13:20, Márton Balassi wrote:
>
> Thanks for writing this up, Gábor. As Aljoscha suggested chaining changes
> all of these and makes it very tricky to work with these which should be
> clearly doc
Hi Stephan,
I figured it out. The problem was that the date/time was different on all
3 nodes. Zookeeper thought that it hadn’t heard from the other nodes for
longer than the allowed period and dropped them, therefore causing the
other two task managers in the cluster to fail. I synchronized the t
Gyula Fora created FLINK-3168:
-
Summary: Make checkpointing properties configurable from
flink-conf.yaml
Key: FLINK-3168
URL: https://issues.apache.org/jira/browse/FLINK-3168
Project: Flink
Issu
Thanks for writing this up, Gábor. As Aljoscha suggested chaining changes
all of these and makes it very tricky to work with these which should be
clearly documented. That was the reason while some time ago the streaming
API always copied the output of a UDF by default to avoid this ambiguous
cases
I guess chaining happens so often, that we should just write this doc
assuming that there is chaining, and not even describe the rules for
the non-chaining case. I mean I would never risk writing a UDF that
only works when there is no chaining, and then constantly worry about
when do I accidentally
oh, sorry, I misread. Just my +1 to renaming OperatorState then :-)
On Mon, Dec 14, 2015 at 11:38 AM, Aljoscha Krettek
wrote:
> As I mentioned in my previous mail, I think that OperatorState would need
> be replaced by more specific types of state (ValueState, ListState, …).
>
> > On 14 Dec 2015
+1 to all changes proposed, that is a reasonable step towards incremental
snapshots and proper reconfiguration support. What is more interesting though
is the actual implementations of the KVState derivatives, I am looking forward
to see what you have in mind there. The operator/UDF KV namespace
As I mentioned in my previous mail, I think that OperatorState would need be
replaced by more specific types of state (ValueState, ListState, …).
> On 14 Dec 2015, at 11:34, Maximilian Michels wrote:
>
>>
>> On a side not, why would you call it KvState? And what would be called
>> KvState?
>
>
> On a side not, why would you call it KvState? And what would be called
> KvState?
The OperatorState interface would be called KvState.
On Mon, Dec 14, 2015 at 11:18 AM, Aljoscha Krettek
wrote:
> Yes, as Kostas said, it would initially nor provide more functionality but
> it would enable u
Good write up. You could extend the Table of 1) a/b 2) a/b at the top with
“chaining” (but you already know this, I guess). Chaining changes all of these
and I think it can be tricky to know whether stuff is chained or not (for
users, and even for us developers…).
> On 13 Dec 2015, at 19:24, G
Yes, as Kostas said, it would initially nor provide more functionality but it
would enable us to add it later.
On a side not, why would you call it KvState? And what would be called KvState?
> On 14 Dec 2015, at 11:14, Kostas Tzoumas wrote:
>
> I suppose that they can start as sugar and evolve
I suppose that they can start as sugar and evolve to a different
implementation.
I would +1 the name change to KVState, OperatorState is indeed somewhat
confusing, and it will only get harder to rename later.
On Mon, Dec 14, 2015 at 11:09 AM, Gyula Fóra wrote:
> Would the Reducing/Folding state
Would the Reducing/Folding states just be some API sugar on top of what we
have know (ValueState) or does it have some added functionality (like
incremental checkpoints for list states)?
Gyula
Aljoscha Krettek ezt írta (időpont: 2015. dec. 14.,
H, 11:03):
> While enhancing the state interfaces
While enhancing the state interfaces we would also need to introduce new types
of state. I was thinking of these, for a start:
- ValueState (works like OperatorState works now, i.e. provides methods to
get/set one state value
- ListState, proves methods to add one element to a list of elements
A lot of this makes sense, but I am not sure about renaming
"OperatorState". The other name is nicer, but why make users' life hard
just for a name?
On Mon, Dec 14, 2015 at 10:46 AM, Maximilian Michels wrote:
> Hi Aljoscha,
>
> Thanks for the informative technical description.
>
> > - function
>
> Regarding Max suggestion to have version compatible connectors: I'm not
> sure if we are able to maintain all connectors across different releases.
>
That was not my suggestion. Whenever we release, existing connectors should
be compatible with that release. Otherwise, they should be removed f
Hi Aljoscha,
Thanks for the informative technical description.
> - function state: this is the state that you get when a user function
> implements the Checkpointed interface. it is not partitioned
> - operator state: This is the state that a StreamOperator can snapshot, it
> is similar to th
Regarding Max suggestion to have version compatible connectors: I'm not
sure if we are able to maintain all connectors across different releases. I
think its okay to have a document describing the minimum required Flink
version for each connector.
With the interface stability guarantees from 1.0 o
Yes, absolutely. Setting up another repository for Flink ML would be no problem.
On Sat, Dec 12, 2015 at 1:52 AM, Henry Saputra wrote:
> I had small chat with Till about how to help manage Flink ML Libraries
> contributions, which use Flink ML as dependencies.
>
> I suppose if this approached is
27 matches
Mail list logo
