Hi!
This issue has been fixed very recently and the fix will go into the
upcoming bugfix release. (0.10.1)
Should be out in the next few days :)
Cheers
Gyula
On Tue, Nov 24, 2015 at 4:49 AM Welly Tambunan wrote:
> Hi All,
>
> Finally i've found the solution for killing the job manager.
>
>
> h
Hi All,
Finally i've found the solution for killing the job manager.
https://flink.apache.org/faq.html#i-cant-stop-flink-with-the-provided-stop-scripts-what-can-i-do
But i do really hope that we have that cancel button for restarting job.
Cheers
On Tue, Nov 24, 2015 at 8:30 AM, Welly Tambuna
Hi
Is it possible to run Flink in Cloudfoundry Environment? if yes, How can we
achive this?
Any help is appreciated. Thanks in Advance.
Thanks,
Madhu
Hi All,
Is there any way to stop/cancel the job that's restarting ?
I have already stop the cluster and start it again but seems it's still
restarting in dashboard.
I also try to cancel the job via CLI by running bin/flink cancel
but it's not working.
Cheers
--
Welly Tambunan
Triplelands
Hello Robert,
Added following
env.java.opts: "-Dstack.name=phd -Dstack.version=3.0.0.0-249"
Same Error
Is there any config which allows to pass special java opts to actual yarn
containers?
Thanks,
Jagat Singh
On Mon, Nov 23, 2015 at 9:21 PM, Robert Metzger wrote:
> Hi,
>
> In Flink the
We are not yet using HA in our cluster instances.
But yes, we will have to change the zookeeper.path.root ☺
We package our jobs with their own config folder (we don’t rely on flink’s
config folder); we can put the maven project name into this property then they
will have different values ☺
Fro
The problem is the execution graph handle which is stored in ZooKeeper. You
can manually remove it via the ZooKeeper shell by simply deleting
everything below your `recovery.zookeeper.path.root` ZNode. But you should
be sure that the cluster has been stopped before.
Do you start the different clus
OK, I understand.
Maybe we are not really using flink as you intended. The way we are using it,
one cluster equals one job. That way we are sure to isolate the different jobs
as much as possible and in case of crashes / bugs / (etc) can completely kill
one cluster without interfering with the o
Hi All,
this is the first bugfix release for the 0.10 series of Flink.
I've CC'ed the user@ list if users are interested in helping to verify the
release.
It contains fixes for critical issues, in particular:
- FLINK-3021 Fix class loading issue for streaming sources
- FLINK-2974 Add periodic off
One addition: You can set the system to use "ingestion time", which gives
you event time with auto-generated timestamps and watermarks, based on the
time that the events are seen in the sources.
That way you have the same simplicity as processing time, and you get the
window alignment that Aljosch
Hi,
@Konstantin: are you using event-time or processing-time windows. If you are
using processing time, then you can only do it the way Fabian suggested. The
problem here is, however, that the .keyBy().reduce() combination would emit a
new maximum for every element that arrives there and you nev
Hey Gwenhaël,
the restarting jobs are most likely old job submissions. They are not cleaned
up when you shut down the cluster, but only when they finish (either regular
finish or after cancelling).
The workaround is to use the command line frontend:
bin/flink cancel JOBID
for each RESTARTING
Yes, you are right I think we should have some nice abstractions for doing
this.
Before the rewrite of the windowing runtime to support out-of-order events,
we had abstractions for supporting this but that code was not feasible
from performance perspective. (The result of a keyed window reduce u
Thanks!
@Fabian: Yepp, but this still results in multiple outputs per window,
because the maximum is emitted for every key.
@Gyula: Yepp, that's the second bullet point from my question ;) The way
I implemented it, it basically doubles the latency, because the
timeWindowAll has to wait for the ne
Hi,
Alright it seems there are multiple ways of doing this.
I would do something like:
ds.keyBy(key)
.timeWindow(w)
.reduce(...)
.timeWindowAll(w)
.reduce(...)
Maybe Aljoscha could jump in here :D
Cheers,
Gyula
Fabian Hueske ezt írta (időpont: 2015. nov. 23., H,
11:21):
> If you set the key
Hi,
In Flink the configuration parameter for passing custom JVM options is
"env.java.opts". I would recommend to put it into the
conf/flink-config.yaml like this:
env.java.opts: "-Dhdp.version=2.3.0.0-2557 -Dhdp.version=2.3.0.0-2557"
Please let me know if this works.
Maybe you are the first user
If you set the key to the time attribute, the "old" key is no longer valid.
The streams are organized by time and only one aggregate for each
window-time should be computed.
This should do what you are looking for:
DataStream
.keyBy(_._1) // key by orginal key
.timeWindow(..)
.apply(...) /
Hi,
Can't you use a second keyed window (with the same size) and apply
.max(...)?
-Matthias
On 11/23/2015 11:00 AM, Konstantin Knauf wrote:
> Hi Fabian,
>
> thanks for your answer. Yes, that's what I want.
>
> The solution you suggest is what I am doing right now (see last of the
> bullet poin
Hi Fabian,
thanks for your answer. Yes, that's what I want.
The solution you suggest is what I am doing right now (see last of the
bullet point in my question).
But given your example. I would expect the following output:
(key: 1, w-time: 10, agg: 17)
(key: 2, w-time: 10, agg: 20)
(key: 1, w-ti
Hi Konstantin,
let me first summarize to make sure I understood what you are looking for.
You computed an aggregate over a keyed event-time window and you are
looking for the maximum aggregate for each group of windows over the same
period of time.
So if you have
(key: 1, w-time: 10, agg: 17)
(key
Hi Nick,
This is easily achievable using the framework I provide.
createDataStream(Input input) does actually return a DataStreamSource.
And the call of assertStream(DataStream datastream, OutputMatcher
matcher) just attaches a TestSink to the datastream, but you can create
the test sink manually
There is also this guide:
https://cwiki.apache.org/confluence/display/FLINK/Migration+Guide%3A+0.9.x+to+0.10.x
On Monday, 23 November 2015, Stephan Ewen wrote:
> Hi Arnaud!
>
> In 0.10 , we renamed the dependency to "flink-streaming-java" (and
> flink-streaming-scala"), to be more in line with t
Hi Arnaud!
In 0.10 , we renamed the dependency to "flink-streaming-java" (and
flink-streaming-scala"), to be more in line with the structure of the
dependencies on the batch side.
Just replace "flink-streaming-core" with "flink-streaming-java"...
Greetings,
Stephan
On Mon, Nov 23, 2015 at 9:07
This is what I used for a Flink Streaming talk and demo at a meetup last
week, this is with Flink 0.10.0
org.apache.flink
flink-core
${flink.version}
org.apache.flink
flink-streaming-java
${flink.version}
org.apache.flink
flink-runtime
${flink.version}
org.apa
Hello,
Small question: I can't find the Streaming Core component in 0.10 version in
the maven repo :
http://mvnrepository.com/artifact/org.apache.flink/flink-streaming-core
Thus in my pom file this artifact is the only part of my Flink's dependencies
to stay in 0.10-SNAPSHOT version.
Is there s
25 matches
Mail list logo
