ut it did not help.
>
> Regards,
> Vijay
>
> Sent from my iPhone
>
>> On Mar 23, 2016, at 5:39 AM, Aljoscha Krettek wrote:
>>
>> Ok, then you should be able to change the log level to DEBUG in
>> conf/log4j.properties.
>>
>>> On 23
changes include both
> Sytem.out as well as logger statements.
>
> Updated log4j property file to DEBUG
>
> Regards,
> Vijay
>
> Sent from my iPhone
>
>> On Mar 23, 2016, at 6:48 AM, Aljoscha Krettek wrote:
>>
>> Hi,
>> what where the step
> Regards,
> Vijay
>
> Sent from my iPhone
>
>> On Mar 23, 2016, at 7:18 AM, Aljoscha Krettek wrote:
>>
>> Hi,
>> did you update the log4j.properties file on all nodes where the TaskManagers
>> run and did you restart the whole cluster?
>>
>>
t; implementation class, I am seeing the part file with empty content though the
> file is renamed from "in-progress" to actual file name.
> Question: Do you know if RollingSink implementation is tested with any Hadoop
> Compatible File System like GlusterFS, etc.,?
> Regard
med from "in-progress" to actual file name.
> Question: Do you know if RollingSink implementation is tested with any
> Hadoop Compatible File System like GlusterFS, etc.,?
> RegardsVijay
>
> On Wednesday, March 23, 2016 7:42 AM, Aljoscha Krettek <
> aljos...@apach
Hi,
the current watermark position is not persisted. The sources will start
emitting based on the elements that they are seeing. So if the source is
reset to some previous reading position (say with Kafka). Then it will emit
the watermarks based on those earlier elements again.
Cheers,
Aljoscha
O
Hi,
it should already be possible to use the Ignite FileSystem to store state
since we just use the HDFS FileSystem interface for that. Of course, one
would have to properly set up the jars and paths and everything for Flink
to pick up the IGFS classes.
Cheers,
Aljoscha
On Wed, 30 Mar 2016 at 16:
Hi,
yes you can output the stages to several different Kafka Topics. If you
don't want to call addSink inside the run() method you somehow have to
return the handle to your stage3 DataStream, for example:
private val env = StreamExecutionEnvironment.getExecutionEnvironment
private val src = env.ad
I ran the usual tests and the release seems fine. A user, however, found
this bug which seems to be a blocker:
https://issues.apache.org/jira/browse/FLINK-3688?jql=project%20%3D%20FLINK
Should we maybe wait for the fix and then to another RC?
On Thu, 31 Mar 2016 at 18:10 Ufuk Celebi wrote:
> De
perfect
On Fri, 1 Apr 2016 at 15:51 Bobby Evans wrote:
> Today I am happy to announce a prototype of an Apache Storm based backend
> for Apache Flink called S-link-y. Storm offers a number of advantages over
> Flink including enterprise level security and a distributed cache with in
> place upd
Yes, that seems reasonable.
On Sat, 2 Apr 2016 at 13:14 Ufuk Celebi wrote:
> Thanks for the initial tests.
>
> I would say let's continue with this RC and wait for the user to
> provide the fix, which we can then include in 1.0.2 soon.
>
> On Fri, Apr 1, 2016 at 2:
Hi,
I like it. Very dense and very focused on the example but I think it should
be good for the Flink Blog.
--
aljoscha
On Fri, 1 Apr 2016 at 15:56 Till Rohrmann wrote:
> Hi Flink community,
>
> I've written a short blog [1] post about Flink's new CEP library which
> basically showcases its fun
ne quick question regarding the release:
> >> > Are you intentionally releasing a 1.0.1 version of the force-shading
> >> module?
> >> >
> >> > Since it is really a work around, I wounder if we should not release
> the
> >> > force-shading anym
Comments are not enabled.
On Mon, 4 Apr 2016 at 13:58 Till Rohrmann wrote:
> Hi Flink community,
>
> I recently started working on dynamic scaling. As a first step we want to
> introduce state sharding which is a requirement for partitioned state to be
> re-distributable. The following design do
Hi Folks,
as part of my effort to improve the windowing in Flink [1] I also thought
about lateness, accumulating/discarding and window cleanup. I have some
ideas on this but I would love to get feedback from the community as I
think that these things are important for everyone doing event-time
wind
al case it ...
On Tue, 5 Apr 2016 at 12:23 Aljoscha Krettek wrote:
> Hi Folks,
> as part of my effort to improve the windowing in Flink [1] I also thought
> about lateness, accumulating/discarding and window cleanup. I have some
> ideas on this but I would love to get feedback from the
I also found an issue in the RocksDB backend (not introduced by me... ;-):
https://issues.apache.org/jira/browse/FLINK-3730
It's not that critical, however.
On Mon, 11 Apr 2016 at 18:40 Till Rohrmann wrote:
> Given that it's low overhead, then +1 for new bug fix release. If we find
> more stuff
first RC
> today.
>
> – Ufuk
>
> On Mon, Apr 11, 2016 at 5:55 PM, Aljoscha Krettek
> wrote:
> > I also found an issue in the RocksDB backend (not introduced by me...
> ;-):
> > https://issues.apache.org/jira/browse/FLINK-3730
> >
> > It's not that cr
Thanks, sorry for the inconvenience :-O
On Thu, 14 Apr 2016 at 16:11 Ufuk Celebi wrote:
> Just started the vote but let's include it. No biggie. I will re-run the
> script later today.
>
> On Thursday, 14 April 2016, Aljoscha Krettek wrote:
>
> > Hi,
> > a
-1
A user just discovered this bug:
https://issues.apache.org/jira/browse/FLINK-3760 which seems somewhat
critical because there is no workaround.
On Thu, 14 Apr 2016 at 16:05 Ufuk Celebi wrote:
> Dear Flink community,
>
> Please vote on releasing the following candidate as Apache Flink version
Hi,
Flink does not make any guarantees about the order of arriving elements
except in the case of one-to-one forwarding patterns. That is, only for
map/flatMap/filter and such operations will the order in which two
successive operations see their elements be the same.
Could you please describe in
There is also InputFormat.configure() which is called before any split
processing happens. But I see your point about a missing close() method
that is called after all input splits have been processed.
On Mon, 18 Apr 2016 at 09:44 Stefano Bortoli wrote:
> Of course there is one already. We'll lo
eed to do is probably manage the window myself in
> operator state (because in many cases I expect that I will not need the
> whole window, so it may be interesting to be able to evaluate that
> eagerly), but I think I really need the events to arrive in order.
>
> On 14 April 2016 at
+1, since others are blocked on this
On Mon, 18 Apr 2016 at 12:10 Ufuk Celebi wrote:
> I am fine with it since it only touches a utility class. So +1 to
> include it as well.
>
> – Ufuk
>
> On Mon, Apr 18, 2016 at 12:07 PM, Fabian Hueske wrote:
> > There is also FLINK-3657 which was recently me
+1
I eyeballed the changes and nothing looks suspicious.
On Wed, 20 Apr 2016 at 13:21 Fabian Hueske wrote:
> Thanks Ufuk for preparing the RC.
>
> - Checked the diff against release 1.0.1. No dependencies were added or
> modified.
> - Checked signatures and hashes of all release artifacts.
>
>
Hi,
setting the uid on the result of the .apply() call is sufficient for the
whole window operation, including the windowing and trigger.
Could you maybe post some example code of the topology before and after the
change and restore from savepoint?
Cheers,
Aljoscha
On Tue, 19 Apr 2016 at 07:33 O
> http://trevorgrant.org
> >
> > *"Fortunate is he, who is able to know the causes of things." -Virgil*
> >
> >
> > On Wed, Apr 20, 2016 at 6:51 AM, Aljoscha Krettek
> > wrote:
> >
> >> +1
> >>
> >> I eyeballed the
gt; > thanks
> > > > to
> > > > >> > > > Chiwan and Aljoscha for looking into it as well.
> > > > >> > > >
> > > > >> > > > I had an offline chat with Gyula about the rolling file sink
> > > >
Hi Team,
I'm currently thinking about how we can bring the broadcast set/broadcast
input feature form the DataSet API to the DataStream API. I think this
would be a valuable addition since it would enable use cases that join
streams with constant (or slowly changing) side information.
For this pur
nd ds.connect() to many streams
>
> But in any case this is awesome initiative :)
>
> Cheers,
> Gyula
>
>
> Aljoscha Krettek ezt írta (időpont: 2016. ápr. 21.,
> Cs, 15:56):
>
> > Hi Team,
> > I'm currently thinking about how we can bring the broadcast
sers:
> >> https://issues.apache.org/jira/browse/FLINK-3701
> >>
> >> On Thu, Apr 21, 2016 at 5:53 PM, Till Rohrmann
> >> wrote:
> >> > I found another critical issue for the next bug fix release:
> >> > https://issues.apache.org/jira/b
https://issues.apache.org/jira/browse/FLINK-3701 does not affect the 1.0.x
series of releases, I think we also discussed this in the 1.0.2 release
thread.
On Fri, 22 Apr 2016 at 10:39 Kostas Kloudas
wrote:
> I am working on:
> https://issues.apache.org/jira/browse/FLINK-2314?filter=-1 <
> https:
assigned and also verify that it is actually used (in contrast
> to the isEventTime() method). Would that be an option or would it
> break the API?
>
> Cheers,
> Max
>
> On Tue, Apr 5, 2016 at 12:29 PM, Aljoscha Krettek
> wrote:
> > By the way. The way I see to fixing
Hi,
I think what we did for now is this: When merging a PR onto the master that
we also consider to be required in a future release of an older release
branch we cherry-pick it there and make the required changes. This only
works for committers, of course, since we can just do that.
I think in you
Hi,
I've put down some words about how we would go about adding support for
side inputs to the streaming API. People are increasingly asking for such a
feature, which would allow joining a main stream with some slowly changing
inputs that can be used for enriching data, among other things.
The des
Hi Folks,
I added a section about "Implementation Details" to the doc, it is still
very rough but I'm afraid this is somewhat tricky and might require some
changes to how we handle state.
As always, feedback is very welcome!
Cheers,
Aljoscha
On Mon, 2 May 2016 at 11:28 Aljoscha
Hi,
I'm afraid there is no nice API way of doing it but you should be able to
access the REST interface of the JobManager to get information about all
TaskManagers. For example on a locally started cluster on my machine I get
this when accessing http://localhost:8081/taskmanagers:
{"taskmanagers":
Hi Gábor,
I used it, yes, but I never encountered such a problem. Let's hope that the
error message Márton discovered helps. :-)
Cheers,
Aljoscha
On Mon, 9 May 2016 at 11:38 Márton Balassi wrote:
> Hi Gabor,
>
> I have checked out your branch and tried debugging WordCountPojo to
> reproduce the
I think Till mistyped, sinks are always parallel, you don't need the
ParallelSinkFunction.
On Tue, 10 May 2016 at 18:56 M Singh wrote:
> Hi Till, Folks:
> I found ParallelSourceFunction/RichParallelSourceFunctions in the
> flink-streaming-java_2.10 module (1.1-SNAPSHOT), but could not find
> Par
I think RequiredParameters is meant to be used with ParameterTool. For
example, check out RequiredParametersTest.
On Wed, 11 May 2016 at 11:05 Flavio Pompermaier
wrote:
> Hi flinkers,
> I was converting my parameter validation code to the new (and very useful)
> RequiredParameters APIs. However
I favor the one-cluster-per job approach. If this becomes the dominant
approach to doing things we could also think about introducing a separate
component that would allow monitoring the jobs in these per-job clusters as
is now possible when running multiple jobs in a single cluster.
On Thu, 12 Ma
Hi,
I agree that this would be very nice. Unfortunately Flink does only allow
one output from an operation right now. Maybe we can extends this somehow
in the future.
Cheers,
Aljoscha
On Thu, 12 May 2016 at 17:27 CPC wrote:
> Hi Gabor,
>
> Yes functionally this helps. But in this case i am proc
+1
The ideas seem good and the proposed number of components seems reasonable.
With this, we should also then cleanup the JIRA to make it actually usable.
On Thu, 12 May 2016 at 18:09 Stephan Ewen wrote:
> All maintainer candidates are only proposals so far. No indication of lead
> or anything
Hi,
with how the window API currently works this can only be done for
non-parallel windows. For keyed windows everything that happens is scoped
to the key of the elements: window contents are kept in per-key state,
triggers fire on a per-key basis. Therefore a count-min sketch cannot be
used becaus
A great post!
I had some small comments on the doc.
On Sat, 21 May 2016 at 16:52 Robert Metzger wrote:
> Thanks a lot for the great blog post!
>
> +1 for publishing it on the Flink blog.
>
> On Fri, May 20, 2016 at 5:12 PM, Fabian Hueske wrote:
>
> > Hi everybody,
> >
> > I wrote a blog post a
that).
> I will also vote for closing the old issue too since there is no other way
> at least for the time being...
>
> Thanx,
> Stavros
>
> On Fri, May 20, 2016 at 7:02 PM, Aljoscha Krettek
> wrote:
>
> > Hi,
> > with how the window API currently works thi
streaming setting use redis for
> its hyperlolog algo.
>
> What are other areas you find more important or of higher priority for the
> time being?
>
> Best,
> Stavros
>
> On Mon, May 23, 2016 at 6:18 PM, Aljoscha Krettek
> wrote:
>
> > Hi,
> >
Hi,
what we are doing in most internal tests is to verify in a sink whether the
data is correct and then throw a SuccessException. This brings down the job
and we check whether we catch a SuccessException to verify that the test
was successful. Look, for example, at the ValidatingSink in
EventTimeW
Hi,
I'm afraid this is a bug. I open a Jira Issue for it:
https://issues.apache.org/jira/browse/FLINK-3977
Cheers,
Aljoscha
On Thu, 26 May 2016 at 09:13 Erik wrote:
> Hi,
>
> I get a "No initial value was serialized for the fold window function.
> Probably the setOutputmethod was not called" wh
Hi,
theoretically, it could work (if I'm not missing anything). But the
savepoint would be lost as soon as the JobManager is shut down.
Cheers,
Aljoscha
On Fri, 27 May 2016 at 18:09 Gyula Fóra wrote:
> Hey!
>
> The flink docs on savepoints say that they cannot be used in combination
> with the
Not sure about that. Maybe Ufuk can chime in.
On Fri, 27 May 2016 at 19:40 Gyula Fóra wrote:
> Not if the savepoint backend is Filesystem, right? So it shouldnt really be
> a matter of the statebackend, only the savepoint backend.
>
> Gyula
>
> On Fri, May 27, 2016, 18:4
that we always discard
when firing. Please take a look at the new doc to see what I'm talking
about there.
Feedback very welcome!
Cheers,
Aljoscha
On Tue, 26 Apr 2016 at 16:45 Aljoscha Krettek wrote:
> Hi Max,
> thanks for the Feedback and suggestions! I'll try and address
Thanks for the feedback! :-) I already read the comments on the file.
On Mon, 30 May 2016 at 11:10 Gyula Fóra wrote:
> Thanks Aljoscha :) I added some comments that might seem relevant from the
> users point of view.
>
> Gyula
>
> Aljoscha Krettek ezt írta (időpont: 2016. má
That last section is a really good Idea! I have several design docs
floating around that were announced on the ML. Without a central place to
store them they are hard to find, though.
-Aljoscha
On Tue, 31 May 2016 at 11:27 Stephan Ewen wrote:
> Hi!
>
> There was some preliminary work on this.
Hi,
I think this is an artifact from the past. Using the "non-reuse"
deserialize seems more correct, especially in the presence of subclasses.
Best,
Aljoscha
On Mon, 30 May 2016 at 19:13 Gábor Horváth wrote:
> Hi!
>
> While I was working on code generation support for PojoComparators, I
> stumb
Hi,
I think maintainer is also fine if we clearly specify that they are not
meant as dictators or gatekeepers of the component that they are
responsible for.
-Aljoscha
On Wed, 1 Jun 2016 at 09:48 Vasiliki Kalavri
wrote:
> Hi,
>
> we could go for something like "sponsor" or "champion" :)
> I'm f
Btw, in Jira, if we clean up our components we can also set a component
Lead that would get notified of issues for that component.
On Wed, 1 Jun 2016 at 10:43 Chesnay Schepler wrote:
> I'd also go with maintainer.
>
> On 01.06.2016 10:32, Aljoscha Krettek wrote:
> > Hi,
&g
I just wrote an answer for this on Stackoverflow:
The problem is in this function:
@Override
public TypeInformation getProducedType() {
// TODO Auto-generated method stub
return null;
}
you cannot return null here.
On Thu, 2 Jun 2016 at 17:57 Akshay Thaka Shingote
wrote:
> Hello,
>
>
r is not a good idea.
> > > >>> The Spark community had a maintainer process which they just voted
> to
> > > >>> remove. From my understanding, a maintainer in Spark had a more
> > active
> > > >> role
> > > >>> than the r
t; > >>
> > > > > >> On Wed, Jun 1, 2016 at 7:11 PM, Fabian Hueske <
> fhue...@gmail.com>
> > > > > wrote:
> > > > > >>
> > > > > >>> I think calling the role maintainer is not a good idea.
> > > > >
Hi,
the static field will be null after the code has been shipped to the
cluster. You can use a RichMapFunction instead, there you can define an
open() method to initialize such a field. Please see this part of the doc
for rich functions:
https://ci.apache.org/projects/flink/flink-docs-master/apis/
Hi,
+1 for releasing what we have now and making it rock solid.
The support for merging windows (session windows) is also new, btw. :-)
Cheers,
Aljoscha
On Thu, 16 Jun 2016 at 11:12 Maximilian Michels wrote:
> Hi Robert, hi Henry,
>
> +1 for a 1.1.0 release soon! We have enough new features th
Hi,
could you please provide a complete minimal example and the commands that
you use to start the cluster and run the job?
Cheers,
Aljoscha
On Fri, 17 Jun 2016 at 10:53 Adrian Portabales
wrote:
> Hi everyone,
> I wrote this post in stackoverflow
>
> http://stackoverflow.com/questions/37841342/
Hi,
you can also take a look at this:
https://techblog.king.com/rbea-scalable-real-time-analytics-king/. From a
high level it seems Drools implements something similar to the system that
they developed on top of Flink.
Cheers,
Aljoscha
On Tue, 21 Jun 2016 at 12:04 Robert Metzger wrote:
> Hi Ant
n Fri, Jun 17, 2016 at 11:36 AM, Adrian Portabales <
> > adrianportaba...@gmail.com> wrote:
> >
> > > Other jobs works fine in Flink (like batch wordcount) but this doesn't.
> > > I use "bin/flink run -m analytics1:6123 -c KafkaWordCountFlink
> > &
Hi Marius,
the chaining code is still somewhat fragile and some stuff in there are
leftovers. For example, StreamGroupedFold can only be used on a
KeyedStream, which means that it can never be within a chain because the
shuffle always breaks a chain. Specifying HEAD here is therefore redundant.
I t
Hi,
you are right, this seems a bit strange. The reason why this works is that
selectChannels of ForwardPartitioner is never called.
In StreamingJobGraphGenerator.connect() operators are connected by
"physical" edges. If the ForwardPartitioner is set on a stream this will
choose the POINTWISE conn
Hi,
I just ran a "mvn clean verify" and it passed on my machine (latest master,
OS X El Capitan, Java 1.8.0_40, Maven 3.3.9). What's your environment?
Cheers,
Aljoscha
On Fri, 24 Jun 2016 at 16:47 Vijay Srinivasaraghavan
wrote:
> I am seeing below failure consistently with the latest trunk code
Hi,
what timestamps are you assigning? Is it guaranteed that all of them would
fall into the same 30 second window?
The issue with duplicate printing in the ElementSelector is strange? Could
you post a more complete code example so that I can reproduce the problem?
Cheers,
Aljoscha
On Mon, 27 Ju
on Ubuntu 16.x, Java OpenJDK 1.8.0_91.
>
> Can you try below commands and see if its working with the latest trunk
> code.
>
> mvn clean verify -pl flink-yarn-tests -Pinclude-yarn-tests
> -Dtest=YARNSessionFIFOITCase#testJavaAPI
>
> mvn clean install
>
> Regards
>
ElementSelector returns tuple.f0 which is the key.
> >>
> >> I am generating the timestamp based on a field from the DTO which is
> >> guaranteed to be in order.
> >>
> >> Will using the triggers help here ?
> >>
> >>
> >> Reg
> Regards,
> Vinay Patil
>
> On Tue, Jun 28, 2016 at 2:41 PM, Aljoscha Krettek
> wrote:
>
> > Hi,
> > first regarding tumbling windows: even if you have 5 minute windows it
> can
> > happen that elements that are only seconds apart go into d
I'm proposing to add a formal process for how we deal with (major)
improvements to Flink and design docs. This has been mentioned several
times recently but we never took any decisive action to actually implement
such a process so here we go.
Right now, we have Jira issues and we sometimes we have
it will differ in real time
> streams.
>
> Regards,
> Vinay Patil
>
> On Tue, Jun 28, 2016 at 5:07 PM, Aljoscha Krettek
> wrote:
>
> > Hi,
> > ingestion time can only be used if you don't care about the timestamp in
> > the elements. So if you have t
Watermarks and generated the
> watermark for each event, still getting one record less.
>
>
> Regards,
> Vinay Patil
>
> On Wed, Jun 29, 2016 at 2:21 PM, Aljoscha Krettek
> wrote:
>
> > Hi,
> > the reason why the last element might never be emitted is the
der.
>
> Late Data : does it have a threshold after which it does not accept late
> data ?
>
>
> Regards,
> Vinay Patil
>
> On Wed, Jun 29, 2016 at 5:15 PM, Aljoscha Krettek
> wrote:
>
> > Hi,
> > the element will be kept around indefinitely if no new wa
Hi,
yes, you are definitely right that allowing to run multiple user code tasks
in the same TaskManager JVM is not good for stability. This mode is still
there from the very early days of Flink where Yarn was not yet available.
In a production environment I would now recommend to always run one
Fli
Hi,
no, it is not just for prototyping. Since it is actually the oldest mode of
execution it is also the most stable. You should have no problem using it.
Cheers,
Aljoscha
On Thu, 30 Jun 2016 at 09:54 Longda Feng
wrote:
>
>
> This means Standalone mode is just for prototype.
> But I think we ne
.createSerializer(env.getConfig()).apply(new JoinStreams());
>
> I am getting error cannot convert from DeltaTrigger to Trigger CoGroupedStreams...
> What am I doing wrong here, I have referred the sample example.
>
> Regards,
> Vinay Patil
>
> On Wed, Jun 29, 2016 at 7:15 PM,
I think it's not a question of easy-of-use but one of licensing. I don't
think anyone really knows what code ends up in vendor.js, so it is very
hard to figure out what we have to put into our LICENSE file.
On Thu, 30 Jun 2016 at 12:14 Maximilian Michels wrote:
> Hi Till,
>
> Thanks for checking
oes. In the end, we just have to properly check out licenses which
> takes time.
>
> In what sense do you think it would make licensing easier?
>
> On Thu, Jun 30, 2016 at 1:47 PM, Aljoscha Krettek
> wrote:
> > I think it's not a question of easy-of-use but one of
Hi,
you didn't post the relevant part of the log where we see the actual
failing test but just seconds ago I pushed a fix to master that fixes a
problem in one of the Storm tests. I think this should fix the failure you
are seeing.
Cheers,
Aljoscha
On Thu, 30 Jun 2016 at 15:26 F. Beligianni wrot
gt; > Tests in error:
> > WrapperSetupHelperTest.testCreateTopologyContext:185 » ClassCast class
> > sun.sec...
> > Tests run: 129, Failures: 0, Errors: 1, Skipped: 0
>
>
> I will rebase and check if the failure is fixed.
> Thanks!
>
> 2016-06-30 16:36
I'm not aware of any plans to change this. It would be possible though, to
add a mode where a standalone cluster only accepts a single job.
On Fri, 1 Jul 2016 at 04:50 Jark Wu wrote:
> As standalone mode has the disadvantage that the TaskManager JVM can’t
> isolate different jobs. Does we have a
t; > When the node comes up will it resume from the same state or will it
> > resume from the last checkpointed state ?
> >
> > Can we explicitly checkpoint inside the window , may be at the start of
> > the window or before we are applying window ?
> >
> >
> >
of including a monolithic file of javascript code in our code base. The
> > > Flink community might release code which violates other people's
> > copyright.
> > >
> > > Additionally, it might be worth the effort to harden our build system
> > > anyway
IMHO, the fix for this should also go in:
https://issues.apache.org/jira/browse/FLINK-4115. This is blocking for
users that want to use the RocksDB backend or FsStateBackend on Amazon EMR
with S3.
There is already an open PR that I'm hoping to get in this week.
On Mon, 4 Jul 2016 at 13:48 Ufuk Ce
way to achieve maximum CPU utilization and parallelism ? Does complete
> > stream chaining into a single box achieve maximum parallelism ?
> >
> > The data we are processing is huge volume of data (60,000 records per
> > second), so wanted to be sure what we can correct to achieve
Hi,
If we have some high-profile users that a worthwhile putting there and that
are OK with us putting up their logos then this would be great.
Cheers,
Aljoscha
On Mon, 4 Jul 2016 at 16:58 Stephan Ewen wrote:
> Hi all!
>
> I was wondering if we want to put some names / logos of Flink users on t
Vinay Patil
>
> On Mon, Jul 4, 2016 at 7:13 PM, Aljoscha Krettek
> wrote:
>
> > Hi,
> > this is true, yes. If the number of Kafka partitions is less than the
> > parallelism then some of the sources might not be utilized. If you
> insert a
> > rebalance a
Flink 1.1-SNAPSHOT is not a released version, this is the name of the
latest master builds of what will eventually be released as Flink 1.1.
On Mon, 4 Jul 2016 at 18:08 Vinay Patil wrote:
> Hi,
>
> Can you please tell how do I download flink1.1-SNAPSHOT for running the job
> on cluster, on the f
to
> test it on cluster with version 1.1
>
> Regards,
> Vinay Patil
>
> On Tue, Jul 5, 2016 at 2:56 PM, Aljoscha Krettek
> wrote:
>
> > Flink 1.1-SNAPSHOT is not a released version, this is the name of the
> > latest master builds of what will eventually be rel
Btw, another blocking issue, IMHO:
https://issues.apache.org/jira/browse/FLINK-4149
I'm working on a fix.
On Tue, 5 Jul 2016 at 17:08 Till Rohrmann wrote:
> I found another critical issue [1]. The murmur hash correction introduced
> between Flink 1.0 and 1.1 breaks the backwards compatibility
I think it's not strictly required that all code be in Scala. There is
already some Java code in there so we shouldn't force people to write Scala
code if they make a valuable contribution in Java.
On Tue, 5 Jul 2016 at 17:33 Flavio Pompermaier wrote:
> Hi to all,
> if Flink 1.1 will introduce u
via subtasks.
> >>
> >> If you insert a "rebalance()" between the Kafka Source and the
> >> Map/Filter/etc it makes sure that the data distribution in the
> >> Map/Filter/etc operators has best utilization independent of how the
> data
> >> w
Hi,
are you running this on Yarn. If yes, the EMR Yarn installation might
already have some of the AWS jars in the classpath and that might interact
badly with the Jars that you manually put into the flink/lib folder.
Cheers,
Aljoscha
P.S. In the future, please use the user mailing list for reque
don't give
enough space for expressing more complex opinions.
Cheers,
Aljoscha
On Mon, 30 May 2016 at 11:23 Aljoscha Krettek wrote:
> Thanks for the feedback! :-) I already read the comments on the file.
>
> On Mon, 30 May 2016 at 11:10 Gyula Fóra wrote:
>
>> Tha
I did:
https://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3ccanmxww0abttjjg9ewdxrugxkjm7jscbenmvrzohpt2qo3pq...@mail.gmail.com%3e
;-)
On Wed, 6 Jul 2016 at 15:31 Ufuk Celebi wrote:
> On Wed, Jul 6, 2016 at 3:19 PM, Aljoscha Krettek
> wrote:
> > In the future, it migh
ould appreciate any suggestion on
> whether these are viable changes or will there any performance issue if
> these are done. Also any pointer on where to start(e.g, do I create a new
> class similar to EvictingWindowOperator that extends WindowOperator?)
>
> Thanks and Re
Hi,
as mentioned in the thread on improving the Windowing API I also have a
design doc just for improving WindowEvictors. I had this in my head for a
while but was hesitant to publish but since people are asking about this
now might be a good time to post it. Here's the doc:
https://docs.google.com
301 - 400 of 1384 matches
Mail list logo
