Hi Ned,
what does the logs of the JobManager say?
Cheers,
Till
On Apr 14, 2016 9:19 AM, "ned dogg" wrote:
> Hi everybody,
>
> I'm Ned, a young and passionte developer of apache technologies. I have
> been playing with apache flink lastly.
>
> This is what I wanted to do submit a flink topology
ng and its port
> is reachable.
> And it is actaully reachable because I can ping that address.
>
> Ned.
>
> On Thu, Apr 14, 2016 at 8:43 AM, Till Rohrmann
> wrote:
>
> > Hi Ned,
> >
> > what does the logs of the JobManager say?
> >
> > Che
flink@172.31.45.232:6123]
> 2016-04-14 08:29:59,777 WARN akka.remote.ReliableDeliverySupervisor
> - Association with remote system [akka.tcp://
> flink@127.0.0.1:35953] has failed, address is now gated for [5000] ms.
> Reason is: [Disassociated].
>
>
> Wher
Hi Andrea,
have you started the Flink Yarn cluster in HA mode? Then the job manager
address is stored in ZooKeeper and you have to tell your FlinkILoop that it
should retrieve the JobManager address from there. In order to do that you
have to set conf.setString(ConfigConstants.RECOVERY_MODE,
"zook
gt; 2016-04-14 08:23:55,678 INFO
> >> > org.apache.flink.runtime.webmonitor.JobManagerRetriever - New
> >> leader
> >> > reachable under akka.tcp://
> >> flink@172.31.45.232:6123/user/jobmanager:null.
> >> > 2016-04-14 08:23:55,692 INFO
>
://gist.github.com/alkagin/0a0b2670ce77f7d9c0807b1e4ef7239a>
> Interpreter/FlinkILoop log
> <https://gist.github.com/alkagin/23e4cec15904448dd2b400a6a37f7fa7>
>
> I was looking Flink shell and it works similar to the interpreter, do it
> works with HA cluster?
>
> Thank you,
Hi Ned,
what you also could do is to ssh to your remote cluster and submit the job
using the private IP address which is reachable from within your cluster. I
don't know whether that would be applicable to your use case.
Cheers,
Till
On Fri, Apr 15, 2016 at 9:22 AM, Till Rohrmann wrote:
one who
> is supposed to handle the maintainance of the cluster.
>
>
>
>
> Le Vendredi 15 avril 2016 10h34, Till Rohrmann
> a écrit :
>
>
> Hi Ned,
>
> what you also could do is to ssh to your remote cluster and submit the job
> using the private IP addre
Hi Maxim,
I think the corresponding JIRA issue is
https://issues.apache.org/jira/browse/FLINK-456
Cheers,
Till
On Thu, Apr 14, 2016 at 10:50 PM, Maxim wrote:
> I don't have full list of metrics, but everything that is related to
> runtime performance and possible bottlenecks of the system. All
Hi Andrea,
I think your problem should be fixed with the PRs [1,2]. I've tested it
locally on my yarn cluster and it worked.
[1] https://github.com/apache/flink/pull/1904
[2] https://github.com/apache/flink/pull/1914
Cheers,
Till
On Tue, Apr 19, 2016 at 2:16 PM, Till Rohrmann wrote:
+1 for Robert's proposal.
On Thu, Apr 21, 2016 at 9:30 AM, Robert Metzger wrote:
> Hi,
>
> I'm in favor of bringing this RC out as Flink 1.0.2 for the following
> reasons:
> - The vote is over today and we have 3 +1 votes
> - At least one production user (Konstantin) urgently needs a fix contain
>
> > +1 from me as well :)
> >
> > Till Rohrmann ezt írta (időpont: 2016. ápr. 21.,
> > Cs,
> > 10:34):
> >
> > > +1 for Robert's proposal.
> > >
> > > On Thu, Apr 21, 2016 at 9:30 AM, Robert Metzger
> > > wrote:
Hi Andrew,
I think the problem is that you assume that both matrices have the same
partitioning. If you guarantee that this is the case, then you can use the
subtask index as the block index. But in the general case this is not true,
and then you have to calculate the blocks by first assigning a b
Hi Greg,
I think we haven't discussed the opportunity for a parallelized collection
input format, yet. Thanks for bringing this up.
I think it should be possible to implement a generic parallel collection
input format. However, I have two questions here:
1. Is it really a problem for users that
Till
On Wed, Apr 13, 2016 at 5:47 PM, Sergio Ramírez
wrote:
> Hello again:
>
> Any news about this problem with enriched MapPartition function?
>
> Thank you
>
>
> On 06/04/16 17:01, Sergio Ramírez wrote:
>
>> Hello,
>>
>> Ok, please find enclose
e DataSource
> subtasks request InputSplits from the JobManager and the assignment happens
> first-come-first-serve.
> Hence, the subtask ID (or partition ID) of an InputSplit is not
> deterministic and a DataSource might read more than one or also no split at
> all (such as in your ca
Thanks for looking into this problem Mathias. I think the Scala test should
be fixed as you've proposed.
Concerning the StreamExecutionEnvironmentTest.fromElementWithBaseTypeTest2,
I think it shouldn't be changed. The reason is that the class defines the
common base class of the elements. And the
Hi Flink community,
I just wanted to raise awareness that in the last 16 days there was just a
single Travis build of master which passed all tests. This indicates that
we have some serious problems with our test stability or even worse a
problem with the master itself. Having an unstable master m
between assigned and actual priorities.
> > >>
> > >> As a first step, I would volunteer to gather a list of tests, which
> > >> have failed in the last weeks and make sure that we have JIRAs for
> > >> them.
> > >>
> > >
ill fail, as Parentclass is no subtype of
> Subtype -- the call fromElements(new Parentclass() new Subclass()) would
> succeed.
>
> Makes sense?
>
> Still no idea how to make it compile in Eclipse...
>
> -Matthias
>
> On 04/27/2016 10:21 AM, Till Rohrmann wrote:
> &g
Hi Stefano,
in this case I think it's best if you opened a PR against the release
branch so that a committer can pull it in.
Cheers,
Till
On Wed, Apr 27, 2016 at 3:03 PM, Stefano Baghino <
stefano.bagh...@radicalbit.io> wrote:
> Hi Ufuk,
>
> thanks for getting back to me, I understand.
> The pr
fromElemenst(Class,
> X...) requires to specify the correct sub-type and has the additional
> advantage, the the compiler can check the type already (instead of a
> potential later runtime error).
>
>
> -Matthias
>
>
> On 04/27/2016 03:07 PM, Till Rohrmann wrote:
> >
t;
> Regards
>
>
> On 26/04/16 14:25, Till Rohrmann wrote:
>
>> Hi Sergio,
>>
>> sorry for the late reply. I figured out your problem. The reason why you
>> see apparently inconsistent results is that you execute your job multiple
>> times. Ea
Hi Trevor,
the multiple linear regression implementation is quite sensitive to the
initial learning rate. If the value is not set right, it might be the case
that the algorithm alternates between ever increasing values left and right
of the minimum. Could you try to set a smaller initial learning
Hi Maxim,
Flink does not execute the operators in the JobManager but the TaskManager.
The JobManager's role is the orchestration of the Flink job.
Unfortunately, there is currently no way to explicitly control the
deployment of tasks to TaskManagers with different roles. However, Flink
supports s
Hi Mans,
there is no explicit guide for how to write a custom source/sink as far as
I know. There is just a paragraph in the data sources section (
https://ci.apache.org/projects/flink/flink-docs-master/apis/streaming/#data-sources
).
For sources you have to implement the SourceFunction interface
ngh
>
> > wrote:
> >
> >
> > Thanks Till for the pointers.
> > Mans
> >
> > On Tuesday, May 10, 2016 1:07 AM, Till Rohrmann <
> trohrm...@apache.org>
> > wrote:
> >
> >
> > Hi Mans,there is no explicit guide for how
Thanks Ufuk :-)
On Wed, May 11, 2016 at 5:16 PM, Stephan Ewen wrote:
> Thanks for pushing this release Ufuk!
>
> On Wed, May 11, 2016 at 5:12 PM, Fabian Hueske wrote:
>
> > Thanks Ufuk!
> >
> > 2016-05-11 16:39 GMT+02:00 Ufuk Celebi :
> >
> > > This vote has passed with 3 binding +1 votes. Than
+1 for the proposal
On May 12, 2016 12:13 PM, "Stephan Ewen" wrote:
> Yes, Gabor Gevay, that did refer to you!
>
> Sorry for the ambiguity...
>
> On Thu, May 12, 2016 at 10:46 AM, Márton Balassi >
> wrote:
>
> > +1 for the proposal
> > @ggevay: I do think that it refers to you. :)
> >
> > On Thu
Hi Greg,
as far as I know there has not been an exhaustive comparison to what extent
the delta iterations can achieve the same accuracy as bulk iterations or
how much accuracy you'll lose. I think it strongly depends on the problem.
For example, graph algorithms such as connected components should
ion 0 will have data and partition 1
> will
> > have data.
> >
> >
> > @till, I see what you did in ALS, with a Custom partitioner, Is there a
> > way that I can write a custom partitioner to make sure that we have data
> in
> > the 0th and 1st partition?
Thanks Flavio for adding the Eclipse section for remote debugging :-)
On Tue, May 17, 2016 at 5:55 PM, Flavio Pompermaier
wrote:
> Done ;)
>
> On Tue, May 17, 2016 at 5:37 PM, Robert Metzger
> wrote:
>
> > Okay, I gave you permissions.
> >
> > On Tue, May 17, 2016 at 5:22 PM, Flavio Pompermaier
Hi Trevor,
delta iterations are described in this paper [1].
[1]
http://stratosphere.eu/assets/papers/spinningFastIterativeDataFlows_12.pdf
Cheers,
Till
On Thu, May 26, 2016 at 1:57 AM, Trevor Grant
wrote:
> I was wondering where the idea for delta iterations came from.
>
> Many of the featur
Hi,
the directed output via the split and select methods are indeed only
available in the DataStream API. Thus, in order to achieve the same with
the DataSet API, you would have to apply multiple filters, as you've
already written.
The result of the select call will only be sent to the same task
Hi Andreas,
your observation is correct. The data is sent to each slot and the
receiving TM only materializes one copy of the data. The rest of the data
is discarded.
As far as I know, the reason why the broadcast variables are implemented
that way is that the senders would have to know which sub
pply for broadcast
> joins as well.
>
> Is this the case?
>
> Regards,
> Alexander
>
>
> 2016-06-08 17:13 GMT+02:00 Kunft, Andreas :
>
> > Hi Till,
> >
> > thanks for the fast answer.
> > I'll think about a concrete way of implementing
I agree. I could be the third backup if you need help with the component.
On Thu, Jun 9, 2016 at 11:33 AM, Aljoscha Krettek
wrote:
> Should probably, yes.
>
> On Thu, 9 Jun 2016 at 10:53 Stephan Ewen wrote:
>
> > Should state bakends and checkpointing go together?
> >
> > The two of us could be
+1 for the thread safe metrics. This should be a rather low hanging fruit
and easily added.
If we decide to add a histogram, then I would also be in favour of
implementing our own version of a histogram. This avoids adding a hard
dependency on Dropwizard or another metrics library to Flink core. A
y well and used quite
> widely in open source projects, I personally on the side of using
> dropwizard metrics rather than re-implement them, unless for performance
> reasons. Still, I'm +1 for adding a wrapper on top of dropwizard metrics.
>
> On Tue, Jun 14, 2016 at 10:45 PM,
would reduce the
complexity on the Flink side since covering all metrics use cases might be
difficult. Would that work for you? What do the others think about it?
Cheers,
Till
On Thu, Jun 16, 2016 at 3:33 PM, Till Rohrmann wrote:
> I agree that dropwizard already offers a lot of functionalit
Hi Gyula,
the scheduler actually deploys independent tasks in a round-robin fashion
across the cluster. So for example, your source sub tasks should be spread
evenly. However, whenever a sub-task has an input, it tries to deploy this
task on the same machine as one of the input sub-tasks (preferre
Hi,
your observation sounds like a bug to me and we have to further investigate
it. I assume that you’re running a batch job, right? Could you maybe share
your complete configuration and the job to reproduce the problem with us?
I think that your investigation that direct buffers are not properly
> On Jun 20, 2016 1:10 PM, "Till Rohrmann" wrote:
>
> > Hi,
> >
> > your observation sounds like a bug to me and we have to further
> investigate
> > it. I assume that you’re running a batch job, right? Could you maybe
> share
> > your com
uld have expose the internal
>>>> addMetric(String name, Metric metric) method.
>>>>
>>>> Regards,
>>>> Chesnay
>>>>
>>>> On 18.06.2016 04:48, Steve Cosenza wrote:
>>>>
>>>> Hi Till,
>>>>
>
Hi Vinay,
maybe this
http://stackoverflow.com/questions/22150417/hadoop-mapreduce-java-lang-unsatisfiedlinkerror-org-apache-hadoop-util-nativec
might help you.
Cheers,
Till
On Thu, Jun 23, 2016 at 5:15 AM, Vinay Patil
wrote:
> Hi All,
>
> Just an update on this:
>
> Setting the codec using Dat
Maybe we could document this fact in the code base. I think this is not a
very obvious behaviour.
Cheers,
Till
On Mon, Jun 27, 2016 at 11:24 AM, Aljoscha Krettek
wrote:
> Hi,
> you are right, this seems a bit strange. The reason why this works is that
> selectChannels of ForwardPartitioner is n
Hi Chen,
I'm not sure what you mean with a pipeline but Flink supports the
submission of multiple jobs to the same cluster (in standalone as well as
Yarn session mode). You simply have to make sure that there are enough
slots for all jobs to be executed at the same time.
Cheers,
Till
On Jun 28, 2
Hi Flink community,
while reviewing the LICENSE and NOTICE file of Apache Flink, I noticed that
according to the LICENSE file Flink contains many java script files.
However, tracking the corresponding files back was not so easy, because
they are actually all merged into
flink-runtime-web/web-dashb
t; working on the webfrontend. If they do, they can simply use NPM and
> > >> Bower which install the listed dependencies. The disadvantage is a
> > >> slight increase of our repository because we commit a new "vendor.js"
> > >> for every recompile, b
+1, I like the idea :-)
On Tue, Jul 5, 2016 at 3:48 AM, Jark Wu wrote:
> It’s a great idea! I would be happy if I can help something.
>
> In addition, maybe we can move the full “Powered By” wiki page to the
> website to reduce external link.
>
> - Jark Wu
>
> > 在 2016年7月4日，下午11:15，Stephan Ewen
Stefan Richter found the following problem with HA:
https://issues.apache.org/jira/browse/FLINK-4150
I think we should fix it for the 1.1 release.
On Mon, Jul 4, 2016 at 9:05 PM, Robert Metzger wrote:
> +1 to do a RC0 this week, but the master-forking with RC1. I would like to
> reduce the time
I found another critical issue [1]. The murmur hash correction introduced
between Flink 1.0 and 1.1 breaks the backwards compatibility with respect
to savepoints. I think we have to fix this for the release.
@Ufuk, I'm not sure whether I find time this week to work on FLINK-4150. I
could make it a
I talked to Stephan and he pointed out that the flink-dist binary file,
which is Flink's fat jar, is not part of the official Flink release. We do
offer to download this file as part of a zip file from the Flink website.
However, this is only for convenience. In contrast to that, other binary
files
Hi Vinay,
have you tried only specifying the directory path you want to read [1]?
I think in the future these kind of questions should be posted to the user
mailing list u...@flink.apache.org. The dev mailing list should be used for
discussions about Flink's development.
[1]
https://ci.apache.or
Stratified sampling would also be beneficial for the DataSet API. I think
it would be best if this method is also added to DataSetUtils or made
available via the flink-contrib module. Furthermore, I think that it would
be easiest if you created the JIRA for this feature, because you know what
you w
I agree with Robert that it would be a nice to have but not strictly
required. I think it would help to have a list of preferred tags so that
new community members have a place to look them up.
Cheers,
Till
On Fri, Jul 15, 2016 at 11:41 AM, Robert Metzger
wrote:
> Hi Ufuk,
>
> I also saw these
Then +1 :-)
On Fri, Jul 15, 2016 at 7:07 PM, Ufuk Celebi wrote:
> It was intended as Till said... a list of preferred tags.
>
> On Fri, Jul 15, 2016 at 6:52 PM, Till Rohrmann
> wrote:
> > I agree with Robert that it would be a nice to have but not strictly
> > require
+1 for the FLIP and making streaming the common case. Very good proposal :-)
On Mon, Jul 18, 2016 at 11:48 AM, Aljoscha Krettek
wrote:
> +1 I like it a lot!
>
> On Fri, 15 Jul 2016 at 18:43 Stephan Ewen wrote:
>
> > My take would be to take streaming as the common case and make special
> > sect
https://github.com/apache/flink/pull/2257 should almost be mergable. There
is only an issue with one test which only occurs on Travis. But I think I
found the culprit. Currently, testing it on Travis again.
On Thu, Jul 21, 2016 at 3:53 PM, Flavio Pompermaier
wrote:
> Maybe someone could complete
to handle
> >>> it
> >>> differently.
> >>>
> >>> By now, we could definitely handle BC Vars differently (only one slot
> per
> >>> TM requests).
> >>> For BC Joins, the hash tables do not coordinate spilling currently,
> w
Welcome to the community Neelesh :-)
On Mon, Aug 1, 2016 at 3:53 PM, Kevin Jacobs wrote:
> Hi!
>
> Welcome to the community :-)!
>
>
>
> On 01.08.2016 09:51, Ufuk Celebi wrote:
>
>> On Sun, Jul 31, 2016 at 8:07 PM, Neelesh Salian
>> wrote:
>>
>>> I am Neelesh Salian; I recently joined the Flink
I think that FLINK-4094 is nice to fix but not a release blocker since we
know how to prevent this situation (setting preallocation to true).
On Mon, Aug 1, 2016 at 11:56 PM, Aljoscha Krettek
wrote:
> I tried it again now. I did:
>
> rm -r .m2/repository
> mvn clean verify -Dhadoop.version=2.6.0
I can confirm Aljoscha's findings concerning building Flink with Hadoop
version 2.6.0 using Maven 3.3.9. Aljoscha is right that it is indeed a
Maven 3.3 issue. If you build flink-runtime twice, then everything goes
through because the shaded curator Flink dependency is installed in during
the first
alization".
> > > I think that is an important enough aspect to warrant separate docs.
> > >
> > > On Mon, Jul 18, 2016 at 3:36 PM, Till Rohrmann
> > wrote:
> > >
> > >> +1 for the FLIP and making streaming the common case.
The metrics transfer design document looks good to me. Thanks for your work
Chesnay :-)
I think the benefit of registering the metrics at the MetricDumper is that
we don't have to walk through the hierarchy of metric groups to collect the
metric values. Indeed, this comes with increased costs at s
+1 from my side
Checked LICENSE and NOTICE files
Build and tests run for
- Hadoop 2.6.0
SBT quickstarts work
Basic stream SQL features work
Metrics work with Ganglia, Graphite, JMX and StatsD
On Wed, Aug 3, 2016 at 9:56 PM, Stephan Ewen wrote:
> +1 from my side
>
> LICENSE and NOTICE files
+1
On Thu, Aug 4, 2016 at 3:20 AM, Aljoscha Krettek
wrote:
> +1 seems good
>
> On Wed, 3 Aug 2016 at 11:05 Stephan Ewen wrote:
>
> > Hi all!
> >
> > We would like to start working on FLIP-6.
> >
> > Because it is such a big change, I would like to start developing it
> > concurrently to the mas
to Calcite and it worked very
> well.
>
> Von: Till Rohrmann
Hi Felix,
I'm not sure whether PipelinedSubpartition should be readable more than
once because then it would effectively mean that we materialize the
elements of the pipelined subpartition for stragglers. Therefore, I think
that we should make blocking intermediate results readable more than once.
+1 for Hadoop 2.7 and Scala 2.11
On Tue, Aug 9, 2016 at 12:25 PM, Robert Metzger wrote:
> Hi,
>
> Thanks a lot for maintaining the flink homebrew formula.
> I think its a good idea to update to Hadoop 2.7 and Scala 2.11.
>
> On Tue, Aug 9, 2016 at 12:59 AM, Wright, Eron wrote:
>
> > Hello,
> >
I agree with Stephan that the main problem is maintenance overhead for the
Flink community. If we could maintain all connectors ourselves then there
would not be an immediate need to out source the connectors. Thus, the
solution should reduce the workload for the core project.
Personally, I would
That is a tough call but I'm personally leaning slightly towards not
breaking the API and adding a note for the casting workaround.
My main concern is where do we set the limit for future API breaking
issues? How critical does an issue has to be to be allowed to break the
API? Currently, we have 1
r should I mark this jira and FLINK-1713
> <https://issues.apache.org/jira/browse/FLINK-1713> as blocking for the
> broadcast jira? What do you think?
>
> Best regards,
> Felix
>
> 2016-08-09 17:41 GMT+07:00 Till Rohrmann :
>
> > Hi Felix,
> >
> >
+1 from my side as well.
On Tue, Aug 9, 2016 at 9:01 PM, Stephan Ewen wrote:
> +1
>
> This is a crucial fix and the released sources are actually still the same,
> so reduced time should be okay.
>
> On Tue, Aug 9, 2016 at 8:24 PM, Henry Saputra
> wrote:
>
> > Official vote
> > +1 (binding)
> >
gt; > > result-x-checkpoint4, ...) so it may be worth addressing that soon
> > anyways.
> > >
> > > (2) Make iterations not dependent on the special local back channel.
> > > Then we can simply schedule iterations like all other things.
> > >
> > >
Hi Theo,
I think currently, there is nobody actively working on side inputs. Thus,
it might be possible that they don't make it in the 1.2 release.
Cheers,
Till
On Wed, Aug 10, 2016 at 10:41 PM, Theodore Vasiloudis <
theodoros.vasilou...@gmail.com> wrote:
> Hello Aljoscha,
>
> Do you think the
Hi,
I'm also not sure whether we should start teaching Flink by demonstrating
the low-level APIs. According to my experience, people new to Flink should
first learn a very basic set of primitive operations. Usually this is map,
flatmap, join, windows, etc. The semantics of these operations is well
Cool design doc Klou. It's well described with a lot of details. I like it
a lot :-) +1 for implementing the trigger DSL.
Cheers,
Till
On Tue, Aug 16, 2016 at 3:18 PM, Kostas Kloudas wrote:
> Thanks for the feedback Ufuk!
> I will do that.
>
> > On Aug 16, 2016, at 1:41 PM, Ufuk Celebi wrote:
Good initiative @Aljoscha.
- Just a convenience link for FLIP-6: The feature branch can be found here
https://github.com/apache/flink/tree/flip-6.
- My main task is the review of the Mesos PR of Eron (
https://github.com/apache/flink/pull/2315) so that we can continue making
progress there.
On W
+1
On Thu, Aug 18, 2016 at 10:21 AM, Aljoscha Krettek
wrote:
> +1
>
> On Thu, 18 Aug 2016 at 03:54 Jark Wu wrote:
>
> > Good start point. +1
> >
> >
> >
> > - Jark Wu
> >
> > > 在 2016年8月17日，下午11:24，Robert Metzger 写道：
> > >
> > > Hi,
> > >
> > > following the discussions in [1], the community d
Hi Pavel,
I think it's a good point you're raising here. The Flink community isn't
using metrics like test coverage to ensure high quality code yet. I think
that is one thing which we can/should improve. Unfortunately, the ASF does
not allow to use codecov.io (or at least the Apache Infra team) [1
g 2, 2016 at 5:18 PM, Aljoscha Krettek
> > wrote:
> > > +1
> > >
> > > On Tue, 2 Aug 2016 at 03:15 Till Rohrmann
> wrote:
> > >
> > >> +1 :-)
> > >>
> > >> On Tue, Aug 2, 2016 at 6:09 PM, Stephan Ewen
table and does not affect
> build duration too much.
>
> 2016-08-19 12:51 GMT+03:00 Till Rohrmann :
>
> > Hi Pavel,
> >
> > I think it's a good point you're raising here. The Flink community isn't
> > using metrics like test coverage to ensure hig
+1 for the FLIP.
I like the described changes and new functionality.
When looking at the public interface, I was wondering whether we should not
allow the user to specify a TimeUnit for the periodic interval. I think
it's nicer to be able to specify the time unit instead of converting
everything
I agree that it would be good to offer these kind of convenience download
links.
On Wed, Aug 24, 2016 at 5:25 PM, Robert Metzger wrote:
> Maybe we should put a link to maven central. We could parameterize the link
> so that it always links to the current release linked on our downloads
> page.
>
Hi Gyula,
I haven't seen this problem before. Do you have the logs of the failed TMs
so that we have some more context what was going on?
Cheers,
Till
On Thu, Aug 25, 2016 at 9:40 AM, Gyula Fóra wrote:
> Hi guys,
>
> For quite some time now we fairly frequently experience a task manager
> cras
Hi Vinay,
it's not only done by enabling encryption of the Akka messages since the
TaskManager to TaskManager communication utilizes Netty. There is a JIRA
issue which tracks the progress on implementing secured communication
within Flink: https://issues.apache.org/jira/browse/FLINK-3931. Feel fre
If I'm not mistaken then this was a limitation of Akka's ask futures. The
maximum timeout was as specified. It might have changed with a new version,
though. Usually, you should rarely run into a case where this actually
matters.
Cheers,
Till
On Mon, Aug 29, 2016 at 5:44 PM, Greg Hogan wrote:
>
Hi Penguin,
What do you wanna do? If you want to change Flink's scheduling behaviour,
then you can take a look at the implementations of SchedulerNG.
Cheers,
Till
On Wed, Jan 6, 2021 at 6:58 AM penguin. wrote:
> Hello! Do you know how to modify the task scheduling method of Flink?
with this effort.
> >
>
> I hope that Sijie and I addressed your concerns.
>
> Best,
>
> Arvid
>
>
> On Wed, Jan 6, 2021 at 11:28 AM Sijie Guo wrote:
>
> > Hi Till,
> >
> > Thank you for your email! Please find my comments inline.
> >
> >
+1 for canceling the RC to fix FLINK-20632. Thanks a lot for driving this
effort Xintong and Robert!
Cheers,
Till
On Wed, Jan 6, 2021 at 1:01 PM Xintong Song wrote:
> Thanks for driving the docker image efforts, Robert.
>
> +1 for canceling this RC. That should also give us the chance to fix
>
HI Badrul,
FlinkML is unfortunately no longer under active development. However, there
is some new effort to add a machine learning library to Flink [1].
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs
Cheers,
Till
On Wed, Jan 6, 2021 at 7:11 PM Badru
Thanks for drafting this FLIP and starting this discussion Yangze.
I like that defining resource requirements on a slot sharing group makes
the overall setup easier and improves usability of resource requirements.
What I do not like about it is that it changes slot sharing groups from
being a sch
Hi Pramod,
Flink's Kafka connector uses transactions in order to support exactly once
semantic.
Cheers,
Till
On Thu, Jan 7, 2021 at 11:17 PM Pramod Immaneni wrote:
> Is there a Kafka producer that can do exactly once semantic without the use
> of transactions?
>
> Thanks
>
At the moment, this requirement has not come up very often. In general, I
am always a bit cautious when adding functionality which executes user code
in the JobManager because it can easily become a stability problem. On the
other hand, I can't think of a different solution other than polling the
j
7 INFO
> org.apache.flink.yarn.YarnResourceManager -
> Disconnect job manager
>
> @akka.tcp://flink@ip-10-6-0-231.ec2.internal:39039/user/rpc/jobmanager_2
> for job a7cffc31c4aeb01356c5132c908be314 from the resource mana
Congrats and welcome Danny!
Cheers,
Till
On Tue, Jan 12, 2021 at 10:09 AM Dawid Wysakowicz
wrote:
> Congratulations, Danny!
>
> Best,
>
> Dawid
>
> On 12/01/2021 09:52, Paul Lam wrote:
> > Congrats, Danny!
> >
> > Best,
> > Paul Lam
> >
> >> 2021年1月12日 16:48，Tzu-Li (Gordon) Tai 写道：
> >>
> >> H
Hi Jie Wang,
thanks for starting this discussion. To me the SPI approach sounds better
because it is not as brittle as using reflection. Concerning the
configuration, we could think about introducing some Hive specific
configuration options which allow us to specify these paths. How are other
proj
Hi Penguin,
the attached screenshot is not displayed correctly. Maybe you can post the
error to this thread.
Cheers,
Till
On Wed, Jan 13, 2021 at 6:17 AM penguin. wrote:
> Hello,
> When importing the Flink source code into idea, the following error
> occurred.
> And several mirrors were config
Thanks for starting this discussion Matthias. I agree with all of you that
a final 1.10.3 release could be really helpful for our users. Given that CI
passes, it shouldn't be too much overhead either.
Cheers,
Till
On Wed, Jan 13, 2021 at 9:45 AM Xingbo Huang wrote:
> Thanks for starting this di
601 - 700 of 1644 matches
Mail list logo
