Hi Stephan / others interested,
I have been working on the flink-mesos integration and there are definitely
some thoughts that I would like to share some thoughts about the
commonalities with the flink-yarn integration.
* Both flink-mesos and flink-yarn integration as they stand today can be
con
Thanks for opening the pull request!
On Mon, Aug 10, 2015 at 3:53 PM, Stephan Ewen wrote:
> I have a pending fix that uses a linguist git attribute to mark the
> directory with generated files as "vendored", which excludes it.
>
> On Tue, Jul 28, 2015 at 9:25 PM, Stephan Ewen wrote:
>
> > Good
fangfengbin created FLINK-2507:
--
Summary: Rename the function tansformAndEmit in
org.apache.flink.stormcompatibility.wrappers.AbstractStormCollector
Key: FLINK-2507
URL: https://issues.apache.org/jira/browse/FLINK-25
As I know, each TaskManager has NettyConnectionManager component and all the
tasks in the TaskManager will user that to transfer data. In the
PartitionRequestClientFactory, the nettyClient will make a connection based on
connectionId, and the connectionId consists of socketAddress and
connect
Hi!
You can think of it the following way: When you execute a join, the data
shuffled from node A to node B for the left side of the join will have a
separate TCP connection than the data shuffled from node A to node B for
the right side of the join.
That is currently important to avoid distribut
Hi Gyula, hi Slim,
being the core developer of SPQR, I am very glad to see the framework
referenced
on the Apache Flink dev list ;-)
Following the same approach as Flink - using real data streaming instead of
mini-batching - SPQR
adds some ad hoc'ness features to introduce more flexibility into t
Hey Christian,
Thanks for the insider view on SPQR. I have to agree with Gyula that
dynamic topology build is not the highest priority for Flink currently, but
certainly a very interesting feature and one that has already been
requested by a couple of users.
As for none of the open source streami
Hi Christian!
Sounds like a very cool proposal, and it seems that Flink and SPQR could
complement each other very well.
If you are interested to see this on top of Flink, let's have a chat to see
how to best get started with this - what would be the easiest points of
integration.
As said, Flink
Hi Andra,
thanks for offering to add this work to Gelly and for starting the
discussion!
How do you think this would look like from an API point of view? Is it easy
to make it transparent to the application? Could you give us a simple
example of what you have in mind?
Apart from usability, we sh
Hi Stephan and Matthias,
Sorry for replying late.
I`ve double checked that this class StormSpoutWrapper is really exist in my jar
file.
And it got the same trouble when I ran the flink-storm-compatibililty-example-
corresponding word-count-storm.
The way I built my Throughput application was jus
Stephan Ewen created FLINK-2508:
---
Summary: Confusing sharing of StreamExecutionEnvironment
Key: FLINK-2508
URL: https://issues.apache.org/jira/browse/FLINK-2508
Project: Flink
Issue Type: Bug
Hi Vasia,
I shall polish the functions a bit, but this is more or less what I had in
mind:
GSA Jaccard [what we have in Gelly right now]:
https://github.com/andralungu/gelly-partitioning/blob/master/src/main/java/example/GSAJaccardSimilarityMeasure.java
The same version with node split:
https://gi
Hi Andra and nice to meet you btw :)
It sounds like very fancy way to deal with skew, I like the idea even though I
am not a graph analytics expert.
Have you ran any experiments or benchmarks to see when this preferable ? Users
should be aware when they will get benefits by using it since node s
Three comments
1) If StormSpoutWrapper is in your jar, is it located in the correct
directory (must be same as package name)?
2) If you are using FlinkTopologyBuilder, you need to package as shown
in StormWordCountRemoteBySubmitter example, using an additional assembly
file.
(The first examples a
Hi Paris,
Nice to virtually meet you too :)
Maybe it makes sense to share my freshest chart:
https://drive.google.com/file/d/0BwnaKJcSLc43Qm9fZV9RUE5zT1E/view?usp=sharing
This is for the Community Detection algorithm [1] in which you basically
find communities by continuously rescoring vertices.
Stephan Ewen created FLINK-2509:
---
Summary: Improve error messages when user code classes are not
found
Key: FLINK-2509
URL: https://issues.apache.org/jira/browse/FLINK-2509
Project: Flink
Issu
We are seeing these class loader issues a lot as of late.
Seems that packaging the classes is trickier than anticipated.
Here is a pull request to add some diagnostics info on a
"ClassNotFoundException": https://github.com/apache/flink/pull/1008
On Tue, Aug 11, 2015 at 3:29 PM, Matthias J. Sax <
Stephan Ewen created FLINK-2510:
---
Summary: KafkaConnector should access partition metadata from
master/cluster
Key: FLINK-2510
URL: https://issues.apache.org/jira/browse/FLINK-2510
Project: Flink
Dear Andra,
The idea seems pretty nice.
I wonder how you decide the threshold to separate the high degree vertices
from the low degree vertices.
Regards,
Samia
On Tue, Aug 11, 2015 at 3:41 PM, Andra Lungu wrote:
> Hi Paris,
>
> Nice to virtually meet you too :)
>
> Maybe it makes sense to sha
Hi Samia,
A good method to statistically determine skewed vertices was beyond the
purpose of my thesis. Unfortunately, the statistical methods that fit a
power law distribution don't do a good job. So what I do is that I plot the
degree distribution and then visually determine the threshold. That
Ted Yu created FLINK-2511:
-
Summary: Potential resource leak due to unclosed InputStream in
FlinkZooKeeperQuorumPeer.java
Key: FLINK-2511
URL: https://issues.apache.org/jira/browse/FLINK-2511
Project: Flink
I'm writing a utility to split a data set randomly into several parts and
return an Array of data sets. However, whenever I operate on any of
these *subsets,
*the program basically start from the original data set, and the split is
performed again.
To ensure that these subsets are mutually exclusi
fangfengbin created FLINK-2512:
--
Summary: Add client.close() before throw RuntimeException
Key: FLINK-2512
URL: https://issues.apache.org/jira/browse/FLINK-2512
Project: Flink
Issue Type: Bug
23 matches
Mail list logo