>
> It could really be specific to your workload. Some workload may need more
> heap memory while others may need more off-heap.
>
The main 'process' of my project creates a cross product of datasets and
then applies a function to all of them to extract some features.
> Alternatively, you can
>
> Since changing off-heap removes memory from '.task.heap.size' is there a
> ratio that I should follow for better performance?
>
I don't think so. It could really be specific to your workload. Some
workload may need more heap memory while others may need more off-heap.
Also, my guess (since I
Hi Xintong,
Thank you for the quick response.
doing 1), without increasing 'task.off-heap.size' does not change the
issue, but increasing the off-heap alone does.
What should the off-heap value size be? Since changing off-heap removes
memory from '.task.heap.size' is there a ratio that I should
Hi Dimitris,
Regarding your questions.
a) For standalone clusters, the recommended way is to use `.flink.size`
rather than `.process.size`. `.process.size` includes JVM metaspace and
overhead in addition to `.flink.size`, which usually do not really matter
for standalone clusters.
b) In case of
Hello,
I'm having a bit of trouble understanding the memory configuration on
flink.
I'm using flink10.0.0 to read some datasets of edges and extract features.
I run this on a cluster consisting of 4 nodes , with 32cores and 252GB Ram
each, and hopefully I could expand this as long as I can add
4:20 (CEST)
An : guenterh.li...@bluewin.ch, user@flink.apache.org
Cc : guenter.hip...@unibas.ch
Betreff : Re: configuration of standalone cluster
Flink still only works with Java 8 at the moment. It will be a while
until we properly support Java 11.
On 02/05/2019 13:58, Günter Hipler wrote:
> swissbib@sb
I'm trying to set up a standalone cluster. My
current configuration
4 server (1 jobmanger and 3 taskmanager)
a) starting the cluster
swissbib@sb-ust1:/swissbib_index/apps/flink/bin$ ./start-cluster.sh
Starting cluster.
Starting standalonesession daemon on host sb-ust1.
Starting taskexecutor daemon
ipler wrote:
> > Hi,
> >
> > For the first time I'm trying to set up a standalone cluster. My
> > current configuration
> > 4 server (1 jobmanger and 3 taskmanager)
> >
> > a) starting the cluster
> > swissbib@sb-ust1:/swissbib_index/apps/flink/bin$ ./st
e?
Günter
On 02.05.19 13:48, Chesnay Schepler wrote:
Which java version are you using?
On 01/05/2019 21:31, Günter Hipler wrote:
Hi,
For the first time I'm trying to set up a standalone cluster. My
current configuration
4 server (1 jobmanger and 3 taskmanager)
a) starting the cluster
swissb
Which java version are you using?
On 01/05/2019 21:31, Günter Hipler wrote:
Hi,
For the first time I'm trying to set up a standalone cluster. My
current configuration
4 server (1 jobmanger and 3 taskmanager)
a) starting the cluster
swissbib@sb-ust1:/swissbib_index/apps/flink/bin$ ./start
Hi,
For the first time I'm trying to set up a standalone cluster. My current
configuration
4 server (1 jobmanger and 3 taskmanager)
a) starting the cluster
swissbib@sb-ust1:/swissbib_index/apps/flink/bin$ ./start-cluster.sh
Starting cluster.
Starting standalonesession daemon on host sb-ust1
11 matches
Mail list logo
