Hi Borys,
To debug how many containers Flink is requesting, you can look out for the
log
statement below [1]:
Requesting new TaskExecutor container with resources [...]
If you need help debugging, can you attach the full JM logs (preferably on
DEBUG level)? Would it be possible for you to te
Hi Averell,
It is up to the YARN scheduler on which hosts the containers are started.
What Flink version are you using? I assume you are using 1.4 or earlier
because you are specifying a fixed number of TMs. If you launch Flink with
-yn
2, you should be only seeing 2 TMs in total (not 4). Are you
Hi Gary,
Thanks for the information. I didn't know that -yn is obsolete :( I am using
Flink 1.6.
Not sure whether that's a bug when I tried to set -yn explicitly, but I
started only 1 cluster.
Thanks and regards,
Averell
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.na
I need to monitor the latency and other similar metric for the Mysql Sink
that Flink writes to. I'm using JDBCAppendTableSink. Any way how I can
check these. Thanks!
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Hi Borys,
if possible the complete logs of the JM (DEBUG log level) would be helpful
to further debug the problem. Have there been recovery operations lately?
Cheers,
Till
On Sat, Oct 6, 2018 at 11:15 AM Gary Yao wrote:
> Hi Borys,
>
> To debug how many containers Flink is requesting, you can
Hello
I downloaded kafka and followed these instructions step by step:
cd kafka_2.11-2
# start zookeeper server
./bin/zookeeper-server-start.sh ./config/zookeeper.properties
# start broker
./bin/kafka-server-start.sh ./config/server.properties
# create topic “test”
./bin/kafka-topics.sh --cre
Thank you Till, I am able to start the session-cluster now.
Regards,
Vinay Patil
On Fri, Oct 5, 2018 at 8:15 PM Till Rohrmann wrote:
> Hi Vinay,
>
> are you referring to flink-contrib/docker-flink/docker-compose.yml? We
> recently fixed the command line parsing with Flink 1.5.4 and 1.6.1. Due
Hi anil,
It seems that there are no dedicated metrics for jdbc sink.
For latency, Flink allows to track the latency of records traveling through
the system[1].
For other metics, I think you can make use of the system metrics. For
example, the metric of numRecordsInPerSecond[2] may be helpful for y
Hi Till,
Thanks for identifying the issue. My cluster is up and running now.
I have few queries. Can you have to anwer that?
1. Do I need to set below properties in my cluster?
jobmanager.rpc.address
rest.address
rest.bind-address
jobmanager.web.address
1. Is there anything I should
Hi Samir,
1. In your setup (not running on top of Yarn or Mesos) you need to set the
jobmanager.rpc.address such that the JM process knows where to bind to. The
other components use ZooKeeper to find out the addresses. The other
properties should not be needed.
3. You can take a look at the ZooKee
Hi Kostas,
Please help ignore my previous email about the issue with security. It seems
to I had mixed version of shaded and unshaded jars.
However, I'm now facing another issue with writing parquet files: only the
first part is closed. All the subsequent parts are kept in in-progress state
forev
11 matches
Mail list logo
