[jira] [Created] (FLINK-25321) standalone deploy on k8s,pod always OOM killed,actual heap memory usage is normal, gc is normal

2021-12-15 Thread Gao Fei (Jira) 
Gao Fei created FLINK-25321:
---

 Summary: standalone deploy on k8s,pod always OOM killed,actual 
heap memory usage is normal, gc is normal
 Key: FLINK-25321
 URL: https://issues.apache.org/jira/browse/FLINK-25321
 Project: Flink
  Issue Type: Bug
  Components: Deployment / Kubernetes
Affects Versions: 1.11.3
 Environment: Flink 1.11.3

k8s v1.21.0

standlone deployment
Reporter: Gao Fei


Start a cluster on k8s, deploy in standalone mode, a jobmanager pod (1G) and a 
taskmanager pod (3372MB limit), the total memory configuration of the Flink TM 
process is 3072MB, and the managed configuration is 0, both of which are on the 
heap memory. Now the pod It will always be OOM killed, and the total process 
memory will always exceed 3072MB. I saw that the system has adopted jemlloc. 
There is no 64M problem. The application itself has not applied for direct 
memory. It is strange why the process is always killed by OOM after a period of 
time.

 

INFO  [] - Final TaskExecutor Memory configuration:
INFO  [] -   Total Process Memory:          3.000gb (3221225472 bytes)
INFO  [] -     Total Flink Memory:          2.450gb (2630667464 bytes)
INFO  [] -       Total JVM Heap Memory:     2.080gb (2233382986 bytes)
INFO  [] -         Framework:               128.000mb (134217728 bytes)
INFO  [] -         Task:                    1.955gb (2099165258 bytes)
INFO  [] -       Total Off-heap Memory:     378.880mb (397284478 bytes)
INFO  [] -         Managed:                 0 bytes
INFO  [] -         Total JVM Direct Memory: 378.880mb (397284478 bytes)
INFO  [] -           Framework:             128.000mb (134217728 bytes)
INFO  [] -           Task:                  0 bytes
INFO  [] -           Network:               250.880mb (263066750 bytes)
INFO  [] -     JVM Metaspace:               256.000mb (268435456 bytes)
INFO  [] -     JVM Overhead:                307.200mb (322122552 bytes)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (FLINK-22838) Flink Dashboard display incorrect Version in 1.13，actual display 1.12.2

2021-06-01 Thread Gao Fei (Jira) 
Gao Fei created FLINK-22838:
---

 Summary: Flink Dashboard display incorrect Version in 1.13，actual 
display 1.12.2
 Key: FLINK-22838
 URL: https://issues.apache.org/jira/browse/FLINK-22838
 Project: Flink
  Issue Type: Bug
  Components: Runtime / Web Frontend
Affects Versions: 1.13.1, 1.13.0
Reporter: Gao Fei
 Fix For: 1.13.2


Flink Dashboard display incorrect Version in 1.13.1，actual display 1.12.2



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-29647) report stackoverflow when using kryo

2022-10-14 Thread Gao Fei (Jira) 
Gao Fei created FLINK-29647:
---

 Summary: report stackoverflow when using kryo
 Key: FLINK-29647
 URL: https://issues.apache.org/jira/browse/FLINK-29647
 Project: Flink
  Issue Type: Bug
  Components: API / Type Serialization System
Affects Versions: 1.13.2
 Environment: flink 1.13.2 version （kryo 2.24 version）
Reporter: Gao Fei


When using kryo to report stackoverflow, the error is as follows:
{code:java}
java.lang.StackOverflowError at 
com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:43) at 
com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:44) at 
com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:44) at 
com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:44) at 
com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:44) at 
com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:44) at 
com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:44) at 
com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:44) at 
com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:44)
{code}
 I am using two-phase commit to write data to mysql, the following is part of 
the mysql sink code:
{code:java}
public class MySqlTwoPhaseCommitSink extends 
TwoPhaseCommitSinkFunction, Connection,Void> {
    private static final Logger log = 
LoggerFactory.getLogger(MySqlTwoPhaseCommitSink.class);
    public MySqlTwoPhaseCommitSink(){
        super(new KryoSerializer<>(Connection.class,new ExecutionConfig()), 
VoidSerializer.INSTANCE);
    }
    @Override
    public void invoke(Connection connection, Tuple2 tp, 
Context context) throws Exception {
        log.info("start invoke...");
        //TODO
        //omit here
    }
    @Override
    public Connection beginTransaction() throws Exception {
        log.info("start beginTransaction...");
        String url = 
"jdbc:mysql://localhost:3306/bigdata?useUnicode=true&characterEncoding=UTF-8";
        Connection connection = DBConnectUtil.getConnection(url, "root", 
"123456");
        return connection;
    }
    @Override
    public void preCommit(Connection connection) throws Exception {
        log.info("start preCommit...");
    }
    @Override
    public void commit(Connection connection) {
        log.info("start commit...");
        DBConnectUtil.commit(connection);
    }
    @Override
    public void abort(Connection connection) {
        log.info("start abort rollback...");
        DBConnectUtil.rollback(connection);
    }
}{code}

I also found similar problem reports: 
https://github.com/EsotericSoftware/kryo/issues/341



--
This message was sent by Atlassian Jira
(v8.20.10#820010)