[jira] [Updated] (FLINK-22941) support column comment in catalogTable column schema
[ https://issues.apache.org/jira/browse/FLINK-22941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] peng wang updated FLINK-22941: -- Affects Version/s: 1.13.0 > support column comment in catalogTable column schema > > > Key: FLINK-22941 > URL: https://issues.apache.org/jira/browse/FLINK-22941 > Project: Flink > Issue Type: New Feature > Components: Table SQL / Planner >Affects Versions: 1.13.0 >Reporter: peng wang >Priority: Major > > we found that column comment is support in flink ddl syntax, but it was > dropped when SqlCreateTable class convert to CatalogTable class. > would we record column comment in catalogTable? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-22941) support column comment in catalogTable column schema
[ https://issues.apache.org/jira/browse/FLINK-22941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] peng wang updated FLINK-22941: -- Description: we found that column comment is support in flink ddl syntax, but it was dropped when SqlCreateTable class convert to CatalogTable class. would we record column comment in catalogTable? was:we found that column comment is support in flink ddl syntax, but it was dropped when SqlCreateTable class convert to CatalogTable class. > support column comment in catalogTable column schema > > > Key: FLINK-22941 > URL: https://issues.apache.org/jira/browse/FLINK-22941 > Project: Flink > Issue Type: New Feature > Components: Table SQL / Planner >Reporter: peng wang >Priority: Major > > we found that column comment is support in flink ddl syntax, but it was > dropped when SqlCreateTable class convert to CatalogTable class. > would we record column comment in catalogTable? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-22941) support column comment in catalogTable column schema
peng wang created FLINK-22941: - Summary: support column comment in catalogTable column schema Key: FLINK-22941 URL: https://issues.apache.org/jira/browse/FLINK-22941 Project: Flink Issue Type: New Feature Components: Table SQL / Planner Reporter: peng wang we found that column comment is support in flink ddl syntax, but it was dropped when SqlCreateTable class convert to CatalogTable class. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-14091) Job can not trigger checkpoint forever after zookeeper change leader
[
https://issues.apache.org/jira/browse/FLINK-14091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931044#comment-16931044
]
Peng Wang commented on FLINK-14091:
---
{{SharedCountConnectionStateListener already handle it(as shown below), but
ZooKeeperCheckpointIDCounter not. }}
h6. @Override
public void stateChanged(CuratorFramework client, ConnectionState newState) {
if (newState == ConnectionState.SUSPENDED || newState == ConnectionState.LOST)
{
lastState = newState;
}
else{
/* if connectionState is not SUSPENDED and LOST, reset lastState. */
lastState = null;
}
}
> Job can not trigger checkpoint forever after zookeeper change leader
> -
>
> Key: FLINK-14091
> URL: https://issues.apache.org/jira/browse/FLINK-14091
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
>Affects Versions: 1.9.0
>Reporter: Peng Wang
>Priority: Minor
>
> when zk change leader, the state of curator is suspended,job manager can not
> tigger checkpoint.but it doesn't tigger checkpoint after zk resume.
> we found that the lastState in the class ZooKeeperCheckpointIDCounter never
> change back to normal when it fall into SUSPENDED or LOST.
> h6. _/**_
> _* Connection state listener. In case of \{@link ConnectionState#SUSPENDED}
> or {@link_
> _* ConnectionState#LOST} we are not guaranteed to read a current count from
> ZooKeeper._
> _*/_
> _private static class SharedCountConnectionStateListener implements
> ConnectionStateListener {_
> _private volatile ConnectionState lastState;_
> _@Override_
> _public void stateChanged(CuratorFramework client, ConnectionState newState)
> {_
> _if (newState == ConnectionState.SUSPENDED || newState ==
> ConnectionState.LOST) {_
> _lastState = newState;_
> _}_
> _}_
> _private ConnectionState getLastState() {_
> _return lastState;_
> _}_
> _}_
>
> we change the state back. after test, solve the problem.
>
> h6. _/**_
> _* Connection state listener. In case of \{@link ConnectionState#SUSPENDED}
> or {@link_
> _* ConnectionState#LOST} we are not guaranteed to read a current count from
> ZooKeeper._
> _*/_
> _private static class SharedCountConnectionStateListener implements
> ConnectionStateListener {_
> _private volatile ConnectionState lastState;_
> _@Override_
> _public void stateChanged(CuratorFramework client, ConnectionState newState)
> {_
> _if (newState == ConnectionState.SUSPENDED || newState ==
> ConnectionState.LOST) {_
> _lastState = newState;_
> _}_
> _else{_
> _/* if connectionState is not SUSPENDED and LOST, reset lastState. */_
> _lastState = null;_
> _}_
> _}_
> _private ConnectionState getLastState() {_
> _return lastState;_
> _}_
> _}_
>
> log：
> h6. 2019-09-16 13:38:38,020 INFO
> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn - Unable
> to }}{{read}} {{additional data from server sessionid 0x26cff6487c2000e,
> likely server has closed socket, closing socket connection and attempting
> reconnect2019-09-16 13:38:38,122 INFO
> org.apache.flink.shaded.curator.org.apache.curator.framework.state.ConnectionStateManager
> - State change: SUSPENDED2019-09-16 13:38:38,123 WARN
> org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService -
> Connection to ZooKeeper suspended. Can no longer retrieve the leader from
> ZooKeeper.2019-09-16 13:38:38,126 WARN
> org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService -
> Connection to ZooKeeper suspended. Can no longer retrieve the leader from
> ZooKeeper.2019-09-16 13:38:38,126 WARN
> org.apache.flink.runtime.jobmanager.ZooKeeperSubmittedJobGraphStore -
> ZooKeeper connection SUSPENDING. Changes to the submitted job graphs are not
> monitored (temporarily).2019-09-16 13:38:38,128 WARN
> org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService -
> Connection to ZooKeeper suspended. The contender
> akka.tcp:}}{{//flink}}{{@node007224:19115}}{{/user/dispatcher}} {{no longer
> participates }}{{in}} {{the leader election.2019-09-16 13:38:38,128
> WARN org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService
> - Connection to ZooKeeper suspended. The contender
> akka.tcp:}}{{//flink}}{{@node007224:19115}}{{/user/resourcemanager}} {{no
> longer participates }}{{in}} {{the leader election.2019-09-16
> 13:38:38,128 WARN
> org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService -
> Connection to ZooKeeper suspended. Can no longer retrieve the leader from
> ZooKeeper.2019-09-16 13:38:38,128 WARN
> org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService -
> Connection to ZooKeeper suspended. The contender
>
[jira] [Created] (FLINK-14091) Job can not trigger checkpoint forever after zookeeper change leader
Peng Wang created FLINK-14091:
-
Summary: Job can not trigger checkpoint forever after zookeeper
change leader
Key: FLINK-14091
URL: https://issues.apache.org/jira/browse/FLINK-14091
Project: Flink
Issue Type: Bug
Components: Runtime / Checkpointing
Affects Versions: 1.9.0
Reporter: Peng Wang
when zk change leader, the state of curator is suspended,job manager can not
tigger checkpoint.but it doesn't tigger checkpoint after zk resume.
we found that the lastState in the class ZooKeeperCheckpointIDCounter never
change back to normal when it fall into SUSPENDED or LOST.
h6. _/**_
_* Connection state listener. In case of \{@link ConnectionState#SUSPENDED} or
{@link_
_* ConnectionState#LOST} we are not guaranteed to read a current count from
ZooKeeper._
_*/_
_private static class SharedCountConnectionStateListener implements
ConnectionStateListener {_
_private volatile ConnectionState lastState;_
_@Override_
_public void stateChanged(CuratorFramework client, ConnectionState newState) {_
_if (newState == ConnectionState.SUSPENDED || newState ==
ConnectionState.LOST) {_
_lastState = newState;_
_}_
_}_
_private ConnectionState getLastState() {_
_return lastState;_
_}_
_}_
we change the state back. after test, solve the problem.
h6. _/**_
_* Connection state listener. In case of \{@link ConnectionState#SUSPENDED} or
{@link_
_* ConnectionState#LOST} we are not guaranteed to read a current count from
ZooKeeper._
_*/_
_private static class SharedCountConnectionStateListener implements
ConnectionStateListener {_
_private volatile ConnectionState lastState;_
_@Override_
_public void stateChanged(CuratorFramework client, ConnectionState newState) {_
_if (newState == ConnectionState.SUSPENDED || newState ==
ConnectionState.LOST) {_
_lastState = newState;_
_}_
_else{_
_/* if connectionState is not SUSPENDED and LOST, reset lastState. */_
_lastState = null;_
_}_
_}_
_private ConnectionState getLastState() {_
_return lastState;_
_}_
_}_
log：
h6. 2019-09-16 13:38:38,020 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn - Unable to
}}{{read}} {{additional data from server sessionid 0x26cff6487c2000e, likely
server has closed socket, closing socket connection and attempting
reconnect2019-09-16 13:38:38,122 INFO
org.apache.flink.shaded.curator.org.apache.curator.framework.state.ConnectionStateManager
- State change: SUSPENDED2019-09-16 13:38:38,123 WARN
org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService -
Connection to ZooKeeper suspended. Can no longer retrieve the leader from
ZooKeeper.2019-09-16 13:38:38,126 WARN
org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService -
Connection to ZooKeeper suspended. Can no longer retrieve the leader from
ZooKeeper.2019-09-16 13:38:38,126 WARN
org.apache.flink.runtime.jobmanager.ZooKeeperSubmittedJobGraphStore -
ZooKeeper connection SUSPENDING. Changes to the submitted job graphs are not
monitored (temporarily).2019-09-16 13:38:38,128 WARN
org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService -
Connection to ZooKeeper suspended. The contender
akka.tcp:}}{{//flink}}{{@node007224:19115}}{{/user/dispatcher}} {{no longer
participates }}{{in}} {{the leader election.2019-09-16 13:38:38,128
WARN org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService -
Connection to ZooKeeper suspended. The contender
akka.tcp:}}{{//flink}}{{@node007224:19115}}{{/user/resourcemanager}} {{no
longer participates }}{{in}} {{the leader election.2019-09-16
13:38:38,128 WARN
org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService -
Connection to ZooKeeper suspended. Can no longer retrieve the leader from
ZooKeeper.2019-09-16 13:38:38,128 WARN
org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService -
Connection to ZooKeeper suspended. The contender http:}}{{//node007224}}{{:8081
no longer participates }}{{in}} {{the leader election.2019-09-16
13:38:38,128 WARN
org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService -
Connection to ZooKeeper suspended. Can no longer retrieve the leader from
ZooKeeper.2019-09-16 13:38:38,128 WARN
org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService -
Connection to ZooKeeper suspended. The contender
akka.tcp:}}{{//flink}}{{@node007224:19115}}{{/user/jobmanager_2}} {{no longer
participates }}{{in}} {{the leader election.2019-09-16 13:38:38,128
WARN org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService
- Connection to ZooKeeper suspended. Can no longer retrieve the leader from
ZooKeeper.2019-09-16 13:38:38,128 WARN
