[jira] [Commented] (FLINK-22281) flink sql consumer kakfa canal-json message then sum(amount)
[ https://issues.apache.org/jira/browse/FLINK-22281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17321917#comment-17321917 ] xx chai commented on FLINK-22281: - Yes, I have added this parameter correctly > flink sql consumer kakfa canal-json message then sum(amount) > -- > > Key: FLINK-22281 > URL: https://issues.apache.org/jira/browse/FLINK-22281 > Project: Flink > Issue Type: Bug > Components: Table SQL / API >Affects Versions: 1.12.0 > Environment: flink 1.12 local >Reporter: xx chai >Priority: Major > Attachments: screenshot-1.png > > > I use flink sql to consumer kafka canal-json message the sql is > CREATE TABLE kafka_mall_order_info ( > id int, > amount double, >PRIMARY KEY ( id) NOT ENFORCED >) WITH ( > 'connector' = 'kafka', > 'topic' = 'topic_yx-dc-3-102_3306', > 'properties.bootstrap.servers' = 'localhost:9092', > 'properties.group.id' = 'kafka_to_hive', > 'scan.startup.mode' = 'latest-offset', > 'format' = 'canal-json'); > create table t2 (amount double) with ('connector' = 'print'); > > > insert into t2 select sum(amount) from kafka_mall_order_info ; > but the result is not i think > the result in image -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-22281) flink sql consumer kakfa canal-json message then sum(amount)
[ https://issues.apache.org/jira/browse/FLINK-22281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17321878#comment-17321878 ] xx chai commented on FLINK-22281: - thanks [~jark] I solve the question . > flink sql consumer kakfa canal-json message then sum(amount) > -- > > Key: FLINK-22281 > URL: https://issues.apache.org/jira/browse/FLINK-22281 > Project: Flink > Issue Type: Bug > Components: Table SQL / API >Affects Versions: 1.12.0 > Environment: flink 1.12 local >Reporter: xx chai >Priority: Major > Attachments: screenshot-1.png > > > I use flink sql to consumer kafka canal-json message the sql is > CREATE TABLE kafka_mall_order_info ( > id int, > amount double, >PRIMARY KEY ( id) NOT ENFORCED >) WITH ( > 'connector' = 'kafka', > 'topic' = 'topic_yx-dc-3-102_3306', > 'properties.bootstrap.servers' = 'localhost:9092', > 'properties.group.id' = 'kafka_to_hive', > 'scan.startup.mode' = 'latest-offset', > 'format' = 'canal-json'); > create table t2 (amount double) with ('connector' = 'print'); > > > insert into t2 select sum(amount) from kafka_mall_order_info ; > but the result is not i think > the result in image -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-22281) flink sql consumer kakfa canal-json message then sum(amount)
[ https://issues.apache.org/jira/browse/FLINK-22281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17321856#comment-17321856 ] xx chai commented on FLINK-22281: - i add the Parameter streamTableEnvironment.getConfig().addJobParameter("table.exec.source.cdc-events-duplicate","true"); but the result is not change > flink sql consumer kakfa canal-json message then sum(amount) > -- > > Key: FLINK-22281 > URL: https://issues.apache.org/jira/browse/FLINK-22281 > Project: Flink > Issue Type: Bug > Components: Table SQL / API >Affects Versions: 1.12.0 > Environment: flink 1.12 local >Reporter: xx chai >Priority: Major > Attachments: screenshot-1.png > > > I use flink sql to consumer kafka canal-json message the sql is > CREATE TABLE kafka_mall_order_info ( > id int, > amount double, >PRIMARY KEY ( id) NOT ENFORCED >) WITH ( > 'connector' = 'kafka', > 'topic' = 'topic_yx-dc-3-102_3306', > 'properties.bootstrap.servers' = 'localhost:9092', > 'properties.group.id' = 'kafka_to_hive', > 'scan.startup.mode' = 'latest-offset', > 'format' = 'canal-json'); > create table t2 (amount double) with ('connector' = 'print'); > > > insert into t2 select sum(amount) from kafka_mall_order_info ; > but the result is not i think > the result in image -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-22281) flink sql consumer kakfa canal-json message then sum(amount)
[ https://issues.apache.org/jira/browse/FLINK-22281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xx chai updated FLINK-22281: Description: I use flink sql to consumer kafka canal-json message the sql is CREATE TABLE kafka_mall_order_info ( id int, amount double, PRIMARY KEY ( id) NOT ENFORCED ) WITH ( 'connector' = 'kafka', 'topic' = 'topic_yx-dc-3-102_3306', 'properties.bootstrap.servers' = 'localhost:9092', 'properties.group.id' = 'kafka_to_hive', 'scan.startup.mode' = 'latest-offset', 'format' = 'canal-json'); create table t2 (amount double) with ('connector' = 'print'); insert into t2 select sum(amount) from kafka_mall_order_info ; but the result is not i think the result in image was: I use flink sql to consumer kafka canal-json message the sql is CREATE TABLE kafka_mall_order_info ( id int, amount double, PRIMARY KEY ( id) NOT ENFORCED ) WITH ( 'connector' = 'kafka', 'topic' = 'topic_yx-dc-3-102_3306', 'properties.bootstrap.servers' = '192.168.3.100:9092,192.168.3.101:9092,192.168.3.102:9092', 'properties.group.id' = 'kafka_to_hive', 'scan.startup.mode' = 'latest-offset', 'format' = 'canal-json'); create table t2 (amount double) with ('connector' = 'print'); insert into t2 select sum(amount) from kafka_mall_order_info ; but the result is not i think the result in image > flink sql consumer kakfa canal-json message then sum(amount) > -- > > Key: FLINK-22281 > URL: https://issues.apache.org/jira/browse/FLINK-22281 > Project: Flink > Issue Type: Bug > Components: Table SQL / API >Affects Versions: 1.12.0 > Environment: flink 1.12 local >Reporter: xx chai >Priority: Major > Attachments: screenshot-1.png > > > I use flink sql to consumer kafka canal-json message the sql is > CREATE TABLE kafka_mall_order_info ( > id int, > amount double, >PRIMARY KEY ( id) NOT ENFORCED >) WITH ( > 'connector' = 'kafka', > 'topic' = 'topic_yx-dc-3-102_3306', > 'properties.bootstrap.servers' = 'localhost:9092', > 'properties.group.id' = 'kafka_to_hive', > 'scan.startup.mode' = 'latest-offset', > 'format' = 'canal-json'); > create table t2 (amount double) with ('connector' = 'print'); > > > insert into t2 select sum(amount) from kafka_mall_order_info ; > but the result is not i think > the result in image -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-22281) flink sql consumer kakfa canal-json message then sum(amount)
[ https://issues.apache.org/jira/browse/FLINK-22281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xx chai updated FLINK-22281: Description: I use flink sql to consumer kafka canal-json message the sql is CREATE TABLE kafka_mall_order_info ( id int, amount double, PRIMARY KEY ( id) NOT ENFORCED ) WITH ( 'connector' = 'kafka', 'topic' = 'topic_yx-dc-3-102_3306', 'properties.bootstrap.servers' = '192.168.3.100:9092,192.168.3.101:9092,192.168.3.102:9092', 'properties.group.id' = 'kafka_to_hive', 'scan.startup.mode' = 'latest-offset', 'format' = 'canal-json'); create table t2 (amount double) with ('connector' = 'print'); insert into t2 select sum(amount) from kafka_mall_order_info ; but the result is not i think the result in image was: I use flink sql to consumer kafka canal-json message the sql is CREATE TABLE kafka_mall_order_info ( id int, amount double, PRIMARY KEY ( id) NOT ENFORCED ) WITH ( 'connector' = 'kafka', 'topic' = 'topic_yx-dc-3-102_3306', 'properties.bootstrap.servers' = '192.168.3.100:9092,192.168.3.101:9092,192.168.3.102:9092', 'properties.group.id' = 'kafka_to_hive', 'scan.startup.mode' = 'latest-offset', 'format' = 'canal-json'); create table t2 (amount double) with ('connector' = 'print'); insert into t2 select sum(amount) from kafka_mall_order_info ; but the result is not i think > flink sql consumer kakfa canal-json message then sum(amount) > -- > > Key: FLINK-22281 > URL: https://issues.apache.org/jira/browse/FLINK-22281 > Project: Flink > Issue Type: Bug > Components: Table SQL / API >Affects Versions: 1.12.0 > Environment: flink 1.12 local >Reporter: xx chai >Priority: Major > Attachments: screenshot-1.png > > > I use flink sql to consumer kafka canal-json message the sql is > CREATE TABLE kafka_mall_order_info ( > id int, > amount double, >PRIMARY KEY ( id) NOT ENFORCED >) WITH ( > 'connector' = 'kafka', > 'topic' = 'topic_yx-dc-3-102_3306', > 'properties.bootstrap.servers' = > '192.168.3.100:9092,192.168.3.101:9092,192.168.3.102:9092', > 'properties.group.id' = 'kafka_to_hive', > 'scan.startup.mode' = 'latest-offset', > 'format' = 'canal-json'); > create table t2 (amount double) with ('connector' = 'print'); > > > insert into t2 select sum(amount) from kafka_mall_order_info ; > but the result is not i think > the result in image -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-22281) flink sql consumer kakfa canal-json message then sum(amount)
[ https://issues.apache.org/jira/browse/FLINK-22281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xx chai updated FLINK-22281: Attachment: screenshot-1.png > flink sql consumer kakfa canal-json message then sum(amount) > -- > > Key: FLINK-22281 > URL: https://issues.apache.org/jira/browse/FLINK-22281 > Project: Flink > Issue Type: Bug > Components: Table SQL / API >Affects Versions: 1.12.0 > Environment: flink 1.12 local >Reporter: xx chai >Priority: Major > Attachments: screenshot-1.png > > > I use flink sql to consumer kafka canal-json message the sql is > CREATE TABLE kafka_mall_order_info ( > id int, > amount double, >PRIMARY KEY ( id) NOT ENFORCED >) WITH ( > 'connector' = 'kafka', > 'topic' = 'topic_yx-dc-3-102_3306', > 'properties.bootstrap.servers' = > '192.168.3.100:9092,192.168.3.101:9092,192.168.3.102:9092', > 'properties.group.id' = 'kafka_to_hive', > 'scan.startup.mode' = 'latest-offset', > 'format' = 'canal-json'); > create table t2 (amount double) with ('connector' = 'print'); > > > insert into t2 select sum(amount) from kafka_mall_order_info ; > but the result is not i think -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-22281) flink sql consumer kakfa canal-json message then sum(amount)
[ https://issues.apache.org/jira/browse/FLINK-22281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xx chai updated FLINK-22281: Description: I use flink sql to consumer kafka canal-json message the sql is CREATE TABLE kafka_mall_order_info ( id int, amount double, PRIMARY KEY ( id) NOT ENFORCED ) WITH ( 'connector' = 'kafka', 'topic' = 'topic_yx-dc-3-102_3306', 'properties.bootstrap.servers' = '192.168.3.100:9092,192.168.3.101:9092,192.168.3.102:9092', 'properties.group.id' = 'kafka_to_hive', 'scan.startup.mode' = 'latest-offset', 'format' = 'canal-json'); create table t2 (amount double) with ('connector' = 'print'); insert into t2 select sum(amount) from kafka_mall_order_info ; but the result is not i think was:I use flink sql to consumer kafka canal-json message Environment: flink 1.12 local > flink sql consumer kakfa canal-json message then sum(amount) > -- > > Key: FLINK-22281 > URL: https://issues.apache.org/jira/browse/FLINK-22281 > Project: Flink > Issue Type: Bug > Components: Table SQL / API >Affects Versions: 1.12.0 > Environment: flink 1.12 local >Reporter: xx chai >Priority: Major > > I use flink sql to consumer kafka canal-json message the sql is > CREATE TABLE kafka_mall_order_info ( > id int, > amount double, >PRIMARY KEY ( id) NOT ENFORCED >) WITH ( > 'connector' = 'kafka', > 'topic' = 'topic_yx-dc-3-102_3306', > 'properties.bootstrap.servers' = > '192.168.3.100:9092,192.168.3.101:9092,192.168.3.102:9092', > 'properties.group.id' = 'kafka_to_hive', > 'scan.startup.mode' = 'latest-offset', > 'format' = 'canal-json'); > create table t2 (amount double) with ('connector' = 'print'); > > > insert into t2 select sum(amount) from kafka_mall_order_info ; > but the result is not i think -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-22281) flink sql consumer kakfa canal-json message then sum(amount)
[ https://issues.apache.org/jira/browse/FLINK-22281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xx chai updated FLINK-22281: Component/s: Table SQL / API Affects Version/s: 1.12.0 Description: I use flink sql to consumer kafka canal-json message Issue Type: Bug (was: Improvement) Summary: flink sql consumer kakfa canal-json message then sum(amount)(was: flink sql) > flink sql consumer kakfa canal-json message then sum(amount) > -- > > Key: FLINK-22281 > URL: https://issues.apache.org/jira/browse/FLINK-22281 > Project: Flink > Issue Type: Bug > Components: Table SQL / API >Affects Versions: 1.12.0 >Reporter: xx chai >Priority: Major > > I use flink sql to consumer kafka canal-json message -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-22281) flink sql
xx chai created FLINK-22281: --- Summary: flink sql Key: FLINK-22281 URL: https://issues.apache.org/jira/browse/FLINK-22281 Project: Flink Issue Type: Improvement Reporter: xx chai -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-20870) FlinkKafkaSink
[ https://issues.apache.org/jira/browse/FLINK-20870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17260938#comment-17260938 ] xx chai edited comment on FLINK-20870 at 1/8/21, 6:59 AM: -- !1610089153(1).png! was (Author: chaixiaoxue): !1610072517(1).jpg! > FlinkKafkaSink > -- > > Key: FLINK-20870 > URL: https://issues.apache.org/jira/browse/FLINK-20870 > Project: Flink > Issue Type: Bug > Components: Connectors / Kafka >Affects Versions: 1.12.0 > Environment: flink :1.12.0 > kafka 2.2.1 >Reporter: xx chai >Priority: Major > Attachments: 1610089153(1).png > > > I consum from kafka sink to kafka,Then I split the message into then pieces.I > guess the ten message should in one transaction. When the fifth message is > sink kafka ,I throw a exception.but the first four are already in kafka. > I set some parameters : >properties.setProperty("transactional.id", "cxx"); > properties.setProperty("ack", "all"); > properties.put("enable.idempotence",true); > properties.put("max.in.flight.requests.per.connection",5); > properties.put("retries", 2); > properties.setProperty("client.id", "producer-syn-2"); > properties.put("isolation.level","read_committed"); -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-20870) FlinkKafkaSink
[ https://issues.apache.org/jira/browse/FLINK-20870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xx chai updated FLINK-20870: Attachment: 1610089153(1).png > FlinkKafkaSink > -- > > Key: FLINK-20870 > URL: https://issues.apache.org/jira/browse/FLINK-20870 > Project: Flink > Issue Type: Bug > Components: Connectors / Kafka >Affects Versions: 1.12.0 > Environment: flink :1.12.0 > kafka 2.2.1 >Reporter: xx chai >Priority: Major > Attachments: 1610089153(1).png > > > I consum from kafka sink to kafka,Then I split the message into then pieces.I > guess the ten message should in one transaction. When the fifth message is > sink kafka ,I throw a exception.but the first four are already in kafka. > I set some parameters : >properties.setProperty("transactional.id", "cxx"); > properties.setProperty("ack", "all"); > properties.put("enable.idempotence",true); > properties.put("max.in.flight.requests.per.connection",5); > properties.put("retries", 2); > properties.setProperty("client.id", "producer-syn-2"); > properties.put("isolation.level","read_committed"); -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-20870) FlinkKafkaSink
[ https://issues.apache.org/jira/browse/FLINK-20870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xx chai updated FLINK-20870: Attachment: (was: 1610072517(1).jpg) > FlinkKafkaSink > -- > > Key: FLINK-20870 > URL: https://issues.apache.org/jira/browse/FLINK-20870 > Project: Flink > Issue Type: Bug > Components: Connectors / Kafka >Affects Versions: 1.12.0 > Environment: flink :1.12.0 > kafka 2.2.1 >Reporter: xx chai >Priority: Major > > I consum from kafka sink to kafka,Then I split the message into then pieces.I > guess the ten message should in one transaction. When the fifth message is > sink kafka ,I throw a exception.but the first four are already in kafka. > I set some parameters : >properties.setProperty("transactional.id", "cxx"); > properties.setProperty("ack", "all"); > properties.put("enable.idempotence",true); > properties.put("max.in.flight.requests.per.connection",5); > properties.put("retries", 2); > properties.setProperty("client.id", "producer-syn-2"); > properties.put("isolation.level","read_committed"); -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-20870) FlinkKafkaSink
[ https://issues.apache.org/jira/browse/FLINK-20870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17260939#comment-17260939 ] xx chai commented on FLINK-20870: - this is my program > FlinkKafkaSink > -- > > Key: FLINK-20870 > URL: https://issues.apache.org/jira/browse/FLINK-20870 > Project: Flink > Issue Type: Bug > Components: Connectors / Kafka >Affects Versions: 1.12.0 > Environment: flink :1.12.0 > kafka 2.2.1 >Reporter: xx chai >Priority: Major > Attachments: 1610072517(1).jpg > > > I consum from kafka sink to kafka,Then I split the message into then pieces.I > guess the ten message should in one transaction. When the fifth message is > sink kafka ,I throw a exception.but the first four are already in kafka. > I set some parameters : >properties.setProperty("transactional.id", "cxx"); > properties.setProperty("ack", "all"); > properties.put("enable.idempotence",true); > properties.put("max.in.flight.requests.per.connection",5); > properties.put("retries", 2); > properties.setProperty("client.id", "producer-syn-2"); > properties.put("isolation.level","read_committed"); -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-20870) FlinkKafkaSink
[ https://issues.apache.org/jira/browse/FLINK-20870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17260938#comment-17260938 ] xx chai commented on FLINK-20870: - !1610072517(1).jpg! > FlinkKafkaSink > -- > > Key: FLINK-20870 > URL: https://issues.apache.org/jira/browse/FLINK-20870 > Project: Flink > Issue Type: Bug > Components: Connectors / Kafka >Affects Versions: 1.12.0 > Environment: flink :1.12.0 > kafka 2.2.1 >Reporter: xx chai >Priority: Major > Attachments: 1610072517(1).jpg > > > I consum from kafka sink to kafka,Then I split the message into then pieces.I > guess the ten message should in one transaction. When the fifth message is > sink kafka ,I throw a exception.but the first four are already in kafka. > I set some parameters : >properties.setProperty("transactional.id", "cxx"); > properties.setProperty("ack", "all"); > properties.put("enable.idempotence",true); > properties.put("max.in.flight.requests.per.connection",5); > properties.put("retries", 2); > properties.setProperty("client.id", "producer-syn-2"); > properties.put("isolation.level","read_committed"); -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-20870) FlinkKafkaSink
[ https://issues.apache.org/jira/browse/FLINK-20870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xx chai updated FLINK-20870: Attachment: 1610072517(1).jpg > FlinkKafkaSink > -- > > Key: FLINK-20870 > URL: https://issues.apache.org/jira/browse/FLINK-20870 > Project: Flink > Issue Type: Bug > Components: Connectors / Kafka >Affects Versions: 1.12.0 > Environment: flink :1.12.0 > kafka 2.2.1 >Reporter: xx chai >Priority: Major > Attachments: 1610072517(1).jpg > > > I consum from kafka sink to kafka,Then I split the message into then pieces.I > guess the ten message should in one transaction. When the fifth message is > sink kafka ,I throw a exception.but the first four are already in kafka. > I set some parameters : >properties.setProperty("transactional.id", "cxx"); > properties.setProperty("ack", "all"); > properties.put("enable.idempotence",true); > properties.put("max.in.flight.requests.per.connection",5); > properties.put("retries", 2); > properties.setProperty("client.id", "producer-syn-2"); > properties.put("isolation.level","read_committed"); -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-20870) FlinkKafkaSink
[ https://issues.apache.org/jira/browse/FLINK-20870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xx chai updated FLINK-20870: Issue Type: Bug (was: Improvement) > FlinkKafkaSink > -- > > Key: FLINK-20870 > URL: https://issues.apache.org/jira/browse/FLINK-20870 > Project: Flink > Issue Type: Bug > Components: API / DataStream >Affects Versions: 1.12.0 > Environment: flink :1.12.0 > kafka 2.2.1 >Reporter: xx chai >Priority: Major > > I consum from kafka sink to kafka,Then I split the message into then pieces.I > guess the ten message should in one transaction. When the fifth message is > sink kafka ,I throw a exception.but the first four are already in kafka. > I set some parameters : >properties.setProperty("transactional.id", "cxx"); > properties.setProperty("ack", "all"); > properties.put("enable.idempotence",true); > properties.put("max.in.flight.requests.per.connection",5); > properties.put("retries", 2); > properties.setProperty("client.id", "producer-syn-2"); > properties.put("isolation.level","read_committed"); -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-20870) FlinkKafkaSink
xx chai created FLINK-20870: --- Summary: FlinkKafkaSink Key: FLINK-20870 URL: https://issues.apache.org/jira/browse/FLINK-20870 Project: Flink Issue Type: Improvement Components: API / DataStream Affects Versions: 1.12.0 Environment: flink :1.12.0 kafka 2.2.1 Reporter: xx chai I consum from kafka sink to kafka,Then I split the message into then pieces.I guess the ten message should in one transaction. When the fifth message is sink kafka ,I throw a exception.but the first four are already in kafka. I set some parameters : properties.setProperty("transactional.id", "cxx"); properties.setProperty("ack", "all"); properties.put("enable.idempotence",true); properties.put("max.in.flight.requests.per.connection",5); properties.put("retries", 2); properties.setProperty("client.id", "producer-syn-2"); properties.put("isolation.level","read_committed"); -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-20780) Flink-sql-client query hive
[ https://issues.apache.org/jira/browse/FLINK-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255497#comment-17255497 ] xx chai commented on FLINK-20780: - I may have found the reason,sql-client is not shut-down when ctrl+z or ./stop-cluster. I use the JPS command and kill sql-clients process .then resatrt and query hive is succeed > Flink-sql-client query hive > > > Key: FLINK-20780 > URL: https://issues.apache.org/jira/browse/FLINK-20780 > Project: Flink > Issue Type: Bug > Components: Connectors / Hive, Table SQL / Client > Environment: standalone >Reporter: xx chai >Priority: Major > Attachments: 1609127183(1).png, ps-flink-process.txt > > > flink-sql-client query hive is fail. > I've already configured hadoop classpath > and my hadoop is cdh > error : > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConfCaused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConf at > java.net.URLClassLoader.findClass(URLClassLoader.java:382) at > java.lang.ClassLoader.loadClass(ClassLoader.java:418) at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at > java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 56 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-20780) Flink-sql-client query hive
[ https://issues.apache.org/jira/browse/FLINK-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255442#comment-17255442 ] xx chai commented on FLINK-20780: - [^ps-flink-process.txt] the flie is I execute aux | grep flink-process-id You can see from the file that hadoop classpath is loaded > Flink-sql-client query hive > > > Key: FLINK-20780 > URL: https://issues.apache.org/jira/browse/FLINK-20780 > Project: Flink > Issue Type: Bug > Components: Connectors / Hive, Table SQL / Client > Environment: standalone >Reporter: xx chai >Priority: Major > Attachments: 1609127183(1).png, ps-flink-process.txt > > > flink-sql-client query hive is fail. > I've already configured hadoop classpath > and my hadoop is cdh > error : > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConfCaused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConf at > java.net.URLClassLoader.findClass(URLClassLoader.java:382) at > java.lang.ClassLoader.loadClass(ClassLoader.java:418) at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at > java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 56 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-20780) Flink-sql-client query hive
[ https://issues.apache.org/jira/browse/FLINK-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xx chai updated FLINK-20780: Attachment: ps-flink-process.txt > Flink-sql-client query hive > > > Key: FLINK-20780 > URL: https://issues.apache.org/jira/browse/FLINK-20780 > Project: Flink > Issue Type: Bug > Components: Connectors / Hive, Table SQL / Client > Environment: standalone >Reporter: xx chai >Priority: Major > Attachments: 1609127183(1).png, ps-flink-process.txt > > > flink-sql-client query hive is fail. > I've already configured hadoop classpath > and my hadoop is cdh > error : > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConfCaused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConf at > java.net.URLClassLoader.findClass(URLClassLoader.java:382) at > java.lang.ClassLoader.loadClass(ClassLoader.java:418) at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at > java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 56 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-20780) Flink-sql-client query hive
[ https://issues.apache.org/jira/browse/FLINK-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255407#comment-17255407 ] xx chai commented on FLINK-20780: - There is only one node in my cluster,the cluster is my test cluster > Flink-sql-client query hive > > > Key: FLINK-20780 > URL: https://issues.apache.org/jira/browse/FLINK-20780 > Project: Flink > Issue Type: Bug > Components: Connectors / Hive, Table SQL / Client > Environment: standalone >Reporter: xx chai >Priority: Major > Attachments: 1609127183(1).png > > > flink-sql-client query hive is fail. > I've already configured hadoop classpath > and my hadoop is cdh > error : > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConfCaused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConf at > java.net.URLClassLoader.findClass(URLClassLoader.java:382) at > java.lang.ClassLoader.loadClass(ClassLoader.java:418) at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at > java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 56 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-20780) Flink-sql-client query hive
[ https://issues.apache.org/jira/browse/FLINK-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255393#comment-17255393 ] xx chai commented on FLINK-20780: - export HADOOP_HOME=/opt/cloudera/parcels/CDH/lib/hadoop export PATH=${HADOOP_HOME}/bin:$PATH export HADOOP_CLASSPATH=`$HADOOP_HOME/bin/hadoop classpath` I execute hadoop classpath,It prints the correct result > Flink-sql-client query hive > > > Key: FLINK-20780 > URL: https://issues.apache.org/jira/browse/FLINK-20780 > Project: Flink > Issue Type: Bug > Components: Connectors / Hive, Table SQL / Client > Environment: standalone >Reporter: xx chai >Priority: Major > Attachments: 1609127183(1).png > > > flink-sql-client query hive is fail. > I've already configured hadoop classpath > and my hadoop is cdh > error : > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConfCaused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConf at > java.net.URLClassLoader.findClass(URLClassLoader.java:382) at > java.lang.ClassLoader.loadClass(ClassLoader.java:418) at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at > java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 56 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-20780) Flink-sql-client query hive
[ https://issues.apache.org/jira/browse/FLINK-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255387#comment-17255387 ] xx chai commented on FLINK-20780: - Every time i restart the cluster after you change jars in the lib folder > Flink-sql-client query hive > > > Key: FLINK-20780 > URL: https://issues.apache.org/jira/browse/FLINK-20780 > Project: Flink > Issue Type: Bug > Components: Connectors / Hive, Table SQL / Client > Environment: standalone >Reporter: xx chai >Priority: Major > Attachments: 1609127183(1).png > > > flink-sql-client query hive is fail. > I've already configured hadoop classpath > and my hadoop is cdh > error : > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConfCaused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConf at > java.net.URLClassLoader.findClass(URLClassLoader.java:382) at > java.lang.ClassLoader.loadClass(ClassLoader.java:418) at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at > java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 56 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-20780) Flink-sql-client query hive
[ https://issues.apache.org/jira/browse/FLINK-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255375#comment-17255375 ] xx chai commented on FLINK-20780: - I have deleted {{flink-connector-hive_2.11-1.12.0.jar}}, {{hive-exec-2.1.1-cdh6.3.2.jar}}, {{hive-exec-2.2.0.jar.bak, but when I query hive same mistake.}} now lib: flink-connector-jdbc_2.11-1.12.0.jar flink-csv-1.12.0.jar flink-dist_2.11-1.12.0.jar flink-json-1.12.0.jar flink-shaded-hadoop-3-uber-3.1.1.7.1.1.0-565-9.0.jar flink-shaded-zookeeper-3.4.14.jar flink-sql-connector-hive-2.2.0_2.11-1.12.0.jar flink-sql-connector-kafka_2.11-1.12.0.jar flink-table_2.11-1.12.0.jar flink-table-api-java-bridge_2.11-1.12.0.jar flink-table-blink_2.11-1.12.0.jar log4j-1.2-api-2.12.1.jar log4j-api-2.12.1.jar log4j-core-2.12.1.jar log4j-slf4j-impl-2.12.1.jar mysql-connector-java-8.0.22.jar > Flink-sql-client query hive > > > Key: FLINK-20780 > URL: https://issues.apache.org/jira/browse/FLINK-20780 > Project: Flink > Issue Type: Bug > Components: Connectors / Hive, Table SQL / Client > Environment: standalone >Reporter: xx chai >Priority: Major > Attachments: 1609127183(1).png > > > flink-sql-client query hive is fail. > I've already configured hadoop classpath > and my hadoop is cdh > error : > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConfCaused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConf at > java.net.URLClassLoader.findClass(URLClassLoader.java:382) at > java.lang.ClassLoader.loadClass(ClassLoader.java:418) at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at > java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 56 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-20780) Flink-sql-client query hive
[ https://issues.apache.org/jira/browse/FLINK-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255369#comment-17255369 ] xx chai edited comment on FLINK-20780 at 12/28/20, 3:59 AM: Flink version is 1.12 chd version is 6.3.2 hadoop version is 3.0.0+cdh6.3.2 flink lib : flink-connector-hive_2.11-1.12.0.jar flink-connector-jdbc_2.11-1.12.0.jar flink-csv-1.12.0.jar flink-dist_2.11-1.12.0.jar flink-json-1.12.0.jar flink-shaded-hadoop-3-uber-3.1.1.7.1.1.0-565-9.0.jar flink-shaded-zookeeper-3.4.14.jar flink-sql-connector-hive-2.2.0_2.11-1.12.0.jar flink-sql-connector-kafka_2.11-1.12.0.jar flink-table_2.11-1.12.0.jar flink-table-api-java-bridge_2.11-1.12.0.jar flink-table-blink_2.11-1.12.0.jar hive-exec-2.1.1-cdh6.3.2.jar hive-exec-2.2.0.jar.bak log4j-1.2-api-2.12.1.jar log4j-api-2.12.1.jar log4j-core-2.12.1.jar log4j-slf4j-impl-2.12.1.jar mysql-connector-java-8.0.22.jar was (Author: chaixiaoxue): Flink version is 1.12 > Flink-sql-client query hive > > > Key: FLINK-20780 > URL: https://issues.apache.org/jira/browse/FLINK-20780 > Project: Flink > Issue Type: Bug > Components: Connectors / Hive, Table SQL / Client > Environment: standalone >Reporter: xx chai >Priority: Major > Attachments: 1609127183(1).png > > > flink-sql-client query hive is fail. > I've already configured hadoop classpath > and my hadoop is cdh > error : > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConfCaused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConf at > java.net.URLClassLoader.findClass(URLClassLoader.java:382) at > java.lang.ClassLoader.loadClass(ClassLoader.java:418) at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at > java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 56 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-20780) Flink-sql-client query hive
[ https://issues.apache.org/jira/browse/FLINK-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255369#comment-17255369 ] xx chai commented on FLINK-20780: - Flink version is 1.12 > Flink-sql-client query hive > > > Key: FLINK-20780 > URL: https://issues.apache.org/jira/browse/FLINK-20780 > Project: Flink > Issue Type: Bug > Components: Connectors / Hive, Table SQL / Client > Environment: standalone >Reporter: xx chai >Priority: Major > Attachments: 1609127183(1).png > > > flink-sql-client query hive is fail. > I've already configured hadoop classpath > and my hadoop is cdh > error : > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConfCaused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConf at > java.net.URLClassLoader.findClass(URLClassLoader.java:382) at > java.lang.ClassLoader.loadClass(ClassLoader.java:418) at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at > java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 56 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-20780) Flink-sql-client query hive
[ https://issues.apache.org/jira/browse/FLINK-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xx chai updated FLINK-20780: Attachment: 1609127183(1).png > Flink-sql-client query hive > > > Key: FLINK-20780 > URL: https://issues.apache.org/jira/browse/FLINK-20780 > Project: Flink > Issue Type: Bug > Components: Table SQL / Client > Environment: standalone >Reporter: xx chai >Priority: Major > Attachments: 1609127183(1).png > > > flink-sql-client query hive is fail. > I've already configured hadoop classpath > and my hadoop is cdh > error : > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConfCaused by: java.lang.ClassNotFoundException: > org.apache.hadoop.mapred.JobConf at > java.net.URLClassLoader.findClass(URLClassLoader.java:382) at > java.lang.ClassLoader.loadClass(ClassLoader.java:418) at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at > java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 56 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-20780) Flink-sql-client query hive
xx chai created FLINK-20780: --- Summary: Flink-sql-client query hive Key: FLINK-20780 URL: https://issues.apache.org/jira/browse/FLINK-20780 Project: Flink Issue Type: Bug Components: Table SQL / Client Environment: standalone Reporter: xx chai flink-sql-client query hive is fail. I've already configured hadoop classpath and my hadoop is cdh error : Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.JobConfCaused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.JobConf at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 56 more -- This message was sent by Atlassian Jira (v8.3.4#803005)