[ https://issues.apache.org/jira/browse/FLUME-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16290421#comment-16290421 ]
Abhishek Choudhary commented on FLUME-2956: ------------------------------------------- [~hejiang] Thanks :) its working fine with the updated jar. Can you please share the git patch for the same? i will check few more scenarios. > hive sink not sending heartbeat correctly > ----------------------------------------- > > Key: FLUME-2956 > URL: https://issues.apache.org/jira/browse/FLUME-2956 > Project: Flume > Issue Type: Bug > Components: Sinks+Sources > Affects Versions: 1.5.2 > Environment: linux CentOS 6.6 > Reporter: Olivier brobecker > Attachments: flume-hive-sink-1.8.0.jar > > > Flume has been configured in order to populate a hive table as follow : > agentSCDR.sinks.hive1.type = hive > agentSCDR.sinks.hive1.channel = channel1 > agentSCDR.sinks.hive1.hive.metastore = thrift://myserver:9083 > agentSCDR.sinks.hive1.hive.txnsPerBatchAsk = 10 > agentSCDR.sinks.hive1.hive.database = myDatabase > agentSCDR.sinks.hive1.hive.table = my_table > agentSCDR.sinks.hive1.hive.partition = %Y > agentSCDR.sinks.hive1.heartBeatInterval = 60 > agentSCDR.sinks.hive1.serializer = DELIMITED > agentSCDR.sinks.hive1.serializer.delimiter = ; > agentSCDR.sinks.hive1.serializer.serdeSeparator = ; > agentSCDR.sinks.hive1.serializer.fieldnames = field1,field2,field3... > My dataflow is irregular at best and I can have more than 30 minutes of > inactivity, so I have a heartbeatInterval at 60s in order to keep my txn > alive. > The issue is that this heartBeat is only sent when flume is trying to inject > some datas into hive instead of having one every 60s. > # grep -i heartbeat flume-agentSCDR.log > 15 juil. 2016 13:40:43,008 INFO [hive-hive1-call-runner-0] > (org.apache.flume.sink.hive.HiveWriter$2.call:238) - Sending heartbeat on > batch TxnIds=[3755...3764] on endPoint = {metaStoreUri=... > 15 juil. 2016 14:12:21,001 INFO [hive-hive1-call-runner-0] > (org.apache.flume.sink.hive.HiveWriter$2.call:231) - Sending heartbeat on > batch TxnIds=[3785...3794] on endPoint = {metaStoreUri=... > 15 juil. 2016 14:27:56,963 INFO [hive-hive1-call-runner-0] > (org.apache.flume.sink.hive.HiveWriter$2.call:231) - Sending heartbeat on > batch TxnIds=[3795...3804] on endPoint = {metaStoreUri=... > ... -- This message was sent by Atlassian JIRA (v6.4.14#64029)