hi all：
I use flume to import data from syslog to hive,but encount the follow errors.
2015-10-22 10:05:05,115 (SinkRunner-PollingRunner-DefaultSinkProcessor) [WARN -
org.apache.flume.sink.hive.HiveSink.drainOneBatch(HiveSink.java:324)] k2 :
Failed connecting to EndPoint {metaStoreUri='thrift://bigdata1:9083',
database='dnsdb', table='dns_request', partitionVals=[] }
org.apache.flume.sink.hive.HiveWriter$ConnectException: Failed connecting to
EndPoint {metaStoreUri='thrift://bigdata1:9083', database='dnsdb',
table='dns_request', partitionVals=[] }
at org.apache.flume.sink.hive.HiveWriter.<init>(HiveWriter.java:99)
at
org.apache.flume.sink.hive.HiveSink.getOrCreateWriter(HiveSink.java:344)
at org.apache.flume.sink.hive.HiveSink.drainOneBatch(HiveSink.java:296)
at org.apache.flume.sink.hive.HiveSink.process(HiveSink.java:254)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.flume.sink.hive.HiveWriter$ConnectException: Failed
connecting to EndPoint {metaStoreUri='thrift://bigdata1:9083',
database='dnsdb', table='dns_request', partitionVals=[] }
at
org.apache.flume.sink.hive.HiveWriter.newConnection(HiveWriter.java:380)
at org.apache.flume.sink.hive.HiveWriter.<init>(HiveWriter.java:86)
... 6 more
Caused by: java.util.concurrent.TimeoutException
at java.util.concurrent.FutureTask.get(FutureTask.java:201)
at org.apache.flume.sink.hive.HiveWriter.timedCall(HiveWriter.java:431)
at
org.apache.flume.sink.hive.HiveWriter.newConnection(HiveWriter.java:373)
... 7 more
my configuration is:
a1.sources = r1
a1.channels = c1 c2
a1.sinks = k1 k2
a1.sources.r1.type = syslogudp
a1.sources.r1.port = 514
a1.sources.r1.host = 192.168.55.246
a1.sources.r1.channels = c1 c2
a1.sources.r1.interceptors = i1
a1.sources.r1.interceptors.i1.type = regex_extractor
a1.sources.r1.interceptors.i1.regex = Dns(.*)\\[
a1.sources.r1.interceptors.i1.serializers = t1
a1.sources.r1.interceptors.i1.serializers.t1.name = type
a1.sources.r1.selector.type = multiplexing
a1.sources.r1.selector.header = type
a1.sources.r1.selector.mapping.Request = c1
a1.sources.r1.selector.mapping.Answer = c2
a1.sinks.k2.type = hive
a1.sinks.k2.channel = c1
a1.sinks.k2.hive.metastore = thrift://bigdata1:9083
a1.sinks.k2.hive.database = dnsdb
a1.sinks.k2.hive.table = dns_request
a1.sinks.k2.hive.partiton = %Y,%m,%d,%H
a1.sinks.k2.hive.txnsPerBatchAsk = 2
a1.sinks.k2.batchSize = 10
a1.sinks.k2.serializer = delimited
a1.sinks.k2.serializer.delimiter = ,
a1.sinks.k2.serializer.fieldnames = timepoint,random,sip,dip,spt,type,name
a1.sinks.k1.type = hive
a1.sinks.k1.channel = c2
a1.sinks.k1.hive.metastore = thrift://bigdata1:9083
a1.sinks.k1.hive.database = Dnsdb
a1.sinks.k1.hive.table = dns_answer
a1.sinks.k1.hive.partiton = %Y,%m,%d,%H
a1.sinks.k1.hive.txnsPerBatchAsk = 2
a1.sinks.k1.batchSize = 10
a1.sinks.k1.serializer = delimited
a1.sinks.k1.serializer.delimiter = ,
a1.sinks.k1.serializer.fieldnames =
timepoint,random,sip,dip,dpt,name,nosuchname,typemax,typecname,typeaddr,authservername,additionalrecords
help me please,thanks.
[email protected]
