Hi Guys,
I am trying to send logs from solr directly to kafka using log4j. I have made
changes in my log4j.properties file, and am not sure where is the problem.
Also, do I need to include a kafka jar file? Can somebody help me on this?
________________________________
The warning and error I am getting:
WARN - 2015-01-19 12:09:25.545;
org.apache.solr.cloud.Overseer$ClusterStateUpdater; Solr cannot talk to ZK,
exiting Overseer main queue loop
INFO - 2015-01-19 12:09:25.552;
org.apache.solr.cloud.Overseer$ClusterStateUpdater; Overseer Loop exiting :
10.254.120.50:8900_solr
WARN - 2015-01-19 12:09:25.554; org.apache.solr.common.cloud.ZkStateReader$2;
ZooKeeper watch triggered, but Solr cannot talk to ZK
ERROR - 2015-01-19 12:09:25.560;
org.apache.solr.cloud.Overseer$ClusterStateUpdater; could not read the data
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode =
Session expired for /overseer_elect/leader
My Log4j.Properties file:
solr.log=/home/solradmin/solr/latest/logs/
log4j.rootLogger=INFO, file, KAFKA
log4j.logger.KAFKA=INFO, file
log4j.logger.solr=INFO, KAFKA
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%5p [%t] (%F:%L) - %m%n
log4j.appender.KAFKA=kafka.producer.KafkaLog4jAppender
log4j.appender.KAFKA.layout=org.apache.log4j.PatternLayout
log4j.appender.KAFKA.layout.ConversionPattern=%-5p: %c - %m%n
log4j.appender.KAFKA.BrokerList=localhost:9092
log4j.appender.KAFKA.Topic=herpderp
log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.MaxFileSize=100MB
log4j.appender.file.MaxBackupIndex=9
log4j.appender.file.File=${solr.log}/solr.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%-5p - %d{yyyy-MM-dd
HH:mm:ss.SSS}; %C; %m\n
log4j.logger.org.apache.solr=DEBUG
log4j.logger.org.apache.zookeeper=WARN
log4j.logger.org.apache.hadoop=WARN
I tried different configurations using the following sources:
http://kafka.apache.org/07/quickstart.htmland KafkLog4JAppender not pushing
application logs to kafka
topic<http://stackoverflow.com/questions/30171881/kafklog4jappender-not-pushing-application-logs-to-kafka-topic?rq=1>.
____________________________________
This message may contain information that is privileged or confidential. If you
received this transmission in error, please notify the sender by reply e-mail
and delete the message and any attachments. Any duplication, dissemination or
distribution of this message by unintended recipients is prohibited. Please
note: All email sent to this address will be received by the NetDocuments
corporate e-mail system and is subject to archival and review by someone other
than the recipient.
