Hi Daniel, I did the below test and I was able to receive the messages in consumer. Infact, I also tried below steps with toic ‘sandbox_hdfs_audit_log’ and was able to see my test messages.
Still I don’t see the HDFS policy alerts in eagle Thanks Sanjay From: Daniel Zhou [mailto:daniel.z...@dataguise.com] Sent: Tuesday, February 16, 2016 5:28 PM To: dev@eagle.incubator.apache.org; Nagalkar, Sanjay Contractor Subject: RE: Policy Alerts are not generated in Eagle Hi, Nagalkar Could you check if your Kafka works ? Use these steps: 1. Create a topic: bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test 2. Check if topic “test” is created: bin/kafka-topics.sh --list --zookeeper localhost:2181 3. Open a terminal and use producer to send message: bin/kafka-console-producer.sh --broker-list hostname:6667 --topic test // then type some messages 4. Open another terminal and see if consumer get the message: bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning // check if messages received If no messages received, you need to double check the setting of Kafka. Regards, Daniel From: Nagalkar, Sanjay Contractor [mailto:sanjay.nagal...@ssa.gov] Sent: Tuesday, February 16, 2016 1:58 PM To: 'dev@eagle.incubator.apache.org' <dev@eagle.incubator.apache.org<mailto:dev@eagle.incubator.apache.org>> Subject: FW: Policy Alerts are not generated in Eagle Hello, just wondering if there is any update on this issue I reported earlier . Below is the detailed information about the encountered issue and the configuration made in our eagle setup: We are planning to implement some tool for sensitive data monitoring /audit and for this purpose , I am exploring Eagle to see if it fits our requirements. My goal is play with eagle in Horton works sandbox environment and once comfortable then move to production. I have followed the Eagle user guide and had setup Horton works sandbox and eagle service as instructed in the User Docs. (I did not use the Ambari plug-in for eagle as I had some issue with it so I am starting eagle services manually from command line). I have configured HIVE and HDFS policies but I am not seeing any Alerts generated for data access by these policy. For HDFS policy – I have setup policy for /tmp/private file access but here I am receiving error while writing to KAFKA topic. I am getting "connection Refused" error. I have a Kafka topic 'Sandbox_hdfs_audit_log' and hdfs action events are not logged into this topic. when I view kafka.out , I see ERROR Exception emitting metrics - org.apache.hadoop.metrics2.sink.timelineUnableToConnectException:java.net.ConnectException. I have verified that the Kafka broker is up and configured ( Pls. see below log screenshots ) For HIVE policy - I have setup Hive policy for PHONE_NUMBER field setup. When I use hive from Horton works Sandbox and execute the HIVE queries against XADEMO schema to select PHONE_NUMBER field, Eagle is not generating any alerts for accessing PHONE_NUMBER field . Pls. click below link to see document outlining steps for HIVE query and eagle policy. https://community.hortonworks.com/storage/attachments/1964-dtempsanbox-hive-eagle-issue.pdf Please let me know if you need more information on this. Your help with this is very much appreciated. Thanks Sanjay Nagalkar Ph: 410-371-3299 On Wed, Feb 10, 2016 at 12:55 PM, Nagalkar, Sanjay Contractor <sanjay.nagal...@ssa.gov<mailto:sanjay.nagal...@ssa.gov>> wrote: Hello I am having issues generating policy alerts for Apache Eagle. I have configured Hive and HDFS policies as per the instructions in the Eagle DOCS but I am not seeing any alerts for my Hive and HDFS policy. I am executing HIVE queries against XADEMO schema to pull PHONE_NUMBER field via HDP sandbox and I was expecting to see some alerts for this data access in eagle. However, I have no policy alerts . Similar situation with HDFS file access. No alerts for /tmp/private file access. Please let me know if there is something that I am missing. Thanks Sanjay Nagalkar
