Re: [Discuss] Check Style Specification

It is nice. So do we have offline document for us to review Google java style? It is supposed that a lot of warnings will be generated with those code standards. Thanks Edward On 8/7/16, 22:36, "r7raul1...@163.com" wrote: >I fire a jira for checkstyle base on Google Java

Re: Created branch "spark" for supporting alert engine on spark streaming

Is this feature aligned with new alert engine in 0.5? Thanks Edward On 8/4/16, 23:14, "Daniel Zhou" wrote: >This is so exiting! Great to have spark support! > >Regards, >Da > > > >> On Aug 4, 2016, at 10:45 PM, Hao Chen wrote: >> >> During offline

Re: [Discuss] what will be the decent way to remove jars from source code for releases

In 0.3 release, Hemanth uses a patch to work around this issue. Can we use the same approach in 0.4 and in 0.5 we have decided to remove dependency on tomcat. Thanks Edward From: Hao Chen Sent: Sunday, July 10, 2016 8:17:48 PM To:

Re: [Announce] New Apache Eagle Committer: Daniel Zhou

Hi Daniel, Welcome to Eagle community, congratulations! Thanks Edward On 6/13/16, 9:33, "Daniel Zhou" wrote: > >Hi Eagle Community , > >Thank you all :). >My name is Da Zhou. I'm a software engineer working at Dataguise, Inc, >focusing on detecting and protecting

Re: SMTP settings for Email alerts

You can pass auth configuration through configuration, by default this is disabled.Please try it. mail.smtp.auth=true mail.user mail.pwd But I do see there is possibly a bug with this, in AlertEmailSender, those Auth information is not passed into EagleMailClient. Thanks Edward On 5/10/16,

Re: Restful API changed?

Right Daniel, we have made many changes recently in master which is for Eagle 0.4, it may not be stable, but we are also creating documents for that. Because you are doing MapR integration, I would suggest you merge that into master as soon as possible to avoid much conflict in the future as long

Re: Question Regarding the folder/file name.

looks 0.3.0-incubating naming is more informational, like kylin https://dist.apache.org/repos/dist/release/kylin/ On 4/6/16, 2:49, "Dendukuri, Hemanth" wrote: >Hi, > >I want to upload voted "eagle release candidate 3" artifacts to apache >release distribution

Re: [VOTE] Release Eagle Incubating Version 0.3.0

Do we have a checklist which we can run like Julian did, so we can avoid some obvious issues. Thanks Edward On 3/6/16, 8:11, "Dendukuri, Hemanth" wrote: >Thanks Julial,for your valuable comments. > >The build was tested on JDK 1.7 and we don't yet support 1.8. I thing we

Re: Cannot svn commit the release for eagle.

We can skip Hadoop/storm related dependency but need test how to run within sandbox. Thanks Edward On 3/3/16, 19:12, "Dendukuri, Hemanth" wrote: >I got this comment for infrastructure bug. > >That's your problem: we have upload limits in place that defaults to >200MB. > >

Re: Work on Eagle 0.3.0 release has started.

Looks all jira tickets are cleared, we can go ahead for next step for release. Thanks Edward On 2/9/16, 13:43, "Dendukuri, Hemanth" wrote: > >I lately realized that we should use ³fixVersion² instead of >³affectedVersion² to tag a particular jira for release. >Kindly

Re: Policy Alerts are not generated in Eagle

For HDFS audit log, by default Eagle uses Kafka log4j appender to send namenode audit log into Kafka. Could you please try "/usr/hdp/2.2.4.2-2/kafka/bin/kafka-console-consumer.sh --topic sandbox_hdfs_audit_log --zookeeper localhost:2181² to see if message is sent out. For Hive query log, by

Re: Work on Eagle 0.3.0 release has started.

Hi Hemanth, Thanks for this work. I have reviewed my tickets. Now we totally have 17 open tickets, owners should look at them and judge if theirs should be within this release. Thanks Edward On 2/9/16, 13:43, "Dendukuri, Hemanth" wrote: > >I lately realized that we should

Re: Policy based on sensitive types stops working if there are too many sensitive items

can you please show one policy? I thought it would be because policy itself is too complicated for engine to parse and evaluate Thanks Edward On 2/5/16, 15:46, "Daniel Zhou" wrote: >Hi all, > >Anyone test eagle's performance in this situation? > >1. large

Re: [Proposal] Dynamical Topology Manager

hink we could start with a list of what Apache Eagle needed from >> Topology lifecycle management and see if we do cross-post to both dev@ >>list >> of Eagle and Storm to see if enough interest from Storm to work on it or >> accept contributions from Eagle. >> >> - H

Re: Questions about multiple-value sensitivityType

As CEP engine does not support list/nested data structure directly, it is hard to match one value onto one field which contain multiple values. probably we can look at Siddhi extension … So right now, we can do workaround like the following in field value, we store the multiple value as v1|v2|v3

Re: [Proposal] Dynamical Topology Manager

I had a short discussion with Henry about this. We probably need discuss a more graceful way to tackle the problem of whether eagle does this or storm does this. Today we know that Storm also provides topology view/statistics features but does not have topology lifecycle management UI. But can we

Re: [BUG]UnionForAlert test case fail

Hao, is this because of your new code on dsl? why do we keep eagle.Tuple2 if we already use scala.Tuple2? Thanks Edward On 12/28/15, 18:26, "Zhang, Edward (GDI Hadoop)" <yonzh...@ebay.com> wrote: >I found TestStormRunner::UnionForAlert fail with type incompatibility, I >

Re: suggestion: add field "threshold" to current "fileSensitivity structure" in Eagle

Yes, looks we need a schema abstraction which can represent any sensitivity information. sensitivityType and numOfOccurrences are just two common fields of the whole sensitivity information. For hdfs, the sensitivity information also includes filedir, while for hive, the sensitivity information

Re: suggestion: add field "threshold" to current "fileSensitivity structure" in Eagle

can save any data they want in that JSON string field. > >Regards, >Daniel > >-Original Message- >From: Zhang, Edward (GDI Hadoop) [mailto:yonzh...@ebay.com] >Sent: Wednesday, January 13, 2016 1:18 PM >To: dev@eagle.incubator.apache.org >Subject: Re: suggestion: add f

bug after merged EAGLE-79

I saw TestSiddhiEvaluator failed after EAGLE-79 because of incompatibility of message format sent to Siddhi engine. In stream definition, it has the following schema without streamname 2016-01-12 11:30:09,312 INFO [main] siddhi.SiddhiPolicyEvaluator[114]: Siddhi stream definition : define

Re: [Fix Eagle Unit Test] please fix eagle's unit test

I have fixed org.apache.eagle.alert.state.TestSiddhiStateSnapshotAndRestore.persistenceT est7, will create pull request. For https://github.com/apache/incubator-eagle/pull/43, can you look at why it is not passing the checks. We need merge that pull request 43 soon. Thanks edward On 1/11/16,

Re: [Discuss] Hadoop metrics,job,GC monitoring

please review latest design of monitoring on hadoop native metrics. https://cwiki.apache.org/confluence/display/EAG/Hadoop+Native+Metrics+Monit oring Thanks Edward On 12/14/15, 23:48, "Zhang, Edward (GDI Hadoop)" <yonzh...@ebay.com> wrote: >started some docu

Re: [siddhi] Siddhi external time (#76)

This is awesome that Eagle uses Siddhi CEP engine and contributes critical improvements back to community :-) Thanks Edward On 1/5/16, 2:44, "Liangfei.Su" wrote: >FYI > >window.externalTimeBatch originated by eagle team is accepted by siddhi >team. :) > > >--

Re: [GitHub] incubator-eagle pull request: EAGLE-78 add configuration requireme...

-- If you find any issue of using this script, please log a comment on https://github.com/apache/incubator-eagle/pull/42 Thanks, Michael On Thu, Dec 31, 2015 at 2:57 PM, Zhang, Edward (GDI Hadoop) <yonzh...@ebay.com<mailto:yonzh...@ebay.com>> wrote: I am going to merg

[Enhancement]AlertExecutor

In AlertExecutor.flatmap, I don’t think we need this double try/catch. And I did not know what is the difference between EAGLE_POLICY_EVAL_FAIL_COUNT and EAGLE_ALERT_FAIL_COUNT. Today we use siddhi asynchronized model, so alert will never fail in flatmap thread. I will clean them up if no

Re: [GitHub] incubator-eagle pull request: EAGLE-78 add configuration requireme...

ollow our criteria: EAGLE-${jira_number} >${pr_title_content}, please ask the author to revise it or you change it >when you recreate the PR. > >Thanks. >Michael > >On Thu, Dec 31, 2015 at 2:35 PM, Zhang, Edward (GDI Hadoop) < >yonzh...@ebay.com> wrote: > >> I mad

Re: query in merge process

Hi Murali, Right now, probably you don¹t have merging privilege. I can merge the pull request https://github.com/apache/incubator-eagle/pull/39 on behalf of you today. I think you will have privilege once after you are committer. That would need some votes based on some merits according to apache

Re: [GitHub] incubator-eagle pull request: EAGLE-78 add configuration requireme...

I made a mistake to close this pull request because I mistaken pull request id for jira id. Please advise how to reopen this pull request #47 Thanks Edward On 12/30/15, 22:12, "asfgit" wrote: >Github user asfgit closed the pull request at: > >

[BUG]UnionForAlert test case fail

I found TestStormRunner::UnionForAlert fail with type incompatibility, I think that is because Hao has refactored processing layer, but we should fix those unit test cases. object UnionForAlert extends App{ val config : Config = ConfigFactory.load; val env =

Re: Requesting contributor access to Jira

Hi Prasad, Added you to contributor access, thanks for being interested in that. thanks Edward On 12/28/15, 1:34, "Prasad Mujumdar" wrote: >Hello, > > I would be interested in contributing to the project. Requesting >contributor access to the Jira so that I can assign

[Review request]policy distribution log reporter EAGLE-99

Hi, Please review this small feature. This will be part of policy distribution statistics feature. https://github.com/apache/incubator-eagle/pull/36 Thanks Edward

Re: create/update differentiation in eagle service API

Hi Murali, There is no easy way to distinguish create and update. But looks it is not that important to know if policy is created or updated. If we do want to know if it is created or updated, we can look at the policy¹s history and the first change must be create. Thanks Edward On 12/22/15,

Re: What is the API of "Mark Sensitivity Data"?

Marking sensitivity data is nothing but invoking the following API (HTTP POST) http://localhost:9099/eagle-service/rest/entities?serviceName=FileSensitivi tyService [ { "tags":{ "site" : "sandbox", "filedir" : "testFilePath" }, "sensitivityType": "EMAIL" }, { "tags":{

Re: Cluster health monitoring

hadoop native metric monitoring, mapreduce job monitoring, gc activity monitoring etc is going to be open sourced in next 2 months. Eagle dev team is converting/developing code to adapt to current Eagle framework. Roadmap and some discussion are in

Re: [Discuss] Hadoop metrics,job,GC monitoring

here users can >create business rules like "I want an alert when NN is in safemode or RM >is >flipping etc". > >Thanks, >Arun > >On Mon, Dec 14, 2015 at 10:58 AM, Zhang, Edward (GDI Hadoop) < >yonzh...@ebay.com> wrote: > >> Hi Eagle devs/users, &

Re: [DISCUSS] Provide analytic DSL support

dependency since Hao's work impact the API a lot. >pull 26 would quickly decouple this dependency. > >Ralph > > >On Mon, Dec 14, 2015 at 12:44 PM, Zhang, Edward (GDI Hadoop) < >yonzh...@ebay.com> wrote: > >> Thanks for updating. >> Some questions: >

Re: Cygwin do no support npm?

I don¹t think that is related to windows. In linux we should do the same thing npm, git etc. I tried 3 environment, linux, mac osx, and windows. I don¹t see any issue with build. Thanks Edward On 12/13/15, 18:07, "Jilin" wrote: >Hi Daniel, >If you want to use Cygwin to build

Re: [jira] [Commented] (EAGLE-14) Re-assemble high level hdfs commands for better policy

exactly one patch for each file Thanks Edward On 12/11/15, 23:26, "Zhang, Edward (GDI Hadoop)" <yonzh...@ebay.com> wrote: >ok, going forward I can develop feature on feature branch instead of >master. >But in this case, when I apply the patch in apache github, it reports &

Re: [jira] [Commented] (EAGLE-14) Re-assemble high level hdfs commands for better policy

Hi Ralph, I have some issue with merge this code change into master. I think that is because we have multiple commits for one file so it is not good to patch to current master. (I did rebase from upstream/master also) https://patch-diff.githubusercontent.com/raw/apache/incubator-eagle/pull/23

Re: [jira] [Commented] (EAGLE-14) Re-assemble high level hdfs commands for better policy

ill always sync the latest change from apache/incubator-eagle.git to >yonzhang/incubator-eagle, but we won¹t keep all the commit history from >development side on this branch, in such case, you will have conflict on >the master branch. > >Thanks, >Hao > > > > >On 12

Re: [Discuss] Eagle Policy State Management

able from recent history, but from whole topology store >>>point >>> of view, this could not hand bolt status dependency exactly. >>> >>> Another point is should the state restore be triggered not only when >>> topology restarts but also when >>&

Re: about checkin

o-merge script as soon as possible, so that we can work it out more >> > conveniently. >> > >> > Michael >> > >> > On Thu, Dec 10, 2015 at 3:54 PM, Hao Chen <cn.haoc...@gmail.com> >>wrote: >> > >> > > I will fill some

Re: Eagle JIRA's

Notification framework is really hard. I have some thoughts for notification extensibility design, please suggest more. 1. alert notification is one interface which accepts alert entity, forwards that to external system, reports status, ... 2. framework should invoke alert notification instance

Re: About dev process

Ralph, I think it¹s time you join the committers list by submitting ACLA. Thanks Edward On 12/9/15, 16:48, "sul*fei" wrote: >That would be fine. We might not be assignee before become committer.(not >sure) Contributor could also send pull request:) > > > >At 2015-12-09

about checkin

Hi Michael, Can you please help finalize the temporary checkin,merge process? I went through many emails but did not get exactly how to :-) Thanks Edward

Re: About dev process

+1. Looks like picture can’t show up in dev mail list. I think that is good to put that into eagle wiki. My only concern is contributor/committer roles. Is that common practice to divide the 2 roles? Is committer the person who shows up in contributor list in git hub? Thanks Edward Zhang

Re: cannot produce messages with Kafka after reboot

9 s.k.ZkCoordinator [INFO] Task [1/1] Finished >refreshing " > >Regards, >Daniel >-Original Message- >From: Zhang, Edward (GDI Hadoop) [mailto:yonzh...@ebay.com] >Sent: Monday, December 07, 2015 2:51 PM >To: dev@eagle.incubator.apache.org >Subject: Re: cannot

Re: Sensitive Type

That is a very good question and common problem with multi-value attribute. Daniel, could you give us one concrete example of multiple sensitive types for a single file. SensitiveType attribute itself supports multi value with each value segmented by vertical bar ³|², for example if one file is

Re: Apache Ranger integration for Audit Logs...

Hi Bosco, Thanks for creating this ticket. It is very helpful if EAGLE can use Ranger as data source and automatically get monitoring capability in 9 Hadoop components. If a datasource is not from Kafka, and needs a lot of pre-processing, it is not trivial to integrate that data source.

Re: Eagle wiki is ready

Hi Bosco, The apache cwiki looks needs user/password, but isn¹t that required? We are going to put design documentation to Eagle wiki but leave user faced information in Eagle site. Please suggest also. Thanks Edward Zhang On 11/29/15, 11:04, "Don Bosco Durai" wrote: >Seems

Re: apply for checkin privilege for https://github.com/apache/incubator-eagle

Yes, I was confused by the 3 gits, forkedgit, apachegit, githubgit :-) We are figuring out how to do pull requests. looks it¹s forkedgit -> githubgit -> apachegit -> githubgit Thanks Edward Zhang On 11/18/15, 22:10, "Henry Saputra" wrote: >The Github account is just

Re: Best Practice for Code Collaboration in Apache Way

Agree, we should avoid too many feature branches in central repo. Create branch in your own fork branch and rebase frequently Edward On 11/11/15, 0:59, "Julian Hyde" wrote: >What you are describing ‹ creating a branch in the central repository so >that developers can

Re: [Discuss] Minimal General-purpose Monitoring Engine Proposal

. Thanks Edward On 11/10/15, 15:44, "Zhang, Edward (GDI Hadoop)" <yonzh...@ebay.com> wrote: >I think over this problem again, now I want to bring up some >considerations we should take into account while we do design. > >1. Simple general-purpose monitoring frame