[GitHub] [metron] JonZeolla commented on issue #1282: METRON-1881: Update the bro plugin used in metron development

[GitBox]

JonZeolla commented on issue #1282: METRON-1881: Update the bro plugin used in 
metron development
URL: https://github.com/apache/metron/pull/1282#issuecomment-565662697
 
 
   Yeah should be good
   
   Jon Zeolla
   
   On Fri, Dec 13, 2019, 4:40 PM Otto Fowler  wrote:
   
   > So I landed apache/metron-bro-plugin-kafka#36
   > , and I
   > believe this would be good to go now. @JonZeolla
   >  please confirm
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub
   > 
,
   > or unsubscribe
   > 

   > .
   >
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [metron] asfgit closed pull request #1533: METRON-2292: [UI] Manual query can't be saved on the Alerts UI

[GitBox]

asfgit closed pull request #1533: METRON-2292: [UI] Manual query can't be saved 
on the Alerts UI
URL: https://github.com/apache/metron/pull/1533
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [metron] sardell commented on issue #1533: METRON-2292: [UI] Manual query can't be saved on the Alerts UI

[GitBox]

sardell commented on issue #1533: METRON-2292: [UI] Manual query can't be saved 
on the Alerts UI
URL: https://github.com/apache/metron/pull/1533#issuecomment-565620887
 
 
   @ruffle1986 I'm actually unable to reproduce the bug now. I think I might've 
been using an old VM build before to test, though I'm not sure if that would've 
caused the issue. Either way, I've tested this pretty thoroughly and I cannot 
reproduce the bug.
   
   With that said, I'm giving this a +1. Thanks for the contribution, 
@ruffle1986!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [metron] ottobackwards commented on issue #1282: METRON-1881: Update the bro plugin used in metron development

[GitBox]

ottobackwards commented on issue #1282: METRON-1881: Update the bro plugin used 
in metron development
URL: https://github.com/apache/metron/pull/1282#issuecomment-565618330
 
 
   So I landed https://github.com/apache/metron-bro-plugin-kafka/pull/36, and I 
believe this would be good to go now.  @JonZeolla please confirm


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [metron] asfgit closed pull request #1527: METRON-2276: [UI] Performance: Switching back from manual filtering fires a "full query"

[GitBox]

asfgit closed pull request #1527: METRON-2276: [UI] Performance: Switching back 
from manual filtering fires a "full query"
URL: https://github.com/apache/metron/pull/1527
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [metron] justinleet commented on issue #1282: METRON-1881: Update the bro plugin used in metron development

[GitBox]

justinleet commented on issue #1282: METRON-1881: Update the bro plugin used in 
metron development
URL: https://github.com/apache/metron/pull/1282#issuecomment-565605449
 
 
   @JonZeolla @ottobackwards Is this good to go, and is it something we should 
try to get into 0.7.2 (or whatever it winds up being)?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (METRON-2337) [UI] Resolve stale data state / message inconsistencies in Alerts UI

[Shane Ardell (Jira)]

 [ 
https://issues.apache.org/jira/browse/METRON-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shane Ardell updated METRON-2337:
-
Summary: [UI] Resolve stale data state / message inconsistencies in Alerts 
UI  (was: Resolve stale data state / message inconsistencies)

> [UI] Resolve stale data state / message inconsistencies in Alerts UI
> 
>
> Key: METRON-2337
> URL: https://issues.apache.org/jira/browse/METRON-2337
> Project: Metron
>  Issue Type: Bug
>Reporter: Shane Ardell
>Priority: Major
>
> As pointing out in [PR 
> #1527|https://github.com/apache/metron/pull/1527#pullrequestreview-298193000],
>  there are a few scenarios where the state of the alerts table data and the 
> stale data state message are not in sync.
> User scenario #1:
> User switch to manual query mode
> Making changes on the filtering
> Pressing search and receives another set of alerts
> Then moves back to Query Builder mode
> This could turn the UI to a "stale data state" bc the list of alerts filtered 
> by the previously shown manual query and the visible filters and the data no 
> longer in sync.
> User scenario #2:
> User switch to manual query mode
> Making changes in the filtering
> UI should turn to a "stale data state" to keep behavior consistent across ace 
> and manual editors.
> User scenario #3:
> User switch to manual query mode
> Making changes in the filtering
> UI should turn to a "stale data state" to keep behavior consistent across ace 
> and manual editors.
> But when User switch back without updating the data based on the manual filter
> "Stale data state" should be cleared hence the data is in sync with the shown 
> filters again.
> User scenario #4:
> User move to manual mode
> Making changes in the filtering
> Pressing search and receiving another set of data
> Then switching to query builder mode
> UI should be in a "stale data state" as described in scenario #1
> If User switch back to manual mode "stale data state" should be cleared hence 
> the data is in sync with the filters again



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [metron] sardell commented on issue #1527: METRON-2276: [UI] Performance: Switching back from manual filtering fires a "full query"

[GitBox]

sardell commented on issue #1527: METRON-2276: [UI] Performance: Switching back 
from manual filtering fires a "full query"
URL: https://github.com/apache/metron/pull/1527#issuecomment-565509712
 
 
   Great question. We were hoping to get some feedback from users regarding 
whether or not the query builder added any value to them, but, unfortunately, 
we received no responses.
   
   I do think the focus of this PR discussion got off the rails a little bit. 
Your work resolves a pretty big potential issue of users' DOSing the servers 
accidentally. [I've created a 
Jira](https://issues.apache.org/jira/browse/METRON-2337) to track the stale 
state inconsistencies that @tiborm mentioned, and I agree that they can be 
addressed as a separate PR (depending on whether or not we deprecate the query 
builder).
   
   I'm a +1 on this. Tested with Full Dev. Thanks for the contribution, 
@ruffle1986!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (METRON-2337) Resolve stale data state / message inconsistencies

[Shane Ardell (Jira)]

Shane Ardell created METRON-2337:


 Summary: Resolve stale data state / message inconsistencies
 Key: METRON-2337
 URL: https://issues.apache.org/jira/browse/METRON-2337
 Project: Metron
  Issue Type: Bug
Reporter: Shane Ardell


As pointing out in [PR 
#1527|https://github.com/apache/metron/pull/1527#pullrequestreview-298193000], 
there are a few scenarios where the state of the alerts table data and the 
stale data state message are not in sync.

User scenario #1:

User switch to manual query mode
Making changes on the filtering
Pressing search and receives another set of alerts
Then moves back to Query Builder mode
This could turn the UI to a "stale data state" bc the list of alerts filtered 
by the previously shown manual query and the visible filters and the data no 
longer in sync.

User scenario #2:

User switch to manual query mode
Making changes in the filtering
UI should turn to a "stale data state" to keep behavior consistent across ace 
and manual editors.

User scenario #3:

User switch to manual query mode
Making changes in the filtering
UI should turn to a "stale data state" to keep behavior consistent across ace 
and manual editors.
But when User switch back without updating the data based on the manual filter
"Stale data state" should be cleared hence the data is in sync with the shown 
filters again.

User scenario #4:

User move to manual mode
Making changes in the filtering
Pressing search and receiving another set of data
Then switching to query builder mode
UI should be in a "stale data state" as described in scenario #1
If User switch back to manual mode "stale data state" should be cleared hence 
the data is in sync with the filters again



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [metron] ruffle1986 edited a comment on issue #1533: METRON-2292: [UI] Manual query can't be saved on the Alerts UI

[GitBox]

ruffle1986 edited a comment on issue #1533: METRON-2292: [UI] Manual query 
can't be saved on the Alerts UI
URL: https://github.com/apache/metron/pull/1533#issuecomment-565441784
 
 
   According to the gif, I tried to reproduce the problem by following the 
exact same steps. Could you please provide a detailed guide on what you did so 
I can do the same? I might have missed something.
   
   This is what I did:
   
   1. Get all the alerts of he last 1 year.
   2. Click on the `source:type` cell to filter by a certain type like bro.
   3. Switch to manual query and save the search
   4. Then change the query in manual query mode and remove the time related 
stuff
   5. Click on the magnifier to apply the query
   6. Save the new query with a different name
   7. Load the previous query (the time should be there)
   8. Load the recently saved query (the time is not there as expected)
   
   
   ![Kapture 2019-12-13 at 14 35 
14](https://user-images.githubusercontent.com/2196208/70804258-4eb59d80-1db6-11ea-8952-4fbc4981ed2b.gif)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [metron] sardell commented on issue #1575: METRON-2335: [UI] Implement synchronization between browser user state and Hbase user state

[GitBox]

sardell commented on issue #1575: METRON-2335: [UI] Implement synchronization 
between browser user state and Hbase user state
URL: https://github.com/apache/metron/pull/1575#issuecomment-565442931
 
 
   @ruffle1986 Thanks for the thorough explanation. 
   
   > So basically it's an issue on the level of the implementation of these 
features (like auto polling, show/hide, etc.) and not on the new service's 
level.
   
   > So in my opinion, if it's suboptimal now, it was suboptimal before as well 
and I didn't want to optimise the features because it's already a big PR with 
lots of changes and I didn't want to increase the difficulty for the reviewer 
because, in my opinion, it's out of the scope of this issue.
   
   I completely agree with you, and I appreciate your consideration for the 
reviewer(s) of this PR. We can make changes to the actual component 
implementations in another PR to keep this scope of work focused on the task 
you set out to accomplish.
   
   I'm a +1 on this, but I'm going to let it sit for another day to make sure 
others have a chance to weigh in if they want. Thanks for the contribution, 
@ruffle1986!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [metron] ruffle1986 commented on issue #1533: METRON-2292: [UI] Manual query can't be saved on the Alerts UI

[GitBox]

ruffle1986 commented on issue #1533: METRON-2292: [UI] Manual query can't be 
saved on the Alerts UI
URL: https://github.com/apache/metron/pull/1533#issuecomment-565441784
 
 
   According to the gif, I tried to reproduce the problem by following the 
exact same steps. Could you please provide a detailed guide on what you did so 
I can do the same? I might have missed something.
   
   This is what I did:
   
   1. Get all the alerts of he last 1 year.
   2. Click on the `source:type` cell to filter by a certain type like bro.
   3. Switch to manual query and save the search
   4. Then change the query in manual query mode and remove the time related 
stuff
   5. Click on the magnifier to apply the query
   6. Save the new query with a different name
   7. Load the previous query (the time should be there)
   8. Load the recently saved query (the time is not there as expected)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [metron] ruffle1986 commented on issue #1527: METRON-2276: [UI] Performance: Switching back from manual filtering fires a "full query"

[GitBox]

ruffle1986 commented on issue #1527: METRON-2276: [UI] Performance: Switching 
back from manual filtering fires a "full query"
URL: https://github.com/apache/metron/pull/1527#issuecomment-565431032
 
 
   What's the status of this PR?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [metron] ruffle1986 commented on issue #1575: METRON-2335: [UI] Implement synchronization between browser user state and Hbase user state

[GitBox]

ruffle1986 commented on issue #1575: METRON-2335: [UI] Implement 
synchronization between browser user state and Hbase user state
URL: https://github.com/apache/metron/pull/1575#issuecomment-565408917
 
 
   Good points, @sardell . 👍 
   
   This PR is focusing on the replacement of the services which means, instead 
of using the data source class and saving these user settings to local storage, 
let's just persist them in hdfs. I have just changed the service calls in the 
exact same places where they used to be called but with the previous service 
(data source) or where they were directly stored to local storage. So basically 
it's an issue on the level of the implementation of these features (like auto 
polling, show/hide, etc.) and not on the new service's level.
   
   So actually these components or services called the methods multiple times 
to persist the user settings to local storage but it was hardly noticeable but 
now it is noticeable, since we're performing actual http requests. So in my 
opinion, if it's suboptimal now, it was suboptimal before as well and I didn't 
want to optimise the features because it's already a big PR with lots of 
changes and I didn't want to increase the difficulty for the reviewer because, 
in my opinion, it's out of the scope of this issue.
   
   But it's true for sure, it's not nice. Even though there are multiple http 
calls unnecessarily and the server throws an error (it does because we're 
hitting hdfs frequently probably but not sure), it works fine. I'm open to 
introduce enhancements on the features` level in order to get rid of these 
problems. 
   
   Would you like me to do it in this PR or it should be a separate Jira task 
with a separate PR?
   
   ---
   
   Just for the record, here's what's happening on the features` level:
   
   - When you open pane where you can change the rows per table or the refresh 
rate, the show/hide service persists the show/hide dismissed alerts and the 
show/hide resolved alerts when the component is initialised. (2 http calls)
   
   - When you change the refresh rate, auto-polling service and configure-table 
service persists their state (2 calls)
   
   - It's the same when you change the rows per page (2 calls)
   
   - When you switch the hide dismissed alerts, the configure-table service 
persists its state and the auto-polling service persists its state twice (3 
calls)
   
   - The same goes for the hide resolved alerts switch (3 calls)
   
   - When you switch the "Convert timestamps to local time", it works fine (1 
call)
   
   - When you open the table columns settings pane and click on the save 
button, the column-names service and the configure-table service persist their 
state (2 calls)
   
   I don't want to details the calls in the save-search module but you get the 
point.
   
   Also, as I said earlier, these changes are highly opinionated and radically 
changed the way how we handle user settings involving backend parts. So I'm 
really curious about @mmiklavc and @nickwallen 's opinion about this. And they 
might be able to explain the error message given by the server which is the 
following:
   
   ```
   RestException: No lease on /user/metron/user-settings (inode 16861): File 
does not exist. [Lease.  Holder: DFSClient_NONMAPREDUCE_-955476593_1, 
pendingcreates: 1]↵  at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3697)↵
  at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:3498)↵
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNewBlockTargets(FSNamesystem.java:3336)↵
  at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3296)↵
  at 
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:850)↵
   at 
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:504)↵
  at 
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)↵
   at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)↵
  at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)↵ at 
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2351)↵at 
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347)↵at 
java.security.AccessController.doPrivileged(Native Method)↵  at 
javax.security.auth.Subject.doAs(Subject.java:422)↵  at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)↵
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2347)↵
   ```
   
   cc @tiborm 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the 

[GitHub] [metron] MohanDV commented on issue #1576: METRON-2336 Stack advisor provides some components multiple times (santal)

[GitBox]

MohanDV commented on issue #1576: METRON-2336 Stack advisor provides some 
components multiple times (santal)
URL: https://github.com/apache/metron/pull/1576#issuecomment-565393251
 
 
   +1 by verification . Thanks for the fix @sziszo !


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services