[Bro-Dev] [JIRA] (BIT-1368) File type identification fixes
[
https://bro-tracker.atlassian.net/browse/BIT-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=20404#comment-20404
]
Robin Sommer commented on BIT-1368:
---
I'm seeing significant performance improvements after this merge, like 4-7% on
the external tests (in a debug mode compile)
File type identification fixes
--
Key: BIT-1368
URL: https://bro-tracker.atlassian.net/browse/BIT-1368
Project: Bro Issue Tracker
Issue Type: Problem
Components: Bro
Affects Versions: 2.4
Reporter: Seth Hall
Assignee: Robin Sommer
Fix For: 2.4
I have some changes nearly queued up for 2.4 release in the repository
(topic/seth/more-file-type-ident-fixes) in the but a bit more work needs to
be done.
There may be one more breaking change to the files api coming in this branch
too. Jon and I discussed some options and I think that creating a new event
named file_sniff in place of the file_mime_type event makes sense. We can
put the mime type and more sniff originated data in a record on that event
so that we can extend it cleanly (and without breaking APIs) in the future.
I think it will look something like this:
```
type fa_sniff: record {
## Depth sniffed.
depth: count default=0;
## Sniffed mime type if one was discovered.
mime_type: string optional;
};
event file_sniff(f: fa_file, sniff: fa_sniff)
{
if ( sniff?$mime_type )
{
print sniff$mime_type;
}
}
```
One other thing this branch will address is a performance degradation from
certain file signatures interacting with each other poorly.
--
This message was sent by Atlassian JIRA
(v6.5-OD-01-120#65000)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1368) File type identification fixes
[
https://bro-tracker.atlassian.net/browse/BIT-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robin Sommer updated BIT-1368:
--
Resolution: Merged (was: Fixed)
Status: Closed (was: Merge Request)
File type identification fixes
--
Key: BIT-1368
URL: https://bro-tracker.atlassian.net/browse/BIT-1368
Project: Bro Issue Tracker
Issue Type: Problem
Components: Bro
Affects Versions: 2.4
Reporter: Seth Hall
Assignee: Robin Sommer
Fix For: 2.4
I have some changes nearly queued up for 2.4 release in the repository
(topic/seth/more-file-type-ident-fixes) in the but a bit more work needs to
be done.
There may be one more breaking change to the files api coming in this branch
too. Jon and I discussed some options and I think that creating a new event
named file_sniff in place of the file_mime_type event makes sense. We can
put the mime type and more sniff originated data in a record on that event
so that we can extend it cleanly (and without breaking APIs) in the future.
I think it will look something like this:
```
type fa_sniff: record {
## Depth sniffed.
depth: count default=0;
## Sniffed mime type if one was discovered.
mime_type: string optional;
};
event file_sniff(f: fa_file, sniff: fa_sniff)
{
if ( sniff?$mime_type )
{
print sniff$mime_type;
}
}
```
One other thing this branch will address is a performance degradation from
certain file signatures interacting with each other poorly.
--
This message was sent by Atlassian JIRA
(v6.5-OD-01-120#65000)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1368) File type identification fixes
[
https://bro-tracker.atlassian.net/browse/BIT-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seth Hall updated BIT-1368:
---
Status: Merge Request (was: Open)
Assignee: (was: Seth Hall)
File type identification fixes
--
Key: BIT-1368
URL: https://bro-tracker.atlassian.net/browse/BIT-1368
Project: Bro Issue Tracker
Issue Type: Problem
Components: Bro
Affects Versions: 2.4
Reporter: Seth Hall
Fix For: 2.4
I have some changes nearly queued up for 2.4 release in the repository
(topic/seth/more-file-type-ident-fixes) in the but a bit more work needs to
be done.
There may be one more breaking change to the files api coming in this branch
too. Jon and I discussed some options and I think that creating a new event
named file_sniff in place of the file_mime_type event makes sense. We can
put the mime type and more sniff originated data in a record on that event
so that we can extend it cleanly (and without breaking APIs) in the future.
I think it will look something like this:
```
type fa_sniff: record {
## Depth sniffed.
depth: count default=0;
## Sniffed mime type if one was discovered.
mime_type: string optional;
};
event file_sniff(f: fa_file, sniff: fa_sniff)
{
if ( sniff?$mime_type )
{
print sniff$mime_type;
}
}
```
One other thing this branch will address is a performance degradation from
certain file signatures interacting with each other poorly.
--
This message was sent by Atlassian JIRA
(v6.5-OD-01-120#65000)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1368) File type identification fixes
[
https://bro-tracker.atlassian.net/browse/BIT-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=20400#comment-20400
]
Seth Hall commented on BIT-1368:
Thanks Jon! I reverted back to the naming I was using (although I'm already
taking some flak for it).
My topic/seth/more-file-type-ident-fixes is ready for merging. There are
branches of the same name in bro-testing and bro-testing-private as well.
Merging this branch also merges the contents of Jon's topic/jsiwek/bit-1368
branch.
File type identification fixes
--
Key: BIT-1368
URL: https://bro-tracker.atlassian.net/browse/BIT-1368
Project: Bro Issue Tracker
Issue Type: Problem
Components: Bro
Affects Versions: 2.4
Reporter: Seth Hall
Assignee: Seth Hall
Fix For: 2.4
I have some changes nearly queued up for 2.4 release in the repository
(topic/seth/more-file-type-ident-fixes) in the but a bit more work needs to
be done.
There may be one more breaking change to the files api coming in this branch
too. Jon and I discussed some options and I think that creating a new event
named file_sniff in place of the file_mime_type event makes sense. We can
put the mime type and more sniff originated data in a record on that event
so that we can extend it cleanly (and without breaking APIs) in the future.
I think it will look something like this:
```
type fa_sniff: record {
## Depth sniffed.
depth: count default=0;
## Sniffed mime type if one was discovered.
mime_type: string optional;
};
event file_sniff(f: fa_file, sniff: fa_sniff)
{
if ( sniff?$mime_type )
{
print sniff$mime_type;
}
}
```
One other thing this branch will address is a performance degradation from
certain file signatures interacting with each other poorly.
--
This message was sent by Atlassian JIRA
(v6.5-OD-01-120#65000)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1368) File type identification fixes
[
https://bro-tracker.atlassian.net/browse/BIT-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robin Sommer reassigned BIT-1368:
-
Assignee: Robin Sommer
File type identification fixes
--
Key: BIT-1368
URL: https://bro-tracker.atlassian.net/browse/BIT-1368
Project: Bro Issue Tracker
Issue Type: Problem
Components: Bro
Affects Versions: 2.4
Reporter: Seth Hall
Assignee: Robin Sommer
Fix For: 2.4
I have some changes nearly queued up for 2.4 release in the repository
(topic/seth/more-file-type-ident-fixes) in the but a bit more work needs to
be done.
There may be one more breaking change to the files api coming in this branch
too. Jon and I discussed some options and I think that creating a new event
named file_sniff in place of the file_mime_type event makes sense. We can
put the mime type and more sniff originated data in a record on that event
so that we can extend it cleanly (and without breaking APIs) in the future.
I think it will look something like this:
```
type fa_sniff: record {
## Depth sniffed.
depth: count default=0;
## Sniffed mime type if one was discovered.
mime_type: string optional;
};
event file_sniff(f: fa_file, sniff: fa_sniff)
{
if ( sniff?$mime_type )
{
print sniff$mime_type;
}
}
```
One other thing this branch will address is a performance degradation from
certain file signatures interacting with each other poorly.
--
This message was sent by Atlassian JIRA
(v6.5-OD-01-120#65000)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1368) File type identification fixes
[
https://bro-tracker.atlassian.net/browse/BIT-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Siwek reassigned BIT-1368:
--
Assignee: Seth Hall (was: Jon Siwek)
File type identification fixes
--
Key: BIT-1368
URL: https://bro-tracker.atlassian.net/browse/BIT-1368
Project: Bro Issue Tracker
Issue Type: Problem
Components: Bro
Affects Versions: 2.4
Reporter: Seth Hall
Assignee: Seth Hall
Fix For: 2.4
I have some changes nearly queued up for 2.4 release in the repository
(topic/seth/more-file-type-ident-fixes) in the but a bit more work needs to
be done.
There may be one more breaking change to the files api coming in this branch
too. Jon and I discussed some options and I think that creating a new event
named file_sniff in place of the file_mime_type event makes sense. We can
put the mime type and more sniff originated data in a record on that event
so that we can extend it cleanly (and without breaking APIs) in the future.
I think it will look something like this:
```
type fa_sniff: record {
## Depth sniffed.
depth: count default=0;
## Sniffed mime type if one was discovered.
mime_type: string optional;
};
event file_sniff(f: fa_file, sniff: fa_sniff)
{
if ( sniff?$mime_type )
{
print sniff$mime_type;
}
}
```
One other thing this branch will address is a performance degradation from
certain file signatures interacting with each other poorly.
--
This message was sent by Atlassian JIRA
(v6.4-OD-16-006#64014)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1368) File type identification fixes
[
https://bro-tracker.atlassian.net/browse/BIT-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=20256#comment-20256
]
Jon Siwek commented on BIT-1368:
Seth, topic/jsiwek/bit-1368 has the changes to the mime type detection script
API that you can merge in to your branch for finalization when you're ready.
For the naming, I went with:
{code}
## Metadata that's been inferred about a particular file.
type inferred_file_metadata: record {
## The strongest matching mime type if one was discovered.
mime_type: string optional;
## All matching mime types if any were discovered.
mime_types: mime_matches optional;
};
event file_metadata_inferred(f: fa_file, meta: inferred_file_metadata);
{code}
File type identification fixes
--
Key: BIT-1368
URL: https://bro-tracker.atlassian.net/browse/BIT-1368
Project: Bro Issue Tracker
Issue Type: Problem
Components: Bro
Affects Versions: 2.4
Reporter: Seth Hall
Assignee: Seth Hall
Fix For: 2.4
I have some changes nearly queued up for 2.4 release in the repository
(topic/seth/more-file-type-ident-fixes) in the but a bit more work needs to
be done.
There may be one more breaking change to the files api coming in this branch
too. Jon and I discussed some options and I think that creating a new event
named file_sniff in place of the file_mime_type event makes sense. We can
put the mime type and more sniff originated data in a record on that event
so that we can extend it cleanly (and without breaking APIs) in the future.
I think it will look something like this:
```
type fa_sniff: record {
## Depth sniffed.
depth: count default=0;
## Sniffed mime type if one was discovered.
mime_type: string optional;
};
event file_sniff(f: fa_file, sniff: fa_sniff)
{
if ( sniff?$mime_type )
{
print sniff$mime_type;
}
}
```
One other thing this branch will address is a performance degradation from
certain file signatures interacting with each other poorly.
--
This message was sent by Atlassian JIRA
(v6.4-OD-16-006#64014)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1368) File type identification fixes
Seth Hall created BIT-1368:
--
Summary: File type identification fixes
Key: BIT-1368
URL: https://bro-tracker.atlassian.net/browse/BIT-1368
Project: Bro Issue Tracker
Issue Type: Problem
Components: Bro
Affects Versions: 2.4
Reporter: Seth Hall
Assignee: Seth Hall
I have some changes nearly queued up for 2.4 release in the repository
(topic/seth/more-file-type-ident-fixes) in the but a bit more work needs to be
done.
There may be one more breaking change to the files api coming in this branch
too. Jon and I discussed some options and I think that creating a new event
named file_sniff in place of the file_mime_type event makes sense. We can put
the mime type and more sniff originated data in a record on that event so
that we can extend it cleanly (and without breaking APIs) in the future. I
think it will look something like this:
```
type fa_sniff: record {
## Depth sniffed.
depth: count default=0;
## Sniffed mime type if one was discovered.
mime_type: string optional;
};
event file_sniff(f: fa_file, sniff: fa_sniff)
{
if ( sniff?$mime_type )
{
print sniff$mime_type;
}
}
```
One other thing this branch will address is a performance degradation from
certain file signatures interacting with each other poorly.
--
This message was sent by Atlassian JIRA
(v6.4-OD-16-006#64014)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1368) File type identification fixes
[
https://bro-tracker.atlassian.net/browse/BIT-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robin Sommer updated BIT-1368:
--
Fix Version/s: 2.4
File type identification fixes
--
Key: BIT-1368
URL: https://bro-tracker.atlassian.net/browse/BIT-1368
Project: Bro Issue Tracker
Issue Type: Problem
Components: Bro
Affects Versions: 2.4
Reporter: Seth Hall
Assignee: Seth Hall
Fix For: 2.4
I have some changes nearly queued up for 2.4 release in the repository
(topic/seth/more-file-type-ident-fixes) in the but a bit more work needs to
be done.
There may be one more breaking change to the files api coming in this branch
too. Jon and I discussed some options and I think that creating a new event
named file_sniff in place of the file_mime_type event makes sense. We can
put the mime type and more sniff originated data in a record on that event
so that we can extend it cleanly (and without breaking APIs) in the future.
I think it will look something like this:
```
type fa_sniff: record {
## Depth sniffed.
depth: count default=0;
## Sniffed mime type if one was discovered.
mime_type: string optional;
};
event file_sniff(f: fa_file, sniff: fa_sniff)
{
if ( sniff?$mime_type )
{
print sniff$mime_type;
}
}
```
One other thing this branch will address is a performance degradation from
certain file signatures interacting with each other poorly.
--
This message was sent by Atlassian JIRA
(v6.4-OD-16-006#64014)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
