Hi,
I have an use case to sum the calculated values for each entity in the profile
and fetch the values for all the entities from profile in single stellar
command.
As per my understanding, currently in metron we dont have such
data structure to store the profile data to Hbase. If I store the profile the
data with key as each entity and value, I need to iterate over the list of keys
to fetch profile data for all keys. I would like to know if there is any better
way to handle my use case with metron profiler. Please go through below
illustration for more information about the usecase.
Illustration:
I need to calculate the upload bytes per each
user in a profile window and retreive the list of all users with upload bytes
ratio for specific duration.
Currently with the existing scope, I have
written a profile that aggregates the upload bytes per each users and stored to
Hbase with key as user and value as sum of bytes for profiler window duration.
Below is the profile definition for the same.
************ profile definition to calculate
upload bytes per user ***************
{
"profile":"upbcpu",
"onlyif":"(source.type == 'test1' || source.type == 'test2') and
EXISTS(src_user_name) and (IS_EMPTY(src_user_name) == false) and
EXISTS(ip_src_bytes_sent) and ip_src_bytes_sent > 0",
"foreach":"src_user_name",
"init": {
"up_bytes" : 0.0
},
"update": {
"up_bytes":"up_bytes + ip_src_bytes_sent"
},
"result": {
"profile": "up_bytes"
}
}
*****************Profile definition end
******************************************
After storing the profile data, I can only
fetch the upload bytes for all user only by querying for each user in separate
stellar command by providing usernname as below
PROFILE_GET('upbcpu',
'testuser', PROFILE_FIXED(20, 'DAYS'))
The issue here is to fetch list of all users
with their upload bytes for specific duration in single staller command, there
is no way I can acheive it from profile definition. I am looking to have a
profile something like below which stores a map to Hbase with key as username
and value as upload bytes of that user and fetch all users information by
querying with entity 'global'
**************************Expected profile
definition****************************
{
"profile":"upbcpu",
"onlyif":"(source.type == 'test1' || source.type == 'test2') and
EXISTS(src_user_name) and (IS_EMPTY(src_user_name) == false) and
EXISTS(ip_src_bytes_sent) and ip_src_bytes_sent > 0",
"foreach":"'global'",
"init": {
"up_bytes_map" : <Initialize Map>
},
"update": {
"is_user_exists":"<Check by
username as key in Map>",
"up_bytes":"if(is_user_exists then ip_src_bytes_sent + <fetch
existing upload bytes value for user from Map> else ip_src_bytes_sent",
"up_bytes_map" : <update/add
upload up_bytes value with key as user>
},
"result": {
"profile": "up_bytes_map"
}
}
**************************Expected Profile
definition End************************
With some definition like above, I can fetch
the all users upload bytes for window using stellar command
PROFILE_GET('upbcpu', 'global',
PROFILE_FIXED(20, 'DAYS'))
I would like to know the best way to implement the above illustrated usecase
with metron profiler. Please help me if such use case can be implemented with
existing stellar configuration.
Looking forward to hear back with the suggestions.
Thanks,
Anil.
