[jira] [Commented] (MESOS-10041) Libprocess SSL verification can leak memory
[
https://issues.apache.org/jira/browse/MESOS-10041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16980584#comment-16980584
]
Greg Mann commented on MESOS-10041:
---
{noformat}
commit e52d0d1f25a91f9940bea4329eb5359373ee0ed0
Author: Benno Evers
Date: Fri Nov 22 12:00:43 2019 -0800
Fixed memory leak in openssl verification function.
When the hostname validation scheme was set to 'openssl',
the `openssl::verify()` function would return without
freeing a previously allocated `X509*` object.
To fix the leak, a long-standing TODO to switch to
RAII-based memory management for the certificate was
resolved.
Review: https://reviews.apache.org/r/71805/
{noformat}
> Libprocess SSL verification can leak memory
> ---
>
> Key: MESOS-10041
> URL: https://issues.apache.org/jira/browse/MESOS-10041
> Project: Mesos
> Issue Type: Bug
> Components: libprocess
>Affects Versions: 1.9.0
>Reporter: Greg Mann
>Assignee: Benno Evers
>Priority: Major
> Labels: libprocess, ssl
>
> In {{process::network::openssl::verify()}}, when the SSL hostname validation
> scheme is set to "openssl", the function can return without freeing an
> {{X509}} object, leading to a memory leak.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (MESOS-10026) Improve v1 operator API read performance.
[
https://issues.apache.org/jira/browse/MESOS-10026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16980530#comment-16980530
]
Benjamin Mahler commented on MESOS-10026:
-
All of the GET_ reads are done:
{noformat}
commit b275a032f217794f20dae15d86f188e55d43ce59
Author: Benjamin Mahler
Date: Fri Nov 8 16:13:40 2019 -0800
Support jsonifying v0 protobuf to v1 protobuf.
This allows us to jsonify a v0 protobuf directly to a v1 protobuf
efficiently, with no need to `evolve()` the message (which is rather
expensive).
The way this works is by converting all "slave" and "SLAVE" strings
in fields and enum values, respectively, to "agent" and "AGENT".
Our current v0 to v1 conversion for the v1 operator API simply
serializes the v0 message and de-serializes into a v1 message, which
means all field tags and message structures are the same, except
for field names. The only difference with field names is the use
of "agent" in place of "slave".
Review: https://reviews.apache.org/r/71748
{noformat}
{noformat}
commit 8bfbcab09be82d3a443697925fbf3c4f31333060
Author: Benjamin Mahler
Date: Fri Nov 8 16:49:36 2019 -0800
Added a test for AsV1Protobuf.
Review: https://reviews.apache.org/r/71749
{noformat}
{noformat}
commit 715035b24cb90ba17f9d92217f6556a2f66979e8
Author: Benjamin Mahler
Date: Fri Nov 8 16:52:37 2019 -0800
Improved performance of v1 operator API GetAgents call.
This updates the handling to serialize directly to protobuf or json
from the in-memory v0 state, bypassing expensive intermediate
serialization / de-serialization / object construction / object
destruction.
This initial patch shows the approach that will be used for the
other expensive calls. Note that this type of manual writing is
more brittle and complex, but it can be mostly eliminated if we
keep an up-to-date v1 GetState in memory in the future.
When this approach is applied fully to GetState, it leads to the
following improvement:
Before:
v0 '/state' response took 6.55 secs
v1 'GetState' application/x-protobuf response took 24.08 secs
v1 'GetState' application/json response took 22.76 secs
After:
v0 '/state' response took 8.00 secs
v1 'GetState' application/x-protobuf response took 5.73 secs
v1 'GetState' application/json response took 9.62 secs
Review: https://reviews.apache.org/r/71750
{noformat}
{noformat}
commit 4f4dab961bd45ca444d13b831cdb2541dd10ced8
Author: Benjamin Mahler
Date: Fri Nov 8 16:56:16 2019 -0800
Improved performance of v1 operator API GetFrameworks call.
This follow the same approach used in the GetAgents call;
serializing directly to protobuf or json from the in-memory
v0 state.
Review: https://reviews.apache.org/r/71751
{noformat}
{noformat}
commit 6ab835459a452e53fec8982a5aaab7e78094bbcb
Author: Benjamin Mahler
Date: Fri Nov 8 16:57:28 2019 -0800
Improved performance of v1 operator API GetExecutors call.
This follow the same approach used in the GetAgents call;
serializing directly to protobuf or json from the in-memory
v0 state.
Review: https://reviews.apache.org/r/71752
{noformat}
{noformat}
commit d7dd4d0e8493331d7b7a21b504ebeab702ff06d5
Author: Benjamin Mahler
Date: Fri Nov 8 16:58:47 2019 -0800
Improved performance of v1 operator API GetTasks call.
This follow the same approach used in the GetAgents call;
serializing directly to protobuf or json from the in-memory
v0 state.
Review: https://reviews.apache.org/r/71753
{noformat}
{noformat}
commit 1c60f0e4acbac96c34bd90e265150cdd3844f915
Author: Benjamin Mahler
Date: Fri Nov 8 16:59:44 2019 -0800
Improved performance of v1 operator API GetState call.
This follow the same approach used in the GetAgents call;
serializing directly to protobuf or json from the in-memory
v0 state.
Before:
v0 '/state' response took 6.55 secs
v1 'GetState' application/x-protobuf response took 24.08 secs
v1 'GetState' application/json response took 22.76 secs
After:
v0 '/state' response took 8.00 secs
v1 'GetState' application/x-protobuf response took 5.73 secs
v1 'GetState' application/json response took 9.62 secs
Review: https://reviews.apache.org/r/71754
{noformat}
{noformat}
commit 469f2ebaf65b1642d1eb4a1df81abfc2c94889dd
Author: Benjamin Mahler
Date: Fri Nov 8 17:00:37 2019 -0800
Improved performance of v1 operator API GetMetrics call.
This follow the same approach used in the GetAgents call;
serializing directly to protobuf or json from the in-memory
v0 state.
Review: https://reviews.apache.org/r/71755
{noformat}
SUBSCRIBE will be next.
> Improve v1 operator API read performance.
> -
>
> Key: MESOS-10026
> U
[jira] [Assigned] (MESOS-10041) Libprocess SSL verification can leak memory
[
https://issues.apache.org/jira/browse/MESOS-10041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Mann reassigned MESOS-10041:
-
Assignee: Benno Evers
> Libprocess SSL verification can leak memory
> ---
>
> Key: MESOS-10041
> URL: https://issues.apache.org/jira/browse/MESOS-10041
> Project: Mesos
> Issue Type: Bug
> Components: libprocess
>Reporter: Greg Mann
>Assignee: Benno Evers
>Priority: Major
> Labels: libprocess, ssl
>
> In {{process::network::openssl::verify()}}, when the SSL hostname validation
> scheme is set to "openssl", the function can return without freeing an
> {{X509}} object, leading to a memory leak.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (MESOS-10041) Libprocess SSL verification can leak memory
Greg Mann created MESOS-10041:
-
Summary: Libprocess SSL verification can leak memory
Key: MESOS-10041
URL: https://issues.apache.org/jira/browse/MESOS-10041
Project: Mesos
Issue Type: Bug
Components: libprocess
Reporter: Greg Mann
In {{process::network::openssl::verify()}}, when the SSL hostname validation
scheme is set to "openssl", the function can return without freeing an {{X509}}
object, leading to a memory leak.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (MESOS-10035) Implement `enable_http_executor_domain_sockets` agent flag
[
https://issues.apache.org/jira/browse/MESOS-10035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benno Evers reassigned MESOS-10035:
---
Sprint: Studio 4: RI-21 60
Story Points: 5
Assignee: Benjamin Bannier
Description:
Based on the design in
https://docs.google.com/document/d/1RUvjoBvM3UX_lLcq_J_crWpMMn3nO8CY0KWc655ELsM/edit
we need a `--enable_http_executor_domain_sockets[=true|false]` flag for the
mesos agent.
The basic functionality we'd like for this task is, in pseudocode:
{noformat}
DURING task launch
IF launching new executor && enable_http_executor_domain_sockets == True:
Inject MESOS_DOMAIN_SOCKET environment variable pointing to `/agent.sock`
{noformat}
Setting the environment variable can be done in the
`slave.cpp:executorEnvironment()` function.
The code that actually creates the socket and puts it into the location pointed
to by `MESOS_DOMAIN_SOCKET` will be implemented in a separate ticket.
was:
Based on the design in
https://docs.google.com/document/d/1RUvjoBvM3UX_lLcq_J_crWpMMn3nO8CY0KWc655ELsM/edit
we need a `--enable_http_executor_domain_sockets[=true|false]` flag for the
mesos agent.
The basic functionality we'd like for this task is, in pseudocode:
{noformat}
DURING task launch
IF launching new executor && enable_http_executor_domain_sockets == True:
Inject MESOS_DOMAIN_SOCKET environment variable pointing to `/agent.sock`
{noformat}
The code that actually creates the socket and puts it into the location pointed
to by `MESOS_DOMAIN_SOCKET` will be implemented in a separate ticket.
> Implement `enable_http_executor_domain_sockets` agent flag
> --
>
> Key: MESOS-10035
> URL: https://issues.apache.org/jira/browse/MESOS-10035
> Project: Mesos
> Issue Type: Task
>Reporter: Benno Evers
>Assignee: Benjamin Bannier
>Priority: Major
>
> Based on the design in
> https://docs.google.com/document/d/1RUvjoBvM3UX_lLcq_J_crWpMMn3nO8CY0KWc655ELsM/edit
> we need a `--enable_http_executor_domain_sockets[=true|false]` flag for the
> mesos agent.
> The basic functionality we'd like for this task is, in pseudocode:
> {noformat}
> DURING task launch
> IF launching new executor && enable_http_executor_domain_sockets == True:
> Inject MESOS_DOMAIN_SOCKET environment variable pointing to
> `/agent.sock`
> {noformat}
>
> Setting the environment variable can be done in the
> `slave.cpp:executorEnvironment()` function.
> The code that actually creates the socket and puts it into the location
> pointed to by `MESOS_DOMAIN_SOCKET` will be implemented in a separate ticket.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
