from:"Jacob Janco \(JIRA\)"

[jira] [Commented] (MESOS-9770) Add no-new-privileges isolator.

2019-05-29 Thread Jacob Janco (JIRA)



[ 
https://issues.apache.org/jira/browse/MESOS-9770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16851203#comment-16851203
 ] 

Jacob Janco commented on MESOS-9770:


[https://reviews.apache.org/r/70757/]

> Add no-new-privileges isolator.
> ---
>
> Key: MESOS-9770
> URL: https://issues.apache.org/jira/browse/MESOS-9770
> Project: Mesos
>  Issue Type: Improvement
>  Components: containerization
>Reporter: James Peach
>Assignee: Jacob Janco
>Priority: Major
>
> To give security-minded operators more defense in depth, add a {{linux/nnp}} 
> isolator that sets the no-new-privileges bit before starting the executor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (MESOS-9770) Add no-new-privileges isolator.

2019-05-29 Thread Jacob Janco (JIRA)



 [ 
https://issues.apache.org/jira/browse/MESOS-9770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco reassigned MESOS-9770:
--

Assignee: Jacob Janco

> Add no-new-privileges isolator.
> ---
>
> Key: MESOS-9770
> URL: https://issues.apache.org/jira/browse/MESOS-9770
> Project: Mesos
>  Issue Type: Improvement
>  Components: containerization
>Reporter: James Peach
>Assignee: Jacob Janco
>Priority: Major
>
> To give security-minded operators more defense in depth, add a {{linux/nnp}} 
> isolator that sets the no-new-privileges bit before starting the executor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-9760) Decouple Docker runtime isolator manifest configuration from image provider

2019-05-01 Thread Jacob Janco (JIRA)



[ 
https://issues.apache.org/jira/browse/MESOS-9760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16831311#comment-16831311
 ] 

Jacob Janco commented on MESOS-9760:


https://reviews.apache.org/r/70581/

> Decouple Docker runtime isolator manifest configuration from image provider
> ---
>
> Key: MESOS-9760
> URL: https://issues.apache.org/jira/browse/MESOS-9760
> Project: Mesos
>  Issue Type: Improvement
>  Components: agent, docker
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Minor
>
> The Docker runtime isolator propagates manifest configuration metadata. This 
> is not always desirable, e.g. a customer may want to ignore the defined 
> WORKDIR/ENV defined in the manifest. 
> We propose adding a flag `–docker_ignore_manifest_config` to decouple the 
> choice of Docker as an image provider and the need for the Docker runtime 
> isolator. In the background, this flag will conditionally ignore the work of 
> the Docker runtime isolator. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (MESOS-9760) Decouple Docker runtime isolator manifest configuration from image provider

2019-05-01 Thread Jacob Janco (JIRA)

Jacob Janco created MESOS-9760:
--

 Summary: Decouple Docker runtime isolator manifest configuration 
from image provider
 Key: MESOS-9760
 URL: https://issues.apache.org/jira/browse/MESOS-9760
 Project: Mesos
  Issue Type: Improvement
  Components: agent, docker
Reporter: Jacob Janco
Assignee: Jacob Janco


The Docker runtime isolator propagates manifest configuration metadata. This is 
not always desirable, e.g. a customer may want to ignore the defined 
WORKDIR/ENV defined in the manifest. 

We propose adding a flag `–docker_ignore_manifest_config` to decouple the 
choice of Docker as an image provider and the need for the Docker runtime 
isolator. In the background, this flag will conditionally ignore the work of 
the Docker runtime isolator. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-9301) Add flag for per-framework metrics

2018-10-08 Thread Jacob Janco (JIRA)



[ 
https://issues.apache.org/jira/browse/MESOS-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642512#comment-16642512
 ] 

Jacob Janco commented on MESOS-9301:


Add documentation for per-framework metrics flag.
 Review: [https://reviews.apache.org/r/68957]

Add flag to toggle per framework metrics.
 In clusters with high numbers of frameworks, it is
 necessary to control the registration of per
 framework metrics.

Review: [https://reviews.apache.org/r/68956]

Add per framework metrics in member function.

This is a preliminary step for control registering
 per framework metrics.

Review: [https://reviews.apache.org/r/68955]

Fix tests related to allocator options refactor.

Review: [https://reviews.apache.org/r/68954]

Refactor allocator configuration into class.

Review: [https://reviews.apache.org/r/68953]

> Add flag for per-framework metrics
> --
>
> Key: MESOS-9301
> URL: https://issues.apache.org/jira/browse/MESOS-9301
> Project: Mesos
>  Issue Type: Improvement
>  Components: metrics
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Major
>
> Clusters with high amounts of frameworks lead to very high numbers of 
> metrics. Add a flag to toggle per-framework metrics and refactor 
> configuration passed into the allocator. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Issue Comment Deleted] (MESOS-9301) Add flag for per-framework metrics

2018-10-08 Thread Jacob Janco (JIRA)



 [ 
https://issues.apache.org/jira/browse/MESOS-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco updated MESOS-9301:
---
Comment: was deleted

(was: 
+++---+
| Status | Review Request |Branch |
+++===+
| Draft  | r/68957 -  | HEAD  |
+++---+
| Draft  | r/68956 -  | HEAD  |
+++---+
| Draft  | r/68955 -  | HEAD  |
+++---+
| Draft  | r/68954 -  | HEAD  |
+++---+
| Draft  | r/68953 -  | for-review/final-remove-per-framework-metrics |
+++---+)

> Add flag for per-framework metrics
> --
>
> Key: MESOS-9301
> URL: https://issues.apache.org/jira/browse/MESOS-9301
> Project: Mesos
>  Issue Type: Improvement
>  Components: metrics
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Major
>
> Clusters with high amounts of frameworks lead to very high numbers of 
> metrics. Add a flag to toggle per-framework metrics and refactor 
> configuration passed into the allocator. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-9301) Add flag for per-framework metrics

2018-10-08 Thread Jacob Janco (JIRA)



[ 
https://issues.apache.org/jira/browse/MESOS-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642502#comment-16642502
 ] 

Jacob Janco commented on MESOS-9301:


+++---+
| Status | Review Request |Branch |
+++===+
| Draft  | r/68957 -  | HEAD  |
+++---+
| Draft  | r/68956 -  | HEAD  |
+++---+
| Draft  | r/68955 -  | HEAD  |
+++---+
| Draft  | r/68954 -  | HEAD  |
+++---+
| Draft  | r/68953 -  | for-review/final-remove-per-framework-metrics |
+++---+

> Add flag for per-framework metrics
> --
>
> Key: MESOS-9301
> URL: https://issues.apache.org/jira/browse/MESOS-9301
> Project: Mesos
>  Issue Type: Improvement
>  Components: metrics
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Major
>
> Clusters with high amounts of frameworks lead to very high numbers of 
> metrics. Add a flag to toggle per-framework metrics and refactor 
> configuration passed into the allocator. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-9301) Add flag for per-framework metrics

2018-10-08 Thread Jacob Janco (JIRA)



[ 
https://issues.apache.org/jira/browse/MESOS-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642500#comment-16642500
 ] 

Jacob Janco commented on MESOS-9301:


| 
[+++---+|https://reviews.apache.org/+++---+]
 |  |
| 
[|Status|ReviewRequest|Branch||https://reviews.apache.org/|Status|ReviewRequest|Branch|]
 |  |
| 
[+++===+|https://reviews.apache.org/+++===+]
 |  |
| [|Draft|r/68957|https://reviews.apache.org/|Draft|r/68957] |  | HEAD  
| |
| 
[+++---+|https://reviews.apache.org/+++---+]
 |  |
| [|Draft|r/68956|https://reviews.apache.org/|Draft|r/68956] |  | HEAD  
| |
| 
[+++---+|https://reviews.apache.org/+++---+]
 |  |
| [|Draft|r/68955|https://reviews.apache.org/|Draft|r/68955] |  | HEAD  
| |
| 
[+++---+|https://reviews.apache.org/+++---+]
 |  |
| [|Draft|r/68954|https://reviews.apache.org/|Draft|r/68954] |  | HEAD  
| |
| 
[+++---+|https://reviews.apache.org/+++---+]
 |  |
| [|Draft|r/68953|https://reviews.apache.org/|Draft|r/68953] |  | 
for-review/final-remove-per-framework-metrics | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 |  |
| [|https://reviews.apache.org/] |  |


> Add flag for per-framework metrics
> --
>
> Key: MESOS-9301
> URL: https://issues.apache.org/jira/browse/MESOS-9301
> Project: Mesos
>  Issue Type: Improvement
>  Components: metrics
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Major
>
> Clusters with high amounts of frameworks lead to very high numbers of 
> metrics. Add a flag to toggle per-framework metrics and refactor 
> configuration passed into the allocator. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Issue Comment Deleted] (MESOS-9301) Add flag for per-framework metrics

2018-10-08 Thread Jacob Janco (JIRA)



 [ 
https://issues.apache.org/jira/browse/MESOS-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco updated MESOS-9301:
---
Comment: was deleted

(was: | 
[+++---+|https://reviews.apache.org/+++---+]
 |  |
| 
[|Status|ReviewRequest|Branch||https://reviews.apache.org/|Status|ReviewRequest|Branch|]
 |  |
| 
[+++===+|https://reviews.apache.org/+++===+]
 |  |
| [|Draft|r/68957|https://reviews.apache.org/|Draft|r/68957] |  | HEAD  
| |
| 
[+++---+|https://reviews.apache.org/+++---+]
 |  |
| [|Draft|r/68956|https://reviews.apache.org/|Draft|r/68956] |  | HEAD  
| |
| 
[+++---+|https://reviews.apache.org/+++---+]
 |  |
| [|Draft|r/68955|https://reviews.apache.org/|Draft|r/68955] |  | HEAD  
| |
| 
[+++---+|https://reviews.apache.org/+++---+]
 |  |
| [|Draft|r/68954|https://reviews.apache.org/|Draft|r/68954] |  | HEAD  
| |
| 
[+++---+|https://reviews.apache.org/+++---+]
 |  |
| [|Draft|r/68953|https://reviews.apache.org/|Draft|r/68953] |  | 
for-review/final-remove-per-framework-metrics | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 |  |
| [|https://reviews.apache.org/] |  |
)

> Add flag for per-framework metrics
> --
>
> Key: MESOS-9301
> URL: https://issues.apache.org/jira/browse/MESOS-9301
> Project: Mesos
>  Issue Type: Improvement
>  Components: metrics
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Major
>
> Clusters with high amounts of frameworks lead to very high numbers of 
> metrics. Add a flag to toggle per-framework metrics and refactor 
> configuration passed into the allocator. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (MESOS-9301) Add flag for per-framework metrics

2018-10-08 Thread Jacob Janco (JIRA)



[ 
https://issues.apache.org/jira/browse/MESOS-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642498#comment-16642498
 ] 

Jacob Janco edited comment on MESOS-9301 at 10/8/18 9:16 PM:
-

| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| 
[|Status|ReviewRequest|Branch||https://reviews.apache.org/|Status|ReviewRequest|Branch|]
 | |
| 
[+++===+|https://reviews.apache.org/+++===+]
 | |
| [|Draft|r/68957|https://reviews.apache.org/|Draft|r/68957] | | HEAD | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|Draft|r/68956|https://reviews.apache.org/|Draft|r/68956] | | HEAD | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|Draft|r/68955|https://reviews.apache.org/|Draft|r/68955] | | HEAD | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|Draft|r/68954|https://reviews.apache.org/|Draft|r/68954] | | HEAD | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|Draft|r/68953|https://reviews.apache.org/|Draft|r/68953] | | 
for-review/final-remove-per-framework-metrics | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|https://reviews.apache.org/] | |


was (Author: jjanco):
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| 
[|Status|ReviewRequest|Branch||https://reviews.apache.org/|Status|ReviewRequest|Branch|]
 | |
| 
[+++===+|https://reviews.apache.org/+++===+]
 | |
| [|Draft|r/68957|https://reviews.apache.org/|Draft|r/68957] | | HEAD | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|Draft|r/68956|https://reviews.apache.org/|Draft|r/68956] | | HEAD | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|Draft|r/68955|https://reviews.apache.org/|Draft|r/68955] | | HEAD | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|Draft|r/68954|https://reviews.apache.org/|Draft|r/68954] | | HEAD | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|Draft|r/68953|https://reviews.apache.org/|Draft|r/68953] | | 
for-review/final-remove-per-framework-metrics | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|https://reviews.apache.org/] | |

> Add flag for per-framework metrics
> --
>
> Key: MESOS-9301
> URL: https://issues.apache.org/jira/browse/MESOS-9301
> Project: Mesos
>  Issue Type: Improvement
>  Components: metrics
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Major
>
> Clusters with high amounts of frameworks lead to very high numbers of 
> metrics. Add a flag to toggle per-framework metrics and refactor 
> configuration passed into the allocator. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Issue Comment Deleted] (MESOS-9301) Add flag for per-framework metrics

2018-10-08 Thread Jacob Janco (JIRA)



 [ 
https://issues.apache.org/jira/browse/MESOS-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco updated MESOS-9301:
---
Comment: was deleted

(was: | 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| 
[|Status|ReviewRequest|Branch||https://reviews.apache.org/|Status|ReviewRequest|Branch|]
 | |
| 
[+++===+|https://reviews.apache.org/+++===+]
 | |
| [|Draft|r/68957|https://reviews.apache.org/|Draft|r/68957] | | HEAD | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|Draft|r/68956|https://reviews.apache.org/|Draft|r/68956] | | HEAD | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|Draft|r/68955|https://reviews.apache.org/|Draft|r/68955] | | HEAD | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|Draft|r/68954|https://reviews.apache.org/|Draft|r/68954] | | HEAD | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|Draft|r/68953|https://reviews.apache.org/|Draft|r/68953] | | 
for-review/final-remove-per-framework-metrics | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|https://reviews.apache.org/] | |)

> Add flag for per-framework metrics
> --
>
> Key: MESOS-9301
> URL: https://issues.apache.org/jira/browse/MESOS-9301
> Project: Mesos
>  Issue Type: Improvement
>  Components: metrics
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Major
>
> Clusters with high amounts of frameworks lead to very high numbers of 
> metrics. Add a flag to toggle per-framework metrics and refactor 
> configuration passed into the allocator. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-9301) Add flag for per-framework metrics

2018-10-08 Thread Jacob Janco (JIRA)



[ 
https://issues.apache.org/jira/browse/MESOS-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642498#comment-16642498
 ] 

Jacob Janco commented on MESOS-9301:


| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| 
[|Status|ReviewRequest|Branch||https://reviews.apache.org/|Status|ReviewRequest|Branch|]
 | |
| 
[+++===+|https://reviews.apache.org/+++===+]
 | |
| [|Draft|r/68957|https://reviews.apache.org/|Draft|r/68957] | | HEAD | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|Draft|r/68956|https://reviews.apache.org/|Draft|r/68956] | | HEAD | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|Draft|r/68955|https://reviews.apache.org/|Draft|r/68955] | | HEAD | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|Draft|r/68954|https://reviews.apache.org/|Draft|r/68954] | | HEAD | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|Draft|r/68953|https://reviews.apache.org/|Draft|r/68953] | | 
for-review/final-remove-per-framework-metrics | |
| 
[+++---+|https://reviews.apache.org/+++---+]
 | |
| [|https://reviews.apache.org/] | |

> Add flag for per-framework metrics
> --
>
> Key: MESOS-9301
> URL: https://issues.apache.org/jira/browse/MESOS-9301
> Project: Mesos
>  Issue Type: Improvement
>  Components: metrics
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Major
>
> Clusters with high amounts of frameworks lead to very high numbers of 
> metrics. Add a flag to toggle per-framework metrics and refactor 
> configuration passed into the allocator. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (MESOS-9301) Add flag for per-framework metrics

2018-10-08 Thread Jacob Janco (JIRA)

Jacob Janco created MESOS-9301:
--

 Summary: Add flag for per-framework metrics
 Key: MESOS-9301
 URL: https://issues.apache.org/jira/browse/MESOS-9301
 Project: Mesos
  Issue Type: Improvement
  Components: metrics
Reporter: Jacob Janco
Assignee: Jacob Janco


Clusters with high amounts of frameworks lead to very high numbers of metrics. 
Add a flag to toggle per-framework metrics and refactor configuration passed 
into the allocator. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (MESOS-7437) cross domain file-theft in the web-ui

2017-05-02 Thread Jacob Janco (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-7437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco updated MESOS-7437:
---
Priority: Major  (was: Minor)

> cross domain file-theft in the web-ui
> -
>
> Key: MESOS-7437
> URL: https://issues.apache.org/jira/browse/MESOS-7437
> Project: Mesos
>  Issue Type: Bug
>  Components: security, webui
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>
> {code:javascript}
> x=document.createElement('script')
> x.src='http://$AGENT_URI/files/read?path=$PATH_TO_FILE=0=5=console.log&_=1490306716903'
> document.body.appendChild(x)
> {code}
> The above code pasted into the web console on http://example.com/, for 
> example, will yield the contents of the requested file. Basic auth is cached 
> and resent in browser tabs/windows as long as the user has authenticated 
> during the browser session. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7437) cross domain file-theft in the web-ui

2017-04-28 Thread Jacob Janco (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-7437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco updated MESOS-7437:
---
Component/s: webui
 security

> cross domain file-theft in the web-ui
> -
>
> Key: MESOS-7437
> URL: https://issues.apache.org/jira/browse/MESOS-7437
> Project: Mesos
>  Issue Type: Bug
>  Components: security, webui
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Minor
>
> {code:javascript}
> x=document.createElement('script')
> x.src='http://$AGENT_URI/files/read?path=$PATH_TO_FILE=0=5=console.log&_=1490306716903'
> document.body.appendChild(x)
> {code}
> The above code pasted into the web console on http://example.com/, for 
> example, will yield the contents of the requested file. Basic auth is cached 
> and resent in browser tabs/windows as long as the user has authenticated 
> during the browser session. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7437) cross domain file-theft in the web-ui

2017-04-28 Thread Jacob Janco (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-7437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco updated MESOS-7437:
---
Description: 
{code:javascript}
x=document.createElement('script')
x.src='http://$AGENT_URI/files/read?path=$PATH_TO_FILE=0=5=console.log&_=1490306716903'
document.body.appendChild(x)
{code}

The above code pasted into the web console on http://example.com/, for example, 
will yield the contents of the requested file. Basic auth is cached and resent 
in browser tabs/windows as long as the user has authenticated during the 
browser session. 

  was:
`x=document.createElement('script')
x.src='http://$AGENT_URI/files/read?path=$PATH_TO_FILE=0=5=console.log&_=1490306716903'
document.body.appendChild(x)`

The above code pasted into the web console on http://example.com/, for example, 
will yield the contents of the requested file. Basic auth is cached and resent 
in browser tabs/windows as long as the user has authenticated during the 
browser session. 


> cross domain file-theft in the web-ui
> -
>
> Key: MESOS-7437
> URL: https://issues.apache.org/jira/browse/MESOS-7437
> Project: Mesos
>  Issue Type: Bug
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Minor
>
> {code:javascript}
> x=document.createElement('script')
> x.src='http://$AGENT_URI/files/read?path=$PATH_TO_FILE=0=5=console.log&_=1490306716903'
> document.body.appendChild(x)
> {code}
> The above code pasted into the web console on http://example.com/, for 
> example, will yield the contents of the requested file. Basic auth is cached 
> and resent in browser tabs/windows as long as the user has authenticated 
> during the browser session. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7437) cross domain file-theft in the web-ui

2017-04-28 Thread Jacob Janco (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-7437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco updated MESOS-7437:
---
Description: 
`x=document.createElement('script')
x.src='http://$AGENT_URI/files/read?path=$PATH_TO_FILE=0=5=console.log&_=1490306716903'
document.body.appendChild(x)`

The above code pasted into the web console on http://example.com/, for example, 
will yield the contents of the requested file. Basic auth is cached and resent 
in browser tabs/windows as long as the user has authenticated during the 
browser session. 

  was:
```
x=document.createElement('script')
x.src='http://$AGENT_URI/files/read?path=$PATH_TO_FILE=0=5=console.log&_=1490306716903'
document.body.appendChild(x)
```

The above code pasted into the web console on http://example.com/, for example, 
will yield the contents of the requested file. Basic auth is cached and resent 
in browser tabs/windows as long as the user has authenticated during the 
browser session. 


> cross domain file-theft in the web-ui
> -
>
> Key: MESOS-7437
> URL: https://issues.apache.org/jira/browse/MESOS-7437
> Project: Mesos
>  Issue Type: Bug
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Minor
>
> `x=document.createElement('script')
> x.src='http://$AGENT_URI/files/read?path=$PATH_TO_FILE=0=5=console.log&_=1490306716903'
> document.body.appendChild(x)`
> The above code pasted into the web console on http://example.com/, for 
> example, will yield the contents of the requested file. Basic auth is cached 
> and resent in browser tabs/windows as long as the user has authenticated 
> during the browser session. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7437) cross domain file-theft in the web-ui

2017-04-28 Thread Jacob Janco (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-7437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco updated MESOS-7437:
---
Description: 
```
x=document.createElement('script')
x.src='http://$AGENT_URI/files/read?path=$PATH_TO_FILE=0=5=console.log&_=1490306716903'
document.body.appendChild(x)
```

The above code pasted into the web console on http://example.com/, for example, 
will yield the contents of the requested file. Basic auth is cached and resent 
in browser tabs/windows as long as the user has authenticated during the 
browser session. 

  was:
x=document.createElement('script')
x.src='http://$AGENT_URI/files/read?path=$PATH_TO_FILE=0=5=console.log&_=1490306716903'
document.body.appendChild(x)

The above code pasted into the web console on http://example.com/, for example, 
will yield the contents of the requested file. Basic auth is cached and resent 
in browser tabs/windows as long as the user has authenticated during the 
browser session. 


> cross domain file-theft in the web-ui
> -
>
> Key: MESOS-7437
> URL: https://issues.apache.org/jira/browse/MESOS-7437
> Project: Mesos
>  Issue Type: Bug
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Minor
>
> ```
> x=document.createElement('script')
> x.src='http://$AGENT_URI/files/read?path=$PATH_TO_FILE=0=5=console.log&_=1490306716903'
> document.body.appendChild(x)
> ```
> The above code pasted into the web console on http://example.com/, for 
> example, will yield the contents of the requested file. Basic auth is cached 
> and resent in browser tabs/windows as long as the user has authenticated 
> during the browser session. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (MESOS-7437) cross domain file-theft in the web-ui

2017-04-28 Thread Jacob Janco (JIRA)

Jacob Janco created MESOS-7437:
--

 Summary: cross domain file-theft in the web-ui
 Key: MESOS-7437
 URL: https://issues.apache.org/jira/browse/MESOS-7437
 Project: Mesos
  Issue Type: Bug
Reporter: Jacob Janco
Assignee: Jacob Janco
Priority: Minor


x=document.createElement('script')
x.src='http://$AGENT_URI/files/read?path=$PATH_TO_FILE=0=5=console.log&_=1490306716903'
document.body.appendChild(x)

The above code pasted into the web console on http://example.com/, for example, 
will yield the contents of the requested file. Basic auth is cached and resent 
in browser tabs/windows as long as the user has authenticated during the 
browser session. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (MESOS-5918) Replace jsonp with a more secure alternative

2017-04-27 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15983568#comment-15983568
 ] 

Jacob Janco edited comment on MESOS-5918 at 4/27/17 8:35 PM:
-

[~greggomann] [~anandmazumdar] [~mlunoe] [~xujyan] [~haosdent] Reopening a bit 
of discussion on replacing the jsonp workaround with CORS handling server side. 
An initial idea is to have a configurable regex for domains available for cross 
origin requests which will match against sent Origin headers. At this point I 
don't think we'll have to support preflighting requests to add this 
functionality. Another consideration, should this be a libprocess level 
configuration or perhaps a flag set on masters and agents?


was (Author: jjanco):
[~greggomann] [~anandmazumdar] [~mlunoe] [~xujyan] Reopening a bit of 
discussion on replacing the jsonp workaround with CORS handling server side. An 
initial idea is to have a configurable regex for domains available for cross 
origin requests which will match against sent Origin headers. At this point I 
don't think we'll have to support preflighting requests to add this 
functionality. Another consideration, should this be a libprocess level 
configuration or perhaps a flag set on masters and agents?

> Replace jsonp with a more secure alternative
> 
>
> Key: MESOS-5918
> URL: https://issues.apache.org/jira/browse/MESOS-5918
> Project: Mesos
>  Issue Type: Improvement
>  Components: webui
>Reporter: Yan Xu
>
> We currently use the {{jsonp}} technique to bypass CORS check. This practice 
> has many security concerns (see discussions on MESOS-5911) so we should 
> replace it with a better alternative.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (MESOS-5918) Replace jsonp with a more secure alternative

2017-04-25 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15983568#comment-15983568
 ] 

Jacob Janco edited comment on MESOS-5918 at 4/25/17 8:40 PM:
-

[~greggomann] [~anandmazumdar][~mlunoe][~xujyan] Reopening a bit of discussion 
on replacing the jsonp workaround with CORS handling server side. An initial 
idea is to have a configurable regex for domains available for cross origin 
requests which will match against sent Origin headers. At this point I don't 
think we'll have to support preflighting requests to add this functionality. 
Another consideration, should this be a libprocess level configuration or 
perhaps a flag set on masters and agents?


was (Author: jjanco):
[~greggomann] [~anandmazumdar][~mlunoe] Reopening a bit of discussion on 
replacing the jsonp workaround with CORS handling server side. An initial idea 
is to have a configurable regex for domains available for cross origin requests 
which will match against sent Origin headers. At this point I don't think we'll 
have to support preflighting requests to add this functionality. Another 
consideration, should this be a libprocess level configuration or perhaps a 
flag set on masters and agents?

> Replace jsonp with a more secure alternative
> 
>
> Key: MESOS-5918
> URL: https://issues.apache.org/jira/browse/MESOS-5918
> Project: Mesos
>  Issue Type: Improvement
>  Components: webui
>Reporter: Yan Xu
>
> We currently use the {{jsonp}} technique to bypass CORS check. This practice 
> has many security concerns (see discussions on MESOS-5911) so we should 
> replace it with a better alternative.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (MESOS-5918) Replace jsonp with a more secure alternative

2017-04-25 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15983568#comment-15983568
 ] 

Jacob Janco edited comment on MESOS-5918 at 4/25/17 8:40 PM:
-

[~greggomann] [~anandmazumdar] [~mlunoe] [~xujyan] Reopening a bit of 
discussion on replacing the jsonp workaround with CORS handling server side. An 
initial idea is to have a configurable regex for domains available for cross 
origin requests which will match against sent Origin headers. At this point I 
don't think we'll have to support preflighting requests to add this 
functionality. Another consideration, should this be a libprocess level 
configuration or perhaps a flag set on masters and agents?


was (Author: jjanco):
[~greggomann] [~anandmazumdar][~mlunoe][~xujyan] Reopening a bit of discussion 
on replacing the jsonp workaround with CORS handling server side. An initial 
idea is to have a configurable regex for domains available for cross origin 
requests which will match against sent Origin headers. At this point I don't 
think we'll have to support preflighting requests to add this functionality. 
Another consideration, should this be a libprocess level configuration or 
perhaps a flag set on masters and agents?

> Replace jsonp with a more secure alternative
> 
>
> Key: MESOS-5918
> URL: https://issues.apache.org/jira/browse/MESOS-5918
> Project: Mesos
>  Issue Type: Improvement
>  Components: webui
>Reporter: Yan Xu
>
> We currently use the {{jsonp}} technique to bypass CORS check. This practice 
> has many security concerns (see discussions on MESOS-5911) so we should 
> replace it with a better alternative.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (MESOS-5918) Replace jsonp with a more secure alternative

2017-04-25 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15983568#comment-15983568
 ] 

Jacob Janco commented on MESOS-5918:


[~greggomann] [~anandmazumdar][~mlunoe] Reopening a bit of discussion on 
replacing the jsonp workaround with CORS handling server side. An initial idea 
is to have a configurable regex for domains available for cross origin requests 
which will match against sent Origin headers. At this point I don't think we'll 
have to support preflighting requests to add this functionality. Another 
consideration, should this be a libprocess level configuration or perhaps a 
flag set on masters and agents?

> Replace jsonp with a more secure alternative
> 
>
> Key: MESOS-5918
> URL: https://issues.apache.org/jira/browse/MESOS-5918
> Project: Mesos
>  Issue Type: Improvement
>  Components: webui
>Reporter: Yan Xu
>
> We currently use the {{jsonp}} technique to bypass CORS check. This practice 
> has many security concerns (see discussions on MESOS-5911) so we should 
> replace it with a better alternative.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (MESOS-6969) Use clipboard.js for copy/paste webui functionality

2017-01-20 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-6969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832558#comment-15832558
 ] 

Jacob Janco commented on MESOS-6969:


https://reviews.apache.org/r/55791/

> Use clipboard.js for copy/paste webui functionality
> ---
>
> Key: MESOS-6969
> URL: https://issues.apache.org/jira/browse/MESOS-6969
> Project: Mesos
>  Issue Type: Bug
>  Components: webui
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Minor
>  Labels: webui
>
> XSS Vulnerabilities exist for zeroclipboard: 
> https://www.cvedetails.com/cve/CVE-2014-1869/
> This patch listed below replaces this library with the pure js library 
> clipboard.js: https://clipboardjs.com/ which also removes the webui flash 
> dependency to support clipboard functions. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MESOS-6969) Use clipboard.js for copy/paste webui functionality

2017-01-20 Thread Jacob Janco (JIRA)

Jacob Janco created MESOS-6969:
--

 Summary: Use clipboard.js for copy/paste webui functionality
 Key: MESOS-6969
 URL: https://issues.apache.org/jira/browse/MESOS-6969
 Project: Mesos
  Issue Type: Bug
  Components: webui
Reporter: Jacob Janco
Assignee: Jacob Janco
Priority: Minor


XSS Vulnerabilities exist for zerclipboard: 
https://www.cvedetails.com/cve/CVE-2014-1869/

This patch listed below replaces this library with the pure js library 
clipboard.js: https://clipboardjs.com/ which also removes the webui flash 
dependency to support clipboard functions. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MESOS-6969) Use clipboard.js for copy/paste webui functionality

2017-01-20 Thread Jacob Janco (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-6969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco updated MESOS-6969:
---
Description: 
XSS Vulnerabilities exist for zeroclipboard: 
https://www.cvedetails.com/cve/CVE-2014-1869/

This patch listed below replaces this library with the pure js library 
clipboard.js: https://clipboardjs.com/ which also removes the webui flash 
dependency to support clipboard functions. 

  was:
XSS Vulnerabilities exist for zerclipboard: 
https://www.cvedetails.com/cve/CVE-2014-1869/

This patch listed below replaces this library with the pure js library 
clipboard.js: https://clipboardjs.com/ which also removes the webui flash 
dependency to support clipboard functions. 


> Use clipboard.js for copy/paste webui functionality
> ---
>
> Key: MESOS-6969
> URL: https://issues.apache.org/jira/browse/MESOS-6969
> Project: Mesos
>  Issue Type: Bug
>  Components: webui
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Minor
>  Labels: webui
>
> XSS Vulnerabilities exist for zeroclipboard: 
> https://www.cvedetails.com/cve/CVE-2014-1869/
> This patch listed below replaces this library with the pure js library 
> clipboard.js: https://clipboardjs.com/ which also removes the webui flash 
> dependency to support clipboard functions. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MESOS-6947) Fix pailer XSS vulnerability

2017-01-18 Thread Jacob Janco (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-6947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco updated MESOS-6947:
---
Shepherd: haosdent

> Fix pailer XSS vulnerability
> 
>
> Key: MESOS-6947
> URL: https://issues.apache.org/jira/browse/MESOS-6947
> Project: Mesos
>  Issue Type: Improvement
>  Components: webui
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>
> There exists a XSS vulnerability in pailer.html.
> `window.name` can be set to an external domain serving js which is wrapped in 
>

[jira] [Updated] (MESOS-6947) Fix pailer XSS vulnerability

2017-01-18 Thread Jacob Janco (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-6947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco updated MESOS-6947:
---
Shepherd:   (was: Yan Xu)

> Fix pailer XSS vulnerability
> 
>
> Key: MESOS-6947
> URL: https://issues.apache.org/jira/browse/MESOS-6947
> Project: Mesos
>  Issue Type: Improvement
>  Components: webui
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>
> There exists a XSS vulnerability in pailer.html.
> `window.name` can be set to an external domain serving js which is wrapped in 
>

[jira] [Created] (MESOS-6947) Fix pailer XSS vulnerability

2017-01-18 Thread Jacob Janco (JIRA)

Jacob Janco created MESOS-6947:
--

 Summary: Fix pailer XSS vulnerability
 Key: MESOS-6947
 URL: https://issues.apache.org/jira/browse/MESOS-6947
 Project: Mesos
  Issue Type: Improvement
  Components: webui
Reporter: Jacob Janco
Assignee: Jacob Janco


There exists a XSS vulnerability in pailer.html.

`window.name` can be set to an external domain serving js which is wrapped in

[jira] [Commented] (MESOS-6947) Fix pailer XSS vulnerability

2017-01-18 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-6947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828978#comment-15828978
 ] 

Jacob Janco commented on MESOS-6947:


https://reviews.apache.org/r/55691

> Fix pailer XSS vulnerability
> 
>
> Key: MESOS-6947
> URL: https://issues.apache.org/jira/browse/MESOS-6947
> Project: Mesos
>  Issue Type: Improvement
>  Components: webui
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>
> There exists a XSS vulnerability in pailer.html.
> `window.name` can be set to an external domain serving js which is wrapped in 
>

[jira] [Commented] (MESOS-6904) Track resource allocation candidates and batch allocation work

2017-01-10 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817026#comment-15817026
 ] 

Jacob Janco commented on MESOS-6904:


Reviews currently in progress: 
https://reviews.apache.org/r/51027/
https://reviews.apache.org/r/51028/
https://reviews.apache.org/r/52534/
WIP from [~gyliu]
https://reviews.apache.org/r/51621/

> Track resource allocation candidates and batch allocation work
> --
>
> Key: MESOS-6904
> URL: https://issues.apache.org/jira/browse/MESOS-6904
> Project: Mesos
>  Issue Type: Bug
>  Components: allocation
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>  Labels: allocator
>
> "Our deployment environments have a lot of churn, with many short-live 
> frameworks that often revive offers. Running the allocator takes a long time 
> (from seconds up to minutes).
> In this situation, event-triggered allocation causes the event queue in the 
> allocator process to get very long, and the allocator effectively becomes 
> unresponsive (eg. a revive offers message takes too long to come to the head 
> of the queue)." - MESOS-3157 
> To remedy the above scenario, it is proposed to track allocation candidates 
> and only dispatch allocation work if there is no pending allocation in the 
> allocator queue. When an enqueued allocation is processed, the tracked set of 
> candidates is cleared. 
> Current behavior will trigger allocation work on cluster events (e.g. 
> `addSlave()`, `addFramework()`, etc) as well as during the periodic batched 
> allocation running at a defined time interval. 
> This ticket tracks the new direction the work has taken since discussion in 
> MESOS-3157 where a previous solution by [~jamespeach] introduced batched 
> allocation only (which we currently run) as well as an approach to reduce 
> redundancy of work in the queue. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MESOS-6904) Track resource allocation candidates and batch allocation work

2017-01-10 Thread Jacob Janco (JIRA)

Jacob Janco created MESOS-6904:
--

 Summary: Track resource allocation candidates and batch allocation 
work
 Key: MESOS-6904
 URL: https://issues.apache.org/jira/browse/MESOS-6904
 Project: Mesos
  Issue Type: Bug
  Components: allocation
Reporter: Jacob Janco
Assignee: Jacob Janco


"Our deployment environments have a lot of churn, with many short-live 
frameworks that often revive offers. Running the allocator takes a long time 
(from seconds up to minutes).
In this situation, event-triggered allocation causes the event queue in the 
allocator process to get very long, and the allocator effectively becomes 
unresponsive (eg. a revive offers message takes too long to come to the head of 
the queue)." - MESOS-3157 

To remedy the above scenario, it is proposed to track allocation candidates and 
only dispatch allocation work if there is no pending allocation in the 
allocator queue. When an enqueued allocation is processed, the tracked set of 
candidates is cleared. 

Current behavior will trigger allocation work on cluster events (e.g. 
`addSlave()`, `addFramework()`, etc) as well as during the periodic batched 
allocation running at a defined time interval. 

This ticket tracks the new direction the work has taken since discussion in 
MESOS-3157 where a previous solution by [~jamespeach] introduced batched 
allocation only (which we currently run) as well as an approach to reduce 
redundancy of work in the queue. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (MESOS-6549) Asynchronous dir removal in agent GC

2016-11-04 Thread Jacob Janco (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-6549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco updated MESOS-6549:
---
Comment: was deleted

(was: https://reviews.apache.org/r/53479/)

> Asynchronous dir removal in agent GC
> 
>
> Key: MESOS-6549
> URL: https://issues.apache.org/jira/browse/MESOS-6549
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>  Labels: gc
>
> In src/slave/gc.cpp: 
>   // TODO(bmahler): Other dispatches can block waiting for a removal
>   // operation. To fix this, the removal operation can be done
>   // asynchronously in another thread.
> We did see this occur in our clusters where rmdir operations can take seconds 
> to complete, blocking other queued events leading to, for example, long 
> latencies to task launch. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-6549) Asynchronous dir removal in agent GC

2016-11-04 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-6549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15637001#comment-15637001
 ] 

Jacob Janco commented on MESOS-6549:


https://reviews.apache.org/r/53479/

> Asynchronous dir removal in agent GC
> 
>
> Key: MESOS-6549
> URL: https://issues.apache.org/jira/browse/MESOS-6549
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>  Labels: gc
>
> In src/slave/gc.cpp: 
>   // TODO(bmahler): Other dispatches can block waiting for a removal
>   // operation. To fix this, the removal operation can be done
>   // asynchronously in another thread.
> We did see this occur in our clusters where rmdir operations can take seconds 
> to complete, blocking other queued events leading to, for example, long 
> latencies to task launch. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-6549) Asynchronous dir removal in agent GC

2016-11-04 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-6549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15636997#comment-15636997
 ] 

Jacob Janco commented on MESOS-6549:


https://reviews.apache.org/r/53479/

> Asynchronous dir removal in agent GC
> 
>
> Key: MESOS-6549
> URL: https://issues.apache.org/jira/browse/MESOS-6549
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>  Labels: gc
>
> In src/slave/gc.cpp: 
>   // TODO(bmahler): Other dispatches can block waiting for a removal
>   // operation. To fix this, the removal operation can be done
>   // asynchronously in another thread.
> We did see this occur in our clusters where rmdir operations can take seconds 
> to complete, blocking other queued events leading to, for example, long 
> latencies to task launch. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MESOS-6549) Asynchronous dir removal in agent GC

2016-11-04 Thread Jacob Janco (JIRA)

Jacob Janco created MESOS-6549:
--

 Summary: Asynchronous dir removal in agent GC
 Key: MESOS-6549
 URL: https://issues.apache.org/jira/browse/MESOS-6549
 Project: Mesos
  Issue Type: Improvement
Reporter: Jacob Janco
Assignee: Jacob Janco


In src/slave/gc.cpp: 
  // TODO(bmahler): Other dispatches can block waiting for a removal
  // operation. To fix this, the removal operation can be done
  // asynchronously in another thread.

We did see this occur in our clusters where rmdir operations can take seconds 
to complete, blocking other queued events leading to, for example, long 
latencies to task launch. 




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (MESOS-3157) only perform batch resource allocations

2016-10-04 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-3157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423829#comment-15423829
 ] 

Jacob Janco edited comment on MESOS-3157 at 10/4/16 3:14 PM:
-

New reviews: 

changes to allocator: https://reviews.apache.org/r/51027/
fixes to tests: https://reviews.apache.org/r/51028/




was (Author: jjanco):
New reviews: 

changes to allocator: https://reviews.apache.org/r/51027/
benchmark: https://reviews.apache.org/r/49617/
fixes to tests: https://reviews.apache.org/r/51028/



> only perform batch resource allocations
> ---
>
> Key: MESOS-3157
> URL: https://issues.apache.org/jira/browse/MESOS-3157
> Project: Mesos
>  Issue Type: Bug
>  Components: allocation
>Reporter: James Peach
>Assignee: Jacob Janco
>
> Our deployment environments have a lot of churn, with many short-live 
> frameworks that often revive offers. Running the allocator takes a long time 
> (from seconds up to minutes).
> In this situation, event-triggered allocation causes the event queue in the 
> allocator process to get very long, and the allocator effectively becomes 
> unresponsive (eg. a revive offers message takes too long to come to the head 
> of the queue).
> We have been running a patch to remove all the event-triggered allocations 
> and only allocate from the batch task 
> {{HierarchicalAllocatorProcess::batch}}. This works great and really improves 
> responsiveness.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (MESOS-3157) only perform batch resource allocations

2016-10-04 Thread Jacob Janco (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-3157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco updated MESOS-3157:
---
Comment: was deleted

(was: Some interesting output from the benchmark listed in the reviews: 

Sample output without 51027:
[ RUN  ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
Using 1 agents and 3000 frameworks
Added 3000 frameworks in 57251us
Added 1 agents in 3.2134535333mins
allocator settled after 1.6123603833mins
[   OK ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
 (290578 ms)

Sample output with 51027:
[ RUN  ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
Using 1 agents and 3000 frameworks
Added 3000 frameworks in 39817us
Added 1 agents in 3.2286054167mins
allocator settled after 25.525654secs
[   OK ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
 (220137 ms)

Any input on the benchmark would be greatly appreciated as well. Thanks!)

> only perform batch resource allocations
> ---
>
> Key: MESOS-3157
> URL: https://issues.apache.org/jira/browse/MESOS-3157
> Project: Mesos
>  Issue Type: Bug
>  Components: allocation
>Reporter: James Peach
>Assignee: Jacob Janco
>
> Our deployment environments have a lot of churn, with many short-live 
> frameworks that often revive offers. Running the allocator takes a long time 
> (from seconds up to minutes).
> In this situation, event-triggered allocation causes the event queue in the 
> allocator process to get very long, and the allocator effectively becomes 
> unresponsive (eg. a revive offers message takes too long to come to the head 
> of the queue).
> We have been running a patch to remove all the event-triggered allocations 
> and only allocate from the batch task 
> {{HierarchicalAllocatorProcess::batch}}. This works great and really improves 
> responsiveness.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-5874) Only send ShutdownFrameworkMessage to agents associated with framework.

2016-09-29 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-5874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533158#comment-15533158
 ] 

Jacob Janco commented on MESOS-5874:


Thanks Alex!

> Only send ShutdownFrameworkMessage to agents associated with framework.
> ---
>
> Key: MESOS-5874
> URL: https://issues.apache.org/jira/browse/MESOS-5874
> Project: Mesos
>  Issue Type: Improvement
>  Components: master
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Minor
>  Labels: mesosphere
> Fix For: 1.1.0
>
>
> slave.cpp:2079] Asked to shut down framework ${framework} by master@${master}
> slave.cpp:2094] Cannot shut down unknown framework ${framework} 
> For high framework/churn clusters this saturates agent logs with these 
> messages. When a framework terminates a ShutdownFrameworkMessage is sent to 
> every registered slave in a for loop. This patch proposes sending this 
> message to agents with executors associated with the framework. 
> Also proposed is moving the logline to VLOG(1). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-5874) Only send ShutdownFrameworkMessage to agents associated with framework.

2016-09-28 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-5874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15531343#comment-15531343
 ] 

Jacob Janco commented on MESOS-5874:


Closed the review, and posted https://reviews.apache.org/r/52371/
Will close ticket if that is satisfactory. Thanks!

> Only send ShutdownFrameworkMessage to agents associated with framework.
> ---
>
> Key: MESOS-5874
> URL: https://issues.apache.org/jira/browse/MESOS-5874
> Project: Mesos
>  Issue Type: Improvement
>  Components: master
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Minor
>  Labels: mesosphere
> Fix For: 1.1.0
>
>
> slave.cpp:2079] Asked to shut down framework ${framework} by master@${master}
> slave.cpp:2094] Cannot shut down unknown framework ${framework} 
> For high framework/churn clusters this saturates agent logs with these 
> messages. When a framework terminates a ShutdownFrameworkMessage is sent to 
> every registered slave in a for loop. This patch proposes sending this 
> message to agents with executors associated with the framework. 
> Also proposed is moving the logline to VLOG(1). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (MESOS-5874) Only send ShutdownFrameworkMessage to agents associated with framework.

2016-09-26 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-5874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524201#comment-15524201
 ] 

Jacob Janco edited comment on MESOS-5874 at 9/26/16 9:25 PM:
-

[~alexr], I can do that, but I had a quick question - would it make sense to 
move this logline to VLOG(1) as this is extremely verbose for our use case? 


was (Author: jjanco):
[~ruklet...@gmail.com], I can do that, but I had a quick question - would it 
make sense to move this logline to VLOG(1) as this is extremely verbose for our 
use case? 

> Only send ShutdownFrameworkMessage to agents associated with framework.
> ---
>
> Key: MESOS-5874
> URL: https://issues.apache.org/jira/browse/MESOS-5874
> Project: Mesos
>  Issue Type: Improvement
>  Components: master
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Minor
>  Labels: mesosphere
> Fix For: 1.1.0
>
>
> slave.cpp:2079] Asked to shut down framework ${framework} by master@${master}
> slave.cpp:2094] Cannot shut down unknown framework ${framework} 
> For high framework/churn clusters this saturates agent logs with these 
> messages. When a framework terminates a ShutdownFrameworkMessage is sent to 
> every registered slave in a for loop. This patch proposes sending this 
> message to agents with executors associated with the framework. 
> Also proposed is moving the logline to VLOG(1). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (MESOS-5874) Only send ShutdownFrameworkMessage to agents associated with framework.

2016-09-26 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-5874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524201#comment-15524201
 ] 

Jacob Janco edited comment on MESOS-5874 at 9/26/16 9:24 PM:
-

[~ruklet...@gmail.com], I can do that, but I had a quick question - would it 
make sense to move this logline to VLOG(1) as this is extremely verbose for our 
use case? 


was (Author: jjanco):
Alexander Rukletsov, I can do that, but I had a quick question - would it make 
sense to move this logline to VLOG(1) as this is extremely verbose for our use 
case? 

> Only send ShutdownFrameworkMessage to agents associated with framework.
> ---
>
> Key: MESOS-5874
> URL: https://issues.apache.org/jira/browse/MESOS-5874
> Project: Mesos
>  Issue Type: Improvement
>  Components: master
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Minor
>  Labels: mesosphere
> Fix For: 1.1.0
>
>
> slave.cpp:2079] Asked to shut down framework ${framework} by master@${master}
> slave.cpp:2094] Cannot shut down unknown framework ${framework} 
> For high framework/churn clusters this saturates agent logs with these 
> messages. When a framework terminates a ShutdownFrameworkMessage is sent to 
> every registered slave in a for loop. This patch proposes sending this 
> message to agents with executors associated with the framework. 
> Also proposed is moving the logline to VLOG(1). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-5874) Only send ShutdownFrameworkMessage to agents associated with framework.

2016-09-26 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-5874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524201#comment-15524201
 ] 

Jacob Janco commented on MESOS-5874:


Alexander Rukletsov, I can do that, but I had a quick question - would it make 
sense to move this logline to VLOG(1) as this is extremely verbose for our use 
case? 

> Only send ShutdownFrameworkMessage to agents associated with framework.
> ---
>
> Key: MESOS-5874
> URL: https://issues.apache.org/jira/browse/MESOS-5874
> Project: Mesos
>  Issue Type: Improvement
>  Components: master
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Minor
>  Labels: mesosphere
> Fix For: 1.1.0
>
>
> slave.cpp:2079] Asked to shut down framework ${framework} by master@${master}
> slave.cpp:2094] Cannot shut down unknown framework ${framework} 
> For high framework/churn clusters this saturates agent logs with these 
> messages. When a framework terminates a ShutdownFrameworkMessage is sent to 
> every registered slave in a for loop. This patch proposes sending this 
> message to agents with executors associated with the framework. 
> Also proposed is moving the logline to VLOG(1). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3157) only perform batch resource allocations

2016-08-17 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-3157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425080#comment-15425080
 ] 

Jacob Janco commented on MESOS-3157:


Some interesting output from the benchmark listed in the reviews: 

Sample output without 51027:
[ RUN  ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
Using 1 agents and 3000 frameworks
Added 3000 frameworks in 57251us
Added 1 agents in 3.2134535333mins
allocator settled after 1.6123603833mins
[   OK ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
 (290578 ms)

Sample output with 51027:
[ RUN  ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
Using 1 agents and 3000 frameworks
Added 3000 frameworks in 39817us
Added 1 agents in 3.2286054167mins
allocator settled after 25.525654secs
[   OK ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22
 (220137 ms)

> only perform batch resource allocations
> ---
>
> Key: MESOS-3157
> URL: https://issues.apache.org/jira/browse/MESOS-3157
> Project: Mesos
>  Issue Type: Bug
>  Components: allocation
>Reporter: James Peach
>Assignee: Jacob Janco
>
> Our deployment environments have a lot of churn, with many short-live 
> frameworks that often revive offers. Running the allocator takes a long time 
> (from seconds up to minutes).
> In this situation, event-triggered allocation causes the event queue in the 
> allocator process to get very long, and the allocator effectively becomes 
> unresponsive (eg. a revive offers message takes too long to come to the head 
> of the queue).
> We have been running a patch to remove all the event-triggered allocations 
> and only allocate from the batch task 
> {{HierarchicalAllocatorProcess::batch}}. This works great and really improves 
> responsiveness.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3157) only perform batch resource allocations

2016-08-16 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-3157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423829#comment-15423829
 ] 

Jacob Janco commented on MESOS-3157:


New reviews: 

changes to allocator: https://reviews.apache.org/r/51027/
benchmark: https://reviews.apache.org/r/49617/
fixes to tests: https://reviews.apache.org/r/51028/



> only perform batch resource allocations
> ---
>
> Key: MESOS-3157
> URL: https://issues.apache.org/jira/browse/MESOS-3157
> Project: Mesos
>  Issue Type: Bug
>  Components: allocation
>Reporter: James Peach
>Assignee: Jacob Janco
>
> Our deployment environments have a lot of churn, with many short-live 
> frameworks that often revive offers. Running the allocator takes a long time 
> (from seconds up to minutes).
> In this situation, event-triggered allocation causes the event queue in the 
> allocator process to get very long, and the allocator effectively becomes 
> unresponsive (eg. a revive offers message takes too long to come to the head 
> of the queue).
> We have been running a patch to remove all the event-triggered allocations 
> and only allocate from the batch task 
> {{HierarchicalAllocatorProcess::batch}}. This works great and really improves 
> responsiveness.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3157) only perform batch resource allocations

2016-07-28 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-3157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15397923#comment-15397923
 ] 

Jacob Janco commented on MESOS-3157:


Yup, will post a new review soon. 

> only perform batch resource allocations
> ---
>
> Key: MESOS-3157
> URL: https://issues.apache.org/jira/browse/MESOS-3157
> Project: Mesos
>  Issue Type: Bug
>  Components: allocation
>Reporter: James Peach
>Assignee: Jacob Janco
>
> Our deployment environments have a lot of churn, with many short-live 
> frameworks that often revive offers. Running the allocator takes a long time 
> (from seconds up to minutes).
> In this situation, event-triggered allocation causes the event queue in the 
> allocator process to get very long, and the allocator effectively becomes 
> unresponsive (eg. a revive offers message takes too long to come to the head 
> of the queue).
> We have been running a patch to remove all the event-triggered allocations 
> and only allocate from the batch task 
> {{HierarchicalAllocatorProcess::batch}}. This works great and really improves 
> responsiveness.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MESOS-5911) Webui redirection to leader in browser does not work

2016-07-26 Thread Jacob Janco (JIRA)

Jacob Janco created MESOS-5911:
--

 Summary: Webui redirection to leader in browser does not work
 Key: MESOS-5911
 URL: https://issues.apache.org/jira/browse/MESOS-5911
 Project: Mesos
  Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Jacob Janco


We deployed 1.0.0-rc4 in a small test cluster with 3 masters/5 agents. 
Redirection to master curling for state.json returned the expected 307, 
however, in browser (chrome/safari/firefox), redirection failed with: 

{code}
XMLHttpRequest cannot load :5050/master/state. No 
'Access-Control-Allow-Origin' header is present on the requested resource. 
Origin ':5050' is therefore not allowed access.
{code}

This is the patch that introduced the redirect on /state (and HTTP calls): 
https://reviews.apache.org/r/34646

The issue is that before this change, the server side does not redirect, the 
web UI controller.js decides from the content of the state.json which leader to 
redirect and then invoke redirection itself. Browsers allow this but not the 
server side initiated redirect without 'Access-Control-Allow-Origin' header?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-5874) Only send ShutdownFrameworkMessage to agents associated with framework.

2016-07-20 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-5874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386714#comment-15386714
 ] 

Jacob Janco commented on MESOS-5874:


https://reviews.apache.org/r/50268/

> Only send ShutdownFrameworkMessage to agents associated with framework.
> ---
>
> Key: MESOS-5874
> URL: https://issues.apache.org/jira/browse/MESOS-5874
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>Priority: Minor
>
> slave.cpp:2079] Asked to shut down framework ${framework} by master@${master}
> slave.cpp:2094] Cannot shut down unknown framework ${framework} 
> For high framework/churn clusters this saturates agent logs with these 
> messages. When a framework terminates a ShutdownFrameworkMessage is sent to 
> every registered slave in a for loop. This patch proposes sending this 
> message to agents with executors associated with the framework. 
> Also proposed is moving the logline to VLOG(1). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MESOS-5874) Only send ShutdownFrameworkMessage to agents associated with framework.

2016-07-20 Thread Jacob Janco (JIRA)

Jacob Janco created MESOS-5874:
--

 Summary: Only send ShutdownFrameworkMessage to agents associated 
with framework.
 Key: MESOS-5874
 URL: https://issues.apache.org/jira/browse/MESOS-5874
 Project: Mesos
  Issue Type: Improvement
Reporter: Jacob Janco
Assignee: Jacob Janco
Priority: Minor


slave.cpp:2079] Asked to shut down framework ${framework} by master@${master}
slave.cpp:2094] Cannot shut down unknown framework ${framework} 

For high framework/churn clusters this saturates agent logs with these 
messages. When a framework terminates a ShutdownFrameworkMessage is sent to 
every registered slave in a for loop. This patch proposes sending this message 
to agents with executors associated with the framework. 

Also proposed is moving the logline to VLOG(1). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MESOS-5781) Benchmark allocation with framework suppression.

2016-07-08 Thread Jacob Janco (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco updated MESOS-5781:
---
Description: 
Benchmarks effects of framework suppression on allocation time. Frameworks are 
suppressed and resources recovered each iteration and allocation time is 
measured as we move to suppress all frameworks in the test case. Referencing 
MESOS-4694. 

Sample run at top of tree: 
[ RUN  ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/35
Using 5000 agents and 6000 frameworks
allocate() took 3.7728787mins to make 5000 offers with 1200 out of 6000 
frameworks suppressing offers
allocate() took 3.8739297333mins to make 5000 offers with 2400 out of 6000 
frameworks suppressing offers
allocate() took 3.7972409833mins to make 5000 offers with 3600 out of 6000 
frameworks suppressing offers
allocate() took 3.85926955mins to make 5000 offers with 4800 out of 6000 
frameworks suppressing offers
allocate() took 23.114991secs to make 0 offers with 6000 out of 6000 frameworks 
suppressing offers
[   OK ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/35 
(1157073 ms)

Sample run with MESOS-4694 (https://reviews.apache.org/r/43666/):
[ RUN  ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/35
Using 5000 agents and 6000 frameworks
allocate() took 2.8807476167mins to make 5000 offers with 1200 out of 6000 
frameworks suppressing offers
allocate() took 2.0985669833mins to make 5000 offers with 2400 out of 6000 
frameworks suppressing offers
allocate() took 1.3209152mins to make 5000 offers with 3600 out of 6000 
frameworks suppressing offers
allocate() took 36.852357secs to make 5000 offers with 4800 out of 6000 
frameworks suppressing offers
allocate() took 76064us to make 0 offers with 6000 out of 6000 frameworks 
suppressing offers
[   OK ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/35 
(624856 ms)

  was:
Benchmarks effects of framework suppression on allocation time. Frameworks are 
suppressed and resources recovered each iteration and allocation time is 
measured as we move to suppress all frameworks in the test case. Referencing 
MESOS-4694. 

Sample run at top of tree: 
Using 2000 agents and 200 frameworks
round 0 allocate took 2.630963secs to make 199 offers
round 1 allocate took 2.640694secs to make 198 offers
round 2 allocate took 2.642664secs to make 197 offers
...
round 197 allocate took 2.433047secs to make 2 offers
round 198 allocate took 2.409804secs to make 1 offers
round 199 allocate took 252270us to make 0 offers

Sample run with MESOS-4694 (https://reviews.apache.org/r/43666/):
Using 2000 agents and 200 frameworks
round 0 allocate took 2.626182secs to make 199 offers
round 1 allocate took 2.62286secs to make 198 offers
round 2 allocate took 2.591389secs to make 197 offers
...
round 101 allocate took 1.494164secs to make 98 offers
round 102 allocate took 1.491371secs to make 97 offers
round 103 allocate took 1.491969secs to make 96 offers
...
round 197 allocate took 534780us to make 2 offers
round 198 allocate took 501947us to make 1 offers
round 199 allocate took 24929us to make 0 offers


> Benchmark allocation with framework suppression.
> 
>
> Key: MESOS-5781
> URL: https://issues.apache.org/jira/browse/MESOS-5781
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>  Labels: allocator, benchmark
>
> Benchmarks effects of framework suppression on allocation time. Frameworks 
> are suppressed and resources recovered each iteration and allocation time is 
> measured as we move to suppress all frameworks in the test case. Referencing 
> MESOS-4694. 
> Sample run at top of tree: 
> [ RUN  ] 
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/35
> Using 5000 agents and 6000 frameworks
> allocate() took 3.7728787mins to make 5000 offers with 1200 out of 6000 
> frameworks suppressing offers
> allocate() took 3.8739297333mins to make 5000 offers with 2400 out of 
> 6000 frameworks suppressing offers
> allocate() took 3.7972409833mins to make 5000 offers with 3600 out of 
> 6000 frameworks suppressing offers
> allocate() took 3.85926955mins to make 5000 offers with 4800 out of 6000 
> frameworks suppressing offers
> allocate() took 23.114991secs to make 0 offers with 6000 out of 6000 
> frameworks suppressing offers
> [   OK ] 
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/35 
> (1157073 ms)
> Sample run with MESOS-4694 (https://reviews.apache.org/r/43666/):
> [ RUN  ] 
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.SuppressOffers/35
> Using 5000 agents and 6000 frameworks
> allocate() took

[jira] [Commented] (MESOS-5781) Benchmark allocation with framework suppression.

2016-07-07 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367193#comment-15367193
 ] 

Jacob Janco commented on MESOS-5781:


Relating another review to bump the amount of frameworks in benchmark 
parameters. 
https://reviews.apache.org/r/49784

> Benchmark allocation with framework suppression.
> 
>
> Key: MESOS-5781
> URL: https://issues.apache.org/jira/browse/MESOS-5781
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>  Labels: allocator, benchmark
>
> Benchmarks effects of framework suppression on allocation time. Frameworks 
> are suppressed and resources recovered each iteration and allocation time is 
> measured as we move to suppress all frameworks in the test case. Referencing 
> MESOS-4694. 
> Sample run at top of tree: 
> Using 2000 agents and 200 frameworks
> round 0 allocate took 2.630963secs to make 199 offers
> round 1 allocate took 2.640694secs to make 198 offers
> round 2 allocate took 2.642664secs to make 197 offers
> ...
> round 197 allocate took 2.433047secs to make 2 offers
> round 198 allocate took 2.409804secs to make 1 offers
> round 199 allocate took 252270us to make 0 offers
> Sample run with MESOS-4694 (https://reviews.apache.org/r/43666/):
> Using 2000 agents and 200 frameworks
> round 0 allocate took 2.626182secs to make 199 offers
> round 1 allocate took 2.62286secs to make 198 offers
> round 2 allocate took 2.591389secs to make 197 offers
> ...
> round 101 allocate took 1.494164secs to make 98 offers
> round 102 allocate took 1.491371secs to make 97 offers
> round 103 allocate took 1.491969secs to make 96 offers
> ...
> round 197 allocate took 534780us to make 2 offers
> round 198 allocate took 501947us to make 1 offers
> round 199 allocate took 24929us to make 0 offers



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-5780) Benchmark framework failover.

2016-07-07 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-5780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367194#comment-15367194
 ] 

Jacob Janco commented on MESOS-5780:


Relating another review to bump the amount of frameworks in benchmark 
parameters. 
https://reviews.apache.org/r/49784

> Benchmark framework failover.
> -
>
> Key: MESOS-5780
> URL: https://issues.apache.org/jira/browse/MESOS-5780
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>  Labels: allocator, benchmark, framework
>
> Benchmarking disconnection and reconnection of all frameworks in cluster 
> proves useful in gauging the efficiency of the allocator's handling of a 
> flooded event queue. I'd also like to reference MESOS-3157.
> Sample run at top of tree: 
> [ RUN  ] 
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/10
> Allocator settled after 11.8424527mins for 5000 agents and 500 frameworks
> Sample run with MESOS-3157: 
> [ RUN  ] 
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/10
> Allocator settled after 5.98023secs for 5000 agents and 500 frameworks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-5781) Benchmark allocation with framework suppression.

2016-07-04 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361832#comment-15361832
 ] 

Jacob Janco commented on MESOS-5781:


https://reviews.apache.org/r/49616/

> Benchmark allocation with framework suppression.
> 
>
> Key: MESOS-5781
> URL: https://issues.apache.org/jira/browse/MESOS-5781
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>  Labels: allocator, benchmark
>
> Benchmarks effects of framework suppression on allocation time. Frameworks 
> are suppressed and resources recovered each iteration and allocation time is 
> measured as we move to suppress all frameworks in the test case. Referencing 
> MESOS-4694. 
> Sample run at top of tree: 
> Using 2000 agents and 200 frameworks
> round 0 allocate took 2.630963secs to make 199 offers
> round 1 allocate took 2.640694secs to make 198 offers
> round 2 allocate took 2.642664secs to make 197 offers
> ...
> round 197 allocate took 2.433047secs to make 2 offers
> round 198 allocate took 2.409804secs to make 1 offers
> round 199 allocate took 252270us to make 0 offers
> Sample run with MESOS-4694 (https://reviews.apache.org/r/43666/):
> Using 2000 agents and 200 frameworks
> round 0 allocate took 2.626182secs to make 199 offers
> round 1 allocate took 2.62286secs to make 198 offers
> round 2 allocate took 2.591389secs to make 197 offers
> ...
> round 101 allocate took 1.494164secs to make 98 offers
> round 102 allocate took 1.491371secs to make 97 offers
> round 103 allocate took 1.491969secs to make 96 offers
> ...
> round 197 allocate took 534780us to make 2 offers
> round 198 allocate took 501947us to make 1 offers
> round 199 allocate took 24929us to make 0 offers



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MESOS-5781) Benchmark allocation with framework suppression.

2016-07-04 Thread Jacob Janco (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco updated MESOS-5781:
---
Description: 
Benchmarks effects of framework suppression on allocation time. Frameworks are 
suppressed and resources recovered each iteration and allocation time is 
measured as we move to suppress all frameworks in the test case. Referencing 
MESOS-4694. 

Sample run at top of tree: 
Using 2000 agents and 200 frameworks
round 0 allocate took 2.630963secs to make 199 offers
round 1 allocate took 2.640694secs to make 198 offers
round 2 allocate took 2.642664secs to make 197 offers
...
round 197 allocate took 2.433047secs to make 2 offers
round 198 allocate took 2.409804secs to make 1 offers
round 199 allocate took 252270us to make 0 offers

Sample run with MESOS-4694 (https://reviews.apache.org/r/43666/):
Using 2000 agents and 200 frameworks
round 0 allocate took 2.626182secs to make 199 offers
round 1 allocate took 2.62286secs to make 198 offers
round 2 allocate took 2.591389secs to make 197 offers
...
round 101 allocate took 1.494164secs to make 98 offers
round 102 allocate took 1.491371secs to make 97 offers
round 103 allocate took 1.491969secs to make 96 offers
...
round 197 allocate took 534780us to make 2 offers
round 198 allocate took 501947us to make 1 offers
round 199 allocate took 24929us to make 0 offers

  was:
Benchmarks effects of framework suppression on allocation time. Frameworks are 
suppressed and resources recovered each iteration and allocation time is 
measured as we move to suppress all frameworks in the test case. Referencing 
MESOS-4694. 

Sample run at top of tree: 
Using 2000 agents and 200 frameworks
round 0 allocate took 2.630963secs to make 199 offers
round 1 allocate took 2.640694secs to make 198 offers
round 2 allocate took 2.642664secs to make 197 offers
...
round 197 allocate took 2.433047secs to make 2 offers
round 198 allocate took 2.409804secs to make 1 offers
round 199 allocate took 252270us to make 0 offers

Sample run with MESOS-4694 (https://reviews.apache.org/r/43666/):
round 0 allocate took 2.626182secs to make 199 offers
round 1 allocate took 2.62286secs to make 198 offers
round 2 allocate took 2.591389secs to make 197 offers
...
round 101 allocate took 1.494164secs to make 98 offers
round 102 allocate took 1.491371secs to make 97 offers
round 103 allocate took 1.491969secs to make 96 offers
...
round 197 allocate took 534780us to make 2 offers
round 198 allocate took 501947us to make 1 offers
round 199 allocate took 24929us to make 0 offers


> Benchmark allocation with framework suppression.
> 
>
> Key: MESOS-5781
> URL: https://issues.apache.org/jira/browse/MESOS-5781
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>  Labels: allocator, benchmark
>
> Benchmarks effects of framework suppression on allocation time. Frameworks 
> are suppressed and resources recovered each iteration and allocation time is 
> measured as we move to suppress all frameworks in the test case. Referencing 
> MESOS-4694. 
> Sample run at top of tree: 
> Using 2000 agents and 200 frameworks
> round 0 allocate took 2.630963secs to make 199 offers
> round 1 allocate took 2.640694secs to make 198 offers
> round 2 allocate took 2.642664secs to make 197 offers
> ...
> round 197 allocate took 2.433047secs to make 2 offers
> round 198 allocate took 2.409804secs to make 1 offers
> round 199 allocate took 252270us to make 0 offers
> Sample run with MESOS-4694 (https://reviews.apache.org/r/43666/):
> Using 2000 agents and 200 frameworks
> round 0 allocate took 2.626182secs to make 199 offers
> round 1 allocate took 2.62286secs to make 198 offers
> round 2 allocate took 2.591389secs to make 197 offers
> ...
> round 101 allocate took 1.494164secs to make 98 offers
> round 102 allocate took 1.491371secs to make 97 offers
> round 103 allocate took 1.491969secs to make 96 offers
> ...
> round 197 allocate took 534780us to make 2 offers
> round 198 allocate took 501947us to make 1 offers
> round 199 allocate took 24929us to make 0 offers



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MESOS-5781) Benchmark allocation with framework suppression.

2016-07-04 Thread Jacob Janco (JIRA)

Jacob Janco created MESOS-5781:
--

 Summary: Benchmark allocation with framework suppression.
 Key: MESOS-5781
 URL: https://issues.apache.org/jira/browse/MESOS-5781
 Project: Mesos
  Issue Type: Improvement
Reporter: Jacob Janco
Assignee: Jacob Janco


Benchmarks effects of framework suppression on allocation time. Frameworks are 
suppressed and resources recovered each iteration and allocation time is 
measured as we move to suppress all frameworks in the test case. Referencing 
MESOS-4694. 

Sample run at top of tree: 
Using 2000 agents and 200 frameworks
round 0 allocate took 2.630963secs to make 199 offers
round 1 allocate took 2.640694secs to make 198 offers
round 2 allocate took 2.642664secs to make 197 offers
...
round 197 allocate took 2.433047secs to make 2 offers
round 198 allocate took 2.409804secs to make 1 offers
round 199 allocate took 252270us to make 0 offers

Sample run with MESOS-4694 (https://reviews.apache.org/r/43666/):
round 0 allocate took 2.626182secs to make 199 offers
round 1 allocate took 2.62286secs to make 198 offers
round 2 allocate took 2.591389secs to make 197 offers
...
round 101 allocate took 1.494164secs to make 98 offers
round 102 allocate took 1.491371secs to make 97 offers
round 103 allocate took 1.491969secs to make 96 offers
...
round 197 allocate took 534780us to make 2 offers
round 198 allocate took 501947us to make 1 offers
round 199 allocate took 24929us to make 0 offers



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MESOS-5780) Benchmark framework failover.

2016-07-04 Thread Jacob Janco (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-5780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco updated MESOS-5780:
---
Description: 
Benchmarking disconnection and reconnection of all frameworks in cluster proves 
useful in gauging the efficiency of the allocator's handling of a flooded event 
queue. I'd also like to reference MESOS-3157.

Sample run at top of tree: 
[ RUN  ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/10
Allocator settled after 11.8424527mins for 5000 agents and 500 frameworks

Sample run with MESOS-3157: 
[ RUN  ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/10
Allocator settled after 5.98023secs for 5000 agents and 500 frameworks


  was:
Benchmarking disconnection and reconnection of all frameworks in cluster proves 
useful in gauging the efficiency of the allocator's handling a flooded event 
queue. I'd also like to reference MESOS-3157.

Sample run at top of tree: 
[ RUN  ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/10
Allocator settled after 11.8424527mins for 5000 agents and 500 frameworks

Sample run with MESOS-3157: 
[ RUN  ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/10
Allocator settled after 5.98023secs for 5000 agents and 500 frameworks



> Benchmark framework failover.
> -
>
> Key: MESOS-5780
> URL: https://issues.apache.org/jira/browse/MESOS-5780
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>  Labels: allocator, benchmark, framework
>
> Benchmarking disconnection and reconnection of all frameworks in cluster 
> proves useful in gauging the efficiency of the allocator's handling of a 
> flooded event queue. I'd also like to reference MESOS-3157.
> Sample run at top of tree: 
> [ RUN  ] 
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/10
> Allocator settled after 11.8424527mins for 5000 agents and 500 frameworks
> Sample run with MESOS-3157: 
> [ RUN  ] 
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/10
> Allocator settled after 5.98023secs for 5000 agents and 500 frameworks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MESOS-5780) Benchmark framework failover.

2016-07-04 Thread Jacob Janco (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-5780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco updated MESOS-5780:
---
Description: 
Benchmarking disconnection and reconnection of all frameworks in cluster proves 
useful in gauging the efficiency of the allocator's handling a flooded event 
queue. I'd also like to reference MESOS-3157.

Sample run at top of tree: 
[ RUN  ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/10
Allocator settled after 11.8424527mins for 5000 agents and 500 frameworks

Sample run with MESOS-3157: 
[ RUN  ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/10
Allocator settled after 5.98023secs for 5000 agents and 500 frameworks


  was:
Benchmarking disconnection and reconnection of all frameworks in cluster proves 
useful in gauging the efficiency of the allocator's efficiency in handling a 
flooded event queue. I'd also like to reference MESOS-3157.

Sample run at top of tree: 
[ RUN  ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/10
Allocator settled after 11.8424527mins for 5000 agents and 500 frameworks

Sample run with MESOS-3157: 
[ RUN  ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/10
Allocator settled after 5.98023secs for 5000 agents and 500 frameworks



> Benchmark framework failover.
> -
>
> Key: MESOS-5780
> URL: https://issues.apache.org/jira/browse/MESOS-5780
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>  Labels: allocator, benchmark, framework
>
> Benchmarking disconnection and reconnection of all frameworks in cluster 
> proves useful in gauging the efficiency of the allocator's handling a flooded 
> event queue. I'd also like to reference MESOS-3157.
> Sample run at top of tree: 
> [ RUN  ] 
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/10
> Allocator settled after 11.8424527mins for 5000 agents and 500 frameworks
> Sample run with MESOS-3157: 
> [ RUN  ] 
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/10
> Allocator settled after 5.98023secs for 5000 agents and 500 frameworks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-5780) Benchmark framework failover.

2016-07-04 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-5780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15361828#comment-15361828
 ] 

Jacob Janco commented on MESOS-5780:


https://reviews.apache.org/r/49617

> Benchmark framework failover.
> -
>
> Key: MESOS-5780
> URL: https://issues.apache.org/jira/browse/MESOS-5780
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>  Labels: allocator, benchmark, framework
>
> Benchmarking disconnection and reconnection of all frameworks in cluster 
> proves useful in gauging the efficiency of the allocator's efficiency in 
> handling a flooded event queue. I'd also like to reference MESOS-3157.
> Sample run at top of tree: 
> [ RUN  ] 
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/10
> Allocator settled after 11.8424527mins for 5000 agents and 500 frameworks
> Sample run with MESOS-3157: 
> [ RUN  ] 
> SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/10
> Allocator settled after 5.98023secs for 5000 agents and 500 frameworks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MESOS-5780) Benchmark framework failover.

2016-07-04 Thread Jacob Janco (JIRA)

Jacob Janco created MESOS-5780:
--

 Summary: Benchmark framework failover.
 Key: MESOS-5780
 URL: https://issues.apache.org/jira/browse/MESOS-5780
 Project: Mesos
  Issue Type: Improvement
Reporter: Jacob Janco
Assignee: Jacob Janco


Benchmarking disconnection and reconnection of all frameworks in cluster proves 
useful in gauging the efficiency of the allocator's efficiency in handling a 
flooded event queue. I'd also like to reference MESOS-3157.

Sample run at top of tree: 
[ RUN  ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/10
Allocator settled after 11.8424527mins for 5000 agents and 500 frameworks

Sample run with MESOS-3157: 
[ RUN  ] 
SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/10
Allocator settled after 5.98023secs for 5000 agents and 500 frameworks




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3157) only perform batch resource allocations

2016-06-21 Thread Jacob Janco (JIRA)


[ 
https://issues.apache.org/jira/browse/MESOS-3157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15343076#comment-15343076
 ] 

Jacob Janco commented on MESOS-3157:


Np, thanks for the initial work, happy to see this through. 

> only perform batch resource allocations
> ---
>
> Key: MESOS-3157
> URL: https://issues.apache.org/jira/browse/MESOS-3157
> Project: Mesos
>  Issue Type: Bug
>  Components: allocation
>Reporter: James Peach
>Assignee: Jacob Janco
>
> Our deployment environments have a lot of churn, with many short-live 
> frameworks that often revive offers. Running the allocator takes a long time 
> (from seconds up to minutes).
> In this situation, event-triggered allocation causes the event queue in the 
> allocator process to get very long, and the allocator effectively becomes 
> unresponsive (eg. a revive offers message takes too long to come to the head 
> of the queue).
> We have been running a patch to remove all the event-triggered allocations 
> and only allocate from the batch task 
> {{HierarchicalAllocatorProcess::batch}}. This works great and really improves 
> responsiveness.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (MESOS-4126) Construct the error string in `MethodNotAllowed`.

2016-02-08 Thread Jacob Janco (JIRA)


 [ 
https://issues.apache.org/jira/browse/MESOS-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacob Janco reassigned MESOS-4126:
--

Assignee: Jacob Janco

> Construct the error string in `MethodNotAllowed`.
> -
>
> Key: MESOS-4126
> URL: https://issues.apache.org/jira/browse/MESOS-4126
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Alexander Rukletsov
>Assignee: Jacob Janco
>  Labels: http, mesosphere, newbie++
>
> Consider constructing the error string in {{MethodNotAllowed}} rather than at 
> the invocation site. Currently we want all error messages follow the same 
> pattern, so instead of writing
> {code}
> return MethodNotAllowed({"POST"}, "Expecting 'POST', received '" + 
> request.method + "'");
> {code}
> we can write something like
> {code}
> MethodNotAllowed({"POST"}, request.method)`
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

61 matches

Mail list logo