[jira] [Updated] (YUNIKORN-352) when child queue capacity greater than parent, the configmap update is rejected but not notified to end user

2020-10-09 Thread Kinga Marton (Jira) 


 [ 
https://issues.apache.org/jira/browse/YUNIKORN-352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kinga Marton updated YUNIKORN-352:
--
Priority: Major  (was: Critical)

> when child queue capacity greater than parent, the configmap update is 
> rejected but not notified to end user
> 
>
> Key: YUNIKORN-352
> URL: https://issues.apache.org/jira/browse/YUNIKORN-352
> Project: Apache YuniKorn
>  Issue Type: Bug
>  Components: core - scheduler
>Reporter: Ayub Pathan
>Assignee: Kinga Marton
>Priority: Major
>
> Create a nested static queue like below.
> {noformat}
> partitions:
>   -
> name: default
> placementrules:
>   - name: tag
> value: namespace
> create: true
> queues:
>   - name: root
> submitacl: '*'
> queues:
>   - name: queue2
> resources:
>   guaranteed:
> memory: 300
> cpu: 300
>   max:
> memory: 1000
> cpu: 1000
> queues:
>   - name: queue3
> resources:
>   guaranteed:
> memory: 300
> cpu: 300
>   max:
> memory: 2000
> cpu: 2000
> {noformat}
> Validate the same through rest API /queues - queues3 is not even shown in the 
> response.
> {noformat}
> {
> "capacity": {
> "capacity": "map[attachable-volumes-aws-ebs:75 
> ephemeral-storage:94992122100 hugepages-1Gi:0 hugepages-2Mi:0 memory:18966 
> pods:87 vcore:4875]",
> "usedcapacity": "0"
> },
> "nodes": null,
> "partitionName": "[mycluster]default",
> "queues": {
> "capacities": {
> "absusedcapacity": "[memory:0 vcore:2]",
> "capacity": "[]",
> "maxcapacity": "[attachable-volumes-aws-ebs:75 
> ephemeral-storage:94992122100 hugepages-1Gi:0 hugepages-2Mi:0 memory:18966 
> pods:87 vcore:4875]",
> "usedcapacity": "[memory:1 vcore:110]"
> },
> "properties": {},
> "queuename": "root",
> "queues": [
> {
> "capacities": {
> "absusedcapacity": "[]",
> "capacity": "[]",
> "maxcapacity": "[]",
> "usedcapacity": "[memory:1]"
> },
> "properties": {},
> "queuename": "monitoring",
> "queues": null,
> "status": "Active"
> },
> {
> "capacities": {
> "absusedcapacity": "[]",
> "capacity": "[]",
> "maxcapacity": "[]",
> "usedcapacity": "[vcore:110]"
> },
> "properties": {},
> "queuename": "kube-system",
> "queues": null,
> "status": "Active"
> },
> {
> "capacities": {
> "absusedcapacity": "[]",
> "capacity": "[cpu:300 memory:300]",
> "maxcapacity": "[cpu:1000 memory:1000]",
> "usedcapacity": "[]"
> },
> "properties": {},
> "queuename": "queue2",
> "queues": null,
> "status": "Active"
> }
> ],
> "status": "Active"
> }
> }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: issues-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-442) Data race in resources.go

2020-10-09 Thread Adam Antal (Jira) 
Adam Antal created YUNIKORN-442:
---

 Summary: Data race in resources.go
 Key: YUNIKORN-442
 URL: https://issues.apache.org/jira/browse/YUNIKORN-442
 Project: Apache YuniKorn
  Issue Type: Bug
  Components: core - scheduler
Affects Versions: 0.9
Reporter: Adam Antal


I added multiple pods in my docker-desktop environment and got the following 
data race:
{noformat}
==
WARNING: DATA RACE
Write at 0x00c00a8fa030 by goroutine 75:
  runtime.mapassign_faststr()
  /usr/local/go/src/runtime/map_faststr.go:202 +0x0
  
github.com/apache/incubator-yunikorn-core/pkg/common/resources.(*Resource).AddTo()
  
/Users/adamantal/git/yunikorn/yunikorn-core/pkg/common/resources/resources.go:136
 +0x1c7
  
github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*SchedulingApplication).addAllocationAsk()
  
/Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduling_application.go:243
 +0x3ed
  
github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*Scheduler).updateSchedulingRequest()
  
/Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduler.go:163 
+0x10b
  
github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*Scheduler).processAllocationUpdateEvent()
  
/Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduler.go:415 
+0x276
  
github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*Scheduler).handleSchedulerEvent()
  
/Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduler.go:597 
+0x38b

Previous read at 0x00c00a8fa030 by goroutine 77:
  runtime.mapiterinit()
  /usr/local/go/src/runtime/map.go:797 +0x0
  
github.com/apache/incubator-yunikorn-core/pkg/common/resources.StrictlyGreaterThanZero()
  
/Users/adamantal/git/yunikorn/yunikorn-core/pkg/common/resources/resources.go:690
 +0xb1
  
github.com/apache/incubator-yunikorn-core/pkg/scheduler.filterOnPendingResources()
  /Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/sorters.go:85 
+0x168
  github.com/apache/incubator-yunikorn-core/pkg/scheduler.sortApplications()
  /Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/sorters.go:61 
+0x2a4
  
github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*SchedulingQueue).sortApplications()
  
/Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduling_queue.go:446
 +0xe8
  
github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*SchedulingQueue).tryAllocate()
  
/Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduling_queue.go:556
 +0xb6
  
github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*SchedulingQueue).tryAllocate()
  
/Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduling_queue.go:569
 +0x6da
  
github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*partitionSchedulingContext).tryAllocate()
  
/Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduling_partition.go:371
 +0x93
  
github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*Scheduler).schedule()
  
/Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduler.go:649 
+0x9d2
  
github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*Scheduler).internalSchedule()
  
/Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduler.go:103 +0x38

Goroutine 75 (running) created at:
  
github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*Scheduler).StartService()
  /Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduler.go:67 
+0x9d
  
github.com/apache/incubator-yunikorn-core/pkg/entrypoint.startAllServicesWithParameters()
  
/Users/adamantal/git/yunikorn/yunikorn-core/pkg/entrypoint/entrypoint.go:87 
+0x64c
  github.com/apache/incubator-yunikorn-core/pkg/entrypoint.StartAllServices()
  
/Users/adamantal/git/yunikorn/yunikorn-core/pkg/entrypoint/entrypoint.go:43 
+0x77
  main.main()
  /Users/adamantal/git/yunikorn/yunikorn-k8shim/pkg/shim/main.go:45 +0x4e9

Goroutine 77 (running) created at:
  
github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*Scheduler).StartService()
  /Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduler.go:74 
+0x1b9
  
github.com/apache/incubator-yunikorn-core/pkg/entrypoint.startAllServicesWithParameters()
  
/Users/adamantal/git/yunikorn/yunikorn-core/pkg/entrypoint/entrypoint.go:87 
+0x64c
  github.com/apache/incubator-yunikorn-core/pkg/entrypoint.StartAllServices()
  
/Users/adamantal/git/yunikorn/yunikorn-core/pkg/entrypoint/entrypoint.go:43 
+0x77
  main.main()
  /Users/adamantal/git/yunikorn/yunikorn-k8shim/pkg/shim/main.go:45 +0x4e9
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: issues-h...@yunikorn.apache.org

[jira] [Updated] (YUNIKORN-429) Set default logging level to INFO

2020-10-09 Thread Adam Antal (Jira) 


 [ 
https://issues.apache.org/jira/browse/YUNIKORN-429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam Antal updated YUNIKORN-429:

Attachment: INFO.txt

> Set default logging level to INFO
> -
>
> Key: YUNIKORN-429
> URL: https://issues.apache.org/jira/browse/YUNIKORN-429
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>Reporter: Weiwei Yang
>Assignee: Adam Antal
>Priority: Major
>  Labels: pull-request-available
> Attachments: INFO.txt
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: issues-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-429) Set default logging level to INFO

2020-10-09 Thread Weiwei Yang (Jira) 


 [ 
https://issues.apache.org/jira/browse/YUNIKORN-429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Weiwei Yang resolved YUNIKORN-429.
--
Fix Version/s: 0.10
   Resolution: Fixed

> Set default logging level to INFO
> -
>
> Key: YUNIKORN-429
> URL: https://issues.apache.org/jira/browse/YUNIKORN-429
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>Reporter: Weiwei Yang
>Assignee: Adam Antal
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.10
>
> Attachments: INFO.txt
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: issues-h...@yunikorn.apache.org

[jira] [Updated] (YUNIKORN-442) Data race in resources.go

2020-10-09 Thread Weiwei Yang (Jira) 


 [ 
https://issues.apache.org/jira/browse/YUNIKORN-442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Weiwei Yang updated YUNIKORN-442:
-
Target Version: 0.10

Thanks [~adam.antal] this needs to be fixed in the next release.
Could you please add the steps on how to reproduce this? Any idea what causes 
the data race?

> Data race in resources.go
> -
>
> Key: YUNIKORN-442
> URL: https://issues.apache.org/jira/browse/YUNIKORN-442
> Project: Apache YuniKorn
>  Issue Type: Bug
>  Components: core - scheduler
>Affects Versions: 0.9
>Reporter: Adam Antal
>Priority: Major
>
> I added multiple pods in my docker-desktop environment and got the following 
> data race:
> {noformat}
> ==
> WARNING: DATA RACE
> Write at 0x00c00a8fa030 by goroutine 75:
>   runtime.mapassign_faststr()
>   /usr/local/go/src/runtime/map_faststr.go:202 +0x0
>   
> github.com/apache/incubator-yunikorn-core/pkg/common/resources.(*Resource).AddTo()
>   
> /Users/adamantal/git/yunikorn/yunikorn-core/pkg/common/resources/resources.go:136
>  +0x1c7
>   
> github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*SchedulingApplication).addAllocationAsk()
>   
> /Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduling_application.go:243
>  +0x3ed
>   
> github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*Scheduler).updateSchedulingRequest()
>   
> /Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduler.go:163 
> +0x10b
>   
> github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*Scheduler).processAllocationUpdateEvent()
>   
> /Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduler.go:415 
> +0x276
>   
> github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*Scheduler).handleSchedulerEvent()
>   
> /Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduler.go:597 
> +0x38b
> Previous read at 0x00c00a8fa030 by goroutine 77:
>   runtime.mapiterinit()
>   /usr/local/go/src/runtime/map.go:797 +0x0
>   
> github.com/apache/incubator-yunikorn-core/pkg/common/resources.StrictlyGreaterThanZero()
>   
> /Users/adamantal/git/yunikorn/yunikorn-core/pkg/common/resources/resources.go:690
>  +0xb1
>   
> github.com/apache/incubator-yunikorn-core/pkg/scheduler.filterOnPendingResources()
>   /Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/sorters.go:85 
> +0x168
>   github.com/apache/incubator-yunikorn-core/pkg/scheduler.sortApplications()
>   /Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/sorters.go:61 
> +0x2a4
>   
> github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*SchedulingQueue).sortApplications()
>   
> /Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduling_queue.go:446
>  +0xe8
>   
> github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*SchedulingQueue).tryAllocate()
>   
> /Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduling_queue.go:556
>  +0xb6
>   
> github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*SchedulingQueue).tryAllocate()
>   
> /Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduling_queue.go:569
>  +0x6da
>   
> github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*partitionSchedulingContext).tryAllocate()
>   
> /Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduling_partition.go:371
>  +0x93
>   
> github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*Scheduler).schedule()
>   
> /Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduler.go:649 
> +0x9d2
>   
> github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*Scheduler).internalSchedule()
>   
> /Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduler.go:103 
> +0x38
> Goroutine 75 (running) created at:
>   
> github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*Scheduler).StartService()
>   
> /Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduler.go:67 
> +0x9d
>   
> github.com/apache/incubator-yunikorn-core/pkg/entrypoint.startAllServicesWithParameters()
>   
> /Users/adamantal/git/yunikorn/yunikorn-core/pkg/entrypoint/entrypoint.go:87 
> +0x64c
>   github.com/apache/incubator-yunikorn-core/pkg/entrypoint.StartAllServices()
>   
> /Users/adamantal/git/yunikorn/yunikorn-core/pkg/entrypoint/entrypoint.go:43 
> +0x77
>   main.main()
>   /Users/adamantal/git/yunikorn/yunikorn-k8shim/pkg/shim/main.go:45 +0x4e9
> Goroutine 77 (running) created at:
>   
> github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*Scheduler).StartService()
>   
> /Users/adamantal/git/yunikorn/yunikorn-core/pkg/scheduler/scheduler.go:74 
> +0x1b9
>   
> github.com/apache/incubator-yunikorn-core/pkg/entrypoint.startAllServicesWithParameters()
>   
> /Users/adamantal/git/yunikorn/yunikorn-core/pkg/entrypoint/entrypoint.go:87 
> +0x