Re: Apache YuniKorn (Incubating) - Community Graduation Vote

[Holden Karau]

For "The project provides a well-documented, secure and private channel to
report security issues, along with a documented way of responding to them.'
the standard that I've seen used is to tell people to e-mail private@ when
they think they might have a security related issue. I think that would
probably work well for Yunikorn too.


On Mon, Jan 10, 2022 at 7:04 AM Chenya Zhang 
wrote:

> Hi Weiwei,
>
> Thanks for driving this! The evaluation is quite comprehensive overall. I
> checked our Apache project maturity guidelines and noticed the below three
> items. Not sure if we already have them but they are not blockers to our
> graduation. We could think more about them along the way.
>
> QU30
>
> The project provides a well-documented, secure and private channel to
> report security issues, along with a documented way of responding to them.
>
> QU40
>
> The project puts a high priority on backwards compatibility and aims to
> document any incompatible changes and provide tools and documentation to
> help users transition to new features.
>
> CO50
>
> The project documents how contributors can earn more rights such as commit
> access or decision power, and applies these principles consistently.
>
>
> Thanks,
>
> Chenya
>
>
>
> On Mon, Jan 10, 2022 at 12:00 AM Weiwei Yang  wrote:
>
>> Hi YuniKorn community and mentors
>>
>> Based on the discussion thread [1], after 2 years time of incubating, it
>> is
>> considered that now is a good time to graduate YuniKorn from the ASF
>> incubator and become a top-level Apache project. We have reviewed the ASF
>> project maturity model [2] and provided some assessment of the project's
>> maturity based on the guidelines. Details are included as the following.
>> Please read this and share your thoughts by replying to this email, your
>> feedback will be much appreciated!!!
>>
>> *Code, License, and Copyright*
>>
>> All code is maintained on github, under Apache 2.0 license. We have
>> reviewed all the dependencies and ensured they do not bring any license
>> issues. All the status files, license headers, and copyright are up to
>> date.
>>
>> *Release*
>>
>> The community has released 5 releases in the past 2 years, i.e v0.8, v0.9,
>> v0.10, v0,11, and v0.12. These releases were done by 5 different release
>> managers [3] and indicate the community can create releases independently.
>> We have also a well-documented release process, automated tools to help
>> new
>> release managers with the process.
>>
>> *Quality*
>>
>> The community has developed a comprehensive CI/CD pipeline as a guard of
>> the code quality. The pipeline runs per-commit license check, code-format
>> check, code-coverage check, UT, and end-to-end tests. All these are built
>> as automated github actions, new contributors can easily trigger and view
>> results when submitting patches.
>>
>> *Community*
>>
>> The community has developed an easy-to-read homepage for the project [4],
>> the website hosts all the materials related to the project including
>> versioned documentation, user docs, developer docs, design docs,
>> performance docs. It provides the top-level navigation to the software
>> download page, where links to all our previous releases. It also has the
>> pages for the new contributors on-boarding with the project, such as how
>> to
>> join community meetings, events links, etc.
>>
>> The community shows appreciation to all contributors and welcomes all
>> kinds
>> of contributions (not just for code). We have built an open, diverse
>> community and gathered many people to work together. With that, we have 41
>> unique code contributors and some non-code contributors as well. Many of
>> them have becoming to be committers and PPMC members while working with
>> the
>> community. There were 2 new mentors, 8 new committers, 2 new PPMC from 6
>> different organizations [5] added in the incubating phase. And in total,
>> the project has 6 mentors, 21 PPMC, and 27 committers from at least 14
>> different organizations. Community collaboration was done in a
>> wide-public,
>> open manner, we leverage regular bi-weekly/weekly community meetings for 2
>> different timezones [6] and dev/user slack channels, mailing lists for
>> offline discussions.
>>
>> *Independence*
>>
>> The project was initially donated by Cloudera, but with a diverse open
>> source community, it has been operated as an independent project since it
>> entered into ASF incubator. The committers and PPMC members are a group of
>> passionate people from at least 14 different organizations, such as
>> Alibaba, Apple, Cloudera, Databricks, LinkedIn, Microsoft, Snowflake, etc.
>> The project's success is not depending on any single entity.
>>
>> I have enough reasons to believe the project has done sustainable
>> development successfully in the Apache way. Again, please share your
>> thoughts, all YuniKorn contributors, committers, PPMC, and mentors. Thank
>> you!
>>
>> [1] 

[jira] [Created] (YUNIKORN-727) Run Spark example RBACs are missing NS in required loc

[Holden Karau (Jira)]

Holden Karau created YUNIKORN-727:
-

 Summary: Run Spark example RBACs are missing NS in required loc
 Key: YUNIKORN-727
 URL: https://issues.apache.org/jira/browse/YUNIKORN-727
 Project: Apache YuniKorn
  Issue Type: Improvement
Reporter: Holden Karau






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-726) Quick start results in admission controller crashloop back off.

[Holden Karau (Jira)]

Holden Karau created YUNIKORN-726:
-

 Summary: Quick start results in admission controller crashloop 
back off.
 Key: YUNIKORN-726
 URL: https://issues.apache.org/jira/browse/YUNIKORN-726
 Project: Apache YuniKorn
  Issue Type: Improvement
Reporter: Holden Karau


Following the quick start guide at [https://yunikorn.apache.org/docs/] the 
admission controller fails start with an error indicating that it can't read 
the PEM data in the certificate input. Looking at the webhook-server-tls there 
is data in the key.pem but not cert.pem.

 

This is on K3s 1.21

 
{code:java}
2021-06-23T23:09:33.107Z INFO log/logger.go:93 scheduler configuration, pretty 
print {"configs": "{\n \"clusterId\": \"my-kube-cluster\",\n 
\"clusterVersion\": \"0.1\",\n \"policyGroup\": \"queues\",\n 
\"schedulingIntervalSecond\": 10,\n \"absoluteKubeConfigFilePath\": 
\"\",\n \"loggingLevel\": 0,\n \"logEncoding\": \"console\",\n \"logFilePath\": 
\"\",\n \"volumeBindTimeout\": 100,\n \"testMode\": false,\n 
\"eventChannelCapacity\": 1048576,\n \"dispatchTimeout\": 3000,\n 
\"kubeQPS\": 1000,\n \"kubeBurst\": 1000,\n \"predicates\": \"\",\n 
\"operatorPlugins\": \"general,yunikorn-app\",\n \"enableConfigHotRefresh\": 
false\n}"}2021-06-23T23:09:33.107Z INFO log/logger.go:93 scheduler 
configuration, pretty print {"configs": "{\n \"clusterId\": 
\"my-kube-cluster\",\n \"clusterVersion\": \"0.1\",\n \"policyGroup\": 
\"queues\",\n \"schedulingIntervalSecond\": 10,\n 
\"absoluteKubeConfigFilePath\": \"\",\n \"loggingLevel\": 0,\n \"logEncoding\": 
\"console\",\n \"logFilePath\": \"\",\n \"volumeBindTimeout\": 100,\n 
\"testMode\": false,\n \"eventChannelCapacity\": 1048576,\n 
\"dispatchTimeout\": 3000,\n \"kubeQPS\": 1000,\n \"kubeBurst\": 
1000,\n \"predicates\": \"\",\n \"operatorPlugins\": 
\"general,yunikorn-app\",\n \"enableConfigHotRefresh\": 
false\n}"}2021-06-23T23:09:33.107Z FATAL webhook/webhook.go:56 Failed to load 
key pair {"error": "tls: failed to find any PEM data in certificate 
input"}main.main 
/Users/boyuan/go/src/yunikorn-release-v0.10.0/incubator-yunikorn-release/staging/apache-yunikorn-0.10.0-incubating-src/k8shim/pkg/plugin/admissioncontrollers/webhook/webhook.go:56runtime.main
 /Users/boyuan/go/install/go1.14.14/src/runtime/proc.go:203{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-725) Support arm64

[Holden Karau (Jira)]

Holden Karau created YUNIKORN-725:
-

 Summary: Support arm64
 Key: YUNIKORN-725
 URL: https://issues.apache.org/jira/browse/YUNIKORN-725
 Project: Apache YuniKorn
  Issue Type: Improvement
Reporter: Holden Karau


It would be good to support arm64, this is probably not too painful and can be 
done by swapping docker build with docker buildx, but there's often edge cases 
where some code changes are needed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-723) Add support for PodGroup (e.g. from kube-batch)

[Holden Karau (Jira)]

Holden Karau created YUNIKORN-723:
-

 Summary: Add support for PodGroup (e.g. from kube-batch)
 Key: YUNIKORN-723
 URL: https://issues.apache.org/jira/browse/YUNIKORN-723
 Project: Apache YuniKorn
  Issue Type: Improvement
Reporter: Holden Karau


To simplify on-boarding new batch apps we could try and support the kube-batch 
APIs starting with PodGroup.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-380) Deploying YUNIKORN 0.9 RC2 on Minikube 1.3.1 results in port conflict

[Holden Karau (Jira)]

Holden Karau created YUNIKORN-380:
-

 Summary: Deploying YUNIKORN 0.9 RC2 on Minikube 1.3.1 results in 
port conflict
 Key: YUNIKORN-380
 URL: https://issues.apache.org/jira/browse/YUNIKORN-380
 Project: Apache YuniKorn
  Issue Type: Improvement
Reporter: Holden Karau


Following the instructions in `README.md` against my currently installed 
minikube I got:

 
{code:java}
 
holden@hkdesktop:~/yunikron_scratch/apache-yunikorn-0.9.0-incubating-src$ 
kubectl logs 
yunikorn-admission-controller-88c5966d9-q672kholden@hkdesktop:~/yunikron_scratch/apache-yunikorn-0.9.0-incubating-src$
 kubectl logs 
yunikorn-admission-controller-88c5966d9-q672k2020-08-23T22:56:23.349Z INFO 
log/logger.go:86 scheduler configuration, pretty print {"configs": "{\n 
\"clusterId\": \"my-kube-cluster\",\n \"clusterVersion\": \"0.1\",\n 
\"schedulerName\": \"yunikorn\",\n \"policyGroup\": \"queues\",\n 
\"schedulingIntervalSecond\": 10,\n \"absoluteKubeConfigFilePath\": 
\"\",\n \"loggingLevel\": 0,\n \"logEncoding\": \"console\",\n \"logFilePath\": 
\"\",\n \"volumeBindTimeout\": 100,\n \"testMode\": false,\n 
\"eventChannelCapacity\": 1048576,\n \"dispatchTimeout\": 3000,\n 
\"kubeQPS\": 1000,\n \"kubeBurst\": 1000,\n \"predicates\": \"\",\n 
\"operatorPlugins\": \"general\"\n}"}2020-08-23T22:56:23.349Z INFO 
webhook/webhook.go:83 the admission controller started {"port": 8443, 
"listeningOn": ["/mutate", "/validate-conf"]}2020-08-23T22:56:23.349Z FATAL 
webhook/webhook.go:79 failed to start admission controller {"error": "listen 
tcp :8443: bind: address already in use"}main.main.func1 
/Users/wyang/workspace/apache/yunikorn/incubator-yunikorn-k8shim/pkg/plugin/admissioncontrollers/webhook/webhook.go:79
{code}
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Added a project status page

[Holden Karau]

Hi folks, I added a project status page (
https://incubator.apache.org/projects/yunikorn.html ) if you notice
anything wrong with it feel free to let me know and I'll be happy to update
it (or help someone else update it).

-- 
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9  
YouTube Live Streams: https://www.youtube.com/user/holdenkarau