[GitHub] [kafka] stevenbooke commented on a diff in pull request #13842: KAFKA-14995: Automate asf.yaml collaborators refresh

2023-06-28 Thread via GitHub 


stevenbooke commented on code in PR #13842:
URL: https://github.com/apache/kafka/pull/13842#discussion_r1245887810


##
refresh-collaborators.py:
##
@@ -0,0 +1,44 @@
+import os
+from bs4 import BeautifulSoup
+from github import Github
+import yaml
+
+### GET THE NAMES OF THE KAFKA COMMITTERS FROM THE apache/kafka-site REPO ###
+github_token = os.environ.get('GITHUB_TOKEN')
+g = Github(github_token)
+repo = g.get_repo("apache/kafka-site")
+contents = repo.get_contents("committers.html")
+content = contents.decoded_content
+soup = BeautifulSoup(content, "html.parser")
+committer_logins = [login.text for login in soup.find_all('div', 
class_='github_login')]
+
+### GET THE CONTRIBUTORS OF THE apache/kafka REPO ###
+n = 10
+repo = g.get_repo("apache/kafka")
+contributors = repo.get_contributors()

Review Comment:
   Hello @vvcephei, I understand your concern. I will make changes to account 
for this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [kafka] stevenbooke commented on a diff in pull request #13842: KAFKA-14995: Automate asf.yaml collaborators refresh

2023-06-28 Thread via GitHub 


stevenbooke commented on code in PR #13842:
URL: https://github.com/apache/kafka/pull/13842#discussion_r1245890763


##
refresh-collaborators.py:
##
@@ -0,0 +1,44 @@
+import os
+from bs4 import BeautifulSoup
+from github import Github
+import yaml
+
+### GET THE NAMES OF THE KAFKA COMMITTERS FROM THE apache/kafka-site REPO ###
+github_token = os.environ.get('GITHUB_TOKEN')
+g = Github(github_token)
+repo = g.get_repo("apache/kafka-site")
+contents = repo.get_contents("committers.html")
+content = contents.decoded_content
+soup = BeautifulSoup(content, "html.parser")
+committer_logins = [login.text for login in soup.find_all('div', 
class_='github_login')]
+
+### GET THE CONTRIBUTORS OF THE apache/kafka REPO ###
+n = 10
+repo = g.get_repo("apache/kafka")
+contributors = repo.get_contributors()
+collaborators = []
+for contributor in contributors:
+if contributor.login not in committer_logins:
+collaborators += [contributor.login]
+refreshed_collaborators = collaborators[:n]
+
+### UPDATE asf.yaml ###
+file_path = ".asf.yaml"
+file = repo.get_contents(file_path)
+yaml_content = yaml.safe_load(file.decoded_content)
+
+# Update 'github_whitelist' list
+github_whitelist = refreshed_collaborators[:10]  # New users to be added
+yaml_content["jenkins"]["github_whitelist"] = github_whitelist
+
+# Update 'collaborators' list
+collaborators = refreshed_collaborators[:10]  # New collaborators to be added
+yaml_content["github"]["collaborators"] = collaborators
+
+# Convert the updated content back to YAML
+updated_yaml = yaml.safe_dump(yaml_content)
+
+# Commit and push the changes
+commit_message = "Update .asf.yaml file with refreshed github_whitelist, and 
collaborators"

Review Comment:
   Will do.



##
refresh-collaborators.py:
##
@@ -0,0 +1,44 @@
+import os
+from bs4 import BeautifulSoup
+from github import Github
+import yaml
+
+### GET THE NAMES OF THE KAFKA COMMITTERS FROM THE apache/kafka-site REPO ###
+github_token = os.environ.get('GITHUB_TOKEN')
+g = Github(github_token)
+repo = g.get_repo("apache/kafka-site")
+contents = repo.get_contents("committers.html")
+content = contents.decoded_content
+soup = BeautifulSoup(content, "html.parser")
+committer_logins = [login.text for login in soup.find_all('div', 
class_='github_login')]
+
+### GET THE CONTRIBUTORS OF THE apache/kafka REPO ###
+n = 10
+repo = g.get_repo("apache/kafka")
+contributors = repo.get_contributors()
+collaborators = []
+for contributor in contributors:
+if contributor.login not in committer_logins:
+collaborators += [contributor.login]
+refreshed_collaborators = collaborators[:n]
+
+### UPDATE asf.yaml ###
+file_path = ".asf.yaml"
+file = repo.get_contents(file_path)
+yaml_content = yaml.safe_load(file.decoded_content)
+
+# Update 'github_whitelist' list
+github_whitelist = refreshed_collaborators[:10]  # New users to be added

Review Comment:
   Will fix that.



##
refresh-collaborators.py:
##
@@ -0,0 +1,44 @@
+import os

Review Comment:
   Wil do.



##
.github/workflows/refresh-collaborators.yaml:
##
@@ -0,0 +1,24 @@
+name: Refresh asf.yaml collaborators every 3 months

Review Comment:
   Will do.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [kafka] stevenbooke commented on a diff in pull request #13842: KAFKA-14995: Automate asf.yaml collaborators refresh

2023-07-18 Thread via GitHub 


stevenbooke commented on code in PR #13842:
URL: https://github.com/apache/kafka/pull/13842#discussion_r1266991049


##
refresh-collaborators.py:
##
@@ -0,0 +1,67 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import os
+from bs4 import BeautifulSoup
+from github import Github
+import yaml
+from datetime import datetime, timedelta
+
+### GET THE NAMES OF THE KAFKA COMMITTERS FROM THE apache/kafka-site REPO ###
+github_token = os.environ.get('GITHUB_TOKEN')
+g = Github(github_token)
+repo = g.get_repo("apache/kafka-site")
+contents = repo.get_contents("committers.html")
+content = contents.decoded_content
+soup = BeautifulSoup(content, "html.parser")
+committer_logins = [login.text for login in soup.find_all('div', 
class_='github_login')]
+
+### GET THE CONTRIBUTORS AND THEIR COMMIT VOLUME OVER THE LAST YEAR TO THE 
apache/kafka REPO ###
+n = 10
+contributors_login_to_commit_volume = {}
+end_date = datetime.now()
+start_date = end_date - timedelta(days=365)
+for commit in repo.get_commits(since=start_date, until=end_date):

Review Comment:
   Yes, you are correct. Will change.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [kafka] stevenbooke commented on a diff in pull request #13842: KAFKA-14995: Automate asf.yaml collaborators refresh

2023-07-18 Thread via GitHub 


stevenbooke commented on code in PR #13842:
URL: https://github.com/apache/kafka/pull/13842#discussion_r1267168635


##
refresh-collaborators.py:
##
@@ -0,0 +1,67 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import os
+from bs4 import BeautifulSoup
+from github import Github
+import yaml
+from datetime import datetime, timedelta
+
+### GET THE NAMES OF THE KAFKA COMMITTERS FROM THE apache/kafka-site REPO ###
+github_token = os.environ.get('GITHUB_TOKEN')
+g = Github(github_token)
+repo = g.get_repo("apache/kafka-site")
+contents = repo.get_contents("committers.html")
+content = contents.decoded_content
+soup = BeautifulSoup(content, "html.parser")
+committer_logins = [login.text for login in soup.find_all('div', 
class_='github_login')]
+
+### GET THE CONTRIBUTORS AND THEIR COMMIT VOLUME OVER THE LAST YEAR TO THE 
apache/kafka REPO ###
+n = 10
+contributors_login_to_commit_volume = {}
+end_date = datetime.now()
+start_date = end_date - timedelta(days=365)
+for commit in repo.get_commits(since=start_date, until=end_date):
+login = commit.author.login

Review Comment:
   Thank you for pointing that out. I will make the appropriate changes.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [kafka] stevenbooke commented on a diff in pull request #13842: KAFKA-14995: Automate asf.yaml collaborators refresh

2023-07-20 Thread via GitHub 


stevenbooke commented on code in PR #13842:
URL: https://github.com/apache/kafka/pull/13842#discussion_r1270123123


##
refresh-collaborators.py:
##
@@ -0,0 +1,83 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import os
+import io
+from bs4 import BeautifulSoup
+from github import Github
+from ruamel.yaml import YAML
+from datetime import datetime, timedelta
+
+### GET THE NAMES OF THE KAFKA COMMITTERS FROM THE apache/kafka-site REPO ###
+github_token = os.environ.get('GITHUB_TOKEN')
+g = Github(github_token)
+repo = g.get_repo("apache/kafka-site")

Review Comment:
   We would not be able to retrieve the organization from the environment for 
"apache/kafka-site" due to the fact that "At the start of each workflow run, 
GitHub automatically creates a unique GITHUB_TOKEN secret to use in your 
workflow. You can use the GITHUB_TOKEN to authenticate in a workflow run.
   
   When you enable GitHub Actions, GitHub installs a GitHub App on your 
repository. The GITHUB_TOKEN secret is a GitHub App installation access token. 
You can use the installation access token to authenticate on behalf of the 
GitHub App installed on your repository. The token's permissions are limited to 
the repository that contains your workflow." See reference 
[here](https://docs.github.com/en/actions/security-guides/automatic-token-authentication#about-the-github_token-secret).
   
   We would only be able to  retrieve the organization from the environment for 
"apache/kafka".



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [kafka] stevenbooke commented on a diff in pull request #13842: KAFKA-14995: Automate asf.yaml collaborators refresh

2023-07-20 Thread via GitHub 


stevenbooke commented on code in PR #13842:
URL: https://github.com/apache/kafka/pull/13842#discussion_r1270124376


##
refresh-collaborators.py:
##
@@ -35,7 +36,10 @@
 contributors_login_to_commit_volume = {}
 end_date = datetime.now()
 start_date = end_date - timedelta(days=365)
+repo = g.get_repo("apache/kafka")
 for commit in repo.get_commits(since=start_date, until=end_date):
+if commit.author is None and commit.author.login is None:

Review Comment:
   Correct, will change.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [kafka] stevenbooke commented on a diff in pull request #13842: KAFKA-14995: Automate asf.yaml collaborators refresh

2023-07-20 Thread via GitHub 


stevenbooke commented on code in PR #13842:
URL: https://github.com/apache/kafka/pull/13842#discussion_r1270163490


##
refresh-collaborators.py:
##
@@ -59,9 +64,20 @@
 yaml_content["github"]["collaborators"] = collaborators
 
 # Convert the updated content back to YAML
-updated_yaml = yaml.safe_dump(yaml_content)
+updated_yaml = io.StringIO()
+yml.dump(yaml_content, updated_yaml)
+updated_yaml_str = updated_yaml.getvalue()
 
-# Commit and push the changes
+# Create a new branch for the changes
+new_branch_name = "update-asf.yaml-github-whitelist-and-collaborators"

Review Comment:
   I look into this more in the coming days and post an update.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [kafka] stevenbooke commented on a diff in pull request #13842: KAFKA-14995: Automate asf.yaml collaborators refresh

2023-08-11 Thread via GitHub 


stevenbooke commented on code in PR #13842:
URL: https://github.com/apache/kafka/pull/13842#discussion_r1291838865


##
refresh-collaborators.py:
##
@@ -59,9 +64,20 @@
 yaml_content["github"]["collaborators"] = collaborators
 
 # Convert the updated content back to YAML
-updated_yaml = yaml.safe_dump(yaml_content)
+updated_yaml = io.StringIO()
+yml.dump(yaml_content, updated_yaml)
+updated_yaml_str = updated_yaml.getvalue()
 
-# Commit and push the changes
+# Create a new branch for the changes
+new_branch_name = "update-asf.yaml-github-whitelist-and-collaborators"

Review Comment:
   @mimaison If the PR is merged the branch will not be deleted. 
   
   I have updated the script to check if branch already exists. If so, the 
script will commit the changes to the branch, and open a pull request with the 
updated `.asf.yaml` file. Otherwise, the script will create a new branch for 
the changes, commit the changes to the new branch, and open a pull request with 
the updated `.asf.yaml` file.
   
   Here is an example of what happens if changes are made to a branch that 
already has a pull request opened and merged:
   
   A pull request is opened for the branch.
   The pull request is merged.
   Changes are made to the branch.
   Another pull request is opened for the branch.
   The second pull request will be based on the latest commit on the branch, 
which will include all of the changes that have been made to the branch since 
the first pull request was merged.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

RE: [GitHub] [kafka] stevenbooke commented on a diff in pull request #13842: KAFKA-14995: Automate asf.yaml collaborators refresh

2023-07-20 Thread miltan 
Hi Team,

Greetings,

We actually reached out to you for Oracle/ IT / SAP / Infor / Microsoft "VOTEC 
IT SERVICE PARTNERSHIP"  "IT SERVICE OUTSOURCING" " "PARTNER SERVICE 
SUBCONTRACTING"

We have very attractive newly introduce reasonably price PARTNER IT SERVICE ODC 
SUBCONTRACTING MODEL in USA, Philippines, India and Singapore etc with White 
Label Model.

Our LOW COST IT SERVICE ODC MODEL eliminate the cost of expensive employee 
payroll, Help partner to get profit more than 50% on each project.. ..We really 
mean it.

We are already working with platinum partner like NTT DATA, NEC Singapore, 
Deloitte, Hitachi consulting. ACCENTURE, Abeam Singapore etc.

Are u keen to understand VOTEC IT PARTNERSHIP offerings? Looping KB 
kail...@votecgroup.com | Partnership In charge |

Let us know your availability this week OR Next week??




-Original Message-
From: stevenbooke (via GitHub) [mailto:g...@apache.org] 
Sent: 21 July 2023 07:48
To: jira@kafka.apache.org
Subject: [GitHub] [kafka] stevenbooke commented on a diff in pull request 
#13842: KAFKA-14995: Automate asf.yaml collaborators refresh


stevenbooke commented on code in PR #13842:
URL: https://github.com/apache/kafka/pull/13842#discussion_r1270163490


##
refresh-collaborators.py:
##
@@ -59,9 +64,20 @@
 yaml_content["github"]["collaborators"] = collaborators
 
 # Convert the updated content back to YAML -updated_yaml = 
yaml.safe_dump(yaml_content)
+updated_yaml = io.StringIO()
+yml.dump(yaml_content, updated_yaml)
+updated_yaml_str = updated_yaml.getvalue()
 
-# Commit and push the changes
+# Create a new branch for the changes
+new_branch_name = "update-asf.yaml-github-whitelist-and-collaborators"

Review Comment:
   I look into this more in the coming days and post an update.



--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the URL above to go 
to the specific comment.

To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

RE: [GitHub] [kafka] stevenbooke commented on a diff in pull request #13842: KAFKA-14995: Automate asf.yaml collaborators refresh

2023-07-20 Thread miltan 
Hi Team,

Greetings,

We actually reached out to you for Oracle/ IT / SAP / Infor / Microsoft "VOTEC 
IT SERVICE PARTNERSHIP"  "IT SERVICE OUTSOURCING" " "PARTNER SERVICE 
SUBCONTRACTING"

We have very attractive newly introduce reasonably price PARTNER IT SERVICE ODC 
SUBCONTRACTING MODEL in USA, Philippines, India and Singapore etc with White 
Label Model.

Our LOW COST IT SERVICE ODC MODEL eliminate the cost of expensive employee 
payroll, Help partner to get profit more than 50% on each project.. ..We really 
mean it.

We are already working with platinum partner like NTT DATA, NEC Singapore, 
Deloitte, Hitachi consulting. ACCENTURE, Abeam Singapore etc.

Are u keen to understand VOTEC IT PARTNERSHIP offerings? Looping KB 
kail...@votecgroup.com | Partnership In charge |

Let us know your availability this week OR Next week??




-Original Message-
From: stevenbooke (via GitHub) [mailto:g...@apache.org] 
Sent: 21 July 2023 06:05
To: jira@kafka.apache.org
Subject: [GitHub] [kafka] stevenbooke commented on a diff in pull request 
#13842: KAFKA-14995: Automate asf.yaml collaborators refresh


stevenbooke commented on code in PR #13842:
URL: https://github.com/apache/kafka/pull/13842#discussion_r1270124376


##
refresh-collaborators.py:
##
@@ -35,7 +36,10 @@
 contributors_login_to_commit_volume = {}  end_date = datetime.now()  
start_date = end_date - timedelta(days=365)
+repo = g.get_repo("apache/kafka")
 for commit in repo.get_commits(since=start_date, until=end_date):
+if commit.author is None and commit.author.login is None:

Review Comment:
   Correct, will change.



--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the URL above to go 
to the specific comment.

To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org