[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title Stephen Connolly closed an issue as Duplicate Closing as a duplicate of JENKINS-36121. The cache directory stuff caused endless issues as I understand it, so it was dropped (that made the rate limit stuff worse). We may look into a different caching approach that can be unified in the branch-api or scm-api plugin so that all SCM systems can benefit (also the cache needs are different as we know hashes are immutable, so if we have checked hash cafebabedeadbeef for a /Jenkinsfile that answer will never change and can probably be stored more efficiently than a Http request cache Jenkins / JENKINS-38937 GitHub API cache is not working Change By: Stephen Connolly Status: Open Closed Resolution: Duplicate Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title Stephen Connolly commented on JENKINS-38937 Re: GitHub API cache is not working Please verify if this issue is an issue with GitHub Branch Source 2.0.0-beta-1 (available from the experimental update center now or 2.0.0 (available in early January 2017) Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title Kirill Merkushev assigned an issue to Unassigned Jenkins / JENKINS-38937 GitHub API cache is not working Change By: Kirill Merkushev Assignee: Kirill Merkushev Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title Kirill Merkushev commented on JENKINS-38937 Re: GitHub API cache is not working Yep, gh-branch-source creates and uses own gh client, so i can't help in gh-plugin with issue Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title Kirill Merkushev updated an issue Jenkins / JENKINS-38937 GitHub API cache is not working Change By: Kirill Merkushev Component/s: github-branch-source-plugin Component/s: github-plugin Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title
Neil Rhine edited a comment on JENKINS-38937
Re: GitHub API cache is not working
The cache is being written to after all, just in a very minor capacity, before it is 's cleared it is 's only at 16kb and since the rate limit is being hit , I expected to see more usage. I think it is more related to JENKINS-36121 and less this bug after all, but this is my list of plugins:{code}Pipeline: API (workflow-api): 2.4Gradle Plugin (gradle): 1.25Job Configuration History Plugin (jobConfigHistory): 2.15Git client plugin (git-client): 2.1.0_javascript_ GUI Lib: ACE Editor bundle plugin (ace-editor): 1.1Jackson 2 API Plugin (jackson2-api): 2.7.3LDAP Plugin (ldap): 1.12Branch API Plugin (branch-api): 1.11.1Pipeline: SCM Step (workflow-scm-step): 2.2OWASP Markup Formatter Plugin (antisamy-markup-formatter): 1.5MapDB API Plugin (mapdb-api): 1.0.9.0GitHub Pull Request Builder (ghprb): 1.33.1Structs Plugin (structs): 1.5Node and Label parameter plugin (nodelabelparameter): 1.7.2Checkstyle Plug-in (checkstyle): 3.46Ant Plugin (ant): 1.4Icon Shim Plugin (icon-shim): 2.0.3Xvnc plugin (xvnc): 1.24Pipeline: Shared Groovy Libraries (workflow-cps-global-lib): 2.4Job DSL (job-dsl): 1.50Git plugin (git): 3.0.0Groovy Postbuild (groovy-postbuild): 2.3.1Throttle Concurrent Builds Plug-in (throttle-concurrents): 1.9.0Credentials Binding Plugin (credentials-binding): 1.9Audit Trail (audit-trail): 2.2xUnit plugin (xunit): 1.102Pipeline: Step API (workflow-step-api): 2.4GitHub Organization Folder Plugin (github-organization-folder): 1.5_javascript_ GUI Lib: Handlebars bundle plugin (handlebars): 1.1.1Post-Build Script Plug-in (postbuildscript): 0.17Email Extension Plugin (email-ext): 2.47Display URL API (display-url-api): 0.5Jira Issue Updater (jenkins-jira-issue-updater): 1.18CloudBees Amazon Web Services Credentials Plugin (aws-credentials): 1.16Pipeline: Basic Steps (workflow-basic-steps): 2.1build timeout plugin (build-timeout): 1.17.1TAP Plugin (tap): 2.0.1Pipeline: Supporting APIs (workflow-support): 2.8GitHub plugin (github): 1.22.4ShiningPanda Plugin (shiningpanda): 0.23JIRA plugin (jira): 2.2.1Xvfb plugin (xvfb): 1.1.3Pipeline: Build Step (pipeline-build-step): 2.2Artifactory Plugin (artifactory): 2.6.0Email Extension Template Plugin (emailext-template): 1.0Pipeline: Stage Step (pipeline-stage-step): 2.2GIT server Plugin (git-server): 1.7Environment Injector Plugin (envinject): 1.92.1OWASP Dependency-Check Plugin (dependency-check-jenkins-plugin): 1.4.2Pipeline: Job (workflow-job): 2.6Token Macro Plugin (token-macro): 1.12.1Matrix Authorization Strategy Plugin (matrix-auth): 1.4Pipeline Graph Analysis Plugin (pipeline-graph-analysis): 1.1Amazon Web Services SDK (aws-java-sdk): 1.11.37GitHub Integration Plugin (github-pullrequest): 0.1.0-rc12_javascript_ GUI Lib: jQuery bundles (jQuery and jQuery UI) plugin (jquery-detached): 1.2.1Timestamper (timestamper): 1.8.5Folders Plugin (cloudbees-folder): 5.12Dashboard View (dashboard-view): 2.9.10External Monitor Job Type Plugin (external-monitor-job): 1.6EZ Templates (ez-templates): 1.2.1SSH Slaves plugin (ssh-slaves): 1.11Copy Artifact Plugin (copyartifact): 1.38.1Pipeline (workflow-aggregator): 2.3JiraTestResultReporter plugin (JiraTestResultReporter): 2.0.3HTML Publisher plugin (htmlpublisher): 1.11Pipeline: Multibranch (workflow-multibranch): 2.9Port Allocator Plug-in (port-allocator): 1.9-SNAPSHOT (private-61b7bdb7-neil.rhine)Script Security Plugin (script-security): 1.23PMD Plug-in (pmd): 3.45JUnit Plugin (junit): 1.18Pipeline: REST API Plugin (pipeline-rest-api): 2.0SSH Credentials Plugin (ssh-credentials): 1.12Javadoc Plugin (javadoc): 1.4Workspace Cleanup Plugin (ws-cl
[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title Neil Rhine commented on JENKINS-38937 Re: GitHub API cache is not working The cache is being written to after all, just in a very minor capacity, before it is cleared it is only at 16kb and since the rate limit is being hit, I expected to see more usage. I think it is more related to JENKINS-36121 and less this bug after all, but this is my list of plugins: Pipeline: API (workflow-api): 2.4 Gradle Plugin (gradle): 1.25 Job Configuration History Plugin (jobConfigHistory): 2.15 Git client plugin (git-client): 2.1.0 _javascript_ GUI Lib: ACE Editor bundle plugin (ace-editor): 1.1 Jackson 2 API Plugin (jackson2-api): 2.7.3 LDAP Plugin (ldap): 1.12 Branch API Plugin (branch-api): 1.11.1 Pipeline: SCM Step (workflow-scm-step): 2.2 OWASP Markup Formatter Plugin (antisamy-markup-formatter): 1.5 MapDB API Plugin (mapdb-api): 1.0.9.0 GitHub Pull Request Builder (ghprb): 1.33.1 Structs Plugin (structs): 1.5 Node and Label parameter plugin (nodelabelparameter): 1.7.2 Checkstyle Plug-in (checkstyle): 3.46 Ant Plugin (ant): 1.4 Icon Shim Plugin (icon-shim): 2.0.3 Xvnc plugin (xvnc): 1.24 Pipeline: Shared Groovy Libraries (workflow-cps-global-lib): 2.4 Job DSL (job-dsl): 1.50 Git plugin (git): 3.0.0 Groovy Postbuild (groovy-postbuild): 2.3.1 Throttle Concurrent Builds Plug-in (throttle-concurrents): 1.9.0 Credentials Binding Plugin (credentials-binding): 1.9 Audit Trail (audit-trail): 2.2 xUnit plugin (xunit): 1.102 Pipeline: Step API (workflow-step-api): 2.4 GitHub Organization Folder Plugin (github-organization-folder): 1.5 _javascript_ GUI Lib: Handlebars bundle plugin (handlebars): 1.1.1 Post-Build Script Plug-in (postbuildscript): 0.17 Email Extension Plugin (email-ext): 2.47 Display URL API (display-url-api): 0.5 Jira Issue Updater (jenkins-jira-issue-updater): 1.18 CloudBees Amazon Web Services Credentials Plugin (aws-credentials): 1.16 Pipeline: Basic Steps (workflow-basic-steps): 2.1 build timeout plugin (build-timeout): 1.17.1 TAP Plugin (tap): 2.0.1 Pipeline: Supporting APIs (workflow-support): 2.8 GitHub plugin (github): 1.22.4 ShiningPanda Plugin (shiningpanda): 0.23 JIRA plugin (jira): 2.2.1 Xvfb plugin (xvfb): 1.1.3 Pipeline: Build Step (pipeline-build-step): 2.2 Artifactory Plugin (artifactory): 2.6.0 Email Extension Template Plugin (emailext-template): 1.0 Pipeline: Stage Step (pipeline-stage-step): 2.2 GIT server Plugin (git-server): 1.7 Environment Injector Plugin (envinject): 1.92.1 OWASP Dependency-Check Plugin (dependency-check-jenkins-plugin): 1.4.2 Pipeline: Job (workflow-job): 2.6 Token Macro Plugin (token-macro): 1.12.1 Matrix Authorization Strategy Plugin (matrix-auth): 1.4 Pipeline Graph Analysis Plugin (pipeline-graph-analysis): 1.1 Amazon Web Services SDK (aws-java-sdk): 1.11.37 GitHub Integration Plugin (github-pullrequest): 0.1.0-rc12 _javascript_ GUI Lib: jQuery bundles (jQuery and jQuery UI) plugin (jquery-detached): 1.2.1 Timestamper (timestamper): 1.8.5 Folders Plugin (cloudbees-folder): 5.12 Dashboard View (dashboard-view): 2.9.10 External Monitor Job Type Plugin (external-monitor-job): 1.6 EZ Templates (ez-templates): 1.2.1 SSH Slaves plugin (ssh-slaves): 1.11 Copy Artifact Plugin (copyartifact): 1.38.1 Pipeline (workflow-aggregator): 2.3 JiraTestResultReporter plugin (JiraTestResultReporter): 2.0.3 HTML Publisher plugin (htmlpublisher): 1.11 Pipeline: Multibranch (workflow-multibranch): 2.9 Port Allocator Plug-in (port-allocator): 1.9-SNAPSHOT (private-61b7bdb7-neil.rhine) Script Security Plugin (s
[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title
Kirill Merkushev edited a comment on JENKINS-38937
Re: GitHub API cache is not working
You can check that cache is used by following steps:1. Click on "Reregister hooks for all jobs"2. Check that there is a folder in cache:{code}$ du -h /opt/jenkins/org.jenkinsci.plugins.github.GitHubPlugin.cache/796K /opt/jenkins/org.jenkinsci.plugins.github.GitHubPlugin.cache/76c844e5800K /opt/jenkins/org.jenkinsci.plugins.github.GitHubPlugin.cache/{code}3. Set cache size to 0, then save global config4. Now cache should be cleared:{code}$ du -h /opt/jenkins/org.jenkinsci.plugins.github.GitHubPlugin.cache/4.0K /opt/jenkins/org.jenkinsci.plugins.github.GitHubPlugin.cache/{code}5. Set cache to any positive num back and resave global config. Then check steps 1-2.Then please write results . Also please provide full list of plugins (maybe some of them eat your limits without using cache from GitHub plugin's client?)
Add Comment
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title Kirill Merkushev commented on JENKINS-38937 Re: GitHub API cache is not working You can check that cache is used by following steps: 1. Click on "Reregister hooks for all jobs" 2. Check that there is a folder in cache: $ du -h /opt/jenkins/org.jenkinsci.plugins.github.GitHubPlugin.cache/ 796K /opt/jenkins/org.jenkinsci.plugins.github.GitHubPlugin.cache/76c844e5 800K /opt/jenkins/org.jenkinsci.plugins.github.GitHubPlugin.cache/ 3. Set cache size to 0, then save global config 4. Now cache should be cleared: $ du -h /opt/jenkins/org.jenkinsci.plugins.github.GitHubPlugin.cache/ 4.0K /opt/jenkins/org.jenkinsci.plugins.github.GitHubPlugin.cache/ 5. Set cache to any positive num back and resave global config. Then check steps 1-2. Then please write results Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues
[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title Neil Rhine commented on JENKINS-38937 Re: GitHub API cache is not working Any updates here or suggestions to mitigate? Our organization is also constantly hitting the API limit on all of the indexing. Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title
Greg Smith commented on JENKINS-38937
Re: GitHub API cache is not working
With 1.22.2 installed, the problem listed in JENKINS-38935 was resolved. However, I still do not see any directory in my $ {JENKINS_HOME} /org.jenkinsci.plugins.github.GitHubPlugin.cache It does not appear that fixing the problem with loading the configuration load solved the problem with using / writing the cache. Is there some other logging I could provide to determine why the cache file is not being created, or confirm if the cache is being used?
Add Comment
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title Kirill Merkushev started work on JENKINS-38937 Change By: Kirill Merkushev Status: Open In Progress Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title Kirill Merkushev stopped work on JENKINS-38937 Change By: Kirill Merkushev Status: In Progress Open Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title Kirill Merkushev assigned an issue to Kirill Merkushev Jenkins / JENKINS-38937 GitHub API cache is not working Change By: Kirill Merkushev Assignee: Kirill Merkushev Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title Kirill Merkushev updated an issue Jenkins / JENKINS-38937 GitHub API cache is not working Change By: Kirill Merkushev Component/s: github-api-plugin Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title Kirill Merkushev commented on JENKINS-38937 Re: GitHub API cache is not working shipped fix in 1.22.2 - try it please Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title Greg Smith commented on JENKINS-38937 Re: GitHub API cache is not working I did find the name of the directory that should be created: org.jenkinsci.plugins.github.GitHubPlugin.cache Though, it appears that this bug should not be filed under github api plugin, that it should instead be moved to the github plugin – as the cache functions are not managed by the API directly, but the github plugin that wrappers the api plugin. Adding that to the components list - please remove github api plugin if that is indeed the case. Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title Greg Smith updated an issue Jenkins / JENKINS-38937 GitHub API cache is not working Change By: Greg Smith Component/s: github-plugin Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-38937) GitHub API cache is not working
Title: Message Title
Greg Smith created an issue
Jenkins / JENKINS-38937
GitHub API cache is not working
Issue Type:
Bug
Assignee:
Unassigned
Components:
github-api-plugin
Created:
2016/Oct/12 5:58 PM
Environment:
Jenkins 2.19.1 (LTS) GitHub API Plugin 1.7.7 GitHub plugin 1.22.1
Priority:
Major
Reporter:
Greg Smith
After moving to Jenkins new LTS release, we found that we were constantly running into our rate limit with github. To start the investigation of this issue, I contacted github support, and requested any information they could provide about our use of the API over the last week. This was the information github support sent back. (Org and projects redacted)
Hi Greg,
Here are some stats for your account, for the past week.
The top 10 API endpoints being hit:
/repositories/:repository_id/contents/?*196,620 25.632%
/repositories/:repository_id109,038 14.215%
/repositories/:repository_id/collaborators 103,631 13.51%
/user 79,102 10.312%
/ 78,743 10.265%
/repositories/:repository_id/branches 77,789 10.141%
/repositories/:repository_id/pulls 77,474 10.1%
/user/:user_id 14,014 1.827%
/repositories/:repository_id/git/refs/* 11,154 1.454%
/repositories/:repository_id/pulls/:id 10,582 1.38%
Top 10 specific resources:
/ 97,571 10.37%
/user 96,932 10.302%
/repos/{org}/{projectA}/contents/22,552 2.397%
/repos/{org}/{projectB}/contents/ 21,249 2.258%
/repositories/{repoA}/collaborators20,004 2.126%
/repositories/{repoB}/collaborators18,058 1.919%
/users/{org} 17,241 1.832%
/repositories/{repoC}/collaborators
