The code for this runs in http://spark-prs.appspot.com (see https://github.com/databricks/spark-pr-dashboard/blob/1e799c9e510fa8cdc9a6c084a777436bebeabe10/sparkprs/controllers/tasks.py#L137 )
I checked the AppEngine logs and it looks like we're getting error responses, possibly due to a credentials issue: Exception when starting progress on JIRA issue SPARK-27355 ( > /base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/controllers/tasks.py:142 > <https://console.cloud.google.com/debug/fromlog?appModule=default&appVersion=live&file=%2Fbase%2Fdata%2Fhome%2Fapps%2Fs~spark-prs%2Flive.412416057856832734%2Fsparkprs%2Fcontrollers%2Ftasks.py&line=142&logInsertId=5cc1483600029309a7af76d5&logNanos=1556170805012269000&nestedLogIndex=3&project=spark-prs&src=ac>) > Traceback (most recent call last): File > Traceback (most recent call last): > File > "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/controllers/tasks.py", > line 138 > <https://console.cloud.google.com/debug/fromlog?appModule=default&appVersion=live&file=%2Fbase%2Fdata%2Fhome%2Fapps%2Fs~spark-prs%2Flive.412416057856832734%2Fsparkprs%2Fcontrollers%2Ftasks.py&line=138&logInsertId=5cc1483600029309a7af76d5&logNanos=1556170805012269000&nestedLogIndex=3&project=spark-prs&src=ac>, > in update_pr start_issue_progress("%s-%s" % (app.config['JIRA_PROJECT'], > issue_number)) File > start_issue_progress("%s-%s" % (app.config['JIRA_PROJECT'], issue_number)) > File > "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/jira_api.py", > line 27 > <https://console.cloud.google.com/debug/fromlog?appModule=default&appVersion=live&file=%2Fbase%2Fdata%2Fhome%2Fapps%2Fs~spark-prs%2Flive.412416057856832734%2Fsparkprs%2Fjira_api.py&line=27&logInsertId=5cc1483600029309a7af76d5&logNanos=1556170805012269000&nestedLogIndex=3&project=spark-prs&src=ac>, > in start_issue_progress jira_client = get_jira_client() File > jira_client = get_jira_client() > File > "/base/data/home/apps/s~spark-prs/live.412416057856832734/sparkprs/jira_api.py", > line 18 > <https://console.cloud.google.com/debug/fromlog?appModule=default&appVersion=live&file=%2Fbase%2Fdata%2Fhome%2Fapps%2Fs~spark-prs%2Flive.412416057856832734%2Fsparkprs%2Fjira_api.py&line=18&logInsertId=5cc1483600029309a7af76d5&logNanos=1556170805012269000&nestedLogIndex=3&project=spark-prs&src=ac>, > in get_jira_client app.config['JIRA_PASSWORD'])) File > app.config['JIRA_PASSWORD'])) > File > "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", > line 472 > <https://console.cloud.google.com/debug/fromlog?appModule=default&appVersion=live&file=%2Fbase%2Fdata%2Fhome%2Fapps%2Fs~spark-prs%2Flive.412416057856832734%2Flib%2Fjira%2Fclient.py&line=472&logInsertId=5cc1483600029309a7af76d5&logNanos=1556170805012269000&nestedLogIndex=3&project=spark-prs&src=ac>, > in __init__ si = self.server_info() File > si = self.server_info() > File > "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", > line 2133 > <https://console.cloud.google.com/debug/fromlog?appModule=default&appVersion=live&file=%2Fbase%2Fdata%2Fhome%2Fapps%2Fs~spark-prs%2Flive.412416057856832734%2Flib%2Fjira%2Fclient.py&line=2133&logInsertId=5cc1483600029309a7af76d5&logNanos=1556170805012269000&nestedLogIndex=3&project=spark-prs&src=ac>, > in server_info j = self._get_json('serverInfo') File > j = self._get_json('serverInfo') > File > "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/client.py", > line 2549 > <https://console.cloud.google.com/debug/fromlog?appModule=default&appVersion=live&file=%2Fbase%2Fdata%2Fhome%2Fapps%2Fs~spark-prs%2Flive.412416057856832734%2Flib%2Fjira%2Fclient.py&line=2549&logInsertId=5cc1483600029309a7af76d5&logNanos=1556170805012269000&nestedLogIndex=3&project=spark-prs&src=ac>, > in _get_json r = self._session.get(url, params=params) File > r = self._session.get(url, params=params) > File > "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", > line 151 > <https://console.cloud.google.com/debug/fromlog?appModule=default&appVersion=live&file=%2Fbase%2Fdata%2Fhome%2Fapps%2Fs~spark-prs%2Flive.412416057856832734%2Flib%2Fjira%2Fresilientsession.py&line=151&logInsertId=5cc1483600029309a7af76d5&logNanos=1556170805012269000&nestedLogIndex=3&project=spark-prs&src=ac>, > in get return self.__verb('GET', url, **kwargs) File > return self.__verb('GET', url, **kwargs) > File > "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", > line 147 > <https://console.cloud.google.com/debug/fromlog?appModule=default&appVersion=live&file=%2Fbase%2Fdata%2Fhome%2Fapps%2Fs~spark-prs%2Flive.412416057856832734%2Flib%2Fjira%2Fresilientsession.py&line=147&logInsertId=5cc1483600029309a7af76d5&logNanos=1556170805012269000&nestedLogIndex=3&project=spark-prs&src=ac>, > in __verb raise_on_error(response, verb=verb, **kwargs) File > raise_on_error(response, verb=verb, **kwargs) > File > "/base/data/home/apps/s~spark-prs/live.412416057856832734/lib/jira/resilientsession.py", > line 57 > <https://console.cloud.google.com/debug/fromlog?appModule=default&appVersion=live&file=%2Fbase%2Fdata%2Fhome%2Fapps%2Fs~spark-prs%2Flive.412416057856832734%2Flib%2Fjira%2Fresilientsession.py&line=57&logInsertId=5cc1483600029309a7af76d5&logNanos=1556170805012269000&nestedLogIndex=3&project=spark-prs&src=ac>, > in raise_on_error r.status_code, error, r.url, request=request, response=r, > **kwargs) JIRAError: JiraError HTTP 403 url: > https://issues.apache.org/jira/rest/api/2/serverInfo text: > CAPTCHA_CHALLENGE; login-url=https://issues.apache.org/jira/login.jsp > r.status_code, > error, r.url, request=request, response=r, **kwargs) > JIRAError: JiraError HTTP 403 url: > https://issues.apache.org/jira/rest/api/2/serverInfo > text: CAPTCHA_CHALLENGE; login-url= > https://issues.apache.org/jira/login.jsp It looks like ASF JIRA was throwing a captcha challenge at us, so I used the credentials to manually log in and complete the challenge. Hopefully that's enough to fix things, but to prevent re-occurrence we might need to change the login credential type from username + password to instead use OAuth tokens. On Wed, Apr 24, 2019 at 10:42 PM Hyukjin Kwon <gurwls...@gmail.com> wrote: > Can anyone take a look for this one? OPEN status JIRAs are being rapidly > increased (from around 2400 to 2600) > > 2019년 4월 19일 (금) 오후 8:05, Hyukjin Kwon <gurwls...@gmail.com>님이 작성: > >> Hi all, >> >> Looks 'spark/dev/github_jira_sync.py' is not running correctly somewhere. >> Usually the JIRA's status should be updated to "IN PROGRESS" when >> somebody opens a PR against a JIRA. >> Looks now it only leaves a link and does not change JIRA's status. >> >> Can someone else who knows where it's running can check this? >> >> FWIW, I check every PR and JIRA almost every day but ever since this >> happened, this makes (at least to me) duplicately check the JIRAs. >> Previously, if I check all the PRs and JIRAs, they were not duplicated >> because JIRAs having PRs have different status, "IN PROGRESS" but now all >> JIRAs have "OPEN" status. >> >> Thanks. >> >
