Status not updated sometimes

[asmeurer]

It's because the added/deleted comment fails with

2020-02-25T21:16:11.385238+00:00 app[web.1]: Received pull_request event with delivery_id=0b46c900-5814-11ea-9fd3-9f90b3d5839b
2020-02-25T21:16:11.385406+00:00 app[web.1]: PR #16149 was edited.
2020-02-25T21:16:11.688004+00:00 heroku[router]: at=info method=POST path="/" host=sympy-bot.herokuapp.com request_id=f17fe55f-7626-4676-aac6-9ac190f00f3c fwd="140.82.115.249" dyno=web.1 connect=2ms service=308ms status=500 bytes=330 protocol=https
2020-02-25T21:16:11.686004+00:00 app[web.1]: Error handling request
2020-02-25T21:16:11.686013+00:00 app[web.1]: Traceback (most recent call last):
2020-02-25T21:16:11.686013+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 390, in start
2020-02-25T21:16:11.686014+00:00 app[web.1]: resp = await self._request_handler(request)
2020-02-25T21:16:11.686014+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/aiohttp/web_app.py", line 366, in _handle
2020-02-25T21:16:11.686015+00:00 app[web.1]: resp = await handler(request)
2020-02-25T21:16:11.686015+00:00 app[web.1]: File "/app/sympy_bot/webapp.py", line 40, in main_post
2020-02-25T21:16:11.686016+00:00 app[web.1]: result = await router.dispatch(event, gh)
2020-02-25T21:16:11.686016+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/gidgethub/routing.py", line 80, in dispatch
2020-02-25T21:16:11.686016+00:00 app[web.1]: await callback(event, *args, **kwargs)
2020-02-25T21:16:11.686017+00:00 app[web.1]: File "/app/sympy_bot/webapp.py", line 68, in pull_request_edited
2020-02-25T21:16:11.686017+00:00 app[web.1]: await pull_request_comment(event, gh)
2020-02-25T21:16:11.686018+00:00 app[web.1]: File "/app/sympy_bot/webapp.py", line 89, in pull_request_comment
2020-02-25T21:16:11.686018+00:00 app[web.1]: com = await gh.getitem(commit['url'])
2020-02-25T21:16:11.686019+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/gidgethub/abc.py", line 99, in getitem
2020-02-25T21:16:11.686019+00:00 app[web.1]: jwt=jwt, oauth_token=oauth_token)
2020-02-25T21:16:11.686020+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/gidgethub/abc.py", line 82, in _make_request
2020-02-25T21:16:11.686020+00:00 app[web.1]: data, self.rate_limit, more = sansio.decipher_response(*response)
2020-02-25T21:16:11.686020+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/gidgethub/sansio.py", line 326, in decipher_response
2020-02-25T21:16:11.686021+00:00 app[web.1]: raise exc_type(*args)
2020-02-25T21:16:11.686095+00:00 app[web.1]: gidgethub.BadRequest: Not Found

[asmeurer]

For PR sympy/sympy#16149 It is trying to load https://api.github.com/repos/sympy/sympy/commits/413dfe7ba58d0ddac413e350a6961eb831ce759f, which is for sympy/sympy@413dfe7. But this is for the wrong PR. The SHA in the commit object is correct be528926594053c2abe761922fb832e34e047119. This must be a bug in the GitHub API.

[asmeurer]

I will push a workaround to use the SHA for now. We can revert it later if the bug is fixed.

[asmeurer]

That didn't fix it. For some reason, it can't access the url for the commit. I am going to just push a workaround for now that ignores it when this happens. Hopefully it is just a bug in the GitHub API and will go away.

[asmeurer]

Also after #83 the release notes stuff will all happen before any added/deleted stuff, so if this were to happen there, it would not break the status.

[oscarbenjamin]

I think github has been partially broken for a while today

[asmeurer]

Yeah, I'm hoping this is somehow related to the outage that happened earlier. The outage is marked fixed https://www.githubstatus.com/. I'll keep an eye on the Heroku logs and if I don't see any warnings in a while we can revert this. This isn't the first time that weird bugs in the GitHub API have caused issues for the bot (e.g., #49).

We might also look into having some kind of notification if the bot fails. Heroku offers a notification service, but only for their paid plan (but it might be worth it).

We can also probably be writing the bot in a more robust way somehow. I'm not an expert in writing this kinds of applications.