Cannot read annotations for a previous attempt of a workflow

[timmc-edx]

Select Topic Area

Product Feedback

Body

Summary

The API endpoint https://github.com/openedx/edx-platform/actions/runs/#####/attempts/# only allows accessing annotations for the most recent attempt, no matter which attempt you select. This appears to be a bug in the API and does not match the UI.

Details

I'm trying to get statistics on how often we're seeing a certain failure in GitHub Actions runs. I should be able to use the Check Runs API to get the annotations for the workflow run. The problem is that people often re-run their CI checks when they see this failure, so the most recent attempt of the workflow is a success, and the API apparently only allows you to see the last check run output. (The web UI allows you to see all of the check runs.)

Here's an example:

• https://github.com/openedx/edx-platform/actions/runs/5476653975/attempts/1 shows the target failure. Job python-3.8,django-pinned,lms-5 has the annotation "The self-hosted runner: edx-platform-openedx-ci-runner-deployment-7xdl7-twcfc lost communication with the server." If you switch to attempt # 2 using the UI, you'll see that it succeeded.
• When I ask for https://api.github.com/repos/openedx/edx-platform/actions/runs/5476653975/attempts/1 and .../attempts/2, the results differ in only a few places. Here are the important differences between the attempts:

  @@ -12,1 +12,1 @@
  -  "conclusion": "failure",
  +  "conclusion": "success",
  @@ -295,1 +295,1 @@
  -  "run_attempt": 1,
  +  "run_attempt": 2,
  @@ -321,1 +321,1 @@
  -  "logs_url": "https://api.github.com/repos/openedx/edx-platform/actions/runs/5476653975/attempts/1/logs",
  +  "logs_url": "https://api.github.com/repos/openedx/edx-platform/actions/runs/5476653975/attempts/2/logs",
  @@ -326 +326 @@
  -  "previous_attempt_url": null,
  +  "previous_attempt_url": "https://api.github.com/repos/openedx/edx-platform/actions/runs/5476653975/attempts/1",

  Most notably, they share a check_suite_url, which only shows the successful checks from the second attempt. (They do have different log URLs, but the logs don't show this kind of failure.)
• Asking for all check runs of the commit (https://api.github.com/repos/openedx/edx-platform/commits/ab3d322bc9/check-runs) again shows this most recent check suite, but not the first attempt.

(This is a re-post of https://github.com/orgs/community/discussions/103026 into "Product Feedback" rather than "Questions"—it was originally a "how do I do this" question, but given the lack of responses I think the answer is "you can't".)

[timmc-edx]

Can you show an example of that workaround in use? As far as I can tell, there is no such workaround.