Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

v3: Refactor attempt creation to be worker requested #1077

Merged
merged 68 commits into from
May 30, 2024
Merged

v3: Refactor attempt creation to be worker requested #1077

merged 68 commits into from
May 30, 2024

Commits on May 1, 2024

  1. WIP worker TaskRunAttempt creation

    @ericallam
    ericallam committed May 1, 2024
    Configuration menu
    Copy the full SHA
    5ed700d View commit details
    Browse the repository at this point in the history

  2. Handling failing task runs that cannot create an attempt for whatever… 

    … reason
    @ericallam
    ericallam committed May 1, 2024
    Configuration menu
    Copy the full SHA
    d1bdd0c View commit details
    Browse the repository at this point in the history

  3. Move the visibility queue stuff into a graphile job

    @ericallam
    ericallam committed May 1, 2024
    Configuration menu
    Copy the full SHA
    e24631e View commit details
    Browse the repository at this point in the history

  4. Fixed task runs with unsanitized queue names

    @ericallam
    ericallam committed May 1, 2024
    Configuration menu
    Copy the full SHA
    f124d94 View commit details
    Browse the repository at this point in the history

  5. “Borrow” the code from alerts PR to get self hosted deployments working

    @ericallam
    ericallam committed May 1, 2024
    Configuration menu
    Copy the full SHA
    3b3b07a View commit details
    Browse the repository at this point in the history

  6. Add an admin API endpoint to get info about the shared marqs queue

    @ericallam
    ericallam committed May 1, 2024
    Configuration menu
    Copy the full SHA
    5ca9e56 View commit details
    Browse the repository at this point in the history

  7. Allow admins to view any project metrics

    @ericallam
    ericallam committed May 1, 2024
    Configuration menu
    Copy the full SHA
    1bba5d5 View commit details
    Browse the repository at this point in the history

  8. start adding lazy attempts to prod

    @nicktrn
    nicktrn committed May 1, 2024
    Configuration menu
    Copy the full SHA
    14992f4 View commit details
    Browse the repository at this point in the history

Commits on May 2, 2024

  1. lazy attempt creation for prod workers

    @nicktrn
    nicktrn committed May 2, 2024
    Configuration menu
    Copy the full SHA
    c75bbfd View commit details
    Browse the repository at this point in the history

  2. resurrect prod stack traces

    @nicktrn
    nicktrn committed May 2, 2024
    Configuration menu
    Copy the full SHA
    f53004d View commit details
    Browse the repository at this point in the history

  3. add exception event to failed run spans

    @nicktrn
    nicktrn committed May 2, 2024
    Configuration menu
    Copy the full SHA
    1919b6f View commit details
    Browse the repository at this point in the history

  4. simplify dependency resumes

    @nicktrn
    nicktrn committed May 2, 2024
    Configuration menu
    Copy the full SHA
    a86b2e6 View commit details
    Browse the repository at this point in the history

  5. fix typecheck

    @nicktrn
    nicktrn committed May 2, 2024
    Configuration menu
    Copy the full SHA
    dcc9745 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' into v3/worker-attempt-creation

    @nicktrn
    nicktrn committed May 2, 2024
    Configuration menu
    Copy the full SHA
    4e18a42 View commit details
    Browse the repository at this point in the history

  7. fix merge

    @nicktrn
    nicktrn committed May 2, 2024
    Configuration menu
    Copy the full SHA
    90153c3 View commit details
    Browse the repository at this point in the history

Commits on May 3, 2024

  1. fresh process for all attempts

    @nicktrn
    nicktrn committed May 3, 2024
    Configuration menu
    Copy the full SHA
    0552a8e View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into v3/worker-attempt-creation

    @nicktrn
    nicktrn committed May 3, 2024
    Configuration menu
    Copy the full SHA
    c7fee76 View commit details
    Browse the repository at this point in the history

  3. always try sigterm first

    @nicktrn
    nicktrn committed May 3, 2024
    Configuration menu
    Copy the full SHA
    1286147 View commit details
    Browse the repository at this point in the history

  4. stop heartbeat timeout on non-inplace replace message

    @nicktrn
    nicktrn committed May 3, 2024
    Configuration menu
    Copy the full SHA
    30b6c2c View commit details
    Browse the repository at this point in the history

  5. add missing ack on checkpoint creation service failure

    @nicktrn
    nicktrn committed May 3, 2024
    Configuration menu
    Copy the full SHA
    4ace3a4 View commit details
    Browse the repository at this point in the history

  6. bypass dequeue for retries with running worker

    @nicktrn
    nicktrn committed May 3, 2024
    Configuration menu
    Copy the full SHA
    78a1e57 View commit details
    Browse the repository at this point in the history

  7. respect retry delays

    @nicktrn
    nicktrn committed May 3, 2024
    Configuration menu
    Copy the full SHA
    1f11944 View commit details
    Browse the repository at this point in the history

Commits on May 7, 2024

  1. crash runs with invalid run status for execution

    @nicktrn
    nicktrn committed May 7, 2024
    Configuration menu
    Copy the full SHA
    ba72219 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into v3/worker-attempt-creation

    @nicktrn
    nicktrn committed May 7, 2024
    Configuration menu
    Copy the full SHA
    d60cf55 View commit details
    Browse the repository at this point in the history

  3. remove debug logs

    @nicktrn
    nicktrn committed May 7, 2024
    Configuration menu
    Copy the full SHA
    5dfaf99 View commit details
    Browse the repository at this point in the history

  4. fix nack message

    @nicktrn
    nicktrn committed May 7, 2024
    Configuration menu
    Copy the full SHA
    93dca36 View commit details
    Browse the repository at this point in the history

Commits on May 8, 2024

  1. fix version locking

    @nicktrn
    nicktrn committed May 8, 2024
    Configuration menu
    Copy the full SHA
    bf79e6b View commit details
    Browse the repository at this point in the history

  2. fresh attempt processes in dev and prod

    @nicktrn
    nicktrn committed May 8, 2024
    Configuration menu
    Copy the full SHA
    6ad28b6 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into v3/worker-attempt-creation

    @nicktrn
    nicktrn committed May 8, 2024
    Configuration menu
    Copy the full SHA
    c6b1a29 View commit details
    Browse the repository at this point in the history

  4. improve handling of ipc timeouts

    @nicktrn
    nicktrn committed May 8, 2024
    Configuration menu
    Copy the full SHA
    0c0eb02 View commit details
    Browse the repository at this point in the history

  5. consider checkpoint failures on cancellation

    @nicktrn
    nicktrn committed May 8, 2024
    Configuration menu
    Copy the full SHA
    091f1d8 View commit details
    Browse the repository at this point in the history

  6. add basic chaos monkey to checkpointer

    @nicktrn
    nicktrn committed May 8, 2024
    Configuration menu
    Copy the full SHA
    2a43a21 View commit details
    Browse the repository at this point in the history

  7. changeset

    @nicktrn
    nicktrn committed May 8, 2024
    Configuration menu
    Copy the full SHA
    e6cea79 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'main' into v3/worker-attempt-creation

    @nicktrn
    nicktrn committed May 8, 2024
    Configuration menu
    Copy the full SHA
    55cd522 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'v3/fix-checkpoint-failures' into v3/worker-attempt-crea… 

    …tion
    @nicktrn
    nicktrn committed May 8, 2024
    Configuration menu
    Copy the full SHA
    2d30f86 View commit details
    Browse the repository at this point in the history

  10. control forced checkpoint simulation via env var

    @nicktrn
    nicktrn committed May 8, 2024
    Configuration menu
    Copy the full SHA
    7a9cd8d View commit details
    Browse the repository at this point in the history

Commits on May 20, 2024

  1. Merge branch 'main' into v3/worker-attempt-creation

    @nicktrn
    nicktrn committed May 20, 2024
    Configuration menu
    Copy the full SHA
    e0423cb View commit details
    Browse the repository at this point in the history

Commits on May 21, 2024

  1. Merge branch 'main' into v3/worker-attempt-creation

    @nicktrn
    nicktrn committed May 21, 2024
    Configuration menu
    Copy the full SHA
    334cf0e View commit details
    Browse the repository at this point in the history

  2. fix merge

    @nicktrn
    nicktrn committed May 21, 2024
    Configuration menu
    Copy the full SHA
    19f6568 View commit details
    Browse the repository at this point in the history

  3. kill old attempt processes before checkpointing

    @nicktrn
    nicktrn committed May 21, 2024
    Configuration menu
    Copy the full SHA
    181641d View commit details
    Browse the repository at this point in the history

  4. detailed perf logging for checkpointing

    @nicktrn
    nicktrn committed May 21, 2024
    Configuration menu
    Copy the full SHA
    0837825 View commit details
    Browse the repository at this point in the history

  5. add coordinator otlp endpoint example

    @nicktrn
    nicktrn committed May 21, 2024
    Configuration menu
    Copy the full SHA
    59a6476 View commit details
    Browse the repository at this point in the history

Commits on May 24, 2024

  1. improve prod run cancellation

    @nicktrn
    nicktrn committed May 24, 2024
    Configuration menu
    Copy the full SHA
    833a0f1 View commit details
    Browse the repository at this point in the history

  2. rename supports lazy attempts migration

    @nicktrn
    nicktrn committed May 24, 2024
    Configuration menu
    Copy the full SHA
    92e257f View commit details
    Browse the repository at this point in the history

  3. fix graceful exit

    @nicktrn
    nicktrn committed May 24, 2024
    Configuration menu
    Copy the full SHA
    1e8743d View commit details
    Browse the repository at this point in the history

  4. fix retry mechanics

    @nicktrn
    nicktrn committed May 24, 2024
    Configuration menu
    Copy the full SHA
    e913c41 View commit details
    Browse the repository at this point in the history

  5. clear paused state before retry

    @nicktrn
    nicktrn committed May 24, 2024
    Configuration menu
    Copy the full SHA
    6aba347 View commit details
    Browse the repository at this point in the history

  6. remove checkpoint image after push

    @nicktrn
    nicktrn committed May 24, 2024
    Configuration menu
    Copy the full SHA
    40a99f8 View commit details
    Browse the repository at this point in the history

  7. crash worker on unrecoverable errors

    @nicktrn
    nicktrn committed May 24, 2024
    Configuration menu
    Copy the full SHA
    5e4b4a3 View commit details
    Browse the repository at this point in the history

  8. refactor unrecoverable error emit

    @nicktrn
    nicktrn committed May 24, 2024
    Configuration menu
    Copy the full SHA
    bc71e2c View commit details
    Browse the repository at this point in the history

  9. switch to do hosted busybox image

    @nicktrn
    nicktrn committed May 24, 2024
    Configuration menu
    Copy the full SHA
    48aadea View commit details
    Browse the repository at this point in the history

  10. increase wait for duration ipc timeout

    @nicktrn
    nicktrn committed May 24, 2024
    Configuration menu
    Copy the full SHA
    127d1aa View commit details
    Browse the repository at this point in the history

  11. add changeset for misc fixes

    @nicktrn
    nicktrn committed May 24, 2024
    Configuration menu
    Copy the full SHA
    02ae3f8 View commit details
    Browse the repository at this point in the history

  12. Merge branch 'main' into v3/worker-attempt-creation

    @nicktrn
    nicktrn committed May 24, 2024
    Configuration menu
    Copy the full SHA
    ef100ad View commit details
    Browse the repository at this point in the history

  13. fix merge

    @nicktrn
    nicktrn committed May 24, 2024
    Configuration menu
    Copy the full SHA
    0ad7b83 View commit details
    Browse the repository at this point in the history

Commits on May 27, 2024

  1. fix retry delay span runId

    @nicktrn
    nicktrn committed May 27, 2024
    Configuration menu
    Copy the full SHA
    8e5b71d View commit details
    Browse the repository at this point in the history

Commits on May 28, 2024

  1. fix dev retries

    @nicktrn
    nicktrn committed May 28, 2024
    Configuration menu
    Copy the full SHA
    ee660a3 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into v3/worker-attempt-creation

    @nicktrn
    nicktrn committed May 28, 2024
    Configuration menu
    Copy the full SHA
    b6e105a View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into v3/worker-attempt-creation

    @nicktrn
    nicktrn committed May 28, 2024
    Configuration menu
    Copy the full SHA
    fed79e7 View commit details
    Browse the repository at this point in the history

  4. improve prod worker logging

    @nicktrn
    nicktrn committed May 28, 2024
    Configuration menu
    Copy the full SHA
    8f378b2 View commit details
    Browse the repository at this point in the history

  5. log checkpoint sizes

    @nicktrn
    nicktrn committed May 28, 2024
    Configuration menu
    Copy the full SHA
    839b349 View commit details
    Browse the repository at this point in the history

  6. add lazy attempts catalog entries

    @nicktrn
    nicktrn committed May 28, 2024
    Configuration menu
    Copy the full SHA
    16a365f View commit details
    Browse the repository at this point in the history

  7. Fixed merge issue: use zodFetch, not wrapZodFetch

    @matt-aitken
    matt-aitken committed May 28, 2024
    Configuration menu
    Copy the full SHA
    d137e4e View commit details
    Browse the repository at this point in the history

  8. Revert "Fixed merge issue: use zodFetch, not wrapZodFetch" 

    This reverts commit d137e4e.
    @matt-aitken
    matt-aitken committed May 28, 2024
    Configuration menu
    Copy the full SHA
    79b47fd View commit details
    Browse the repository at this point in the history

  9. importEnvVars uses wrapZodFetch now

    @matt-aitken
    matt-aitken committed May 28, 2024
    Configuration menu
    Copy the full SHA
    23eb918 View commit details
    Browse the repository at this point in the history

Commits on May 29, 2024

  1. add backwards compat for retries without checkpoints

    @nicktrn
    nicktrn committed May 29, 2024
    Configuration menu
    Copy the full SHA
    0e7e0df View commit details
    Browse the repository at this point in the history

  2. handle more cases of unrecoverable runs

    @nicktrn
    nicktrn committed May 29, 2024
    Configuration menu
    Copy the full SHA
    66c9186 View commit details
    Browse the repository at this point in the history

  3. don't kill the child process if it shouldn't be killed

    @nicktrn
    nicktrn committed May 29, 2024
    Configuration menu
    Copy the full SHA
    2099d91 View commit details
    Browse the repository at this point in the history