GitCaddy/GitCaddy Server
Limited
2
0
Fork 0
Code Issues Releases 4

feat(actions): add stuck job rescue mechanism
Some checks failed
Build and Release / Build Binaries (amd64, darwin, macos) (push) Has been cancelled
Details
Build and Release / Build Binaries (amd64, linux, linux-latest) (push) Has been cancelled
Details
Build and Release / Build Binaries (amd64, windows, windows-latest) (push) Has been cancelled
Details
Build and Release / Build Binaries (arm64, darwin, macos) (push) Has been cancelled
Details
Build and Release / Build Binary (linux/arm64) (push) Has been cancelled
Details
Build and Release / Integration Tests (PostgreSQL) (push) Has been cancelled
Details
Build and Release / Create Release (push) Has been cancelled
Details
Build and Release / Unit Tests (push) Has been cancelled
Details
Build and Release / Lint (push) Has been cancelled
Details

Browse Source 
Introduce a cron task that rescues waiting jobs stuck due to version-sync issues by bumping the task version for affected scopes. Also bump version after each successful job pick to ensure runners re-poll for remaining waiting jobs. Configurable via STUCK_JOB_TIMEOUT (default: 5 minutes).
This commit is contained in:
David H. Friedel Jr.
2026-01-27 09:00:36 -05:00
parent 8745968b83
commit 43adbaeffe
4 changed files with 57 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
modules/setting/actions.go
View File

@@ -25,6 +25,7 @@ var (
ZombieTaskTimeout time.Duration `ini:"ZOMBIE_TASK_TIMEOUT"`
EndlessTaskTimeout time.Duration `ini:"ENDLESS_TASK_TIMEOUT"`
AbandonedJobTimeout time.Duration `ini:"ABANDONED_JOB_TIMEOUT"`
StuckJobTimeout time.Duration `ini:"STUCK_JOB_TIMEOUT"`
SkipWorkflowStrings []string `ini:"SKIP_WORKFLOW_STRINGS"`
// Runner health settings
@@ -165,6 +166,7 @@ func loadActionsFrom(rootCfg ConfigProvider) error {
Actions.ZombieTaskTimeout = sec.Key("ZOMBIE_TASK_TIMEOUT").MustDuration(10 * time.Minute)
Actions.EndlessTaskTimeout = sec.Key("ENDLESS_TASK_TIMEOUT").MustDuration(3 * time.Hour)
Actions.AbandonedJobTimeout = sec.Key("ABANDONED_JOB_TIMEOUT").MustDuration(24 * time.Hour)
Actions.StuckJobTimeout = sec.Key("STUCK_JOB_TIMEOUT").MustDuration(5 * time.Minute)
if !Actions.LogCompression.IsValid() {
return fmt.Errorf("invalid [actions] LOG_COMPRESSION: %q", Actions.LogCompression)