Skip to content

Contributing

Contributions are welcome. This page captures the conventions an AI agent or human contributor should follow when working in this repo. The authoritative agent guidance lives in AGENTS.md; this page mirrors the essentials and adds the docs workflow.

What this project is

taskrunner is a Go library (not a service) for distributed, reliable task processing built on Redis Streams. It is consumed by other applications; the public API lives in the runner, contracts, redisstream, and election packages. Module path: github.com/soroosh-tanzadeh/taskrunner. Go 1.24.9 required (see go.mod).

Repository layout

contracts/      Public interfaces & value types (MessageQueue, Message, Lock, errors)
runner/         Core engine: TaskRunner, workers, fetchers, config, timing, tuning
redisstream/    Redis Streams implementation of contracts.MessageQueue
election/       Standalone leader election (Lua-based atomic SET/RENEW/DELETE)
internal/       Unexported helpers -- DO NOT import from outside this module:
                  cache    generic TTL cache
                  locker   redsync-backed DistributedLocker
                  ring     Redis-backed circular buffer for queue metrics
                  safemap  generic RWMutex'd map with Snapshot()
examples/       Runnable examples + tests (simple, scheduler, unique)
cmd/worker/     Local benchmark harness (needs a real Redis on :6379)
vendor/         Vendored deps -- checked in; do NOT hand-edit
.github/        GitHub Actions workflows (Go CI + docs)
docs/           This documentation site (MkDocs Material)
mkdocs.yml      Docs site configuration

Build, test, format, vet

go build ./...          # compile everything (vendor is checked in)
go vet ./...            # static checks -- keep this clean
gofmt -l .              # list unformatted files (ignore vendor/)
go test ./...           # full suite (uses miniredis -- no Redis needed)
go test -race ./...     # race detector; the runner is heavily concurrent
go test ./runner/...    # one package
go test -run TestX -v ./runner/   # single test

The suite relies on github.com/alicebob/miniredis/v2 to run an in-process Redis, so go test ./... works offline. The GitHub Actions CI (.github/workflows/go.yml) additionally runs against a real redis:7.4 container.

Known issue

On master, go test -race ./... currently has one known race in a test (see the "Known: race in one test" note in AGENTS.md). The non-race suite is green. If you touch that test, please also try to resolve it.

Dependencies

Deps are vendored. After changing go.mod/go.sum, run go mod vendor (the CI before_script does this). Do not hand-edit vendor/.

Core deps:

  • github.com/redis/go-redis/v9 -- Redis client.
  • github.com/panjf2000/ants/v2 -- worker pool (PoolWithFunc + Tune).
  • github.com/go-redsync/redsync/v4 -- distributed mutex for unique jobs.
  • github.com/sirupsen/logrus -- logging (import log "...").
  • github.com/google/uuid -- IDs.
  • github.com/Masterminds/semver/v3 -- Redis version gating.
  • github.com/stretchr/testify (+ testify/mock) -- test assertions & mocks.
  • github.com/alicebob/miniredis/v2 -- test Redis.

Coding conventions

  • Standard library style; tabs; idiomatic Go. Keep public APIs small and focused (this is a library).
  • Error types: runner.TaskRunnerError (a string) and contracts.QueueError. Sentinel constants live in runner/task_runner.go and contracts/errors.go. Use errors.Is/errors.As; do not string-compare.
  • Concurrency: state is guarded by sync.RWMutex (safemap, cache) or sync/atomic (status, inFlight, isLeader, counters). The TaskRunner transitions stateInit -> stateStarting -> stateStarted -> stateStopped via atomic.Uint64 CAS.
  • Logging: logrus via import log "github.com/sirupsen/logrus". Structured fields (log.WithFields, WithError) over formatted strings.
  • Runtime errors from fetchers/workers go onto TaskRunner.ErrorChannel() (buffered) rather than being returned.
  • Contexts: long-lived operations take a context.Context for cancellation (Start, StartDelayedSchedule, election loop). The per-message hot path inside a worker uses context.Background() deliberately -- don't "fix" that without understanding the visibility-timeout/heartbeat interaction.

Mocks

Mocks in runner/mock_queue_test.go are generated by mockery v2.40.1. Regenerate if you change contracts.MessageQueue:

mockery --name MessageQueue --output runner --outpkg runner

The generated file is checked in so contributors don't need mockery to build.

Workflow

  1. Branch from master using the naming convention feature/{feature-name} (or fix/..., docs/...).
  2. Make focused changes; run gofmt -w, go vet ./..., go build ./..., and go test ./... before pushing. For concurrency changes, add go test -race ./....
  3. Keep APIs backward-compatible where possible (this is a library). Public signatures in contracts/ affect downstream consumers and the mockery-generated mock.
  4. Open a PR to master; CI runs build + test against a real Redis.
  5. Don't rewrite history on master, don't force-push shared branches, and don't commit secrets or .env.

Documentation changes

The docs site is built from docs/ by MkDocs Material. To preview locally:

# One-time: create a venv and install the doc deps
python -m venv .venv
source .venv/bin/activate
pip install -r requirements-docs.txt

# Serve the site with live reload
mkdocs serve
# -> open http://127.0.0.1:8000

The site is rebuilt and deployed to GitHub Pages automatically on push to master via .github/workflows/docs.yml (only when docs/**, mkdocs.yml, or requirements-docs.txt change).

When adding a new page, also add it to nav: in mkdocs.yml so it appears in the sidebar. The build runs in --strict mode in CI, so broken links or missing nav entries will fail the workflow.

Building docs locally

# Build the static site to ./site (gitignored)
mkdocs build --strict --clean

# The ./site directory is gitignored -- do not commit build output.

Adding a benchmark

Benchmarks live next to the code they exercise. See Benchmarks for the existing suite. A new benchmark looks like:

func BenchmarkMyHotPath(b *testing.B) {
    // setup outside the loop
    b.ResetTimer()
    b.ReportAllocs()
    for i := 0; i < b.N; i++ {
        // the thing under test
    }
}

Run with go test -run '^$' -bench 'BenchmarkMyHotPath' ./runner/....

Releases

  • Tag releases follow semver.
  • Don't commit vendor/ changes that aren't reflected in go.mod/go.sum.
  • The master branch is always buildable and tested.

License

GPL-3.0. See LICENSE. By contributing you agree your contributions are licensed under the same terms.