releases

First TaskFused Release Runs Duplicate Requests Once and Survives Worker Crashes

Some work gets asked for more than once. Someone double-clicks Export. A client times out and sends the request again while the first one is still running. Ten dashboards open at nine and all want last month’s report. Each copy costs the same CPU, the same API credits and the same database time as the first, and when copies overlap they can trip over each other.

TaskFused 0.1 is a small Go engine that runs that work once. You name each piece of work. The first request with a name creates an operation, saved to a local SQLite file before anyone hears about it. Equal requests after that join the same operation, wait with it, and get the same result. Once the work is done, later requests get the stored result straight away.

It doesn’t matter where the copies come from. They can be goroutines in one program, separate processes on the same machine, or a retry that arrives after the first process crashed. They all find the same operation in the file.

A terminal running TaskFused: the same report request submitted twice, the second one joining the first operation, the worker running it once, and wait returning the shared result.

What You Get

  • Joining. Equal requests share one operation and one result. The same name with different input is refused, so different work can never share a result by mistake.
  • Durable operations. Each step is one SQLite transaction, flushed to disk before TaskFused reports it.
  • Leases and recovery. Workers hold each attempt on a lease and renew it while they work. If a worker dies, the lease runs out and another worker takes over under the same operation ID. A late result from the dead worker is turned away.
  • Retries. Failed attempts wait, then run again, with the wait doubling each time. An operator can reopen failed work with a few more attempts.
  • Shared waiting. Any number of callers can wait on one operation. One giving up never cancels the work for the others.
  • Cancel, limits and pruning. Cancel unfinished work, cap how much a namespace keeps, and remove old results when you need the space.

Three Ways In

Go programs import the package and run their own handlers. Everything else can use the taskfused command or the HTTP API it serves on the loopback address, along with a small read-only status page:

taskfused submit -db app.db -namespace demo -handler echo -version v1 \
  -key report-42 -input 'hello' -join
taskfused wait -db app.db -namespace demo -id op_...

Run the submit line twice and the second reply says "joined": true with the same ID. The quick start goes through it step by step.

What It Isn’t

TaskFused 0.1 runs on one host. The SQLite file stays on a local disk, and the HTTP API only listens on 127.0.0.1 or [::1]. It runs each operation at least once: if a worker crashes after doing the work and before saving the result, the work runs again. Anything with outside effects, such as a payment, should use the operation ID as its own idempotency key. And it doesn’t guess which requests are equal. You decide by choosing the name.

Tested by Breaking It

The test suite kills real worker processes in the middle of a job and checks that another worker finishes the same operation. It forces storage failures in 13 kinds of write, both mid-statement and at commit, and checks each time that nothing was half-saved and that the same action then works. In the recorded demo, 100 identical requests produced 1 operation and 1 run, and a killed worker’s job finished on a new worker about 1.1 seconds after the kill. How it was tested has the details.

Get It

Binaries for Linux, macOS and Windows are on the download page, and the Go package is go get github.com/taskfused/taskfused@latest. Everything is open source under the Apache License 2.0, on GitHub.