Client Retries After a Timeout With TaskFused, the Second Request Joins the First

A phone app sends “import my contacts” to your server. The import takes twelve seconds, but the app gives up after ten, because that’s what its HTTP client does by default. It shows a spinner and sends the request again. Now the server is running two imports of the same contacts into the same account, and the user ends up with every contact twice.

Retries like that are normal and mostly good. Networks drop replies, phones switch from Wi-Fi to mobile data, and a client that never retried would leave users stuck. The trouble is a retry that can’t tell it’s a retry. TaskFused gives the server a way to tell.

The Request Carries Its Own Name

The app makes up a key when the user taps Import, a random ID, and sends it with every attempt of that request, usually in an Idempotency-Key header. A new tap makes a new key. A retry reuses the old one. The server passes the key straight to TaskFused.

func importContacts(w http.ResponseWriter, r *http.Request) {
    account := authenticatedAccount(r) // from your login, never from the request body
    key := r.Header.Get("Idempotency-Key")
    body, _ := io.ReadAll(io.LimitReader(r.Body, taskfused.MaxInputBytes+1))
    if len(body) > taskfused.MaxInputBytes {
        http.Error(w, "request too large", http.StatusRequestEntityTooLarge)
        return
    }

    sub, err := store.Submit(r.Context(), taskfused.Request{
        Namespace: "account-" + account, Handler: "import-contacts", Version: "v2",
        Key: key, Input: body, JoinExisting: true,
    })
    if errors.Is(err, taskfused.ErrConflict) {
        http.Error(w, "this key was used for a different request", http.StatusUnprocessableEntity)
        return
    }
    ...
}

The first request creates the operation. The retry, with the same key and the same body, joins it and gets the same operation ID. There’s still only one import. If a buggy client reuses a key for a different body, TaskFused refuses it as a conflict, so the server can answer with an error instead of guessing.

The namespace comes from the logged-in account. Never take it from anything the client sends. TaskFused keeps namespaces apart, but it doesn’t check who’s asking; that’s the server’s job.

Answering Before the Client Gives Up

The server waits a little, less than the client’s own timeout, and then either answers with the result or says the work is still going:

ctx, cancel := context.WithTimeout(r.Context(), 8*time.Second)
defer cancel()
op, err := store.Wait(ctx, "account-"+account, sub.Operation.ID)
if errors.Is(err, context.DeadlineExceeded) {
    w.WriteHeader(http.StatusAccepted) // still running; ask again with the same key
    json.NewEncoder(w).Encode(map[string]string{"id": sub.Operation.ID})
    return
}

Giving up on the wait never stops the import. The app retries, joins, waits again, and the third or fourth try comes back with the finished result. Retries after the import is done get the stored result immediately, so a user who reopens the app an hour later sees “imported 1,240 contacts” without anything running again.

When the Server Itself Crashes

A retry also helps when the server dies mid-import. The import’s operation was saved before the first request was even acknowledged, and the worker running it holds a lease. When the lease runs out, another worker takes the import over under the same ID, and the client’s next retry joins it. A late result from the dead worker is turned away.

The import itself should be safe to repeat from the start, because a crash after some contacts are written means the next attempt starts again. The simplest way is to write contacts keyed by their ID from the phone, so writing one twice changes nothing.

Lost Replies

The worst moment is a reply lost just after the server saved the work. TaskFused’s tests cut the connection at exactly that point and check that the repeat creates nothing new. The same holds here: the client never knows the first attempt worked, retries with the same key, and simply joins it.

The reference has the joining rules in full.