One stream, many viewers

Every open page holds one long-lived GET. Commands change the atom and answer 204. When the atom changes, the server renders each viewer's whole view again and sends it down that viewer's stream, and Datastar's morph applies only the difference. I know. It sounds wildly wasteful. It sounds like the kind of thing a code reviewer circles in red. Measured, it costs about 40 bytes per update, and it never loses the user's focus.

Part 5 left us with a cliffhanger, or at least a stale board. Ada placed an order, and Sam's board, open in another browser, never heard a word about it. That's not a bug in our code; it's how HTTP works. A response can only go to the browser that asked. So the fix is to make every page ask once, at load, with a GET whose response never ends. While that request is open, the server can write to it whenever it feels like it, and commands no longer need to return any HTML at all.

The shape

-- on page load
browser: data-init="@get('/customer/stream')"      -- request stays open
server:  subscribe(viewer, send)                    -- viewer: sid cookie + tab id
server:  send(render(view-for(viewer, state)))     -- the whole #app, right now

-- on any command
browser: @post('/orders')                           -- same request as part 5
server:  swap!(state, handle-command, cmd)          -- same pure function as part 5
server:  respond 204                                -- nothing else
-- the atom's watch runs next
for each subscriber s
  html = render(view-for(s, new-state))
  if html != s.last-sent then send(s, patch-elements(html))

Notice what just happened to a command's response: it's empty. Datastar accepts a 204 and does nothing with it, which is exactly what we want. And command-response, part 5's careful little ledger of "what changed on which page", is gone. Deleted. In its place there's one function per kind of page, view-for, that renders the whole page body from the state, and one rule: after every change, render every view again. That's the entire architecture. You could fit it on an index card and still have room for a doodle.

Holding a stream open in http-kit

Parts 1–5 returned one-shot responses that Datastar read to EOF. Easy. A held-open stream needs a body that doesn't end, and in http-kit that means as-channel.

(hk/as-channel req
  {:on-open  (fn [ch]
               (hk/send! ch {:status 200 :headers (assoc sse/sse-headers "Connection" "keep-alive")}
                         false)                ;; false: keep the channel open
               (on-open #(hk/send! ch % false)))  ;; send-fn: one string in
   :on-close (fn [_ch _status] (on-close))})

(defn stream-lifecycle [hub !state {:keys [sid tabid draft] :as viewer}]
  (let [id (random-uuid)]
    {:on-open  (fn [send-fn]
                 (when tabid                            ;; a customer tab: adopt its draft
                   (swap! !state handle-command {:type :open-tab :sid sid :tabid tabid
                                                 :stream id :draft draft}))
                 (subscribe! hub id send-fn (dissoc viewer :draft)) ;; :last starts nil…
                 (broadcast! hub !state))               ;; …so this sends the whole view now
     :on-close (fn []
                 (unsubscribe! hub id)
                 (when tabid
                   (swap! !state handle-command {:type :close-tab :tabid tabid :stream id})))}))

From lesson06.live (the handler around as-channel is in clojure/src/lesson06/live.clj). stream-lifecycle knows nothing about http-kit, so the tests drive it with a send-fn that conj's onto an atom. No sockets, no sleeps. The tab parts are explained below.

Now, if you're squinting at that and thinking "wait, why are there two :on-opens?", good. That's the right squint. They belong to different layers. http-kit's gets the channel ch; the lifecycle's gets a send-fn. stream-handler is the shim between them: it sends the headers, wraps ch in a closure, and passes the closure on. The figure follows one customer stream through both layers, start to finish.

Solid arrows are calls; dashed arrows are return values. The orange arrows are the only path from the hub to this browser: the send-fn closure, which holds ch. Steps 1 and 2 run once per stream. Step 3 runs on the thread of whatever command changed the state. One subtle bit: :open-tab runs before subscribe!, so its watch does not reach this stream; the broadcast! after subscribe! sends the first view.

http-kit is non-blocking, so an idle stream holds a socket, not a thread. That's the good news. The bad news is that you should check that your server actually tells you when a viewer leaves: on http-kit 2.8.1, :on-close never fired when an SSE client disconnected, so the hub kept dutifully rendering for closed tabs, like a waiter refilling water glasses at an empty table. The course pins 2.9.0-beta4, where it fires. And since on-open sends the whole view at once, the page is correct the moment the stream opens.

So how does a change get from the atom to all those streams? The hub is an atom of subscribers plus a watch on the state atom. That's it:

(defn broadcast! [hub !state]
  (locking hub
    (let [state @!state]                     ;; read inside the lock: see below
      (into []
            (keep (fn [[id {:keys [send-fn viewer last]}]]
                    (try
                      (let [view (html (view-for state viewer))]
                        (when (not= view last)
                          (send-fn (sse/patch-elements view))
                          (swap! hub #(cond-> % (contains? % id)   ;; still subscribed?
                                        (assoc-in [id :last] view)))
                          id))
                      (catch Exception e                  ;; one bad view: see below
                        (binding [*out* *err*] (println "broadcast: skipped" id (ex-message e)))))))
            @hub))))

(add-watch !state ::broadcast (fn [_ _ old new] (when (not= old new) (broadcast! hub !state))))

add-watch calls its function with the old and new value after every change to !state, on the thread that made the change. locking makes broadcasts run one at a time, so two commands that arrive together cannot interleave writes to a stream or race on :last. But (and this is the kind of thing that bites in production at 2 a.m.) one at a time is not the same as in order. If command A changes the state and then command B changes it, B's thread can take the lock first and send the newer view. If A's broadcast then rendered the new value its watch received, it would send the older view, and the page would stay stale until the next change. So broadcast! ignores that value and reads @!state inside the lock. A late broadcast sees the newest state, renders what was already sent, and sends nothing. Late, but harmless.

One more race, and it's sneakier. unsubscribe! runs from http-kit's :on-close, on its own thread, and doesn't wait for the lock. So a tab can close while a broadcast is halfway through the loop. A plain (swap! hub assoc-in [id :last] view) would then cheerfully re-create the departed tab as {:last "…"}, with no send-fn and no viewer, and every later broadcast would throw on it. That's every command failing and every live page frozen, from one badly timed close. The contains? check records :last only for a subscriber that is still there.

And that close race has a cousin, which is what the try is for. Remember where broadcast! runs: inside the watch, inside the swap! of whatever command changed the state. So suppose one viewer's render throws. Say a bug in a view that only Ada's oddball cart ever reaches. Without the try, that one exception doesn't just skip Ada. It stops the loop, so everybody after Ada in the hub misses this change. Then it climbs out of the watch and out of the swap!, and the command answers 500, even though the state already changed. One bad view, everybody's problem. With the try, the failing subscriber is logged and skipped, everyone else gets their view, the command answers 204 as usual, and the next change gives Ada another shot.

The :last check matters more than it looks. Most changes touch one person's view: when Grace adds a mocha to her cart, Ada's page renders to the same string, so nothing is sent to Ada. Ada doesn't care about Grace's mocha, and now neither does Ada's network connection. The figure shows who gets an event for each command.

And here's a little gift. The sold-out command is the one part 4 promised. When Sam marks oat milk sold out, every customer's #drink-options renders with <option value="oat" disabled>Oat (sold out)</option>, so every open customer page gets it, with no code written for it. None! The drop-down is part of the view, and sold-out is state, and that's all it takes. (add-to-cart still rejects a sold-out milk from a page that hasn't updated yet. Trust, but verify.)

The drink choice has to move to the server

Now for the catch. To render Ada's #drink-options, the server needs to know which drink she picked. In part 4 the drink lived only in $drink, and GET /drink-options read it from the request. But a re-render triggered by Sam's command has no request from Ada. Her stream was opened once, at page load, and its signals describe the page as it was then: Datastar sends signals with requests, and the stream makes no new ones until it reconnects. The server is, in effect, looking at a photograph of Ada's page from ten minutes ago.

So here's the rule, and it's a big one: state that shapes server-rendered HTML must live on the server. So in lessons 06–07 and the capstone, all three selects run @post('/draft') on change. :choose-draft fits the draft to the drink and stores it as the tab's draft, [:tabs tabid :draft]. It answers 200 with a patch-signals of whichever of size/milk the fitting changed, or 204. The new option lists arrive in the re-render the swap! triggers. Part 4's GET /drink-options stays a pure query, for pages without a stream.

Why do the size and milk selects post too? Surely the drink is enough? Nope. The server must also render which option is chosen. Predict this one before you peek.

The mechanism is in patchElements.ts, and it's a little devious. Options have no ids, so findBestMatch pairs them in order, and morphNode copies each new option's value and text onto the old one. It sets the selected property only when the presence of the selected attribute differs between old and new. Latte's medium sat at position 2; chai's position 2 is Large, and it keeps the selected property. $size is still medium, valid for chai, so no signal patch arrives and data-bind has nothing to react to. Screen and signal now disagree, and neither one is going to tell you.

So: in a whole-view render, render the true current value of every form control you re-render. drink-options-view takes [drink sold-out {:keys [size milk]}] and puts selected on the stored choice; when chai renders medium as selected, the attribute's presence changes on both options and the morph sets both properties. The stored choice must be real, which is why #size and #milk post too. Text inputs are the exception, happily: the morph leaves .value alone while the value attribute is unchanged, so #customer, rendered with no value, keeps what Ada types.

A cookie identifies a browser, not a tab

Next question: where should the draft be stored? The obvious answer is the sid cookie. It's on every request, and it is the right key for the cart: Ada's cart is Ada's in any tab. But it is the wrong key for the draft, because every tab of a browser sends the same cookie, and the draft belongs to the tab whose drop-downs show it. Obvious answers are like that.

Only the browser knows where one tab ends and another begins, so the page seeds a tab id on <body>:

[:body {:data-signals (json/generate-string draft-signals)
        :data-signals:tabid__ifmissing "crypto.randomUUID()"}
 [:div {:data-init "@get('/customer/stream', {retry: 'always'})"}]
 view]

From lesson06.live/page. tabid has no _, so it rides along on every request: the stream's GET and every command.

Order matters here. Datastar applies the root element first, then its descendants in document order (engine.ts), so $tabid exists before the child's data-init opens the stream. Keep data-init off <body> itself: hiccup sorts attributes, and it would run first. (Alphabetical order, of all things, deciding your boot sequence.) Every page load starts with an empty signal store, so a reload or a duplicated tab gets a fresh id. crypto.randomUUID() exists only in a secure context: https, or localhost in development.

:carts, :orders and (in part 7) :devices stay keyed by sid. Per-tab state lives in :tabs {tabid {:sid … :stream … :draft … :error …}}, and each customer subscriber carries both keys, so view-for renders the browser's cart with the tab's draft.

The server also learns a tab's draft when the stream opens, and this turns out to be a lovely bit of robustness for free. A hidden tab closes its stream and opens a new one when it becomes visible, and a restart forgets everything, but the stream's GET carries the tab's signals as they are now. So opening the stream adopts the draft: :open-tab fits it and stores it before the first render, and :on-close can drop the tab without losing anything. :close-tab checks the stream id, because a reconnect can open the new stream before the old one's close arrives. Networks: never in the order you'd like.

Errors are server state too

A failed command (a sold-out milk, an empty cart) still answers 204. It records the message at [:tabs tabid :error], the re-render draws it with (when error [:p.error error]), and the tab's next command clears it. You might be tempted to use an $_error signal shown with data-text instead. Don't. It would fight the server, which renders that element empty on every re-render, and the server always wins those fights. If the server renders it, the server must know it. (data-ignore-morph, which makes the morph skip an element, is for widgets the client really owns, such as a third-party editor.)

"Sending the whole view on every change is too much"

Alright. Let's have the argument properly, with numbers.

Bytes. A barista view with 12 orders is 2,613 bytes as an SSE event, and the next render is almost all identical. Gzipped on its own, an event is about 540 bytes. Fine, not amazing. But here's the trick: a long-lived stream can share one compression context. Run it through one deflate stream, flushing after each event, and each re-render refers back to the last. Over twelve successive renders the first event was 526 bytes and every later one 39 to 44 bytes. Forty-ish bytes to re-send the whole board. http-kit doesn't compress for you; Datastar's Clojure SDK ships gzip and Brotli "write profiles" for this, and you can wrap the output stream yourself. (Deflate looks back at most 32 KB; Brotli allows larger windows.)

CPU. The cost is subscribers × changes × render time: nothing for a coffee shop. For thousands of viewers, render once per distinct view (all baristas see the same board) and throttle to the latest state every 100 ms, both inside broadcast!. Part 8 compares the whole view with patching only the elements that changed.

Focus and typing are the objection people really lose sleep over. Predict first.

How the morph keeps your place (or: why this isn't innerHTML)

A datastar-patch-elements event with no selector and the default mode outer finds the element with the same id (<main id="app">) and morphs it: it walks the old and new trees together and changes the existing nodes until they match. This is the whole magic trick. A surviving node is the same object, so focus, cursor, typed text, scroll and running transitions survive with it. Reduced to its decisions, patchElements.ts does this:

function morph(old, new)
  persistent = ids present in both trees on the same tag, minus duplicates
  idSets     = for every element: the persistent ids somewhere inside it
  morphChildren(old.parent, [new])

function morphChildren(oldParent, newParent)
  cursor = first old child
  for each newChild in newParent.children
    match = findBestMatch(newChild, from cursor)
      -- 1st choice: same tag, and their id sets share an id (hard match)
      -- 2nd choice: same tag, and the old one has no id or the same id (soft match)
    case match of
      found      -> remove old nodes between cursor and match; morphNode(match, newChild)
      elsewhere  -> if newChild.id is persistent: move that old node here, morphNode it
      none       -> insert a copy of newChild              -- new DOM, no old state
    cursor = next after it
  remove old children left after the cursor

function morphNode(old, new)
  if input: set old.value only if the value ATTRIBUTE differs   -- typing survives
  copy attributes new -> old; remove attributes new lacks
  if not old.isEqualNode(new) then morphChildren(old, new)       -- equal subtrees skipped
  if text node: copy nodeValue if different

Inputs keep what the user typed because morphNode compares the value attribute, not the live value. <input id="customer" data-bind:customer> has no value attribute, so nothing is set and "Ad" stays in the box, cursor and all. Render value="…" from changing server state, though, and you overwrite the user mid-word. They will notice. data-preserve-attr leaves named attributes alone; with data-ignore-morph on both old and new, morphNode returns the old element untouched.

Ids keep identity across moves. A matched element with a persistent id is moved, not rebuilt: when Sam advances order #1, <li id="board-order-1"> moves from "placed" to "making" as the same node. So give ids to anything that holds state and anything that moves, and keep them unique: duplicates drop out of the persistent set, quietly, without so much as a warning.

Switch the figure to replace mode to see the alternative: a fresh copy (replaceWith), correct HTML, but every node is new, so the input is empty and focus is on <body>. That is what an innerHTML re-render does, and it's why so many people believe, deep in their bones, that "send the whole page" can't work. With innerHTML, it can't. With a morph it does.

Reconnects

Let's be realistic: a stream open for an hour will break. Wi-Fi drops, a laptop sleeps, a deploy restarts the server. So what does Datastar do about it? From fetch.ts:

• On a failed connection or stream error, Datastar retries with backoff: retryInterval 1,000 ms, ×retryScaler 2 each time, capped at retryMaxWait 30,000 ms, up to retryMaxCount 10 tries (about 3 minutes in total). A successful connection resets the count; a server retry: line overrides the interval.
• When the server ends the stream cleanly, the default retry: 'auto' does not reconnect; retry: 'always' does. This one is sneaky. A graceful shutdown that ends each stream would leave pages disconnected after a deploy, politely and permanently, so open live streams with @get('/customer/stream', {retry: 'always'}).
• A @get closes when the tab is hidden and reopens when it is visible again (openWhenHidden defaults to false for GET). A phone that locks drops its stream.
• Each retry rebuilds the request, with the signals as they are now.
• The backoff has no jitter. A deploy drops every stream at the same instant, so every page retries on the same schedule, 1 s, 2 s, 4 s, in lockstep, and every reconnect asks for a whole render. That's a stampede you scheduled yourself. Spreading it out (each stream's own random retry: interval, say, or a staggered shutdown) is left as an exercise for the reader.
• Nothing notices a silent stream. Datastar reads the stream with no read timeout. A connection that dies without a close, which mobile networks are especially fond of, looks open to the browser, and the server's :on-close can take minutes to fire. Part 7 has the fix.

And here's where the "wasteful" design pays for itself. Because on-open sends the whole current view, a reconnect needs no event ids and no replay buffer. None of that machinery. The new connection gets the truth, and the morph lands it on whatever the page was showing.

Open the stream from data-init on an element outside #app, so the re-render never touches it. (Sawing off the branch you're sitting on is a classic for a reason.) A second @get to the same URL replaces the first rather than doubling up (Part 8 covers request cancellation).

What the server keeps and what it doesn't

Let's take inventory. The hub holds, per open page, the send function, the viewer (a customer's sid and tabid, or :board), and the last HTML sent. The state atom holds what is shared (orders, stock), per customer (carts), and per tab but rendered (the draft, the error). What the server never renders stays in the browser: typed text, the quantity, focus, scroll. Part 4's split still holds, just refined: state that shapes server-rendered HTML moves to the server, keyed as narrowly as it lives.

Where this goes next

One atom, one watch, one lock. That's the right size for a coffee shop and for a course, and it's a perfectly good way to start a real app. But it's a starting point, not a destination. Serious Datastar apps have grown it in two opposite directions, and there's a third trick that works with either one:

• No connection state at all. Anders Murphy's Hyperlith is stateless on purpose. An action can change what a page shows only by writing to the database and answering 204. Each connection's view is a pure function of the database and the request, re-rendered on every change at a throttled rate, and a view that many users share is rendered once for all of them. Nothing lives in the connection, so a deploy or a missed event loses nothing. The price: per-tab UI state, like an open panel or a wizard step, has to live in the database or in signals.
• A small process per connection. The opposite move keeps per-tab UI state on the server, but gives each connection its own process instead of one shared atom: a session that holds the tab's state and a renderer that draws from the latest state at most once per tick (core.async.flow is one way to build it). A command handler becomes a pure function that returns effects as data. No global lock, a burst of changes collapses into one render, and a slow render or a broken view only hurts its own tab. The price is machinery: a registry of processes, cleanup when a connection goes away, and swapping out the stale session when a tab reconnects. And it's still memory, so a deploy resets it. (Brew Board gets off lightly there for the draft, because every reconnect adopts the tab's signals.)
• A static shell. The page GET returns a cacheable shell, and the stream renders everything, which is what Hyperlith does. "One renderer" stops being a tip you have to remember and becomes a plain fact, because the page GET never renders the view at all. The cost is first paint: nothing shows until the stream opens, and a client without JavaScript, link-preview bots included, sees an empty shell.

cd clojure && clojure -M:ex -d ex -n exercises.lesson06.live-test