HTML over the wire

A button says @get('/hello'). The server answers with an event stream that carries HTML. Datastar finds the element with the matching id and patches it in place. And that's it. No, really: that loop is the whole framework. The other seven parts are about using it well, which, as with most simple things, turns out to be where all the interesting arguments are.

Let's start with the way most web apps built in the last ten years split up the work, because you've almost certainly built one. The server sends JSON, and a JavaScript application in the browser turns the JSON into DOM. That application has its own state, its own router, its own build, and its own copy of half the business rules, which it keeps lovingly out of sync with the other half. Datastar splits it differently. The server renders HTML, the same way a server rendered HTML in 2005. (Yes, 2005. Stick with me.) And a single script on the page (11.89 KiB, according to the v1.0.4 README, which is smaller than the hero image on most landing pages) does two small jobs. It reads data-* attributes and wires them to a small reactive store, which is part 3. And it turns server responses into DOM patches, which is this part and the next.

One more thing before we dive in. This series is about using Datastar from Clojure without an SDK. Datastar publishes an official Clojure SDK, and it's good. Use it in production if you like! But the wire format is a few dozen lines of string building, and once you've written those lines yourself, nothing the browser does is a mystery anymore. That's a surprisingly nice feeling, and it's the feeling we're after. Part 2 writes those lines. This part writes the smallest version that works.

The toy: Brew Board

Every part builds toward one app, Brew Board: a coffee counter with six drinks. Customers (Ada, Grace, and Linus, because of course) configure a drink, add it to a cart, and place an order. The barista (Sam) sees every order on a live board and shepherds each one through placed → making → ready. Prices are in cents, because floating-point money is how you end up explaining yourself to an accountant:

Size adds 0 (single, small), 50 (medium), or 100 (double, large). Oat and almond add 60. So a medium oat latte is 450 + 50 + 60 = 560 cents. Remember that latte. It's going to follow us around.

We'll use all of that later. For now, part 1 needs exactly one button and one div. Humble beginnings.

The page

A Datastar page is ordinary HTML with one script tag. No bundler, no build step, no node_modules folder quietly eating your disk. We render it with hiccup:

(def datastar-cdn
  "https://cdn.jsdelivr.net/gh/starfederation/datastar@v1.0.4/bundles/datastar.js")

(defn page []
  (str "<!DOCTYPE html>"
       (h/html
        [:html
         [:head
          [:meta {:charset "utf-8"}]
          [:title "Part 1: HTML over the wire"]
          [:script {:type "module" :src datastar-cdn}]]
         [:body
          [:h1 "HTML over the wire"]
          [:div#greeting "Nobody has said hello yet."]
          [:button {:data-on:click "@get('/hello')"} "Say hello (SSE)"]
          " "
          [:button {:data-on:click "@get('/hello-html')"} "Say hello (text/html)"]]])))

Two attributes do all the work here. id="greeting" names the element the server will replace. data-on:click="@get('/hello')" tells Datastar: when somebody clicks this button, evaluate the expression @get('/hello'). The @ marks an action. Under the hood, Datastar rewrites @get('/foo') into a call to its fetch plugin before it evaluates the expression. (If you're the kind of person who likes to see the trick, the rewrite rules are listed in a comment in engine.ts.)

Now a couple of hiccup quirks, because there are always quirks. Hiccup renders a keyword with a colon in it, :data-on:click, as the attribute name data-on:click, which is exactly what you want. Use single quotes inside the expression. Hiccup escapes them as ', which the browser decodes when it parses the attribute, so view-source looks a little odd but Datastar sees @get('/hello'). Don't panic when you see it. And here's one hiccup trap you should know about right now, before it costs you an afternoon: never write true as an attribute value. Hiccup renders {:data-bind:drink true} as data-bind:drink="data-bind:drink", and Datastar actually reads that value. Write "" for an attribute that has no value.

The click, step by step

So what actually happens between the click and the new text? Let's slow it way down. Step through it below. Each step is what the browser or the server actually sees, nothing dressed up.

Three details in that request are easy to miss, and every one of them matters later.

The request always carries signals. Even now. The page has no signals yet, but look at the URL anyway: it's still /hello?datastar=%7B%7D. For a GET, Datastar serializes the signals it is allowed to send (part 3 covers which ones) as JSON and tucks the JSON into a query parameter named datastar. For POST, PUT, and PATCH it sends the same JSON as the request body. With no signals, the JSON is {}, which is what that %7B%7D is, wearing its URL-encoding costume.

The request identifies itself. Every Datastar request has the header Datastar-Request: true and Accept: text/event-stream, text/html, application/json. Think of it as a name tag. Part 5 uses the first header as a cheap guard on routes that change state.

The response is a stream, even for one event. This is the one that surprises people. The server answers with Content-Type: text/event-stream, and the body is a single event:

event: datastar-patch-elements
data: elements <div id="greeting"><strong>Hello from the server!</strong> Visit number 1.</div>
                                          -- a blank line ends the event

That's the entire format, and you could teach it to a houseplant. The event has a type line, then one data: line per line of payload, then a blank line. Each data line starts with a key (elements), a space, and the value. The browser reads the event, parses the HTML, notices that its top-level element has id="greeting", finds the element with that id in the page, and morphs the old element into the new one. Morphing means it changes only what differs, and leaves everything else alone. Part 6 leans on that heavily. Very heavily.

The handler

Here's the part that I find genuinely delightful. A one-shot SSE response is an ordinary Ring response. The body is the event text, and the browser reads it until the connection closes. That's it. We don't need an async server, a channel, or a library for that:

(defn sse-event
  "Format one SSE event: an event line, one data line per entry, then a blank line."
  [event-type data-lines]
  (str "event: " event-type "\n"
       (string/join (map #(str "data: " % "\n") data-lines))
       "\n"))

(defn greeting-html
  "The element the server sends back. Its id tells Datastar where it goes."
  [visits]
  (str (h/html [:div#greeting [:strong "Hello from the server!"] (str " Visit number " visits ".")])))

(defn hello-response
  "A one-shot SSE Ring response that patches #greeting."
  [visits]
  {:status  200
   :headers {"Content-Type"  "text/event-stream"
             "Cache-Control" "no-cache"}
   :body    (sse-event "datastar-patch-elements" [(str "elements " (greeting-html visits))])})

And that's the whole server half. Go ahead and scroll back up; I'll wait. The lesson routes with a plain case on [request-method uri], and the /hello branch is (hello-response (swap! visits inc)), which counts clicks so you can watch each patch land. (Part 5 swaps the case for reitit, once there are enough routes to need it. Not before.) Best of all, hello-response is a plain function from a number to a response map, so the test just calls it directly and checks the string. No server needs to be running. No mocks, no fixtures, no ceremony.

Not every response has to be a stream

Now, having just told you the response is a stream, let me walk it back a little. Datastar looks at the response's Content-Type before it decides what to do with it. Remember, the request said it accepts three types. The status code matters too: only a 200 gets applied.

So the lesson's second button, @get('/hello-html'), gets the very same patch as plain old HTML:

(defn hello-html-response
  "The same patch as a text/html response. Datastar morphs each top-level element by id."
  [visits]
  {:status  200
   :headers {"Content-Type" "text/html"}
   :body    (greeting-html visits)})

The browser ends up in exactly the same state. Which raises the obvious question: why bother with SSE at all? Here's why. One HTML response can carry one patch. One event stream can carry many, of different kinds, over time. Part 3 sends an element patch and a signal patch in the same response. Part 6 holds one stream open for the whole life of the page. So use text/html when the answer really is one element, and use the stream for everything else. The rest of this series mostly uses the stream, for a boring and excellent reason: the same writer then covers every case.

What the browser didn't do

I want you to look at the list of things that didn't happen during that click, because it's the most important list in this part. No JSON was parsed into a model. No component tree re-rendered. No client-side router changed. The browser had no idea what a "greeting" even is. It had an id and some HTML, and frankly it was happy that way. Everything that knows about greetings (and later about drinks, carts, and orders) lives in exactly one place: the Clojure process.

That has a consequence that this series comes back to near the end, and it's a big one. Anything that can load a web page and run fetch can host this UI unchanged, and the server decides what that UI looks like on every single response. File that away. You'll want it when you get to part 7.

It also has a cost, and it's only fair to name it now, before you fall in love. Every interaction that needs the server's knowledge pays a round trip. A drop-down whose options depend on stock (part 4) can't fill itself without asking. There's no getting around that, and anyone who tells you otherwise is selling something. What Datastar does do, in part 3, is handle the interactions that don't need the server at all (a counter, a computed total, showing and hiding things) entirely in the browser, with signals.

cd clojure && clojure -M:ex -d ex -n exercises.lesson01.hypermedia-test