SynChat

How a message finds its lane

1. Sending returns immediately. The message is stored and delivered to everyone, with no topic yet.
2. A job is queued for its conversation. Jobs for one conversation run strictly one at a time and in order, so two assignments can never race. Different conversations run side by side.
3. The message is embedded in a worker thread with paraphrase-multilingual-MiniLM-L12-v2 (384 dimensions, about 3 ms per message on CPU), so inference never blocks the event loop and message text never leaves the server to be embedded.
4. The assigner decides inside one transaction that holds the conversation’s lock: join an existing topic, or start a new one.
5. A lanes:update event tells every member. In a three-browser test on one machine, the topic chip appeared on the other screens a median of 136 ms after sending.

Two cases skip scoring entirely: an explicit reply joins the topic of the message it replies to, and a message sent from inside a lane stays in that lane.

The assigner: a ranking model

The assigner is a pure function: a new message plus the last 200 messages in, a decision out. There is no database, model or clock inside it, so the server and the offline evaluation run exactly the same code, and the numbers below describe what runs in production.

Every recently active topic gets a score, a weighted sum of features describing how well the message fits it, and “start a new topic” gets a score of its own. The highest score wins. The features cover:

• Meaning: embedding similarity, weighted by how informative the message is, plus shared distinctive words (“Q3”, “villa”).
• Recency: what was said in the last few messages, with nearer messages counting more.
• Author: people tend to stay in their own threads, and a quick follow-up to your own line usually continues it.
• Turn-taking and questions: whose turn it is, which question is still open, and whether “8 works” answers “what time?”.
• Addressing: naming someone who is taking part in a topic.

The weights are fitted by maximum likelihood with L2 regularisation on labelled chats, then calibrated on replayed conversations, where the model has to live with its own earlier decisions.

Short messages like “ok” or “lol” carry no meaning of their own. Every meaning feature is multiplied by an informativeness score: 0 for “ok”, 1 for a full sentence. That score is computed after removing filler words, including the Hindi and Hinglish fillers people mix in, with stretched spellings like “okkk” normalised. For short messages the conversational features decide.

What measuring taught me

• Embedding similarity is weak evidence for chat messages. Given the true earlier lanes, meaning alone picks the right lane for a substantial message about 70% of the time, where chance is 42%. Three larger models did no better, and plain word overlap did as well as any of them. So the embedding is one feature among many, and a small, fast model is the right choice.
• Raw similarity is not comparable across languages. Any two romanised-Hindi messages scored 0.3 to 0.9 whatever they were about, against 0.1 to 0.3 for English pairs, so no single threshold could serve both. SynChat centres similarity per conversation: it subtracts the mean vector of the conversation’s recent messages before comparing, so unrelated messages score about 0 in any language. pgvector does this in SQL, and a test checks that the SQL and the evaluation’s TypeScript agree to four decimal places.

Evaluation

The metrics are the standard ones for conversation disentanglement: variation of information (VI) and one-to-one overlap. As a check on the implementation, the “link every message to the previous one” baseline reproduces the published figures exactly.

Ubuntu IRC benchmark (Kummerfeld et al., ACL 2019), test split:

System	VI	One-to-one
SynChat assigner, weights fitted on the IRC dev split	91.5	78.3
Kummerfeld et al. 2019, feedforward	91.3	75.6
Kummerfeld et al. 2019, linear	88.9	69.5
Baseline: link to previous message	66.1	27.6

These figures use the same code with weights fitted to IRC, not the weights the product ships, and come from a single run. The assigner is behind the paper’s model on exact-match F1.

Casual group chats, the product’s real domain: 20 hand-labelled chats, about a third in mixed Hindi and English. Fifteen were used for tuning and five were held out:

Held-out chats (207 messages)	One-to-one
Whole chat as one lane (baseline)	40.1
Local assigner	58.0
Local assigner + AI tidy-up at each pause	86.5

Caveats: the held-out set is only five chats, written and labelled for this project, and per-chat results vary widely. Topic starts are the local assigner’s weak point: it missed 8 of 15 on the held-out chats. It leans towards over-splitting rather than mixing, because merging two lanes is one action while untangling a mixed lane is many, and the UI makes merging easy.

The optional AI layer

AI features are off in every group until one of its admins turns them on, because they send the group’s messages to an outside provider. The provider is Gemini or Claude, behind one small interface. When they are on:

• After 90 quiet seconds, a model re-sorts the last 80 messages (the tidy-up) and writes a digest for each changed topic: a two-line catch-up, a decided-or-open status and the questions waiting on each member.
• Only first names and short references are sent, never ids, usernames or email addresses. The prompt treats messages as data, not instructions, and each message sits in its own tag with its text escaped, so it cannot break out and pose as structure.
• The output is treated as untrusted. Responses are constrained to a JSON schema and validated again in code. Anything that refers to a message or member it was not given is dropped, a message a person placed by hand is never moved, and an explicit reply stays with the message it answers.
• Turning AI off deletes the summaries, statuses and questions the model produced for that group. The lanes stay as they are.

One finding shaped the design: showing the model the local assigner’s guesses made it worse (74.4% against 85% when sorting from the text alone). The tidy-up therefore hides those guesses for messages that have not been sorted yet.

Reliable messaging

Each conversation carries two counters, both incremented under its row lock. last_seq orders messages; rev counts every change: a new message, an edit, a delete, a reaction, a topic move. A client remembers the highest rev it applied without a gap and, after reconnecting, asks for exactly what changed since. That is one catch-up mechanism for everything.

Sends carry a client-generated UUID, so retries are idempotent: eight concurrent retries of one send produce one row, and a test does exactly that.

Security

• Sockets are authenticated during the handshake from the same httpOnly session cookie as the API, and the handshake checks Origin, which CORS does not cover for WebSockets.
• Every read and write of a conversation’s data goes through one membership gate. A non-member gets the same 404 as a missing id, over HTTP and over the socket.
• Sessions are random tokens stored only as SHA-256 hashes, so logging out really revokes them.
• Zod validates every body, query and path parameter. Rate limits apply per IP, per account and per user, and uploads are typed from their bytes and capped at 5 MB.
• There is no user directory: people are found only by exact username or an invite link.

Testing

245 backend tests run against a real PostgreSQL that is rebuilt from the migrations each time. They cover the membership gate on every route, ordering under concurrent sends, racing idempotent retries, reconnect catch-up, multi-device presence and hostile model output.

Six Playwright walk-throughs drive two or three browsers through every feature area, the last one against the production build. They caught bugs the unit tests had missed, such as the embedding worker failing to load only in the production build.

Limits and next steps

• Without the AI layer, lane accuracy is the thing to improve, especially topic starts. Every correction a member makes is logged as labelled data for exactly that.
• The tidy-up’s early decisions tend to stick. The next experiment is re-sorting the window from scratch and matching lanes back by id.
• It runs as a single server instance. Events are already addressed to per-user rooms and the job queue is an interface, so a Redis adapter is the path to scaling out.