Calibrating how long a role takes to reply against real chat logs, the first pass pooled more than a year of data and got the conclusion backwards. With the method fixed, it turned out the system was missing an entire middle tier.
Autarkia’s roles answer messages on QQ and in the app. The early rule was simple: apart from deferring while asleep or busy, every message got an immediate reply. Functionally fine; in practice it felt like customer support — always instant, always answering.
So I added “read, no reply”: low-information messages (acknowledgements, sign-offs, a lone emoji) may be skipped; light sharing goes into a revisit queue to be answered later; questions, requests for help, and long messages must be answered. The first parameters were guesses: skip 0.45, revisit 0.2, revisit delay 10 to 90 minutes.
01The first calibration
Guessed parameters need to match a real rhythm. I took a real chat log spanning more than a year, used only timestamps and message types — never the content — and measured the gap from “a block of consecutive messages from the other side” to “the first reply after it”. The median was eight seconds; only 3.5% of replies took longer than ten minutes. The conclusion looked clear: mostly instant, with ignoring the exception. The parameters came down accordingly: skip 0.15, revisit 0.10.
That conclusion was wrong. More than a year of messages had been treated as one sample, and rhythms change. Recomputed month by month: the earliest month had a five-second median with 0.7% over ten minutes; three months later, sixteen seconds and 10.5%; in the most recent month, a 21-second median, and 40% of questions saw no response within a minute. “Median eight seconds” described the earliest period, whose volume flattened everything after it.
The rule now is to segment first, decide which segment represents “now”, and use only that one. Behavioural parameters especially: a person’s reply habits can change a lot in a year.
One quantity can’t be observed at all: “never replied”. No reply means no message, and the log records nothing. All I could do was treat the rate of very late replies as an upper bound and pick a conservative value. The revisit delay ceiling went up to twelve hours, sampled log-uniformly — dense at the short end, sparse in the tail. Sampled shares came out at 26% for 10–30 minutes, 16% for 30–60, 26% for 1–3 hours, and 32% for 3–12; the measured shares were 34%, 19%, 19%, and 28%.
02The missing middle tier
With the method fixed it still felt wrong: either instant, or ignored for half a day. Splitting the most recent month into three bands showed why. The real rhythm was 55% within a minute, 35% within one to ten minutes, and 9.5% after ten minutes or the next day. The system had only two modes: reply now, or go into the revisit queue, whose earliest slot is ten minutes out. The middle 35% — “not near the phone, sees it a few minutes later” — simply didn’t exist.
The added tier is an idle short delay: when the role is idle, with probability 0.35 it waits 60 to 180 seconds before replying. Messages that arrive during the wait join the same batch and get a single reply, which matches the real pattern of the other side sending 2.7 messages per block and receiving 1.7 back.
A 200-run Monte Carlo over the most recent month’s 1,535 real messages: before the change, 90.9% immediate, 0% within one to ten minutes, 5.3% after ten minutes, 3.8% never; after it, 59.1%, 31.9%, 5.2%, 3.8%; the real figures were 55.1%, 35.4%, 9.5%, 0%.
03A few deliberate choices
- The short delay is capped at 180 seconds. Any longer and the client starts to look disconnected; the tail beyond ten minutes belongs to the revisit queue.
- The probability of “still not replying when the revisit comes due” defaults to 0. Raising it to 0.2, 0.3, or 0.5 pushes “never replied” to 4.9%, 5.4%, 6.5% — further from the real 0.
- “Not replying when the revisit comes due” used to be left to the model emitting a skip marker; across 24 samples it never fired once. A model almost never chooses silence on its own, so it has to be executed in the system layer.
- The hint the model gets after a delay states only facts, such as “not near the phone”, never excuses like “just saw this” or “was busy”; otherwise it opens every reply with an explanation.
Some of it still doesn’t line up: the over-ten-minutes band simulates at 5.2% against a real 9.5%, and “never replied” at 3.8% against a real 0. The latter comes from the quantity that can’t be observed, and I don’t yet have a better way to calibrate it.