The prompt said not to ask whether I had eaten, while the code filled a "meals" slot every day. I switched to a short greeting first with the real topic carried into the follow-up conversation, then taught the scheduler when to stay quiet.
Table of Contents
The previous post was about how often proactive messages go out. Once the frequency was under control, the remaining problem was tone: the messages a role in Autarkia sent unprompted read like a checkup. "What did you have for breakfast?" "Do you prefer strong flavours or mild?" "What do you usually do after dinner?" Lunch at noon, then an hour later, what do you want for dinner.
The same logs also had a few that read fine: whether I finally got the noodles I had mentioned, or whether I felt better after saying I was unwell the day before. The difference was obvious. The good ones picked up something specific I had said; the awkward ones had nothing behind them.
01The prompt said don’t ask; the code asked
I read the prompt first, and it was restrained: don’t ask "what are you doing", "have you eaten", "how have you been", and don’t send without a concrete reason. The problem was in the code that assembled the prompt. A "profile completion" module checked six daily slots by time of day (woke up, meals, activity, study, mood, sleep). If one was empty and the user had been silent for twenty minutes, it appended "this round, ask about exactly one profile item" to the instructions and added a health reminder to the available actions.
Two instructions contradicted each other in the same prompt, and the model followed the more specific one. You can’t see this from the prompt files alone; you have to print the full text that actually went to the model on a given round to see who is adding what.
02The top-ranked candidate was silently dropped
The stranger part was one layer down. Each round the system ranks what could be talked about today, with candidates from the day’s study plan, due reviews and open tasks. Across three days of ranking logs, 27 of 28 top items were a plan or a review, mapped to a "planned topic" action, and that action was never in the list of selectable actions. The ranking was discarded every time and the flow fell back to profile completion, which asked about breakfast again.
Even when selected, the topic never reached the step that writes the message: the placeholder passed to generation was [PLANNED_TOPIC_TRIGGER], the template expected [PLANNED_TRIGGER], and the mismatch went into the prompt verbatim. The model call that does the ranking failed 76 times out of 89 and fell back to rule-based ordering, leaving one warning line each time. Put the three together and a person with a study plan, due reviews and open tasks still got asked whether they had eaten.
03Say hi first, bring up the topic later
With the ranking fixed, a proactive message would take the plan and ask outright, "Did you finish this afternoon’s review session?" It had a reason now and still read like a progress check. When a person wants to talk to someone about something, they usually send "you there?" or "what are you up to?" first and steer the conversation once the other side replies.
The flow now has two steps. Besides deciding whether to send, the decision model picks a style: timed reminders and things the user needs to know now are said directly; following up on a plan or checking how something went gets only a short greeting. After the greeting goes out, the topic is saved as an "agenda" item and placed in the main chat prompt. On the user’s first reply the role just responds; after that it looks for a natural opening, and if none appears by the fifth turn it mentions the topic lightly. When the topic has been covered, the user says they are on it, or clearly does not want to talk about it, the model calls a tool to close the agenda.
There was a detour. The sample opener I proposed was "just put the soup on, you there?", and it was rejected flatly: nobody texts like that. Openers are plain "you there?" and "what are you up to?", kept from repeating by checking them against the last few. Trying to make the opener sound natural, I had made it theatrical.
One engineering detail: proactive messages go out over QQ and the user may reply in the phone app, and the two have different conversation IDs. Stored per conversation, the agenda would be lost whenever the user replied from the other client, so it is stored per role. The cost is that the same role’s group chats and role-to-role chats resolve to it too, and those have to be excluded explicitly, or the role would bring up the user’s private business in a group.
04An MDP that could not learn to stay quiet
Which kind of message to send is chosen by a small MDP: the state is time of day, the topic of the last message and whether it got a reply; a reply adds reward, silence subtracts. I assumed it also governed frequency. The Q-table said otherwise: the "do nothing" action was explicitly excluded from learning and sat at its initial 0 forever. Any send action that learned a value slightly above zero kept getting picked. It could learn which kind of message gets more replies, never whether to send this time.
The fix gives "do nothing" a counterfactual reward: whenever a send gets an outcome, "do nothing" in the same state is updated too, +0.5 if ignored, -0.5 if replied to. A send action converges to reply rate minus ignore rate, "do nothing" to half of that with the sign flipped, and they cross exactly at a 50% ignore rate. In any situation where more than half the messages go unanswered, it learns not to send, and there is no extra threshold to tune.
Two bookkeeping bugs turned up along the way. Rewards were recorded against a state computed at reply time, which is the state of the next decision; the state in which the action was actually chosen never got updated. The state is now saved when the message is sent. The second came from code review: every incoming user message added reward to the last proactive message, so five quick replies counted five times, and a message hours later still counted. Now each send’s outcome is recorded once, and only for a reply that comes after the send.
Replaying two weeks of real logs, bucketed by time of day only: one role got replies to more than half its messages both day and evening and keeps sending; another got 2 replies out of 11 in the evening and none out of 2 late at night, and learns to stay quiet in those slots.
Still open: good-morning and good-night messages, farewells when the role’s schedule changes, and timed reminders bypass the MDP, so learning does not affect them; a fallback after long silence can still turn "do nothing" into a message. The backend was down when the new logic was finished, so none of this has run live yet. The replay shows direction only; the actual effect will take a few days of real data.