← All lab updates

To see a model give in, read the answer it gave before

Last time we found that a model squeezed to 2 bits starts agreeing with a user who insists on a falsehood, and that our caving reading barely saw it. The reason was simple: it read each answer alone. We built a new small reader that sees the previous answer, the user's push and the reply together. On the same held-out conversations it now separates caving from holding almost perfectly, with very few false alarms.