Compacting long transcripts

Compacting long transcripts

SH

long threads are slow to open. what is actually expensive

16:40

Not the render. It is that the whole transcript ships in the payload, so a 400 message thread is a 900KB response before anything paints.

Compaction is the fix the product already implies: keep the last N turns verbatim, replace the rest with a summary the model wrote, and show it above the scrollback so nobody thinks messages were lost.

16:41
SH

how many turns stay

16:50

Enough that a follow-up question still has its referent. In practice the last 20 turns covers almost every 'and the other one' style follow-up, and it holds the payload under 100KB.

16:51

ReUI Chat can make mistakes. Check important info.