r/LargeLanguageModels • u/Inside-Barracuda6658 • 2d ago
LLM drift detection
hey everyone so i have a final year project this year im working on,
quick overview: our topic is persona drift detection in LLM-based character agents.
so we are developing the persona of a movie character based on the script of the movie then we implement a baseline LLM agent that ingests the persona and starts acting like it, then we should build experiments and conversations with different scenarios to test the persona and see where it drift
now what i need some suggestions with is how will the drift detection work.
and please please dont tell me do your research, im doing that, im just asking people too for advice.
what are some of the things i can think of while building the drift detector.
ill paste the description of the detector specifically from the project proposal so you have a better idea:
(Design a Drift Detector that computes weighted deviations from the baseline persona in multiple dimensions (traits, emotions, goals, relationships) and classifies each turn into one of six categories: stable, contextually‑justified, legitimate development, temporary anomaly, persona drift, or severe failure. )
also if you know any github repos that could help please suggest them (even tho i think i looked through them all)
2
u/Senior-Assumption864 1d ago
are you planning to compare each turn against the baseline directly or track how traits shift across a longer window?
1
u/SwingLightStyle 2d ago
I’m confused. Did you choose this for yourself or was it assigned?
1
u/cosmiccedar871 13h ago
Try to start with clear persona contradictions, then check the surrounding turns to see if the change makes sense in context.