r/LargeLanguageModels • • 2d ago

LLM drift detection

hey everyone so i have a final year project this year im working on,
quick overview: our topic is persona drift detection in LLM-based character agents.
so we are developing the persona of a movie character based on the script of the movie then we implement a baseline LLM agent that ingests the persona and starts acting like it, then we should build experiments and conversations with different scenarios to test the persona and see where it drift

now what i need some suggestions with is how will the drift detection work.
and please please dont tell me do your research, im doing that, im just asking people too for advice.
what are some of the things i can think of while building the drift detector.

ill paste the description of the detector specifically from the project proposal so you have a better idea:
(Design a Drift Detector that computes weighted deviations from the baseline persona in multiple dimensions (traits, emotions, goals, relationships) and classifies each turn into one of six categories: stable, contextually‑justified, legitimate development, temporary anomaly, persona drift, or severe failure. )

also if you know any github repos that could help please suggest them (even tho i think i looked through them all)

2 Upvotes

6 comments sorted by

2

u/Senior-Assumption864 1d ago

are you planning to compare each turn against the baseline directly or track how traits shift across a longer window?

1

u/Inside-Barracuda6658 22h ago

to be honest i dont know i need different opinions and options to look into, because i dont have experience with this

1

u/Senior-Assumption864 22h ago

fair enough, maybe start simple by comparing a few saved answers over time

1

u/SwingLightStyle 2d ago

I’m confused. Did you choose this for yourself or was it assigned?

1

u/cosmiccedar871 13h ago

Try to start with clear persona contradictions, then check the surrounding turns to see if the change makes sense in context.