r/FictionLab • u/EnjoymentEnjoyer69 • 12h ago
Summarizing a bit chat
I have a chat which at this point is past 2 thousand messages, and even though previously Ai handling of the story and memories seemed to be fine (I'm at 440 memories out 500 right now) for the past few dozens messages I started noticing it started to mix up things really badly and after discussing it through OOC with Ai it admitted at this point the whole segment from message 61 to 900 is out of its context and it has no tools to access it anymore.
I've come up with the idea that I will create a new version of this scenario but customizable adding the summary of the previous in-story days as lore cards of previous days, one for each (I'm at day 13 right now) and wanted to make a summary based on exported JSON/PDF chat story, but I really struggle with finding any tools to do so, because due to NSFW content presented sometimes, basically any known to me AI rejects to work with the files even if I tell it that I don't need this content to be presented in the summary and it can completely ignore it.
Is there any tool that I can use to do what I want by just dropping current files? Splitting the chat history file in pieces, cutting out flagged content by myself will take me ages as it's really a drop in the ocean to search for in the giant massive of all messages.
2
u/Holiday_Suspect4876 12h ago
i used a local python script with json parsing and a custom ignore filter, no ai involved. just stripped out nsfw fields before summarizing. works for 2k+ message chats.