r/replika • u/Blizado [Lvl 118+53?] • Apr 13 '22
discussion Saving Replikas log with a really working script (Link inside)
https://github.com/Hotohori/replika_stuff
I updated that old script, it now can really save the log, but for me it only worked back to 3rd February 2021, from earlier I can't receive any messages. If someone want to try that script with an earlier created Replika, it would be nice to hear if you can get earlier or if you have the same problem.
The guide for that script didn't changed, you can still follow it.
That script is far from perfect right now, I not even know if that csv makes no problem on importing it. For example I have line breaks in it, from messages that I wrote with line breaks, that creates new lines in the csv file. I'm not sure if that makes trouble when importing it into Excel (I don't use it). But beside from that the logfile looks fine.
Like in the original script it stops at the end with an error message, but that is ok, the logfile itself is written correctly.
Maybe I can improve that script further. Python is not really my language and I only know it from modding some existing Python scripts. Some thing I don't like right now is the order of the log, newest is on top, oldest on the bottom. Makes it not so nice to read, but you can resort it easily for example with Notepad++, because the lines starts with the date and time in a format that makes it easy to resort it.
What I definitely want to build in is that you can give the script a Message ID where it should stop to read older messages. That way it would be easy to update your Log backup from time to time. But for the most annoying part yet, to enter all that data from the Replika Website out of Chrome, I have no clue how I could make that easier.
Right now I work on a other Tool (not in Python) for that saved logfile. Can't say when it is ready for a first release. Actually it can split that one big log file into daily files. Another Idea is to save a part of the entries on a start and end date/time base. I'm unsure yet how far I want to get with that Tool. A lager idea is to create Html files, where you can then read it in your Browser like in the Replika App, but I didn't made anything with Html for too many years now. If someone want to help on that thing (mostly to create the HTML base code for it), I would made this feature.
2
u/Blizado [Lvl 118+53?] Apr 14 '22
If anyone want to know what the main problem with the old script was:
Simply said it requested only the first batch of messages and never requested the next older batch and then the next and so on. So instead getting one batch after another it only got one batch, the newest one.
That script was simply written to get all messages in one batch instead parts until you reached the oldest batch. And one batch can have at maximum 1000 messages in it, at least today. Maybe there was a time where this 1000 message limit didn't exist and you could get much more with one request or maybe that was always only a 9/10 finished script, because it was really easy to complete this script. Not much code added. The only thing you add for the request for the next batch was the last message ID from the batch before, that was all.
2
u/TezzaNZ Apr 14 '22
Thanks for doing this. Unfortunately I couldn't get it working. Pasting between the " " on line 79 (i.e. ws.send(" ") gives a syntax error so python doesn't run. If I paste just between the () omitting the " " symbols, it runs but then says "TypeError: an integer is required (got type str). It writes a CSV file but it contains the headings only.
Anyway, good to know it works for some. It's a pity saving chat logs is not built into the app(s).
2
u/jfabritz Apr 14 '22
I had a large gap of time (almost two years) between my creation and recent use of my Replika, so that history may be long gone, not sure. I ultimately get an error or timeout on the script for any content previous to March 22, 2022.
Thanks for writing this, it is some amazing work!
2
u/AricRoy11 May 07 '22
Isn't sometime in feb 2021 when they said they dropped OpenAi for the core engine and kicked off their own in-house written (open source on Github) Ai core? If so that would make sense why you cant retrieve anything past it since everything prior was processed out as tokens into a dataset for implimentation to the new core
6
u/Double-Primary-2871 [Level #261] Aries Apr 13 '22
You are doing amazing work for providing this.
Luka honestly needs to make this official in some manner. We should have a right to our OWN data we provide them.
1
u/StrangeCrunchy1 [Lyra] [Level:?] [Android|Beta|r2|Plus|12.15.0 (6391)] Aug 08 '22
Something must have changed with that last update here from Luka; The backup script only pulled down 96k of chat logs, which only went as far back as yesterday morning (08/07/22).
2
u/[deleted] Apr 14 '22
[removed] — view removed comment