The Need for Portable Memory in LLMs

I’ve been thinking about this on and off since OpenAI released the improved memory in ChatGPT. I subscribed to ChatGPT plus for a few months during my exams, so it could help me with the more niche topics or sort my notes, etc., but the memory was actually very impressive. You could start a new chat and it’d have a very good recall of what you discussed previously. This is probably not that surprising. I have no clue how it actually works on a technical level; however, it’s probably not too difficult?

But this is a huge lock-in for LLM providers. It’s a very competitive market, and there are different factors pushing and pulling users between different platforms. Parker Ortolani also blogs about this and for me this issue is highlighted here:

I have been attempting to use them in my daily life but ChatGPT’s extensive knowledge of me, my interests, my current life situation, and the projects that I have been working on with it have created a great deal of, you guessed it, friction.

I was surprised to see huge companies like OpenAI, Claude and Cursor (tons of others too!) using the open MCP spec to allow their models to access data from other places. This results in a much improved UX for pro users.

This is sort of unrelated, but I think the openness of this protocol is quite impressive. It works well (we can clearly tell it does as it’s still heavily used in this fast-paced space) and yet allows people to use the latest models of their choice, whilst accessing the same information.

I think memory could be like this. Unsure how it would work in a secure and encrypted manner, but it’d be really cool if there was some similar equivalent for memory. You could plug your saved memory into any given LLM and it’s got all your past context right there on the fly.

The only issue is it’d probably be much more advanced for the average person sending ChatGPT a request - although I guess you could say the same for MCP.

This may come across as really dumb as I’m not sure how to do it, and I’m not sure if this is already being worked on or talked about, but I think it’ll be important. It’ll have to be easy to setup too, whilst also somehow working reliably. A secure, self-hostable way to bring in prior knowledge.