I was thinking of setting up another docker container on the same network with API gateway that would use the same instance of weaviate where I store my vector collection as a cache and a middleware to pass the request to open ai when needed. Might actually work out, especially if I use grpc to lookup in cache and http towards open ai.
↧