hi @Cam_Quilici !!
Do you have any resource usage reading? We have some dashboards here for prometheus and grafana: weaviate-local-k8s/manifests/grafana-dashboards at main · weaviate/weaviate-local-k8s · GitHub
Indeed those logs looks normal.
Also, it is important to note that the dynamic batch will calculate it’s size according to the latency from the last batch. This may overwhelm the server in low latency scenarios (dedicated vectorizer in same cloud/k8s, for example)
A good alternative here is creating a fixed sized batch, and tweak those values.
On top of that, ASYNC_INDEXING
can help too, as it will take it’s time to index the recently created vectors while still having horsepower to ingest the data.
It also interesting to check the some other env vars, for example GOMEMLIMIT: Environment variables | Weaviate
But as you mentioned, for those 3k objects, weaviate performance shouldn’t be an issue.
Have you tried stress testing that vectorizer? Is it able to vectorize those 3k objects in some expected rate?