Quantcast
Channel: Weaviate Community Forum - Latest posts
Viewing all articles
Browse latest Browse all 3605

Batch thread died unexpectedly

$
0
0

But then, why does batching not working using async methods using fastapi as web framework?

I think (but not 100% sure) because we also create event loops inside of the batching

using fastapi as web framework

I am not an fastapi expert, but our batching algorithm is mainly aimed for long running tasks eg 1000 objects+ and is not threadsafe.

I would recommend to either:

  • use data.insert/insert_many() and build your own async wrapper around them
  • install weaviate-client==4.7.0-rc.2 and test our new async client (please not directly in production :slight_smile: ) you can then do:
async with weaviate.use_async_with_local() as async_client
    collection = async_client.collections.get(name)
    await collection.data.insert()/insert_many(....)
  • give us feedback if anything does not work as expected :slight_smile:

Viewing all articles
Browse latest Browse all 3605

Trending Articles