Quantcast
Channel: Weaviate Community Forum - Latest posts
Viewing all articles
Browse latest Browse all 3604

Errors: text too long for vectorization. Tokens for text: 10440, max tokens per batch: 8192, ApiKey absolute token limit: 1000000'

$
0
0

Hi @DudaNogueira that’s not my case here What i want if I have three chunks and my search is nearest to the one of the chunks then i wanna get all those three chunks

We are working on a file base system so if we lets say have a large files we converted it into three chunks that is : a,b,c
Now if my query matches with b then I have to fetch the whole files to do some thing on that, and in your example even this thing has been explained that if you convert object into the chunks on match you will get the whole object but I am not sure about the tokenization because the method in documentation mentioned is not working for me as I have pasted code above as well.
I am following this Example part 1 - Chunking | Weaviate can you have look and let me know please :wink:


Viewing all articles
Browse latest Browse all 3604

Trending Articles