collection = client.collections.get(class_name)
response = collection.query.fetch_objects()
object_cnts = len(response.objects)
It shows 20 when I load first file; 15 when I upload 2nd file. After both files are done, cnts is always 20 even after I tried to upload a 3rd file.
Weaviate version is 3.4
I can hardly find any documentation on how to trouble shoot issue like this.
I did dump all objects to local file; and found some chunks from 1st file are missing file_counts = Counter()
all_files= get_all_filenames(pdf_file_path)
for filename in all_files:
count = sum(1 for o in response.objects if o.properties.get(“source”) == filename)
file_counts[filename] = count
logging.info(f" === utils.py counts per file \n {json.dumps(file_counts, indent=2)}")
# Define the path to save the JSON file
output_file_path = "temp.txt" # Update with your desired path
with open(output_file_path, "w") as f:
for i, o in enumerate(response.objects, start=1):
f.write(f"Object {i} properties:\n")
# Access only the properties dictionary of each object
for key, value in o.properties.items():
f.write(f" {key}: {value}\n")
f.write("\n") # Separate objects by a newline
return object_cnts