Quantcast
Channel: Weaviate Community Forum - Latest posts
Viewing all articles
Browse latest Browse all 3605

Embeded weaviate with objects stuck at 20; connect_to_local shows thousands objects

$
0
0

collection = client.collections.get(class_name)
response = collection.query.fetch_objects()
object_cnts = len(response.objects)

It shows 20 when I load first file; 15 when I upload 2nd file. After both files are done, cnts is always 20 even after I tried to upload a 3rd file.
Weaviate version is 3.4

I can hardly find any documentation on how to trouble shoot issue like this.

I did dump all objects to local file; and found some chunks from 1st file are missing file_counts = Counter()
all_files= get_all_filenames(pdf_file_path)
for filename in all_files:
count = sum(1 for o in response.objects if o.properties.get(“source”) == filename)
file_counts[filename] = count

logging.info(f" === utils.py counts per file \n {json.dumps(file_counts, indent=2)}")

# Define the path to save the JSON file
output_file_path = "temp.txt"  # Update with your desired path

with open(output_file_path, "w") as f:
    for i, o in enumerate(response.objects, start=1):
        f.write(f"Object {i} properties:\n")
        # Access only the properties dictionary of each object
        for key, value in o.properties.items():
            f.write(f"  {key}: {value}\n")
        f.write("\n")  # Separate objects by a newline


return object_cnts

Viewing all articles
Browse latest Browse all 3605

Trending Articles