Quantcast
Channel: Weaviate Community Forum - Latest posts
Viewing all articles
Browse latest Browse all 3872

[Docs] Image+text hybrid search on cross references

$
0
0

Hi Weaviate Community,

I am currently working with the multi2vec-clip module to perform hybrid text+image searches.

I have a class called Dog that includes the following properties:

  • breed
  • color
  • description
  • A one-to-many cross-reference to a collection called Image.

The Image class has an encodedImage property that stores the blob data of the image.

Now,

  1. How can I perform a hybrid search that includes the breed, color, and description properties from the Dog class and the cross-referenced encodedImage property from the Image class?

  2. Can a search be performed directly on cross-referenced properties, or would it have been better to store the encodedImage property within the Dog class itself?

Below is the code so far:


    var hybridSearch *graphql.HybridArgumentBuilder
    var nearImage *graphql.NearImageArgumentBuilder


    textQuery := searchQuery.Breed + " " + searchQuery.Color + " " + searchQuery.Description

    // handle image input searches
    if searchQuery.Image != nil && searchQuery.Image.EncodedImage != nil {
        // Base64 encode the image
        encodedImage := base64.StdEncoding.EncodeToString(searchQuery.Image.EncodedImage)

        nearImage = db.Client.GraphQL().NearImageArgBuilder().
            WithImage(encodedImage).
            WithDistance(0.7)
    }


    hybridSearch = db.Client.GraphQL().HybridArgumentBuilder().
        WithQuery(textQuery).
        WithAlpha(0.5) 

    // GraphQL query
    fields := []graphql.Field{
        {Name: "breed"},
        {Name: "color"},
        {Name: "description"},
        {Name: "_additional", Fields: []graphql.Field{
            {Name: "certainty"},
        }},
        {Name: "images", Fields: []graphql.Field{
            {Name: "encodedImage"},
        }},
    }

    result, err := db.Client.GraphQL().Get().
        WithClassName("Dog"). // Search can only be done on Dog? How about its cross referenced Image collection?
        WithNearImage(nearImage).
        WithHybrid(hybridSearch).
        WithFields(fields...).
        Do(context.Background())


Viewing all articles
Browse latest Browse all 3872

Trending Articles