Well, the absolute comparison of distance between models per se doesn’t giving it much as it can be relative to other objects, but the similarity/relevance of the results against the query that is the main thing to look for.
Have you seen this blog post?
Looks like you are in to some benchmarks, so it may be relevant here 
Other than that, unless there is something going wrong on the second batch import, it points to how the model vectorized your dataset 
Let me know if this helps