Isn't the performance (as in the capacity of retrieval, not performance as compute/memory usage) of kNN mostly given by the quality of the vectors/embeddings themselves?

Most vector DBs use (at least) some kind of KNN anyways.