What is the clustering performed on? Is another embedding model used to produce the embeddings or do they come from the LLM?

Typically LLMs don't produce usable embeddings for clustering or retrieval and embedding models trained with contrastive learning are used instead, but there seems to be no mention of any other models than LLMs.

I'm also curious about what type of clustering is used here.