Yes! I think you're right. That's a great connection, never thought of Annihilation in that way. "Color out of Space" is the first HPL story I ever read, and still one of my favorites.