The chocolate has to be shipped cool, requiring more energy. That said: more and more producing countries are doing initial processing (roasting and grinding into liquor), and derivatives (powder and butter).
Processing cacao in North America means I have to pay to get rid of shell which could be brought back to farms. Hopefully one day I can buy good liquor.
It's not clear that shipping the sugar to where the cacao is, then shipping refrigerated chocolate is any better than shipping unrefrigerated cacao mass to the North where it's milled with sugar.
Especially if you look at grid emissions in all those countries. Transport uses energy, but so do milling, tempering and packaging.
Looks to me like the same countries could provide sugar and cacao.
The end result will always have to be shipped refrigerated anyway and is consumed pretty much worldwide so it doesn't look like a decisive factor, especially as it isn't a product that needs to be aged like say, a wine or a cheese. Once it is produced it is ready to ship, no long term storage needed.
Processing cacao in North America means I have to pay to get rid of shell which could be brought back to farms. Hopefully one day I can buy good liquor.