People did complain at the time and denied that Elgin had the right to take them in the first place, which is why parliament investigated the matter at the time. While the controversy has intensified in the last few decades, it's been controversial pretty much since day one. It's not a "meme-age" thing.
Whether Elgin did or did not have permission from the occupying Ottoman forces is of secondary importance. Many people at the time already considered the Greeks to be occupied by the Ottomans, which is one reason why all of this was controversial at the time (Greeks were not viewed with the same racism as the occupied people in Africa or Asia – quite the opposite since many people were huge Greek fanboys).
If some official had legally approved removing huge chunks of St. Paul's Cathedral in 1800 then the British would be up in arms about it today, and rightfully so. Nazis had "legal" permission to remove a lot of art works through occupied Europe. No one today would claim that the painting of the Fallen Madonna with the Big Boobies would belong to the inheritors of Herr Flick, no matter how they were acquired in the first place. There are many examples of things done "legally" where today we no longer acknowledge the legality of it.
This is also why the British government is often so disproportionally testy about the entire matter, and why that law was passed in the first place. They know that is is clearly and obviously the right thing to return one of the most prominent cultural and historical artifacts to Greece, removed by someone for his private garden, with dubious permission to do so, and even if it did exist it was given by someone who had no right to give it in the first place. They know they're wrong and don't want to talk about it.
I suppose we need to have a semantic discussion about what exactly "full of" means. Note that the original person who started the thread used the phrasing "many" (which only moves the semantic discussion to how many artifacts with dubious origins you need to have to speak of "many").
Personally I think that's not a very interesting discussion. Your general point that there are also a great many artifacts are not stolen (the majority): yes, you are correct. And I agree it's worth pointing that out as some people do overstate the case at times.
Complaining - even quite correctly - about the ethics of ownership is not a legal challenge to ownership.
You even point out yourself that other occupying forces had rights of ownership legally, even if they did not have moral and ethical rights.
I'm not arguing whether Elgin should or should not have taken them. I'm not arguing whether he had permission or not. I'm saying he did take them, and that legally pushing back against that was almost impossible at the time because UK law did not prevent him from presenting them to the British Museum, and multiple acts (the most recent being the 1963 act), have made it clear once an item is gifted to the Museum, disposing of it must only be done for very specific reasons that don't apply to the Marbles.
They could have changed the law then, but they didn't. And they didn't, not because of malice or evil or even incompetence. They didn't because changing the law in that context was hard. And changing it today is hard.
I personally am pretty ambivalent about the items in question. They're nice enough to visit, they're interesting, they could stay, they could go, I'm glad they were salvaged from what sounds like a disaster stone (it's not like they were much loved by anybody in Athens at the time), and sure, they could go back if everyone wanted them to.
But that's all irrelevant - the law is the law. We can't just instruct the directors of the Museum to ignore the law. We can't just expect Downing Street to make an edict. An Act of Parliament has to be passed to amend section 5 of the 1963 British Museum Act. End of. There's nothing else to discuss here.
And parliamentarians will be reticent, because the moment that's in the chamber, every other nation on Earth will ask for other material to be returned regardless of the means of procurement (even if legally bought at auction). In essence, to amend the act for the Marbles would be to disperse the entire collection and close the Museum. You might be OK with that, but given its contribution to the studies of historians and archaeologists for centuries, and the fact it's an incredibly popular (and free to access), tourist attraction, it might be a hard pill for most MPs to swallow.
The Greeks could offer a loan deal for a few years, and we could take it from there, but my original point is that no side trusts the other, and without the law being amended, it all seems rather sadly remote, and the chances of the law being amended seem remote too.
Whether Elgin did or did not have permission from the occupying Ottoman forces is of secondary importance. Many people at the time already considered the Greeks to be occupied by the Ottomans, which is one reason why all of this was controversial at the time (Greeks were not viewed with the same racism as the occupied people in Africa or Asia – quite the opposite since many people were huge Greek fanboys).
If some official had legally approved removing huge chunks of St. Paul's Cathedral in 1800 then the British would be up in arms about it today, and rightfully so. Nazis had "legal" permission to remove a lot of art works through occupied Europe. No one today would claim that the painting of the Fallen Madonna with the Big Boobies would belong to the inheritors of Herr Flick, no matter how they were acquired in the first place. There are many examples of things done "legally" where today we no longer acknowledge the legality of it.
This is also why the British government is often so disproportionally testy about the entire matter, and why that law was passed in the first place. They know that is is clearly and obviously the right thing to return one of the most prominent cultural and historical artifacts to Greece, removed by someone for his private garden, with dubious permission to do so, and even if it did exist it was given by someone who had no right to give it in the first place. They know they're wrong and don't want to talk about it.