> "some backend script which a) runs sql deletes on an account ID and all tables and b) fires off scripts that physically deletes all photos across all servers."
Neither should happen. You never actually delete rows from your tables. You mark them as deleted, sure, but the data needs to be able to come back. Space is cheap, there are simply very few cases where outright deletion from a DB table is warranted.
I have a hard time believing Flickr would make a mistake like this. I agree with you - odds are if they actually let an engineer loose on this problem it'll be licked with a DB backfill in 20 minutes.
I bet that in cases of copyright infringement a logical delete isn't legally sufficient. Sure, you can leave the database pointers in place... but you're going to need to actually remove the infringing content. In that case, at least, a true delete tool would be needed. Like any Internet discussion about copyrights or other people's backend scripts, this is all purely speculative.
I don't buy it. Simply making infringing data unavailable has to be legally sufficient, except in cases involving removing secret or confidential data (which would likely require a court case) there's no harm in having the data merely inaccessible. Moreover, it simply must be the case that this is how it works in practice.
Consider how deep this rabbit hole goes. If expunging any copies of copyrighted data were strictly necessary, then:
What of data existing on backup media? Should those be re-hydrated, the target data expunged, and then recreated in abridged form?
What of data existing in various content caches (such as memcached)? Should all of those systems be flushed of any possible contaminant?
What of the actual data on physical media? Deleting a file on any modern form of media does not expunge the data, it merely unlinks the location of the data on disk from the file system directory. It might be necessary, depending on filesystem and drive type, to scan all of the unused sectors on an entire disk to find out if the target data or any part of it existed.
As you see, quite quickly you get into absurdities. Nobody goes to that much trouble to delete merely copyright infringing data.
I can't speak for Flickr, but I can tell you that one of their larger competitors simply does a logical delete. That seems to take care of any legal obligation.
I should note that this not true in the case of child porn. Once the feds give them the signal that is completely purged.
> You never actually delete rows from your tables. You mark them as deleted, sure, but the data needs to be able to come back. Space is cheap, there are simply very few cases where outright deletion from a DB table is warranted.
This comes with a cost, and it's not a small one. Every single query that ever touches this table, in every single piece of code owned by the company, written or maintained by every single developer (consultant or transient or offshore, none of which ever got training sessions on this requirement) needs to be enhanced to respect "where isDeleted <> true".
It's well understandable if the expected cost to recover after the fact from an occasional issue is smaller than the cost of implementing and maintaining that in the first place.
A sane approach would be to do all your table access via views, and the view definitions have the 'where is_deleted = 0' or whatever. Then your database handles it.
Updates to the view work as well, in pretty much every database system, including MySQL even.
That just shifts the problem. Every piece of code and every developer now needs to know to hit that view rather than directly hit the table. Except the code that actually deals with deleting and undeleting which has to hit the table directly (or another view or sp.) And you still have the original problem in writing and managing the views themselves - every developer writing or maintaining one of those views needs to know to respect "where is_deleted = 0".
Using views adds a layer of abstraction. That's often a good idea and a step in the right direction, but does come at the cost of increasing the overall complexity of the system. It's an incremental step to the positive, but no revolutionary solution.
Neither should happen. You never actually delete rows from your tables. You mark them as deleted, sure, but the data needs to be able to come back. Space is cheap, there are simply very few cases where outright deletion from a DB table is warranted.
I have a hard time believing Flickr would make a mistake like this. I agree with you - odds are if they actually let an engineer loose on this problem it'll be licked with a DB backfill in 20 minutes.