I feel like that ship has sailed. Every software company I have ever worked for is dysfunctional in this regard. You might think your "delete my data" request succeeded but there is absolutely zero way to guarantee that it actually did, and chances are it didn't.
Agree, this is pointless. For one thing how many companies have the technical ability to remove specific records from all their database backups and logs? None that I’ve worked at
I’m not expecting my data get deleted from old backups or log files. I can see where that would be an issue.
What I do expect is my data is deleted from the production database and thus won’t be in any future backups/logs/etc. I guess to that end, they would need to keep a record of delete requests to re-delete them if they ever need to restore from backup.
If there is a data breach in a year where the company’s user data ends up on the internet, I expect to not be in that user list.
The problem is - imagine microservices - that data does not exist in one spot. And chances are no one actaually knows 100% where the data lives. It probably lives in a prod db, an ETL data lake type platform (or two/three - and god knows if that has any kind of identifier to actually delete it) and chances are if you are big enough some 3rd party systems. So even if you delete it from prod, it still exists somewhere.
In a perfect world there would be some way to snap your fingers and delete it from every system - but we do not live in a perfect world. There is absolutely no incentive to build systems with this kind of requirement in mind. It's a waste of time and effort. Europeans will say "but hey wait! GDPR!" meanwhile the world keeps spinning and no one gives a shit.
I don't understand. If they want the ability to ever update the data how do they expect to have it propagate if these information systems aren't connected? And if they are connected, why would 'delete entry' be harder than 'update entry'?
You’re assuming the data is actually handled with care, and lives in a system designed for this sort of thing, but it’s probably not.
Some data scientist who wasn’t even supposed to be able to view stuff managed to copy data from prod to dev, maybe by accident. Then an engineer who intended to be testing an in place transform pipeline actually wrote a copy elsewhere, maybe once for each test, and didn’t even notice. This stuff happened off and on for like fifteen years while projects and people all changed and no one cares, because sure there’s some vague gestures in the direction of compliance, security, and privacy but it’s just a token effort because anyone who really starts asking questions is let go for not being a team player.
Syncing updates to some “main” copy won’t make it easy to delete dupes.
Not being able to delete every possible record that may have been used during testing doesn’t mean they shouldn’t delete the data they do know about in the main system. If I can change my username, password, or email address… and those functions work. A delete should also work.
Why a prod db with sensitive user info is being used in testing is a whole separate issue. Those can be deleted in their entirety.
The big scary fine is a absolutely an incentive, unless you are huge company with an army of lawyers on standby. Some customers I've worked with submit GDPR requests on behalf of their users to all of their software vendors; there're even SaaS products for doing exactly that. They also get you to sign documentation to that extent if you want to keep them as a customer.
Not that anyone is disagreeing, but it bears repeating: This is a lack of any real pressure from regulators, not a technical challenge. Or rather, there may be technical challenges but they absolutely can be overcome, and aren’t being tackled right now very simply because the business doesn’t care. As is so often the case, the business must be made to care.
> This is a lack of any real pressure from regulators, not a technical challenge.
Also, I think it's easy to misstep if we start thinking of it as a problem of "better regulators", since some of the blame lies on deeper legal-aspects around (data-)ownership, contracts, and what what happens in bankruptcies.
Even a company with great intentions may have difficulty ensuring the promises they made are kept long-term, especially if a bankruptcy court voids those promises in the name of repaying creditors.
My impression from all that I've heard is that you should have a backup retention policy, but otherwise there's no set upper bound on how long that may be. Not that the text of the GDPR breathes a word of it, though, everything's just a rat's nest of exemptions suggested by various authorities and other parties that haven't been tested in court.
In general I don't particularly care what other people say on this topic and rely on the legal guidance I received during my work from UK ICO and Slovenian office, but even some of your links don't collaborate you. The second one linking to Verasafe's page on which it clearly says that yes, you should delete it.
There's a lot of complainig around how difficult that can be and the fact that EU legislation in general often does not like to precisely prescribe its requirements like what reasonable means, which can indeed be annoying.
You still need to remove it either directly or your retention policy for backups needs to be short enough that keeping it in backups for a while is judged as reasonable.
> In general I don't particularly care what other people say on this topic
Nor do I see why I should particularly listen to what you say on this topic, given that others have similarly claimed authority from their lawyers or from their local jurisdictions.
> The second one linking to Verasafe's page on which it clearly says that yes, you should delete it.
Right before the "But don’t panic! Enforcement authorities know how difficult it is to fulfil this obligation in practice." section, where it elaborates on your ability to claim that stripping data from backups is technically infeasible, in which case you must promise to delete the data on restoration. Just like I've heard from everyone else.
It's always seemed paradoxical to me that the GDPR is branded as this unyielding hammer against companies improperly storing your data, only for it to be riddled with amorphous holes on every axis. "Data is data, period, unless it's not on a live production system, in which case the written vague rules it abides by are swapped out for a new set of totally undefined rules!"
> You still need to remove it either directly or your retention policy for backups needs to be short enough that keeping it in backups for a while is judged as reasonable.
And how might I know a priori what's the longest 'reasonable' retention term that a business might be permitted by its jurisdiction? The whole nature of backups is that they're useless right up until they aren't, so the marginal value of each additional week is difficult to measure in the first place. And when most concrete talk of 'reasonableness' is seemingly done behind closed doors if at all, I have no idea just how far other jurisdictions' ideas of a reasonable term might differ from mine.
On the other hand very few organizations I have worked at could definitely restore backups (at least it was not tested regularly) and logs will eventually roll off.
Exactly this. Especially for a currently failing company that got an incentive to NOT delete your data (because that's the only value they still have).
GDPR and CCPA etc made it easy to send a request for deletion that will most probably be a frontend gimmick. How much effort are they really going to put into going back in their backups and deleting all your entries? I'm pretty sure it must be the lowest roadmap priorities.
And it's amazing how financial liability has a way of getting things on a VP's feature radar that common sense doesn't.
The reason it was haphazardly handled prior was that there was no liability. Who cared? (legally speaking)
From working inside a T25 American retail company, I can say that we went top-to-bottom and rearchitected for traceability and hard deletes as a result of the CCPA.
I have a feeling that it's also quite a difficult problem past some scale of infrastructure.
If I ask Google to delete my data (EU citizen), I have trouble believing that they actually go through all of their cold storage backups where it was stored and make sure it's erased. At best I could believe that the process is designed in such a way that my soft-deleted data is unlikely to be recovered (intentionally or not) and maybe unlikely to be possible to link to my account.
What they should do (I have no idea what they do) is to encrypt every record belonging to a user with an individual key. Live records, backups, everything. If a user wishes to be deleted, that live key is simply obliterated, making any data the user owns unrecoverable.
Since the key is not used for end to end encryption, and backends still have access to the data (as long as the key lives), it has different requirements on how it needs to be protected. The biggest challenge is backing up the key itself, as losing it means losing access to all the user’s data by design. But backing up and obliterating a single key is much, much easier than doing so for a whole set of loosely associated data across many databases.
Practically speaking, it also makes using and querying that data and doing any kind of analytics much, much more expensive. It is done that way in some cases, but in the absence of a technical requirement to do so, there are cheaper approaches.
Those are solvable problems. I could also argue how address space separation and more generally MMU protections make things so, so much more complex (they do!), yet we don’t question that one very much.
There is no end to end encryption involved here, so you don’t need to resort to such voodoo as homomorphic encryption.
> the problem only partially smaller, since you still need to sync and back up the keys.
I mentioned that: It makes the problem much smaller, as you only have one single, small piece of data to backup and and erase, instead of an ever-changing many-faceted blob of distributed data.
> Also, is an encrypted piece of data with a lost key truly deleted? What if the encryption gets cracked?
Oh boy. If simple symmetric encryption gets “cracked”, then you have much larger problems.
> I would say it is more deleted than toggling a `deleted` flag in the db and less deleted than burning the tapes in fire.
For all practical purposes symmetrically encrypted data that lost its keys is considered “random” data. If you “erase” data on a device before you sell it, most often it will just throw away the key to the disk contents nowadays.
They already do this (the encryption-at-rest part). Deleting the data is still a hard requirement. Also, the keys are never seen outside of the centralized encryption service. Deletion is still a must.
Before you make a deletion request, make a subject data request and see what they have on you; then request deletion; then make a subject data request again.
Google-scale companies have very capable people employed, both on the technical and legal side, who do nothing else than look for these kinds of oversights, and are empowered to make sure they get fixed.
Large companies fail in spectacular ways all the time. Google is super successful because they tapped into the biggest cash cow of all times. Not because the employees are somehow very capable and above any oversight.
I can't speak for any other companies, but you don't need to speculate. You can search the internet and find several articles outlining that the correct strategy for businesses here is to delete the data from production systems, and then maintain a record of references to those deleted records such that a restored backup can ensure that deleted records are not put back into production.
There is generally an expectation that data may be retained in backups for a specified retention period, but will not be used or restored. Beyond that, it is up to the regulator to determine if this is meets the standard, but it's worth noting that there are notions baked into the text and the interpretations of the text of GDPR that account for reasonable costs and efforts.
Auditors can and do test and monitor for this, both using audit processes and demanding evidence, and by performing manual testing and experimentation.
Fines for non-compliance with GDPR regarding data of European citizens can amount to 4% of annual revenue:
83(5) GDPR, the fine framework can be up to 20 million euros, or in the case of an undertaking, up to 4 % of their total global turnover of the preceding fiscal year, whichever is higher.
100% this. It's laughable if you believe those requests work as expected. Sure they may "delete" some surface level bs like your account or login, but there is no way it's 100% scrubbed in the way it's supposed to work.
A lot of recourse is around intent and liability. I would like to believe my request is honored; in the event it is later proved to not have been honored, recourse is potentially available through legal and regulatory mechanisms.
23andme didn't implement strong customer identity and auth mechanisms, for example, and it cost them ~$30M to settle their data breach liability [1]. Take action, keep receipts, and failing good faith actions, step back while regulators and the legal system whack whack whack with a hammer.
