Ask HN: How are you implementing GDPR-compliant soft deletes?

The idea: Customer requests account or item deletion, you set it to "deleted" in DB without actually deleting it and it helps for documentation purpose should the dispute arise over some issue in future.

  • Soft deletes as you describe aren't allowed under GDPR.

    But a possible solution may be to disassociate the data from the customer (as long as the data itself isn't considered 'personal data'). For example, if the reason the data falls under GDPR is because it is connected to the customers email address, you could clear the email address. But that wouldn't let you ever re-associate it. But you could maybe (but don't take my word for it) one way hash the email address (bcrpyt, etc.) and if a dispute arises in the future scan the hashes for a match of the person raising the disputes email address in order to re-associate the data.

  • Consult your Data Protection Officer first.

    GDPR says you must delete information about the customer; but there are cases where you still might need to have that data available.

    If your customer can interact with another one inside your app/platform, he/she can commit a crime, and you might be required by court (and by law) to disclose some information (even conversations! inside the platform).

    Setting something to "deleted" might not be the best way to do the actual soft delete.

    Sometimes you can "delete" that user moving it to a separate part of an LDAP branch (where nobody except someone with authority can access).

    In other cases, you can add the "deleted" flag on the table. If so, MAKE SURE your app access the data from a view of the table where the 'deleted' users are not present. Even better: partition the underlying table based on the "deleted" field to physically separate active and deleted users.

    But whatever you do, ask your Data Protection Officer first.

  • > it helps for documentation purpose should the dispute arise over some issue in future.

    If you are required to hold on to the data for legal purposes such as dispute settlement, there is no issue. The customer can request you delete such data but you have no obligation to do so. Issues arise when holding on to the data is no longer "necessary". At that point soft deletion is not enough and you must be able to remove personal data regarding that customer from your systems. Source: IP lawyer I talked with last week on this.

    From what I can tell, one good way of going about this in case you really do not want to throw away data – such as when you're doing event journaling – would be to have all personal data encrypted. When the data reaches its expiration point or is requested for deletion, throw away the decryption key and all you're left with is is the metadata which you are allowed to hold on to for purposes of running your business.

    EDIT: This concept is called 'cryptographic deletion'. Here's one whitepaper on the subject. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.397....

  • I don't think "soft deletes" are actually allowed, are they?

    https://gdpr-info.eu/art-17-gdpr/

    > the controller shall have the obligation to erase personal data without undue delay

  • Some comments on the legal aspect to deletion under the GDPR: 1. deletion can generally only be requested if the personal data is being processed under the individual´s consent. Thus, other personal data such as under legitimate interest or execution of a contract does not fall under it. 2. The rule on deleting the data is not absolute as data retention laws prevail over this rule. Thus, only if no data rentention law mandates the storing of the data (which is often the case for business communication) then you are obliged to delete the data or anonymize it. Hope this clarifies the non-technical aspect. Dominic Staiger https://www.raptorcompliance.com/en

  • OK. Here goes... Advice...

    Firstly, download the regulation itself. At the very least, read article 17. Article 17 concerns something called "The right to be forgotten". It is about 25 lines of legalese. Once you have read it, have a think about it. Then read it again and have a longer thing about its implications. Then, just to be sure you have not gone stark staring bonkers, read the rest. Carefully. Be under no illusion, the GDPR is a game changer in information management terms, let alone anything else.

  • Deletion of backup-data is also an interesting topic

  • We're building a consent framework API so our customers can consent for personal data use. Data is then cleaned and transformed (ETL) from personally identifiable to pseudo-anonymised. The data is also separated into two separate encrypted storages for anonymous and pseudo-anonymised data for generalisation and separation. Random (important) hashed identifiers are created and put into a metadata service which is used as a lookup-table. If right to be forgotten is invoked, the data is disassociated from the pseudo-anonymised and personally identifiable data thus making it anonymous.

    Important is also how you handle data analytics and this is why we're deploying high restrictions on raw data. Analytics will only be able to be done through an analytics service which can give the employees access to only certain parts of the data which is approved for the use-case. We're using Apache Sentry for fine grained role based authorisation to data and metadata and a directory services for user auth.

    Things we've learned:

    * Minimise data usage

    * Don't use personally identifiable data

    * You will need to be able to prove consent when it comes to data usage and it cannot be consent by default, it has to be opt-in

    * Log all data access so that use cases can be proved. This needs to be evaluated and audited

    * Encrypt in transit and at rest

    * Centralise mapping for all data

  • It may be that the future issues you envisage can still be met with anonymous data i.e. some overall audit of your service use. Anonymous data retains a small link to the original but is exempt from GDPR. One method is to generalise records in a database, for example, mask / remove direct identifiers like names, put ages in to age brackets and fuzz spatial data by x distance. So this can be used to deal with erasure subject access requests and sharing data in general. There's a lot of advice on this from the ICO - https://ico.org.uk/media/1061/anonymisation-code.pdf

    Disclaimer: I'm a fan of anonymisation because I'm working on a project to bundle this in to a service - https://anon.ai - would be great to understand more about your use case.

  • All data under GDPR which also falls under another law can be stored, but NOT used for any other purpose than of the overlaying laws. So just because you have the information doesn’t mean you can use it. Having the same information, even “soft deleted” would violate GDPR since you are not allowed to store it like that. Having information grouped like that still implies information which will be under GDPR. Like if you store this information in the “customer” database that will imply that the customer was a customer at some time, and then illegal by GDPR. If you move this information to another database which stores “transactions” which you require to do by law that’s ok. But you are not allowed to query that information to answer the question, “who have been my customers”.

  • Hiya. If I may, a couple more things to add to considerations... Firstly while, rightly, you concentrate on the design impact of article 17. I guess most of you live and work in the US. What that almost certainly means, if you deliver SAAS, is that personal data is exported from the EU for use. You guys really need to understand GDPR Chap 5, articles 44 to 50 inclusive and get hold of something called "treaty 108" which is about the idea of "adequacy".

    And secondly, I am going to ask someone I know who is a legal eagle to sign up to this site. She is very clever and knows this stuff backwards. She also has an aspiration to come and work in the US....

  • What if you outsource the PII? For example use a payment processor and only store their reference. You can always go back to transaction in the payment processor in case of disputes, but you don't store the personal information.

  • because the GDPR is a tad vague, given the interconnected world we live in nowadays (I am a brit living in a small (very) town called Tidworth which is a pin prick o the map) there is something called "Working Party 29" which is providing some fairly detailed and verbose refinements of definition of key phrases and terms. This is an attorneys bean feast as far as I am concerned because of the vagueness, if nothing else..

  • Your idea is not allowed under GDPR. You need to unassociate the account from the user.

    That is: Remove all personal information - email, name, surname, etc..

  • This is how it was explained to me, and the approach we take. YMMV.

    Understand what your lawful basis for storing and processing the data is, and that dictates how you need to handle it. Plenty of people throw around large fines and rights to be forgotten and make sweeping statements about "you can't keep X data/we can hash things!/delete all the things". You have a number of potential reasons for storing or processing data. Legal reasons, contractual reasons, consent based reasons - data subjects have different rights depending on the reason you're storing the data (there is a list of these reasons; see below.)

    I may be keeping data because I have a contract with a client, and I require the data. An example of this would be an email address stored as a login. My legal basis for storing the information is contractual (I have a contract with my client). Does the data subject (in this case, the user who's email it is) have a right to erasure? No. Can I store the information in my Amazon RDS instance in Virginia? Yes. Provided I've explicitly stated it, and been transparent about how and where I'm storing and processing the data, and my client has agreed. Do I need to secure and look after the data? Yes. Obviously. Do I need to get to get consent from the user? No. That wasn't the legal basis for storing the information.

    What about consent based stuff? I may have someone subscribe to my mailing list. I get their consent. Therefore, the legal basis for storing and processing is consent. That consent should be time limited, and I should be transparent about it. I need to give them the means to review, withdraw and act on their consent should they wish to.

    What about keeping a record of a person you've deleted because they have requested it? You can store this. Lawful basis is that it's a legal requirement. If you go and use the data for any other purpose, it's not allowed - because that would require their consent.

    If you want to understand lawful basis, this is a good overview - https://ico.org.uk/for-organisations/guide-to-the-general-da...

    Trying to wade through one half of the GDPR and its requirements without understanding lawful basis leads to confusion, because you'll keep hitting cases that seem completely unreasonable (because they may not be required). Trying to paint the law vague and unreasonable defeats the point - it will become less vague over time. It makes privacy a first class citizen (something we sorely need), and will become more specific as it's tested in the courts, just like any other law in jurisdictions that value legal precedent.

    Get to know it and work with it - this is not your mother's EU cookie law.

  • But it is only part of a series of new bits of legislation that is being introduced in Europe related to computer security Some other acronyms "PECR" something all marketeers should have a look at as it governs things like email distribution

  • Another pattern you might consider is setting an expiry date on a database row. You only show values where the expiry date is null. Every night you have a cronjob or some other process that deletes all objects that have expired.

  • What would be the sort of dispute you envisage?

  • About the GDPR, can anyone recommend a company in the UK they have dealt with, that brought them up to compliance?

  • undefined

  • undefined

  • Ladies, gents... The kinds of things you are saying/asking about GDPR are the same kinds of things being asked on this side of the pond...