July 17, 2018 at 8:35 pm
Comments posted to this topic are about the item Storage that Expires
July 17, 2018 at 9:31 pm
Sounds like a great place to store audit data.
--Jeff Moden
Change is inevitable... Change for the better is not.
July 18, 2018 at 1:51 am
I think I agree that on the whole the GDPR is a good thing. Although as yet I haven't noticed a substantial drop off in my spam. It will be backed to the hilt by the EU which will potentially make a lot of money from those found non-compliant. The issue is the impact it is having on smaller versus larger industries. Larger corporations have resources to throw at updating and fixing all their systems to be compliant. Smaller businesses not so much!
WORM could be a good place to store legal/contractual data.
July 18, 2018 at 1:52 am
My experience has been that data and data objects are retained so long that there is fear and doubt as to whether they can be deprecated or purged. Similarly applications have blocks of code that no-one is quite sure if the functions within that code are still in use or not.
Defining and implementing a retention policy upfront would cut through a painful change management process trying to get permission to deprecate obsolete material. The size of the risk posed by deprecation is perceived differently by different people. For some people all risk is huge and God help you if they are in a position of power when trying to get rid of old stuff. They actually increase the likelihood of the thing they most fear.
For GDPR Subject Access Requests a means is required for an authenticated person to access the totality of their data. The implications of that is that data for a person has to be collated, converted into a universally accessible (by the customer) format and stored so that they can retrieve it. Setting a expiry date on that storage location means that a customer has a limited time, after receiving notification that their subject access request has been fulfilled, to look at their data. Placing all the customers data in a single location represents a risk so expiring it after a period of time mitigates that risk.
Log files, audit info, backups, data with a legally mandated expiry date, all these are candidates for expiring storage.
July 18, 2018 at 2:18 am
Yeah, criminals would definitely like a strong data retention policy.
_____________
Code for TallyGenerator
July 18, 2018 at 2:26 am
Think of those #metoo activists - do the even have a case?
According to the new law it's even illegal to remember events from that distant past.
Wait till they themselves are in court on accusations of breaching data protection legislation.
🙂
_____________
Code for TallyGenerator
July 18, 2018 at 9:59 am
In the securities industry, SEC/FINRA require record retention on WORM storage (https://www.sec.gov/rules/interp/34-44238.htm, or it's a little easier to read here: https://www.17a-4.com/regulations-summary/). I will be interested in watching this develop in Azure.
July 18, 2018 at 11:14 am
I wish we had WORM storage back where I used to work when I worked there. If memory serves, according to regulations, adult data had to be kept for 15 years and adolescent data for 21 years. We just left that data in the database, but I wouldn't have minded it if we'd archived the oldest of it by putting it into some sort of WORM storage.
In a related topic, what happens when the whole places go away? The last 8 years of my previous employer they were laying people off every year. I was one of those let go in 2014. Finally, the whole place went out of business in 2016, laying off everyone that was left. But what happens in those cases? Since I wasn't there, I don't know what they did with the data. They may have archived everything onto tape and gave it to the governing/funding body. Or not. Just curious if regulations specify what's supposed to happen to the data, when a business that collects sensitive data goes out of business?
Where I work now is a large state agency with lots of different types of data. I'm sure that various regulations specify how long data is to stay around. And I believe they are.
Kindest Regards, Rod Connect with me on LinkedIn.
July 18, 2018 at 11:22 am
Sergiy - Wednesday, July 18, 2018 2:25 AMThink of those #metoo activists - do the even have a case?According to the new law it's even illegal to remember events from that distant past.Wait till they themselves are in court on accusations of breaching data protection legislation.:-)
Heh... funny thing... those that want to protect themselves keep data forever and those that have something to hide don't want to keep it at all. Because of laws that have no statute of limitations, there appears to be no sane middle ground for data retention.
--Jeff Moden
Change is inevitable... Change for the better is not.
July 19, 2018 at 6:42 am
Sergiy - Wednesday, July 18, 2018 2:18 AMYeah, criminals would definitely like a strong data retention policy.
Department of Homeland Security encourages cellphone and internet service providers to retain detailed records, as well as making them easily accessible.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
July 19, 2018 at 7:00 am
I've suggested a few times that SQL Server should have a database or table level property whereby the DBA may specify an intended use case as design time, and then the SQL Server automatically self tunes the indexing, statistics, recovery model, isolation level, trace flags and other settings dynamically. For example:
Optimize Workload For: OLTP, OLAP, WORM, ETL
Another advantage to this is that when a database is migrated to a newer release or cumulative update, the performance related settings could dynamically adjust to take advantage of new or updated features. Azure SQL somewhat, and to a greater degree Azure Cosmos DB, already does this.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
July 20, 2018 at 12:11 am
Eric M Russell - Thursday, July 19, 2018 7:00 AMI've suggested a few times that SQL Server should have a database or table level property whereby the DBA may specify an intended use case as design time, and then the SQL Server automatically self tunes the indexing, statistics, recovery model, isolation level, trace flags and other settings dynamically. For example:
Optimize Workload For: OLTP, OLAP, WORM, ETL
Excellent idea! You could use extended properties and build jobs to do this. A TTL setting would be good as well.
July 20, 2018 at 12:16 am
Jeff Moden - Wednesday, July 18, 2018 11:22 AMHeh... funny thing... those that want to protect themselves keep data forever and those that have something to hide don't want to keep it at all. Because of laws that have no statute of limitations, there appears to be no sane middle ground for data retention.
Tough cases make bad laws.
This is why laws are so complicated and verbose. There are so many "Ah but..."s. In fact there are so many that it's impossible to capture all cases so we end up with a discrepancy between the spirit of the law (which people agree with and respect) and the letter of the law (which keeps the legal profession well fed and in BMWs)
July 20, 2018 at 7:17 am
David.Poole - Friday, July 20, 2018 12:11 AMEric M Russell - Thursday, July 19, 2018 7:00 AMI've suggested a few times that SQL Server should have a database or table level property whereby the DBA may specify an intended use case as design time, and then the SQL Server automatically self tunes the indexing, statistics, recovery model, isolation level, trace flags and other settings dynamically. For example:
Optimize Workload For: OLTP, OLAP, WORM, ETLExcellent idea! You could use extended properties and build jobs to do this. A TTL setting would be good as well.
Azure Cosmos DB already has a time-to-live property for record retention. It also has automatic indexing, auto partitioning, SLA based resource scaling, and other turn key style performance features. I believe it's coming to Azure SQL, or perhaps Azure SQL will eventually be folded into the Cosmos DB platform. I think there is an analogy to made between Microsoft's long term strategy for Cosmos DB and their strategy with VisualStudio.NET that started back in the '90s. Cosmos DB will be for data what VS.NET is for programming languages.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
July 20, 2018 at 8:23 am
David.Poole - Friday, July 20, 2018 12:16 AMTough cases make bad laws.This is why laws are so complicated and verbose. There are so many "Ah but..."s. In fact there are so many that it's impossible to capture all cases so we end up with a discrepancy between the spirit of the law (which people agree with and respect) and the letter of the law (which keeps the legal profession well fed and in BMWs)
It also leads to contradictory and absolutely silly and sometimes harmful laws. Like my brother used to say, "Make a law, lose a right". 😀 Heh... just like doing a REORGANIZE on a random GUID key index, the problem is a self perpetuating Catch-22.
--Jeff Moden
Change is inevitable... Change for the better is not.
Viewing 15 posts - 1 through 15 (of 16 total)
You must be logged in to reply to this topic. Login to reply