October 9, 2015 at 3:16 pm
Comments posted to this topic are about the item The Reality of Data Governance
Best wishes,
Phil Factor
October 10, 2015 at 8:37 am
Seriously I found this article one of the best of the last years. The issue described is the one I've been facing over and over in large organizations across the world. And it's extremely normal to face several problems just because some people unconciously sabotage initiatives or hide information.
At the time being I'm working again on a huge project which mainly requires the best data governance we can achieve and it's exactly what's described here.
Congratulations for capturing the essence of one of the most common problems in IT.
October 10, 2015 at 8:59 am
Phil:
Great article. I remember working 20 years ago with a company in Canada. One of their Product Sales Table columns was BrokerId. I asked how that was defined, valued, modified etc. After a very long search through all too many people who didn't really know, it turned out to be the Tax Id assigned by the Canadian "IRS." The company had long considered it a way to geographically identify the location of a product's broker in a specific Province. Thus, there were formulas, programs, etc. written to take the sales of a specific broker (via the Broker Id) and then report on the sales of that geographic location which then was used to compute the commission of the Province Product Manager. Once I found out what the value really meant, I asked, "What happens to the sales report and commission when a broker moves from Yellow Knife to Toronto? After all, their Canadian Tax ID number wouldn't then change.
Again there was a "Deer in the headlights" look back at me.
Now, while what follows is certainly self-serving to my company, please let me point to you all to the Whitemarsh Website (www.wiscorp.com), and to a set of "Short Papers" all dealing with "data." Link is http://www.wiscorp.com//shortpaperseries.html.
Virtually all the papers were written AFTER a consulting assignment, not before one. Hence they result from the wisdom gained from what is really "scar tissue."
October 10, 2015 at 10:36 pm
remember that different people need different data.
Where I work accounting, operations and support data aren't the same. There is no one with any power that wants to communicate about the differences.
October 11, 2015 at 2:36 am
The basic gist of your article as I understood it was one of scepticism directed towards (perhaps unconscious) sabotage to a project. I think that's a point worth keeping in mind; but I think in general my experience has tended to include more these situations:
* Ignorance; the person you are asking is ignorant of the answer or meaning of something (but perhaps unwilling to declare that, so they fudge it);
* Many people find it difficult to distinguish between the way things actually work, from how it should work, from how they would like it to work. Depending on who you are speaking to, and the context in which you speak, you may get their answers couched in any one of those three frameworks without them expressing that!
* Competition between teams for resources; in this instance people actually demand your technical resource and they'll try to get the resource for themselves. The net impact might be that projects get interrupted and / or slowed due to the need to justify it over and over as other teams claim their project is a higher priority for the business!
Speaking for myself, I've constructed some systems from scratch but probably most of my career has been working with pre-existing databases where the data-model is basically complete (but poor) at my arrival... Thus, I have to believe (hope?) that your point that there is only one 'truth' in data modelling complete or wrong (:w00t:); pragmatic and workable solutions are achievable through evolution (even from inefficient structures).
October 11, 2015 at 10:13 am
nigel 69291 (10/11/2015)
Many people find it difficult to distinguish between the way things actually work, from how it should work, from how they would like it to work.
Amen!
Then there is the nightmare of figures generated by a buggy system that everyone knows are a polite fiction but to correct the application would result in a massive upheaval as per Phil's article.
October 12, 2015 at 4:50 am
I think I even forgot "how it used to work last week, last month, or last year" 🙂
October 12, 2015 at 7:15 am
Phil's article assumes that we as DBAs and accidental DBAs have an influence on the actual database structure as designed. In most realities, we deal with what we get given, possibly a schema that has existed for years, if not decades. Or get ignored as Cassandras* when we raise valid constructive critiques.
I'm dealing with design issues that the developers and managers have ignored for years that only become an issue after large amounts of data have accumulated over time. But I pointed this out at the origin of this application that this would happen. Now schema changes are more costly and the burden and solution is bourn by the admin and the users, not the developers and managers.
October 12, 2015 at 8:11 am
When it comes to data governance, in the absence of any preexisting official documentation and governance, there simply is no deterministic truth. You're attempting to measure something that is subjective and evolving from one day to the next. What you're really doing is slowly and methodically creating a truth that will eventually be imposed on the (dis)organization. Simply because things don't appear to add up, it doesn't necessarily mean you're missing something, it's very well possible that it simply doesn't and hasn't ever added up.
It's like building up a political government in a nation that's been in a state of civil war and chaos for decades. It may be important to document how everything currently works (in all it's various and conflicting forms), but make no mistake that the ultimate goal is to impose a more organized system that dictates how things should work, so that can be used as a framework going forward.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
October 13, 2015 at 6:15 am
I found this to be a great and insightful article...but for a slightly different reason.
If a wise IT professional encounters the kind of problems in understanding with business users as has been related above, imagine what happens when some of these business users, often in upper echelons, make IT-centric decisions in searching for solutions to a business need.
IT professionals who have earned their scars and stripes know that knowledge of the business is at least as important as technical dexterity and skill. Oftentimes more. Buying a vendor's shiny product or hiring a bunch of code grunts who are skilled in the latest BI tool is no substitute for people who have innate knowledge of your particular business. And sometimes there is no easy answer to a company's IT needs.
Many a project has floundered not for lack of IT skills but for lack of understanding how the company needs to use its own data. Happy is the business that has a knowledgable and forthcoming base of users and cultivates (as much as is possible or practical) its own technical staff...and encourages them to work together to know each other's spheres. It has been my pleasure to work at some of those.
October 13, 2015 at 6:49 am
I had to go to wikipedia to get a good definition of data governance:
https://en.wikipedia.org/wiki/Data_governance
There appears to be several areas affected by it:
1. Quality through constraints and process controls.
2. Protection through security and tracking.
3. Safety through backups and redundancy.
These tasks fall upon the DBA with some guidance from management and laws. The details would overwhelm managers and therefore get ignored and fall upon us. Luckily we can implement most of this and then evolve it when complaints arise.
The common argument is in not matching authority with responsibility. We must build the framework for that to occur but we don't have responsibility for the authority of others. For example, data entered by users is stamped with their key so they will get blamed for incorrectly entered data, not us.
October 13, 2015 at 7:02 am
Data Governance is a lot simpler when databases and IT teams are not operating in silos. It helps if there is one application for each line of business, one set of web services that support data entry, one data warehouse that feeds BI reporting, one SharePoint site visible to all, and a cross-functional round table meeting that occurs once a week or at least once a month. Not only does it allow the organization to cross-pollinate IT skills and business domain knowledge, it also promotes standards and governance. That exists in concept almost everywhere these days
... but few actually do it in practice.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
October 13, 2015 at 9:07 am
Eric M Russell (10/13/2015)
Data Governance is a lot simpler when databases and IT teams are not operating in silos. It helps if there is one application for each line of business, one set of web services that support data entry, one data warehouse that feeds BI reporting, one SharePoint site visible to all, and a cross-functional round table meeting that occurs once a week or at least once a month. Not only does it allow the organization to cross-pollinate IT skills and business domain knowledge, it also promotes standards and governance. That exists in concept almost everywhere these days... but few actually do it in practice.
I think that depends. I've found that even when you have great collaboration between developers, database professionals and IT, you still have the end user who is separated. Then if you have your teams split across many officials both in the same region and international, it makes things a bit difficult.
Then of course there is the one data warehouse mentality too. I really want this to work, but with the scability issues with SQL Server and the different use cases of the data, it makes it hard to consolidate everything from teams to technology to one single source of truth without the need to partition. This adds to the issues with data governance that need to be considered and addressed.
November 4, 2015 at 6:37 am
I find that it is the unwritten and rarely spoken business driver rules that are the hardest to pin down. I don't mean how an application works with the data, as sometimes this can be backwards engineered, but the data that is manually handled even when kept in a data store including that which drives manual processes.
I too have found that some people are prepare to mislead you so that you never know when you have got it right.
This always has a feel of job protection.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
Viewing 14 posts - 1 through 13 (of 13 total)
You must be logged in to reply to this topic. Login to reply