June 30, 2008 at 9:09 pm
Comments posted to this topic are about the item Data Rich and Information Poor
June 30, 2008 at 10:27 pm
I've always felt the data layer should contain the raw metrics and the reporting (presentation) layer should display vectors. How the presentation obtains the units of measure is up to the BI architect in my opinion. They can certainly be hard-coded into the report. They can reside in a UnitsOfMeasure dimension in a Kimball data warehouse.
Wherever the units of measure reside, I believe they are the first order of converting data into information.
But there's more to information than units of measure. There's communication. Status-at-a-glance is important for triage during a catastrophic enterprise event. But again, it falls to the report designer to effectively format this information. There are studies about that suggest the human mind interprets color quicker than anything else (visually at least - similar studies suggest smell/taste is actually the most powerful and truly fastest sense).
Communication hasn't occurred until the information recipient understands the message. Although the data is important - especially data integrity - presentation plays an important role in communicating information.
:{> Andy
Andy Leonard, Chief Data Engineer, Enterprise Data & Analytics
July 1, 2008 at 1:02 am
What I came across in the last few years doing BI is that most companies don't know their requirements and this is one of the biggest obstacles implementing a BI solution.
From KPI Excel spreadsheets extracting and formatting numbers to a Data Warehouse with functioning reports is a big step and most companies don't seem to understand that "just fudging the numbers into a spreadsheet" just doesn't cut it.
True, a BI project is even at a small company a big undertaking and the pressure to deliver ROI is always high. Along the way the requirements change sometimes to be nothing that is comparable to the start of the way.
Kimball and others have developed nice ways of how to implement the technical aspects of a Data Warehouse but as long as there are no tools for managers to decide what they want from it the Data Warehouse is simply just a data repository.
The morphing from Data Warehouse to Information Repository is what makes a BI project successful and to accomplish this takes a team of quite skilled project members across the business that has the right combination of hard and soft skills. Unfortunately in the immature BI market it seems the focus of most companies is on the hard skills with the thought that data becomes information through miracle.
P.S.: Missing the first few numbers is not fair when presenting the Fibonacci series.
July 1, 2008 at 1:20 am
imho BI implementation is more about the experience of the people involved than the technology. MS SQL 2005 does help bring BI to the masses through SSRS and SSAS, and facilitates it through SSIS.
What it does not do is help to design the data warehouse. Designing a data warehouse is the single most important step of a BI implementation. Yes it takes time, yes it is best done in iterative steps, yes of course it depends on the data available from source systems, yes time is spent cleaning (or rejecting) data.
The terminology BI, I think, takes away from the real work of building the data warehouse. You can get useful "BI" (and "huge benefits and real dollars created in the company") from an Excel pivot sitting over a view of a properly designed data warehouse. Building BI is an iceberg project where the results can only be seen in the last ninth of time elapsed.
My main point though is that it takes someone who understands data warehouse design AND can communicate with the right business sponsor (could be multiple people and formal and informal) to
1) get their requirements
2) ensure they understand how long the task will take
3) gain and keep their trust by continually focussing on 1) and 2) above.
Cheers,
Adam
July 1, 2008 at 4:17 am
I would be a little embarrassed if I did not recognise the Fibonacci Series
I would be just as surprised though, if you could tell me what that series means ...
July 1, 2008 at 5:09 am
Well... I did recognize the Fibonacci sequence 🙂
For those curious about it: Fibonacci numbers
It has a lot of mathematical properties and is also related to the "classical" Golden Ratio 😎
Besides that, I agree with Steve: information is somewhat tied to how you use your data: so why having plenty of numbers if you are not able to extract their meaning and use them?
I think that information has to "shape" data flow and that this design has to be kept in mind when building your "information platform" around that.
BI does require a bit of "intelligence", doesn't it? 🙂
What I mean is that you cannot simply "crunch numbers" and automagically have good information from that, unless you carefully built everything so that these results are what you really need.
Just my 2cents 🙂
PS: @ Knut Boehnert
I've also seen the Fibonacci series starting with 1, 2...
Anyway you're right, it's usually defined starting from 0 and 1.
July 1, 2008 at 5:37 am
Thank-you Luca for proving my point - you provide more info on what the series IS, but not what it would mean if it appeared in a DB context.
July 1, 2008 at 5:49 am
I echo all of your sentiments. Having been a BI consultant for over 10 years, I'm very excited about the opportunity that SQL Server and SharePoint now present many customers ... low cost options, from a licensing perspective, for distributing BI.
Frequently, the BI consultant has to generate and maintain the project sponsor's "appetite" for BI. While it is utimately important to implement a data warehouse in parallel, one has to ensure that the customer is receiving incremental deliverables. This is increasingly important when you consider that many SQL Server implementations may not represent significant investments early on.
July 1, 2008 at 5:58 am
I don't believe everyone followed your link to the utilities blog that you referenced. The author mentioned a particular aspect of analysis twice that I believe to be important - spatially enabled datasets.
In the Geographic Information System (GIS) world, that is our bread and butter. Particularly in government, but to a significant degree in the private sector, including utilities, data often is only related by spatial relationships. Key structures are not always available in disparate datasets but if each separate silo maintains a spatial component - an address, a coordinate set (such as latitude/longitude or local coordinate system) - they can be analyzed together.
That is the true power of spatial analysis - the ability to look at datasets that have no common key but share proximity or some other spatial aspect.
By plotting customers on a map, businesses take a huge leap forward in moving from data to information. I encourage DBA and BI folks to give GIS a look.
July 1, 2008 at 6:48 am
There's a couple of dangers here.
For many businesses there is a practical limit to how much real use additional can provide. It's all cool and geeky to extract various patterns and statistics, but that may not actually help make the business any better.
Even worse is the possibility that invalid conclusions could be derived from data that is innaccurate, inconsistent, misunderstood or interpreted with a preconceived bias.
...
-- FORTRAN manual for Xerox Computers --
July 1, 2008 at 7:11 am
My experience, having worked both business and technical sides of BI, is that it takes superior understanding of business needs (questions needing to be answered) AND technical functionality (how to get the right answer) to produce the best results. IT budgets seem to be stretched so thin that BI implementation can be an add-on paid for by a business segment that is desperate for answers. And this is only the technology-related cost. With the need for business experts to define reporting requirements (if they know what they want), the need for analysts to determine data cleansing and ETL requirements, and project managers coordinating BI needs with other systems development projects, the costs add up rapidly.
Beyond initial BI system development, there needs to be a mindset that plans for continual improvement by involving BI teams in non-BI systems projects that will ultimately feed BI reporting systems. In many cases, operational systems designed without thought of long-term reporting impact require substantial efforts to analyze, cleanse, and integrate data for BI projects, thereby adding to the upfront cost of these projects. Until business and technical management view BI as an ongoing part of systems development, BI will continue to be just an afterthought and will seem to cost a lot up front.
John Yates
BI analyst
July 1, 2008 at 9:02 am
My employer is also data rich and information poor, but it's entirely his own fault.
First, he requests data collection without first determining how he's going to use the data. He just thinks the data will be good to have "if we need it" in the future. Big mistake. With limited time and resources, we shouldn't be developing applications or imports to collect data unless we are collecting it for a specified purpose. As a result, we have a lot of data sitting in our db waiting for a business need.
Second, because of excitement generated over the latest idea, it always gets priority over the idea that we've been working on for weeks or months. As a result, he keeps his limited resources tied up with the new data collection applications and, as a result, doesn't have the resources to develop applications that would turn data already collected into information.
Something tells me that my guy isn't just the exception.
July 1, 2008 at 9:14 am
Fibonacci Sequence of Numbers
July 1, 2008 at 9:16 am
Most collect the data they need. If they do not know that they need more or a better way to look at it, they are not willing to pay for it. If however the business requires some knowledge from the data we should be ready to provide access and tools to get them there.
BI is not the only approach to information, before many invest in the heavy lifting they look into reports. If you can get them interested in the information in the basic reports you can point them eventually into getting analytical analysis on the data and point them down the road to information. It is not hard, might take a little patience but it has and will be done again and again.
If you let them see what they can have they will start to justify it and will later pay for it.
Miles...
Not all gray hairs are Dinosaurs!
July 1, 2008 at 12:20 pm
I'm actually trained in using data to gather information about businesses and turn it into management plans. That and marketing.
My last job, the database I built turned into a huge resource because of BI capabilities I built into it, and analyses I pulled out of it. Very helpful to management. Worth literally millions of dollars per year. But to really pull that off, I had to have access to a detailed understanding of how businesses work, how money works, marketing, the specifics of the exact business sector and the specific company, as well as a general knowledge of scientific methodology, mathematics, statistics, data analysis, and a reasonable understanding of how to get data into and out of an SQL database. That takes more than one person, and if any ingredient is missing, the results will have degraded usefulness.
It can be done, but it has to be understood that it takes a lot of work to do it right. Worth it, but a lot of work. Many people want instant pudding and microwave meals, and aren't even willing to consider the alternative to instant gratification.
- Gus "GSquared", RSVP, OODA, MAP, NMVP, FAQ, SAT, SQL, DNA, RNA, UOI, IOU, AM, PM, AD, BC, BCE, USA, UN, CF, ROFL, LOL, ETC
Property of The Thread
"Nobody knows the age of the human race, but everyone agrees it's old enough to know better." - Anon
Viewing 15 posts - 1 through 15 (of 18 total)
You must be logged in to reply to this topic. Login to reply