client, I had the pleasure of meeting and working with a prominent individual
skilled in the art of designing data warehouses. Due to the nature and
importance of this BI project, we pulled out all stops looking for the right
person with the experience and knowledge that can do the job. All bets
were riding on this high-profile endeavor, as well as the reputation of IT as a
whole on the ability to deliver the end-product to the business on time and in
scope. As we were forewarned by the architects of the system that we were
facing an "impossible data warehouse" situation, we thought why not bring in the
person who wrote the book? So, we did. Why settle for anyone less?
Co-author of the book,
Situations: Solutions from the Experts, Chuck Kelley
- internationally known data warehouse expert - answers questions about himself,
the profession, data-warehousing, Microsoft's venture into the world of Business
Intelligence, and the tools behind it. I've even asked him to
comment on some of the articles relating to data warehousing, appearing here on
First, some background
on Mr., Kelley's bona-fides. Chuck has 30 years of experience in designing and
implementing operational/production systems and data warehouses. He has worked
in some facet of the design and implementation phase of more than 50 data
warehouses and data marts. He also teaches seminars, co-authored four books on
data warehousing and has been published in many trade magazines on database
technology, data warehousing and enterprise data strategies. He is
considered one of the early pioneers in the industry.
Currently, he
participates on the 'Ask the Experts' panel for DMReview, the premier business
intelligence, analytics and data warehousing publication site. I will even
highlight and reprint some of those questions and answers here; that I think
will be of interest to our audience.
So, if you have
questions on any of the aforementioned topics, feel free to post them at
DMReview and chances are good that Chuck or one of his esteemed DW/BI colleagues
will answer. Or, if you have any project work in need of DW expertise, and
would like to secure his services, he can be contacted directly at chuckkelley@usa.net.
As we collaborated
closely on this project, I figured that this would be a great topic for an
article of interest on SSC.com and share with you all things data warehouse from
one of the industry's top professionals. I'd be a fool to let an
opportunity like this go by, so after a few database favors, he graciously
agreed to the interview that I have brought you here today.
RP: What
actually brought you into this field, and in particular data warehousing?
Give us a little background on how you started and evolved into this line of
work
CK: In
the late 80s, I was working at Digital Equipment Corporation (DEC) in the
database group. I noticed there was starting to be some buzz around data
warehousing, but I was more in tune with operational systems. In 1990, I
saw a few books about Data Warehouse by this guy named W. H. Inmon.
I called the editor of
DM Review (at that time) and
asked if he had Bill's number. Turns out we lived in the same state and so
I called to discuss his perspective. Shortly afterward, we wrote a book on
using DEC products in the data warehouse environment. It was entitled
"Rdb/VMS: Developing the Data Warehouse". During that time, I also met
another guy named Ralph Kimball, who was working on
Red Brick Systems. Between these two fine gentlemen, I learned a lot about
business intelligence. I am very honored to know and to have learned from
them both.
[If you've heard the name before, that's Bill Inmon,
world-renowned expert, speaker and author on data warehousing, widely recognized
as the "father of data warehousing" and creator of the Corporate Information
Factory.
Ralph
Kimball, is an author on the
subject of data warehousing and business intelligence. He is known for
long-term convictions that data warehouses must be designed to be understandable
and fast - so, indeed Chuck is in good company!]
RP: When did
you realize the value of the data warehouse to companies and advent of business
intelligence in the enterprise?
CK: I
realized in the early 1990 that the value to enterprises of any size was going
to be fairly high. At the time, I told DEC that I would like to move the
consulting group I managed toward that arena, but they didn't see the benefit at
that time. There was still much to do with transaction systems. And
they were correct. So I decided to depart from them and start my own
consulting company dedicated to database (for the financial side!) and data
warehouse (for the bleeding edge side).
RP: What advice
would you give to others considering becoming a Data Warehouse
architect?
CK: You
should know a lot about transaction system databases -- how they are built, the
tradeoffs on the design, etc. Then learn everything (by pushing to the
back of your mind, but not forgetting, the things you learned about transaction
systems!) you can about dimensional modeling. Then learn how to read
marketing brochures and articles (including mine!) critically, since every one
of them will sound reasonable, but some of the ideas just won't work.
RP: BI is a
broad term, what is the breakdown of functional jobs
CK: I
think there is a number of different type folks needed on BI project.
Project Manager, Data Modeler, DBA, User Interface/Report Developer, Business
Analyst, ETL Developer are a few that come to mind. You will also need
production support type folks.
RP: Recently
appearing on SSC.com, was an article entitled "From DBA to DWA" about the
responsibilities of what he titled the "DWA" or data warehouse
administrator.
- Do you agree
with this categorization as a separation of duties from the dba.
- How do you
think the duties and tasks from the typical DBA vary from the DWA?
- Does this accurately sum up the role and responsibilities of the
DWA?
CK:
Somewhat, I would agree, but I think that the DWA and DBA have very similar
types of tasks and could probably be done by the same person/group. This
of course would be dependent on the size of the team, much like the question of
how many DBA's do you need.
RP: What
do you think of a company charging the DBA to support the data
warehouse?
CK: Why
not? Of course, it depends on the definition of support and whether the
DBAs are separated into Ops and Apps.
[Fellow DBA's: One
thing learned is to be sure that management understands the differences between
database administration/operations support and development and design. The
DBA is often painted with a broad brush, and often we wear many hats, so make
sure that expectations compliment the work you desire to do.]
To set up the next few
questions, let me give the readers a quick overview in the varying paradigms in
the data
warehousing field. Here, we
often hear about discussions on where a person / organization's philosophy falls
into Bill Inmon's camp or into Ralph Kimball's camp. Below is a brief summary on
the difference between the two, and then we'll get Chuck Kelley's take on
this.
Bill Inmon's
paradigm: Data
warehouse is one part of the
overall business intelligence system. An enterprise has one data
warehouse, and data marts source their information from the data warehouse. In
the data warehouse, information is stored in 3rd normal form. Based
on this paradigm, he developed what's known as the Corporate Information Factory
(CIF). CIF is a
logical architecture whose purpose is to deliver business intelligence and
business management capabilities driven by data provided from business
operations
Ralph Kimball's
paradigm: Data warehouse is
the conglomerate of all data
marts
within the enterprise. Information is always stored in the dimensional
model. In fact, the term dimensional modeling is also known as the
Kimball methodology.
RP: Is Bill
Inmon's "Corporate Information Factory" DW model still valid in today's
world?
CK:
Absolutely.
RP: What camp
do you fall into Ralph Kimball vs. Bill Inmon?
CK:
Both. I find very little difference between them other than the point of
view.
RP: Is one
model superior over the other?
CK: I see
them as basically the same. Bill talks about the data from a data
management perspective and Ralph talks about it from the end user's access
perspective. Both speak of taking data from transactions system (and
external data) storing them in a big storage area (data warehouse or
staging). Then from that, you build the data marts. Yes, I know that
there are some differences in how the DW and Staging areas are done, but I
believe that is irrelevant. I am sure if you talk to others, they will
vehemently disagree with me, and that is their prerogative. The great
thing about the computer industry is there are multiple ways of doing
things. As long as it meets the needs of our user community AND has good
performance AND does not cost too much AND fits our architected data
environment, then all is good.
RP: With
respect to your current project, what were some of the challenges in specking
out and coming up with the initial design? How much iteration do you go
through to come up with the final working DW model?
CK: My
current project had some issues with timelines, but we are slowly getting out of
that problem. So far we have gone through about 3 iterations, which is
about where we should be at this time in the project. I wish we had been
through the first couple of iterations earlier, but that's OK.
RP: What are
some of the most common questions you receive on data warehousing and business
intelligence?
CK: I
wrote a book on that exact topic and some friends of mine. The title is
"Impossible Data Warehouse Situations:
Solutions from the Experts"
Here is an excerpt on
the book's stated purpose. "There is no reason that each organization, as it
begins and continues to develop data warehoused projects, must wrestle with many
of the very difficult situations that have confounded other organizations.
The same impossible situations continue to raise their ugly heads, often with
surprisingly little relation to the industry, the size of the organization, or
the organizational structure. In this book we let you know that you are
not alone and your problems are not unique. We also offer hope to the
perplexed who see no obvious solutions to their problems."
[This extremely
detailed and well-organized book offers some very real-world practical
considerations and solutions to embarking on the design and implementation of
the data warehouse in one's organization. You can grab a copy of this
great tome on Amazon.com by clicking the above-entitled link.]
I'd
like to end Part One of this interview with a public service message. For
all you hard-working DBA's out there putting in countless hours in overtime, and
running 24x7 shops ,we owe Mr. Kelley a debt of thanks for his Dr. Phil moment,
boldly answering this one question of damsel in distress vs. DBA - actually
appearing DM Review Online.
Should I be concerned about my
boyfriend is spending 60 to 70 hours per week on a data warehouse
project?
DMR: My boyfriend is a
database administrator working on a data warehouse project. He claims the
project requires that he works 60 to 70 hours/week. Since the project began, he
seems distant and we hardly are intimate any more. I'm worried that he's seeing
someone on the side. Should I be concerned?
CK: First, let me
say that I am not a psychologist, psychiatrist, social worker or counselor of
any kind when it comes to relationships. Just ask my wife! However, 60 to
70 hours per week a few times during major deployment times is not unreasonable.
But, that would only be a week or two. I would be concerned that your boyfriend
is going to kill himself due to overwork. My wife complains about me being
distant during those same times. My suggestion is to ask him. (DM Review
Online, December 24, 2007)
So, honey, I'll be
home as soon as I possibly can! Keep my dinner warm for me.
In the next exciting
installment of this two-part interview with Chuck Kelley, we'll drill down more
into his thoughts on the Microsoft SQL Server Tools Suite, and answer some
common questions about the art of Data Warehousing.
Written by: Robert Pearl,
President
Pearl Knowledge Solutions, Inc.
mailto:rsp05@pearlknows.com
http://www.pearlknows.com/
Copyright ) 2008 - All
Rights Reserved.
Note: Not to be
reprinted or published without express permission of the author.
