Pairing Data Vault and Biml for DW Agility

Question

Pairing Data Vault and Biml for DW Agility

benjamin.seidle

SSC-Addicted

Points: 498
More actions
September 3, 2014 at 10:02 pm

#311346

Comments posted to this topic are about the item Pairing Data Vault and Biml for DW Agility

Viewing 15 posts - 1 through 14 (of 14 total)

You must be logged in to reply to this topic. Login to reply

mark 66658 SSC Journeyman Points: 93 More actions · Answer 1

Hi, thanks for the article on DataVault and BIML! I'd very much like to see a series of articles on the topic if at all possible. A more 'in depth' study if you will. I've done a fair bit of reading but have not implemented this in practice. Would love to be able to learn more.

Koen Verbeeck SSC Guru Points: 259105 More actions · Answer 2

This is a blog post, not a technical article.

Nice story, but I expected at least some scripts...

I hope this is an introduction to a great series about the combination of Data Vault and BIML.

Need an answer? No, you need a question
My blog at https://sqlkover.com.
MCSE Business Intelligence - Microsoft Data Platform MVP

Phil Parkin SSC Guru Points: 246960 More actions · Answer 3

mark 66658 (9/3/2014)
Hi, thanks for the article on DataVault and BIML! I'd very much like to see a series of articles on the topic if at all possible. A more 'in depth' study if you will. I've done a fair bit of reading but have not implemented this in practice. Would love to be able to learn more.

Agreed – this article is both interesting and a tease!:-) Please write more.

The absence of evidence is not evidence of absence
- Martin Rees
The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
- Phil Parkin

hennie7863 SSCommitted Points: 1653 More actions · Answer 4

Please more information. I'm interested in the technical implementation..

Knut Boehnert SSCrazy Points: 2951 More actions · Answer 5

Interesting teaser post. I am working on a similar project currently to rewrite our Data Warehouse and Benjamin made a good point to show that DV is not a good platform for reporting - this needs to be addressed separately.

Technically a DV is pretty much box standard. Each hub and link have their own unique surrogate ID on which you hang and track the business key. The concept is what makes a DV interesting. Instead of combining keys and data into one table you put keys into one table (hub or links) and data into others (satellites). So data is separated from keys. This basic core concept is what alienates lots of people.

The other core concept of DV is "All the data all the time". So no matter whether the data is good or bad or ugly you just load it into the DV and worry about the quality later. This gives the DV a power to be a repository of all the business data. Some other system creates data for invoices that are not in the original source system? Just load it into a new satellite. How to get it into reports? That is a different kettle of fish.

Data Vault is not so much a technical scripting. It is more about the design concept that a new person needs to understand. One very nice description can be found on Hans Hultgren's site (http://hanshultgren.files.wordpress.com/2013/09/data-vault-modeling-guide-2013-v2.pdf). To make an analogy running some scripts to implement a DV is like taking a car out of for a road trip without filling up the tank - you will get somewhere but your mileage will definitely vary and arrival where you want to be is for sure very questionable.

Koen Verbeeck SSC Guru Points: 259105 More actions · Answer 6

Knut Boehnert (9/4/2014)
Interesting teaser post. I am working on a similar project currently to rewrite our Data Warehouse and Benjamin made a good point to show that DV is not a good platform for reporting - this needs to be addressed separately.

Indeed. DV is more of a system of record, after which data marts are created for reporting purposes.

Knut Boehnert (9/4/2014)
The other core concept of DV is "All the data all the time".

I believe the DV community is now more believing in a "All of the data in scope, all of the time" 😀

Knut Boehnert (9/4/2014)
Data Vault is not so much a technical scripting. It is more about the design concept that a new person needs to understand. ... . To make an analogy running some scripts to implement a DV is like taking a car out of for a road trip without filling up the tank - you will get somewhere but your mileage will definitely vary and arrival where you want to be is for sure very questionable.

True, but the DV community usually boasts how easily DV can be generated (at least the Raw layer), so that's why I was very curious to see some BIML scripts to generate DV objects. Quite the dissapointment... 🙂

Need an answer? No, you need a question
My blog at https://sqlkover.com.
MCSE Business Intelligence - Microsoft Data Platform MVP

enders-437268 SSC Enthusiast Points: 155 More actions · Answer 7

For the non technicals, could explain how many tables were involved ? How long did it take to learn BIML ? Had any pre experience with other programing languages ? How often do you have to change the BIML scripts ?

For the techies do you use a staging database ? What is the load speed ? How many rows ? How many columns ? Is it Datavault 1.0 or 2.0 ? How do you do mappings ? How do you handle errors ? Does biml generate the tables in datavault or reads it from datavault ? Do you create SSAS cubes ? How do you implement those cubes ? As views on the Datavault or physical tables ?

Just a few questions

andycao Say Hey Kid Points: 698 More actions · Answer 8

andycao

Say Hey Kid

Points: 698

September 4, 2014 at 7:10 am

#1743191

what ender said! +1

Thomas LeBlanc SSCrazy Eights Points: 8085 More actions · Answer 9

Great description.

But, you need to give some examples and scripts and diagrams to make this a great article.

Now, you got me wanting to read the book, but I probably won't until I see tangible examples to get me to gravitate to a Data Vault.

Thomas

Thomas LeBlanc, MVP Data Platform Consultant

Data Wrangler SSC Enthusiast Points: 156 More actions · Answer 10

I too thought that this was the intro to an in-depth article. While the post may inspire us to consider new ways to design and implement a data warehouse, it would be most helpful to get details, such as trade offs made, roadblocks encountered, what you would do differently next time, and most importantly for this community, some technical implementation details. Please write more!

Jonathan Woodard

benjamin.seidle SSC-Addicted Points: 498 More actions · Answer 11

Thanks for the warm reception! More articles and details to follow soon. Meantime if you have any questions on specifics feel free to hit me up.

benjamin.seidle SSC-Addicted Points: 498 More actions · Answer 12

Beaten to the punch! Pretty good presentation from the BIML experts on how to use BIML to generate Data Vault Models

https://www.youtube.com/watch?v=QVS79-YcPsA

Koen Verbeeck SSC Guru Points: 259105 More actions · Answer 13

benjamin.seidle (12/10/2014)

Beaten to the punch! Pretty good presentation from the BIML experts on how to use BIML to generate Data Vault Models

https://www.youtube.com/watch?v=QVS79-YcPsA%5B/quote%5D

Thanks for the link!

Need an answer? No, you need a question
My blog at https://sqlkover.com.
MCSE Business Intelligence - Microsoft Data Platform MVP

hennie7863 SSCommitted Points: 1653 More actions · Answer 14

hennie7863

SSCommitted

Points: 1653

December 11, 2014 at 12:55 am

#1763962

Interesting...Thnx