June 27, 2023 at 11:36 pm
If it's feasible you could do a Full Update, you would update the original record on the source database with the new data first. This way, your fact table always represents the most up-to-date state of your data.
June 27, 2023 at 11:49 pm
Thanks Jonathan,
Yes, sorry, thought it was clear that the data would be coming from the operational/source system corrected. Basically, there's a variety of drop downs that users can change after the fact. I used the example of a date to keep it simple, but it might a subtype of some sort (for example, it might be a repair reason was changed).
Ultimately, I need to make sure the DW can handle these changes because they happen a few times a week.
June 27, 2023 at 11:57 pm
How long does a full load of the DW take?
June 28, 2023 at 2:08 am
About 25 minutes right now. We have about 10 stars right now and growing. I expect to have another 5 by next month.
June 28, 2023 at 7:07 am
About 25 minutes right now. We have about 10 stars right now and growing. I expect to have another 5 by next month.
25 minutes is quite fast for a full load of a DW. You could schedule a truncate and load of the DW once a week at a time when it is not being used much.
June 28, 2023 at 1:46 pm
Chrissy321 wrote:I probably should have stated my use case. It's not a traditional data warehouse but rather a tabular model on which PowerBI reports are built for analysis/business intelligence. My analysts don't want to see historical attributes, only the current attributes. Perhaps one day they will require point in time capabilities but not currently.
Just to clarify for me, please.... Does this mean that the "current" data lives no where else? That it lives only in your "tabular model" and not some table in a database somewhere else as the original source of the "current data"?
The source data is in an EDM system upstream where it is datetime stamped on creation. So there is some sort of history just not a proper type 2 SCD.
June 28, 2023 at 8:13 pm
What I'm getting at is that if users at the end need to make changes for reporting or whatever, there is something seriously wrong with the source of data and that data may be telling the truth about a problem that people aren't paying attention to or are consciously changing the results for (a nice term for "cookin' the books", an auditor's favorite find, I'm sure).
--Jeff Moden
Change is inevitable... Change for the better is not.
Viewing 7 posts - 16 through 21 (of 21 total)
You must be logged in to reply to this topic. Login to reply