December 23, 2003 at 4:30 pm
I am currently under a time constraint and need to have 24 medium to large cubes updated nightly. Currently when running all 24 cubes serailly it takes between 4 to 6 hours depending on server utilization I believe. The process cannot start prior to 1:00 AM due to some batch processing that needs to occur. I took the existing cubes and made logical packages of three, because in most cases, their are two normal cubes that need to be combined to a virtual cube for summary reporting by customers. I was able to create a master job where it calls the DTS sub packages and broke them down to only run two threads. (which my hope was to cut the time it takes to update all the information from 6 hours to 3.) The package starts out fine with one task to update all the shared dimensions first. Then I have two parallel tasks updating different cubes in the same database. The process doesn't fail but the next two tasks ran for about 3 hours with no progress. Even if I was bogging down the system the two packages by themselves should take no longer than an hour. Can I not update cubes in parallel from the same data base repository???
December 26, 2003 at 8:00 am
This was removed by the editor as SPAM
December 31, 2003 at 9:28 am
Okay - way out on a limb - no time to fact check, you assume all risk, etc.
As I recall, all OLAP tasks "execute on the main package thread", so no parallel possible directly in DTS2000. You even have to mark the "execute on main package thread" on ActiveX tasks that modify OLAP tasks, so the limits are pervasive.
There is a parallel procesing utility available that allows you to pick multiple partitions (even from multiple cubes) and run them in a user-configurable number of streams. Note that we ran into a performance problem going over 2 streams in the 32-bit version. Now we also have very memory intensive cube processing requirements. But 2 was better than serial.
Note also that the parallel processing utility may not be a supported tool.
Hope this helps.
Larry
Larry
January 3, 2004 at 9:52 pm
Larry:
thank you for the response. I have read several books on the subject of parallel processing, but none of the articles of books were very comprehensive. It appears that my server was a capacity since it is quite small. Do you have a name for that utility and if so what the licencing requirements are for that utility.
thank you!
January 5, 2004 at 2:27 am
How about running two DTS packages, both sceduled to run at the same time?
Keith Henry
January 5, 2004 at 10:06 am
From the "announcement" email:
QUOTE:
It is with great please that I announce an updated version of the PPU (V3) is now live.
See the attached readme.doc file for all of the new features and bug fixes.
My thanks to *all* who participated in the beta program and worked with me to address the issues that arose.
It can be downloaded from:
This utility allows you to process Analysis Services objects (cubes, partitions and dimensions) in parallel.
Complete source (VB6) is included.
END QUOTE
Larry
Viewing 6 posts - 1 through 5 (of 5 total)
You must be logged in to reply to this topic. Login to reply