The Problems with SSIS
-
I saw quite a few blogs referencing this entry, and no, the links aren't listed below. That's one of the reasons this newsletter is valuable. I spend the time going through sources and consolidating and limiting what you might be interested in.
The entry is by Oren Eini and lists the 15 things that he hates about the environment. A number of SSIS gurus picked up on the entry and added comments or their own blog entries. I'm someone that hasn't used SSIS very much, and am more in the middle, though I think there are quite a few fundamental problems with the way this ETL tool has changed.
To me, while it's a great, powerful programming environment, the bar has been raised so high that I really see most DBAs struggling with the tool. There were people that struggled in DTS at times, and not just trying to do complicated scripting or loops. There were DBAs just trying to import or export data with some strange rule and had a hard time. They needed a cleaner, easier to configure tool and instead of getting that, they got a Visual-Studio programming environment that most of them can't use.
You might say they have no business being a DBA, but I disagree. There are lots of people that need to do more than the wizard allows, but not much more. They need an intuitive, easy to use tool. For the whiz-bang gurus like Jamie Thomson and Simon Sabin, give them a high tech plug-in for VS that lets them access the full environment. For DBAs, give us something like we have in maintenance plans.
But include Version control and an easy way to migrate stuff from development to production. And STOP HARD CODING ANYTHING. Let every single @#$%@#$$@$ thing be changeable by the developer.
There's also a rebuttal if you're interested. My favorite: Yeah, your point is "It works on my machine".
Soft Skills
I'm very much concerned with the skills and capabilities of employees outside the technical ones; those are the ones you can usually teach someone in this industry. However the softer skills pointed out in this article, like project management, are just as important. I'd put teamwork at the top of my list, but there are definite skills beyond being able to program or administer a server that can set you apart from others.
One of the things that I really stress to people is finding a good fit with your job. Make sure you and the people in the company can work together in addition to being sure the job is something you want to do. It makes finding a job hard, and you might have to turn down some offers, but in the medium to long run, I think it's worth it and it will make every day at work a better one.
Steve's Pick of the Week : Inside the High-Tech Hunt for a Missing Silicon Valley Legend
- I, as do many others, still mourn the passing of Jim Gray. While it's possible he's still alive, it's been a long time and I count myself lucky enough to have heard him speak. Read this very interesting account of how the the tech community made an effort to help. -
http://ayende.com/Blog/archive/2007/07/24/Rhino-ETL-First-Code-Drop.aspx
Oren is developing an ETL product and he slams SSIS? This seems an "interesting" response to the issues he perceives with SSIS.
I applaud the applomb, but I have to wonder which is more cost-effective: learning the proper syntax of SSIS or writing your own ETL engine?
:{> Andy
Andy Leonard, Chief Data Engineer, Enterprise Data & Analytics
-
No it works just not fully documented, using proxy account with the Agent to move data from one location to another worked from SQL Server 7.0. There are issues not documented that I usually address with users like not creating constraint on destination tables or use temp tables as an intermediate location before destination. A user asked me if it is a restriction no it is just the nature of delimited files passed to a table with contraints that sees null values, Microsoft did not document it because it is not related to their product just the nature of data.
Kind regards,
Gift Peddie -
I don't agree with someone slamming SSIS when they have an agenda - such as having their own ETL product in development. I really get a kick out of the list of faults starting with a dislike of the error handling in SSIS and the second remaining problem in his own product being decent error handling.
I have to say that SSIS has it's share of problems. It took me awhile to figure out a good deployment strategy (once I had a good plan, package configurations have done just fine), the UI was confusing at first, and creating custom components was tricky the first time.
Compared to the last ETL product that came with SQL Server, this is a HUGE step forward. Most of the people complaining about SSIS are saying it is too hard to learn to use correctly. Well, new products come with a learning curve. I would guess these people are also having trouble with the service broker. The line between developer and DBA has gotten pretty blurry recently. Perhaps the DBA's that are having trouble with SSIS need to give up ETL control to their developers who will be right at home in Visual Studio with VSS handling their version control.
-
Michael,
That's a good point and I hadn't thought about ETL becoming more a developer's area. However far SSIS has improved the ETL capabilities of SQL Server, it's also left behind a lot of people that NEED to do ETL work. Not complex, fuzzy transforms, but simple move data in and out from multiple formats. Handle loops, etc.
I'm starting to think that we really should have a stripped down, simple SSIS in SSMS. Something more like DTS, but with the capabilities of loops and a few other tricks. Then include the high powered version in BIDS for the people that really need it.
Above all: stop hard coding file paths and figure out a way to make deployment between test and production easy.
Andy, thanks. I hadn't realized he was building his own tool from his blog. I'm not sure this is an agenda to promote his own tool, which seems to be open-source, not for profit, or if it is frustration with SSIS.
-
"And STOP HARD CODING ANYTHING. Let every single @#$%@#$$@$ thing be changeable by the developer."
ROFL. Oh yeah, I'll drink to that!
-
As I replied on Oren's blog, 75% of the "issues" are addressed with training and experience. I've trained hundreds in SSIS - and any of them will readily identify the 75% I reference.
SSIS comes with a steep learning curve at no extra charge. There's a difference between "this is wrong" and "I don't understand this".
There are lots of good training courses out there - I know Steve is involved with a group that provides SSIS training. I make a living teaching folks SSIS for a different company. There's also lots of information available here at SQL Server Central, from books, from the newsletters here, and from blogs here and at Microsoft and elsewhere.
In short, there's little excuse for a capable developer or DBA to claim SSIS is too difficult to learn.
:{> Andy
Andy Leonard, Chief Data Engineer, Enterprise Data & Analytics
-
good resources and you might be right. I know it's worked for me in some simple projects, but the time to get up to speed was long. Much longer than I'd like for relatively simple in/out data movement.
And I am involved with End to End Training. Brian Knight, my former partner here, and the author of a couple SSIS books does an (almost) monthly class in FL. I haven't been to the class, but I hear good things from students that go through it.
Viewing 8 posts - 1 through 7 (of 7 total)
You must be logged in to reply to this topic. Login to reply