This is kind of funny - IT juggernaut vs small mini-mart project...
-
The short of it is I was hired to dev a DB (SQL Srv) for my group, and generally be the IT guy of the group (for automation, etc). Well we need some SQL server space for our 50gb database.
Our IT dept has all these processes in place to limit data duplication and facilitate change control, which is fine, but it turns our 25-50gb mini-mart project into an enterprise wide project due to standardized dev and admin procedures.
I went back to my group after the meeting with IT with these results of "we need to create a project and get our dev and dba teams involved, do some requirements analysis", and the driver of this project in our team wants to find a way to do this thing on our local workstation we use to crunch numbers (MS 2k Pro box lol), and just circumnavigate IT.
So I said we could use MSDE if we want to split the data into 1.5gig sections, and just run seperate ODBC conns depending on which timeframe of data we need to pull in. It would be clunky, and kind of a pain, but it's doable, I think. Unless the box can't handle the stress, although I don't think it will be really huge.
We don't need much, just a place to dump data from multiple sources already in our internal environment, and a couple external environments. I already have the data structure, and we already pull pieces of the data into Excel. It doesn't have to be a huge undertaking.
So even though we have the big SQL EE resources through IT, the buearocracy involved is causing my group to use a mutiple DB structure to solve it's problems.
It's like the DB dev and admin are going to be easy due to it's size, but the IT dept is so big and procedural, it's crushing the projet; slowing it down to a crawl. Too much overhead and meetings.
I'm in the middle as I can see why IT needs those procedures, but my group (customers) still have needs to be met.
Damn.
-
You are not alone
I suppose if your data is nicely cleaved into small time chunks of 1-1.5GB each, then you shouldn't have too much of a problem. You'll run into difficulties when you need to get data from two different time periods though. If you have SQL EE licences spare and you have a spare licence for Windows Server, why not upgrade one of your boxes to server and install EE and then all is well... Fantasy land of course
Good luck - let us know how the performance differs and any troubles/workarounds you come across. Never know when others will have the same question.
-
Wow - not funny at all!
Unless you enjoy banging your head against the IT Juggernaut on a frequent basis you need to figure out a way to make the IT Juggernaut work for/with you rather than against you.
As the "new guy" you could probably get away with doing what you've proposed at least once - maybe more than once but you risk getting branded by the IT group as a rogue/non-conformist/problem which is NOT where you want to be. Best case you're on the outside looking in, worst case the IT folks are actively after your head / waiting for you to fall on your face.Also, think for a moment about your career path at this employer - unless you're dead set on leaving in a few years - you're eventually going to need to move up the food chain, which in your circumstances probably means into the IT group, the big boys & girls w/ all the good toys live there, not in a business unit. Unless your business unit has shown you some sort of career plan they have no clue what to do with you beyond using you to solve their immediate business needs. Outsourcing, etc. notwithstanding your long term opportunity is going to be in the IT group.
Nor do you want to be in a situation where the only thing you can get done on a daily basis is "feed" the monster you've created. Your MSDE hack/work around sounds like it might even work - but I wouldn't want to be the one responsible for it for the long term. Do you really want to spend the next X years being the only person who understands, can or will work with it?
So where does that leave you? I would consider something along these lines:(a) Go back to the boss and say something to the effect that "after thinking about it, I want to try to find a way to involve the IT group on this project - I know they can be difficult to work with/slow but breaking our 50GB database down to work on MSDE as we've discussed could really hurt us in the long term." or something to that effect - try to put as positive a spin as possible on the change of heart. You don't want to fall back on "hey, I'm just the new guy" though - better to say you've been reading up or consulting w/ others, etc. and have changed your mind accordingly.
Some things you may want to think about/quantify before this discussion:
(1) long term cost of supporting the application (e.g. if it breaks once a week and takes 10 hours of your time to fix it every time?)
(2) future needs/growth of the data mart.
(3) importance of the data mart once in "production".
(4) costs of/time to replace/redo the data mart when your little "pc as a server" blows a rod.
(5) data mart = reports & analysis = Reporting Services/Analysis Services rather than custom software hacked together on top of the wrong database engine/version.
(6) budget - make sure you've got some budget for hardware/software if IT wants you to pay for some stuff (or your business unit needs to buy it directly).
(b) Now, here's the fun part - assuming your boss didn't throw a gasket - you've got to figure out the best approach to get the IT group to participate in the project (and future projects) without having to jump through every hoop they can come up with.
From your post I gather that the IT group is pretty set in their ways - but somebody somewhere in the group can tell you how to short-circuit/bend the rules. If you don't know your way around the IT personnel well enough to identify the right person off the top of your head, I would approach the most sympathetic member of the group(s) you have already met with and try to get some private advice (don't even try it in a meeting).
The approach here is just like what you outlined in your post, "I understand the need for the rules, etc. BUT I've been tasked with getting this thing done NOW... how can we make this work? I can't go through months of meetings, etc. for a simple project like this. Can you help me?" If this person can't tell you how to get around the rules, ask him/her who can? Repeat until you find the right person, with the proper authority, who can help you bend the rules to your advantage on "little" projects, preferably on a permanent basis.
Some thoughts/talking points for dealing with the IT group:
(1) you've got to do your job - but want to be a team player.
(2) want their help/advice as appropriate (e.g. getting your hands on the right hardware and software as opposed to being forced to munge together solutions).
(3) can/will meet them in the middle when possible (e.g. "lite" versions of requirements, documentation, etc.)
(4) helping you to do your job is a good thing - in many circumstances like yours IT is viewed as the bad guy who can never do enough, etc. You are effectively a cost-free IT resource that can help make them look good.
(5) look for a R&D/proof of concept/"innovation" group that seems to operate outside of the rules already.
(6) IT groups hate having to clean up other peoples messes - which is a risk for them if you have to go it alone...
(7) maybe you need a "mentor" or dedicated contact in the IT group to help you with your interactions. Ideally a senior staff member with some influence/pull (or even better a Rabbi who can part the red tape).
(8) make it a point to make your manners over in IT frequently - get on their radar screen, preferably as a friendly.
At a minimum your goal in this conversation is to get the right hardware (nominal database server w/ RAID that can grow over time) and the right software for your data mart project. Try to get the IT group to buy off/support this project as a proof of concept or R&D project for which you are solely responsible but they are aware/approve of. If it works, they get some of the credit and a working model for the next data mart... (there may already be data marts in other business units, might not hurt to ask how that is handled).
In the bigger picture, you're trying to establish a working relationship with the IT group that will benefit you both for the short and long term. Benefits to them: (1) eventually you will want to move into the ranks, bringing with you a pretty deep understanding of the business which probably takes their business analysts years to develop and (2) they don't have a mini-IT group churning out work for which they may someday be responsible/called to account for, etc.. For you, establishing a good working relationship with your potential future employers is a good idea - as the sole developer/business analyst in your group how many promotions/salary raises, etc. can your business unit afford?
Worst case, you've done your part and tried to get them involved at a reasonable level before you take the bit between your teeth and get the job done - be careful to not burn any bridges if you do have to go it alone, you're going to want to continue to try to work with the IT group folks even if it's not on this project.
I would also be prepared to go back to your boss and recommend the purchase of some "real" version of SQL Server. SQL Server Standard w/ 5 CALs, is only $1400. Add additional disk drives and a hardware RAID controller to your existing workstation, say another $500 bucks if you go SATA/ATA133. My guess is that the cost of your time to get your MSDE scheme up and running would be far more than buying the appropriate software and reinforcing your hardware. You probably could get the MSDE scheme to work but probably shouldn't. 50GB of data doesn't sound like a whole lot until you have to recreate it from scratch after you lose your single hard drive and 30(?)+ 1.5 GB databases. Just imagine: your single hard drive goes down, the reports that your business unit VP has come to depend on for his/her meetings with his/her boss aren't available for the critical end of quarter meeting - and it's all your fault...
I didn't sit down to write you a book, just got on a roll, sorry.
Joe
-
I completely agree with everything Joe said in his response, especially about how your long term future may be in the IT hierarchy rather than your current business unit.
However, if you do decide to do whatever is necessary to get your app up and running "outside of IT", be aware of SQL Server "Workgroup Edition". This is a relatively new edition, is legal for "production work", and can be installed on a "desktop" operating system like Windows XP Professional. This usually gets your hardware out of the data center, and is a much better solution than MSDE.
-
I've been in this situation before. If the project is truly small in nature, you should whip up quick Visio and Word documents to present to your IT Dept...high-level architecture, requirements, diagrams, etc. Presenting proper documentation might help bypass the formal procedures.
mandav
-
Another twist would be write a proposal/project plan that states this first iteration of the project is a demo and use a dev licence and get IT buy in later down the road. The breaking the app up into desktop chunks is a bad idea.
-
Another bit of knowledge: I was consulting in the IT department as an asset manager before I got this perm job as an analyst/programmer, so luckily I already know a lot of people in that dept, and they generally all like me.
So I do have that advantage.
A second set of pert info is our 50gb db would sit on a box with a couple to a few other sql dbs already in production. Part of the reason for the standarization and testing, ad nausium is to ensure our db doesn't crash the box or soak up all the resources. I definitely see their point there.
So what I talked to my lead about is to create this sorry excuse for a DB, and when that's at least providing the bare nececities, we can then weather an IT project to create the real thing. I told him we are eventually going to need the real deal anyway. There's no way our "slice" solution is going to last.
He really, REALLY hates IT, for whatever reason, but did give the nod, so I'm going to continue to work on the IT thing on the side while creating my frankenstein. Hopefully I can get the IT thing done pretty quick, and I know a few people in IT that want to help me expedite it.
My main goal for this project is to learn SQL Server, so I can add it to my bag of tricks (of which there aren't many right now). I have no idea what my future is in this IT dept, but without experience, I don't have a future in any IT dept. Gotta take this chance to learn (pref using EE in a real environment).
I'm going to go back to one of the guys in IT, and figure out how we can go under the radar until the project is a bit more fleshed out. We don't need to include everybody on the initial push. Before I figure out how to create the project and avoid IT "black holes", there's not much reason to bring it to light. As well, I would rather keep my lead a bit out of the loop until I can show we already have a good thing in the works with IT. Quick action will hopefully win him over.
-
Nethervoid -
Is your "lead" a geek/technical or what?
If there is one common thread I'm seeing here it's that you want to get any version of SQL Server but MSDE for your development machine, that two GB limit is going to absolutely kill you in the long term. Just think, 35+ different databases that have to be tied together or updated one at a time, etc. Spend your time mastering SQL server rather than going through all the brain damage of trying to get the "slice" to work.
One thing to ask your IT guy about would be your companies terms with MS for running SQL Server. I've been in environments where it really didn't matter what version of SQL Server you ran in development - the organization only paid MS for licenses for "customer facing" database servers. Your IT guy may be able to just hand you a copy of EE, after all it's only for development, right?
Joe
-
My group are all business analysts. I'm the only one who has bus analyst and IT experience. They don't know much about IT at all.
I'm going to keep pinging IT and see what we can come up with.
-
My advice is a step back from things: What are your personal midrange goals and long term goals?
Do you want to work at this place forever? Then learn the beaurocracy and how to work within it at the expense of your technical expertise and true success of your projects.
Do you need to get some SQL experience before you move on to better places? Then do what you can while your can, and try and make something work, but be ready to get out of there.
Do you not need this job or can get another without it hurting to bad? Then to quote from a movie, "get out!"
I have many, many stories from over the years but the lesson is always the same: If there is beaurocracy, and it is in even the slightest way a hinderance to the tasks at hand, you're better off not traveling down this path. Believe me. I've had simple three day technical efforts that could not be completed after three months of effort to find a crack that would allow the business problem to be solved.
Beaurocracies grow, they don't shrink or disappear. They are on a growth curve and even if you find a hole or two that allow you to solve the business needs, these holes will be pinched off over time.
-
I would get a copy of developer ED SQL 2000 and find out what stats the IT guys want and then learn how to get them from your development box.
The developer ED SQL 2000 costs about $50.00, but can ONLY be used in development not production.
Tim S
Viewing 11 posts - 1 through 10 (of 10 total)
You must be logged in to reply to this topic. Login to reply