April 7, 2008 at 11:43 pm
Comments posted to this topic are about the item Virtualization for Database Development
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
April 8, 2008 at 1:49 am
Thank you for sharing your struggles ...
Especially the fact that despite the warnings we give, people confirm them not to be a problem (again and again) until they are actually confronted with these warnings and then hell breaks out.
I'm no longer feeling alone :w00t:
Johan
Learn to play, play to learn !
Dont drive faster than your guardian angel can fly ...
but keeping both feet on the ground wont get you anywhere :w00t:
- How to post Performance Problems
- How to post data/code to get the best help[/url]
- How to prevent a sore throat after hours of presenting ppt
press F1 for solution, press shift+F1 for urgent solution 😀
Need a bit of Powershell? How about this
Who am I ? Sometimes this is me but most of the time this is me
April 8, 2008 at 5:35 am
The thing is, I think we could have made it work. But the expectations were so unrealistic that while we had functionality, we didn't have perfection. Sans perfections, they pulled the plug. After a while I got some of the feeling that Cassandra must have had when dealing with the Greeks.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
April 8, 2008 at 5:46 am
Did they pull the plug only for the virtualisation part or for the full ad-hoc regression test infrastructure(s) ?
Johan
Learn to play, play to learn !
Dont drive faster than your guardian angel can fly ...
but keeping both feet on the ground wont get you anywhere :w00t:
- How to post Performance Problems
- How to post data/code to get the best help[/url]
- How to prevent a sore throat after hours of presenting ppt
press F1 for solution, press shift+F1 for urgent solution 😀
Need a bit of Powershell? How about this
Who am I ? Sometimes this is me but most of the time this is me
April 8, 2008 at 5:57 am
Just for the virtualizations. We already do resets from production as a part of our QA testing. We were just hoping to incorporate a similar process into an automated development methodology. No joy.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
April 8, 2008 at 7:34 am
Thank you for the interesting article! I'm just starting to try and get real virtualization going as a development environment and have run into similar issues, too. Since we're still "trying out" utilizing VMs to develop and test things, we also are working against a very limited budget. I.e., VMware Server (free), old retired server hardware (CPU, RAM, and disk storage definitely limit what all we can do), and then trying to get everyone on the same page.
It would be nice if there was a better (free) way to manage everything, too....as we don't have the resources now to go with one of the relatively expensive management tools.
Either way, it's a good learning experience and we're starting to see how powerful it can be. Thanks for the idea of versioning the VMs with source code software, too!
April 8, 2008 at 9:00 am
What were the key factors make to choose virtualized OS (VMware) vs. virtualized SQL (instance)? VMware will introduce some perform overhead, as well as extra admin effort. On the other hand, if using SQL instance, to deploy a standard SQL instance could be a easy routine process.
April 8, 2008 at 10:30 am
Yu Yang (4/8/2008)
What were the key factors make to choose virtualized OS (VMware) vs. virtualized SQL (instance)? VMware will introduce some perform overhead, as well as extra admin effort. On the other hand, if using SQL instance, to deploy a standard SQL instance could be a easy routine process.
The virtual systems represented complete application environments, not just databases and database servers. The trick was to keep all the servers integrated with a particular version of the functioning software including the database code and the servers, BizTalk, client code, etc. Talking about deploying any one database or creating new instances for deploying databases is easy. The hard part was coordinating that process with all the other servers. The parts that we got to work, worked well because everything, all code, moved together. Does that make it a bit more clear?
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
April 8, 2008 at 10:47 am
Grant,
Sorry to hear of your experience with VMware. I recently helped delploy VMware in production. The company had nearly 50 physical SQL Servers, almost all use SQL Enterprise version and under utilized. Two big SQL2000 Servers were in a Active/passive Cluster.
We elliminated the need for all Enterprise versions and clustering by employing VMware. We greatly reduced the amount of hardware and expensive liscensing of the Enterprise version (much was not really needed). VMware will spin up a new SQL box if a hardware failure occurs on another.
Patrick.
April 8, 2008 at 11:17 am
It was basically functional. The key issues, apart from deployment issues & some performance, were around how we could manage the virtual versions of AD. It apparently wasn't surmountable.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
April 8, 2008 at 11:26 am
We have worked with virtual servers extensively, both in the context of cloning entire environments, and in the context of insufficient machines (the live environment is 9 servers, and we have 5 machines in the QA lab, and we're not willing to re-use dev boxes as QA servers, THAT is a nightmare for deployment)
In my experience, project managing deployments onto virtual servers is a very good way of pre-testing deployments onto live machines, mostly because you can save a base state in a way that you cannot do when uninstalling windows - in other words, you can have a clean windows, or windows + SQL, or whatever, install by just "shut down, copy, paste, startup"
We were working MS Virtual Server and Virtual PC, not VMWare, only challenge we experienced is that the servers could not attach to the USB ports, which I understand VMWare can.
Most of our development (lots and lots of sharepoint) happens on VPCs - or at least, sharepoint resides on a VPC, and Visual Studio on a local box, one place we've found performance problems.
But have a look at this post from MSDN blogs about performance:
http://www.hanselman.com/blog/VirtualPCTipsAndHardwareAssistedVirtualization.aspx
https://www.microsoft.com/windowsserversystem/virtualserver/default.aspx
http://blogs.msdn.com/virtual_pc_guy/archive/2006/03/14/550712.aspx
VPC instances can be optimized a number of ways
1. Have a processor that supports hardware virtualization. It does matter! See the diagram below:
2. As you could see, there's an option in the VPC settings that allows hardware-assisted virtualization if your processor supports it. Unfortunately, in my case even my notebook PC does not support that. My processor configuration is as follows:
According to Scott's blog, and I quote him:
If you're running Intel, refer to this table at Intel to see if your chip supports VT. All the Core Duos support VT except the ones that end in "E" like the Intel Core Duo processor T2300E .
3. In no particular order, you should check the Intel Virtualization Technology (VT) web site to find out what processor supports hardware-assisted virtualization, also known as "Virtualization Inside" (cool name). Also check out AMD virtualization technology. I do not take sides, as I have processors from both AMD and Intel.
4. My home PC has an AMD Athlon 64 X2 Dual-Core processor and it supports hardware-assisted virtualization. Whenever I am preparing my demo on a VPC image, I work from home (a valid reason to my boss that I have to work from home)
5. Memory allocation: In the above VPC instance, I had 1.6GB of memory allocated to my VPC instance. In my home PC, I allocate 2GB, or more if I do not run many applications in the background. Size does matter.
6. Do not run your VPC on the same hard disk as your host OS. Run your VPC on an external HDD that has a high RPM speed. Or if you PC has two separate physical HDDs, run and store your VPC on the HDD that is not where your host OS is installed. I am using an external Maxtor HDD with an external power source. Do not use those tiny 2.5" pocket HDD that draws power from another USB port. The power from another USB port is not enough to juice it up, besides most likely the RPM speed of the pocket HDD is low.
7. I compress the folder which contains my VPC image. The reason behind this is not just because I want to optimize my storage space, but even more importantly is the fact that I want to reduce the amount of disk reads from the HDD. It's better to let my CPU work hard rather than my HDD work hard. Everyone knows that the weakest link in terms of the performance of a PC is the HDD.
8. This is how I compress the folder, right-mouse click on the folder and choose Properties:
9. You should also compact your VPC image (I don't do this as often as I should though). Here's how:
10. If you are running your VPC on Windows Server 2003, for instance, use Microsoft Virtual Server 2005 R2 instead.
If you want to read more about VPC optimization, you may check out the following blogs:
Scott Hanselman
The Virtual PC Guy
April 8, 2008 at 11:48 am
Great info. If they relaunch the effort I'll definately use this stuff as a reference. Thanks.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
April 8, 2008 at 11:48 pm
My company is also considering to implement virtual servers. This input will really help us to consider. Thanks Grant Fritchey for sharing your inputs. Also special thanks to Mark Stacey for his valuable comments.
🙂
April 9, 2008 at 1:32 am
This looks like a bad experience. The trick with using VMs is to treat them - logically speaking - as real machines. Then the next question would be: why would you install sql server AND application on the same server? And next: why would you integrate software changes in a development environment? If having real servers, wouldn't you use an integration environment with application and database servers? This would be the right environment to keep security under control because not any developer will deploy in this environment, but the DBA will do (database) deployments and db versioning. As a note, being a DBA, I talk only about the db side of deployments, but the actual work has to be done with a designed person that deploys application changes as a team.
And an important thing to make a DBA's life easier: do not bring developer VMs under domain; you cannot and shouldn't control any AD change.
If you would need to re-create a SQL Server VM with same name as an old one then, again, treat this one as a real server, which means assign a static IP which, by the way, it's not necessary to be the same with the one for the old vm, rename the server ann then rename the sql server instance (if already installed) and finally, decommission the old VM.
There are many other things to be said both pro or against using VMs for sql servers. Hope these lines would help someone in setting up VM environments.
April 9, 2008 at 5:41 am
michaela (4/9/2008)
This looks like a bad experience. The trick with using VMs is to treat them - logically speaking - as real machines. Then the next question would be: why would you install sql server AND application on the same server?
We didn't install in the same machine. The system we were trying to implement involved having a set of virtual servers that could be created as a group. That way, the application server, the web server, the database server, would all be in a common, working state, synchronized like it was a working production environment. Then copies of that environment, all the servers, would be "deployed" for use by various development teams.
And next: why would you integrate software changes in a development environment? If having real servers, wouldn't you use an integration environment with application and database servers? This would be the right environment to keep security under control because not any developer will deploy in this environment, but the DBA will do (database) deployments and db versioning. As a note, being a DBA, I talk only about the db side of deployments, but the actual work has to be done with a designed person that deploys application changes as a team.
The problem with saying that the developers shouldn't have an integrated environment is, what happens when they're developing code that relies on interaction from other sources? This was designed to support a Service Oriented Architecture (SOA) approach to development where each application provides a service to the others. Some apps absolutely required the other apps to be in place to work. So you're faced with one of two choices, each of which we've tried to work with before we tried this alternative. Option 1) all the developers work on the same environment, but then code changes in Service 'A' breaks Application 'B'. Option 2) Have Service 'A' deploy their code to Application 'B' servers when it's "ready" but then you see your developers constantly troubleshooting new deployments and integration inconsistencies instead of developing new code.
And an important thing to make a DBA's life easier: do not bring developer VMs under domain; you cannot and shouldn't control any AD change.
If you would need to re-create a SQL Server VM with same name as an old one then, again, treat this one as a real server, which means assign a static IP which, by the way, it's not necessary to be the same with the one for the old vm, rename the server ann then rename the sql server instance (if already installed) and finally, decommission the old VM.
There are many other things to be said both pro or against using VMs for sql servers. Hope these lines would help someone in setting up VM environments.
Ah, but then you get all these servers communicating out into your general network instead of safe behind a "fence." Suffice to say, it didn't work out, but it was a very good idea.
Also, the DBA's weren't responsible for administering all these servers. We were only part of the whole process & team. We didn't administer active directory or the application servers. We just worked with the databases & database servers as usual.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
Viewing 15 posts - 1 through 15 (of 23 total)
You must be logged in to reply to this topic. Login to reply