May 31, 2010 at 9:48 am
Hello Folks
I am about to work with Designing n Maintaining DBS in a company which is using Amazon Cloud Storage. So i was wondering how different it is from using a External HDD or Network Drive to store and retrieve Data from cloud storage.
I wish it is as simple as placing the data files on a Network Drive... is it so ?
What is the exact interface which we use to access the data from that web service ?
Will cloud services be able to provide the fast data access for fetching queries ?
IS internet speed a major factor that decides the speed of my DB ?
Its a pretty small company with all in one (DB design, developer n Admin) so suggestions are welcome.
[font="Verdana"]
Today is the tomorrow you worried about yesterday:-)[/font]
May 31, 2010 at 11:31 am
How about using sql express 2008 [advanced] on a local machine? If you really have only a few users, that'd be plenty strong enough and you wouldn't have to worry about amazon integration or anything going wrong with the internet or amazon.
May 31, 2010 at 5:34 pm
AFAIK, you can't run SQL Server locally and put the files on Amazon's cloud service. You can contract with them for database services, and in that case, the speed of the database will depend on the network as well as individual machine speed.
June 1, 2010 at 1:44 am
I don't really get what you mean. They are using S3 for storage or they want to have the database in EC2.
If the former then you can forget it sorry. You can use S3 to back up the DB though.
If the latter then that's fine and is pretty easy to set up. Be aware that I/O will be worse than you're used to so factor that into planning.
June 1, 2010 at 2:00 pm
@@Steve
I am using EC2.
Why cannot i run SQL Server Locally and put my DB files on Cloud?
So how can SQL Server and Cloud storage work together?
Reference to any articles and info tin this regard is welcome
Thank You
[font="Verdana"]
Today is the tomorrow you worried about yesterday:-)[/font]
June 1, 2010 at 2:07 pm
SQL Server doesn't map the data files to a network location by default, and it won't map to most transfer protocols and work at any reasonable speed. It really requires very high speed access, at bus speeds, or SAN/HBA speeds, not network speeds. Most times people have issues when they put database files on their local network.
You're asking for trouble trying to do this. Contract with Amazon to run the instance in their cloud, or keep it local.
June 1, 2010 at 2:49 pm
What are you trying to accomplish? Is this for DR, high availability or some other reason?
And how much space do you think you will need? Sorry for all the questions, just trying to get a better idea what your needs are.
June 1, 2010 at 2:55 pm
Firstly, I agree with Steve.
Having said that, if he absolutely must run this in S3 then I've thought of a way that might work. Jungle Disk (www.jungledisk.com) presents a S3 volume as a local volume and uses caching to give good performance. I have my home directory on Jungle Disk and it's quite acceptable, you never really notice that things are being backed up transparently. Whenever I sit at another computer my files are 'just there', whether that computer is elsewhere in the office or elsewhere in the world. It's pretty cool.
That's files... a database, well, Jungle Disk won't know about SQL transactions and I would be stunned to find the remote copy is guaranteed to be in a correct state at all times. Also, I don't know how SQL Server adds data to a file or how Jungle Disk copies it but I'd be surprised if Jungle Disk elegantly handled keeping a many-gigabyte file stored. It might work I suppose... just like running a data warehouse in SQLite might work too 🙂
I think you should go back to whoever the IT manager is at this company and tell them that SQL just doesn't run without fast local storage and you're happy to use S3 for all backups (daily, hourly, etc.) but you can't use it for physically storing the database.
June 1, 2010 at 9:44 pm
One thing to be wary of with remote storage is that SQL doesn't know things are committed, and you could end up with an inconsistent database, especially if the transfer from your local disk to Amazon went down. I know some people that have had this happen with SANs when the SAN had too much stuff written to cache, and SQL was expecting those transactions hardened to disk. If the power drops or there's some issue and the data never gets written to disk, you'll restart SQL in an inconsistent state.
SQL is not recommended for remote storage. The disks must appear local, and I'm not sure this would be a supported config if you had issues and called MS.
June 3, 2010 at 10:04 am
<b>@@Dave Jones</b>
We are looking for high availability and scalability, and by using EC2 we can be less worried about buying our own servers etc.. Since ours is a small business..What all we need is
1.Data being quried out of DB via website
2.Data being written into DB via forms on website
3.Export batches/2 or 3 excel files into few tables
Thats all ..
No Log shipping
Replications
Automated Jobs etc
<b>@@Steve Jones</b>
Thank You Steve. SO what you said is in reference to S3 storage right ?
As i told you we are using EC2 which will have its own virtual servers.
So would that still be a negative aspect to what we are trying to accomplish.
<b>@@Corins</b>
thank You cornis. Jungle-disk is cool. But my problem is
1. My managements asked me to design a DB
2. then integrate it to Web and
3. Once all design is set .. They want to move it to cloud
so that they have less burden of managing the servers..
So do they still need a DBA after Step 2 ?
I don't want myself to be laid off 😛
[font="Verdana"]
Today is the tomorrow you worried about yesterday:-)[/font]
June 3, 2010 at 2:35 pm
Hi,
EC2 is completely different to what we've been answering. EC2 will give you a standard virtual server with EBS (not S3). The only way you could reduce DBA needs is to use their managed MySQL service. Otherwise it's just a server like any other.
Be aware that the I/O wil be lower than you're used to so that will affect your design.
I suggest you just kick off a small SQL server AMI and give it a whirl, it'll cost almost nothing (except time) to learn.
Viewing 11 posts - 1 through 10 (of 10 total)
You must be logged in to reply to this topic. Login to reply