May 2, 2016 at 10:33 am
Where I work now, most instances are virtualized and that's the default configuration for all new instances, so we can scale them up if needed. What that means is that we tend to have databases segregated across a lot of small VM instances (rather than a lot of databases on one big bare metal instance).
However, the downside of this architectural approach is that I see a lot of queries written by the application developing teams that (in all fairness by necessity) use remote table joins (4 part naming convention), which kills performance. Even if the separate source servers are scaled up with more CPU/RAM, the performance for these distributed queries still sucks. There is often times no truely graceful solution other than the change the location of the database, which of course then creates other political and technical problems. Optimizing distributed queries is not the type of task I want to see routinely land in my Kanban queue.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
May 2, 2016 at 11:29 am
Eric M Russell (5/2/2016)
Where I work now, most instances are virtualized and that's the default configuration for all new instances, so we can scale them up if needed. What that means is that we tend to have databases segregated across a lot of small VM instances (rather than a lot of databases on one big bare metal instance).
I hope you got your licensing right 😀
May 2, 2016 at 11:30 am
Eric M Russell (5/2/2016)
However, the downside of this architectural approach is that I see a lot of queries written by the application developing teams that (in all fairness by necessity) use remote table joins (4 part naming convention), which kills performance. Even if the separate source servers are scaled up with more CPU/RAM, the performance for these distributed queries still sucks. There is often times no truely graceful solution other than the change the location of the database, which of course then creates other political and technical problems. Optimizing distributed queries is not the type of task I want to see routinely land in my Kanban queue.
Is this a Microservices architecture?
May 2, 2016 at 11:52 am
stakes (5/2/2016)
Eric M Russell (5/2/2016)
However, the downside of this architectural approach is that I see a lot of queries written by the application developing teams that (in all fairness by necessity) use remote table joins (4 part naming convention), which kills performance. Even if the separate source servers are scaled up with more CPU/RAM, the performance for these distributed queries still sucks. There is often times no truely graceful solution other than the change the location of the database, which of course then creates other political and technical problems. Optimizing distributed queries is not the type of task I want to see routinely land in my Kanban queue.Is this a Microservices architecture?
Honestly, I don't think it was ever "architected" this way at a high level; it just sort of happened in way similar to urban sprawl. If you ask the application developers, then maybe they'll call it a microservices architecture, and there may even be some misguided fool who will proudly take credit for it. However, from the Operational / BI / DBA point of view, distributing joins across instances is all evil and can do no good.
I think maybe it's time to bulldoze most of the these sprawling VM enclaves and replace them with a handful of profressionally atchitected high rises.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
May 6, 2016 at 6:32 am
I think this is the old way of thinking. In a Cloud world you "SHOULD NEVER" have to think about the underlying hardware. With all of the capabilities current (5/5/2015) and being rolled out in SQL in Azure - it is only an uneducated DBA doing things the old school way that will think in terms of hardware. Even though the OP talked about Microsoft and Azure, he had not kept up to date with all of the enhancements in SQL DB as a service and DTU/DWU measurements. On stress tests, yes it is possible to do the same stress tests which you currently use, (like Replay Direct) on a SQL DB in Azure. Remember SQL DB is offered as a service in Azure with the ability to dynamically shrink and grow Databases (both SCALE UP and SCALE out ) on demand. Go and try these things in Azure before exposing your ignorance. Any DBA worth his salt would have to skill up on these new cloud paradigms, and note that in Azure since storage is decoupled from the compute you can pause at will and save money for your company. The OP does not even know this basics of Azure before writing an article posing off as an expert. His original statement about monthly costs is a non issue as you can scale up and down on demand and as well as completely STOP the Data warehouse (first vendor to offer this is Microsoft) and leave the storage intact. I know you find it hard to believe - go and check out this in Azure before responding with more ignorant replies. The cloud is changing the way we operate and Microsoft is way ahead of Amazon/Oracle in these respects. If you are a SQL DBA, you owe it yourself and your employer to know this as it will save the company money and enhance your business value to the org, do read up, fire up Azure and try these things before posing more fud and nonsense.
May 6, 2016 at 8:16 am
girishpillai (5/6/2016)
I think this is the old way of thinking. In a Cloud world you "SHOULD NEVER" have to think about the underlying hardware. With all of the capabilities current (5/5/2015) and being rolled out in SQL in Azure - it is only an uneducated DBA doing things the old school way that will think in terms of hardware. Even though the OP talked about Microsoft and Azure, he had not kept up to date with all of the enhancements in SQL DB as a service and DTU/DWU measurements. On stress tests, yes it is possible to do the same stress tests which you currently use, (like Replay Direct) on a SQL DB in Azure. Remember SQL DB is offered as a service in Azure with the ability to dynamically shrink and grow Databases (both SCALE UP and SCALE out ) on demand. Go and try these things in Azure before exposing your ignorance. Any DBA worth his salt would have to skill up on these new cloud paradigms, and note that in Azure since storage is decoupled from the compute you can pause at will and save money for your company. The OP does not even know this basics of Azure before writing an article posing off as an expert. His original statement about monthly costs is a non issue as you can scale up and down on demand and as well as completely STOP the Data warehouse (first vendor to offer this is Microsoft) and leave the storage intact. I know you find it hard to believe - go and check out this in Azure before responding with more ignorant replies. The cloud is changing the way we operate and Microsoft is way ahead of Amazon/Oracle in these respects. If you are a SQL DBA, you owe it yourself and your employer to know this as it will save the company money and enhance your business value to the org, do read up, fire up Azure and try these things before posing more fud and nonsense.
Put the 'Kool - Aid' down dude, not everyone is in the Cloud nor is it a universal panacea that your salesman told you it is. There are plenty of scenarios (particularly security) where you don't want your data off-premises so the Cloud isn't an option. Additionally, hardware configuration is as relevant a topic to a DBA as it ever was. For example, in the real world, what unfortunately happens is the SAN administrator gives you a file server and you have to educate him/her how to build a DBMS server. How long have you actually been a DBA? I ask because it sounds like you've only worked in a Cloud environment?
May 6, 2016 at 8:21 am
girishpillai (5/6/2016)
I think this is the old way of thinking. In a Cloud world you "SHOULD NEVER" have to think about the underlying hardware. With all of the capabilities current (5/5/2015) and being rolled out in SQL in Azure - it is only an uneducated DBA doing things the old school way that will think in terms of hardware. Even though the OP talked about Microsoft and Azure, he had not kept up to date with all of the enhancements in SQL DB as a service and DTU/DWU measurements. On stress tests, yes it is possible to do the same stress tests which you currently use, (like Replay Direct) on a SQL DB in Azure. Remember SQL DB is offered as a service in Azure with the ability to dynamically shrink and grow Databases (both SCALE UP and SCALE out ) on demand. Go and try these things in Azure before exposing your ignorance. Any DBA worth his salt would have to skill up on these new cloud paradigms, and note that in Azure since storage is decoupled from the compute you can pause at will and save money for your company. The OP does not even know this basics of Azure before writing an article posing off as an expert. His original statement about monthly costs is a non issue as you can scale up and down on demand and as well as completely STOP the Data warehouse (first vendor to offer this is Microsoft) and leave the storage intact. I know you find it hard to believe - go and check out this in Azure before responding with more ignorant replies. The cloud is changing the way we operate and Microsoft is way ahead of Amazon/Oracle in these respects. If you are a SQL DBA, you owe it yourself and your employer to know this as it will save the company money and enhance your business value to the org, do read up, fire up Azure and try these things before posing more fud and nonsense.
What an unbelievably arrogant post. :w00t:
May 6, 2016 at 8:44 am
The facts about the Cloud paradigm and Azure are correct and the truth. since you don't have a proper response you can choose to call it arrogant - but regardless for any one with an open mind especially database developers, DBA and ISVs Database as a service in Cloud, and especially Azure is a way forward from the current messy and costly way we operate. I humbly and strongly urge you to read up, try it and conclude for yourself. It will open more doors for you professionally, and be seen as more valuable by the business - which is what we all DBAs eventually want. This will help you do more cost effectively. Many ISVs and their Apps are already exploring/using this as it goes straight to their bottom lines.
https://azure.microsoft.com/en-us/documentation/articles/sql-database-elastic-pool/
May 6, 2016 at 9:08 am
girishpillai (5/6/2016)
https://azure.microsoft.com/en-us/documentation/articles/sql-database-elastic-pool/
Your response was arrogant. Offering counterarguments or criticism to an article can contribute to a worthwhile discussion, but your response was rude and unprofessional.
In future, try communicating in a clear and courteous manner - it will help you do more to maintain a healthy debate/discussion and people will be more likely to respond in the same way. Insinuating that the writer (and other members of the forum who share his thoughts) is "uneducated" and "posing off as an expert" (eek) is detrimental to a discussion.
You have an obvious passion for cloud solutions and it's great to see someone so passionate for the technology. However as ccd3000 pointed out, Azure isn't necessarily feasible for every scenario. There is no one-size-fits-all solution in this line of work.
🙂
May 6, 2016 at 10:40 am
My experience so far is that some vendor solutions that rely on DBs are not cloud ready. If the apps aren't cloud ready then putting the DB in the cloud isn't going to work very well.
In some cases we can find ways to install and configure the apps in a way that is sympathetic to cloud deployment. The question then becomes whether the vendor will support that configuration. Again my experience is that the vendors are keen to have a case study showing that their app can work in the cloud.
Where they are unlikely to compromise is where there is an SLA involved. Some of the machine specs are considerably higher than instances available in the cloud.
Cloud DBs do offer considerable advantages for many scenarios. Belief in technical robustness lags some way behind actual technical robustness
May 9, 2016 at 5:04 am
The Cloud (Azure, AWS, etc) are great solutions and products. In some scenarios. In others, they don't make sense.
You can scale up and down, and pausing things certainly can work. however you need to be able to pause them, and restart them. That isn't always feasible. I know some people with fairly steady workloads. In that case, it's not even necessarily cost effective to move to the cloud.
As David mentioned, your apps need to work in the cloud, and some of them will work well, some may not. Certainly some may need rewriting, but that's a separate issue.
Do you worry about hardware? Yes and no. You don't have responsibility, but choosing levels of hardware, weighing the costs of those, managing multiple connections to storage, or even managing the federation/sharding of data is your concern, and that can be affected by the performance of the underlying hardware. So while you are abstracted in the cloud, ultimately performance matters and whether I'm sizing the machines I need, perhaps moving data among nodes, or choosing EMC/HP hardware, the effect is the same.
You do need to be aware of how the hardware choices (direct or indirect) you make can affect performance. You will still need to stress test to understand how (and when) you will scale up and down.
June 6, 2020 at 11:49 pm
If you really want to stress test some hardware, just migrate your existing code to it. That's when you'll find out two things... your code really is crap and all the hardware in the world isn't going to fix it. 😀
--Jeff Moden
Change is inevitable... Change for the better is not.
Viewing 12 posts - 16 through 26 (of 26 total)
You must be logged in to reply to this topic. Login to reply