May 8, 2018 at 6:28 am
Sean Lange - Monday, May 7, 2018 3:07 PMsgmunson - Monday, May 7, 2018 2:18 PMSteve Jones - SSC Editor - Monday, May 7, 2018 9:25 AMJeff Moden - Monday, May 7, 2018 7:34 AMsgmunson - Monday, May 7, 2018 6:56 AMI can only conclude that someone forgot about the rather strong and CRITICAL need to manage the database, or that somehow, databases are things to be played with instead of properly cared for. You don't abandon stored procedures because you have developers that don't know how to write them. You either train the ones you have to be able to properly code stored procedures, hire better developers to begin with, or suffer consequences. Your choice... And while agile/scrum advocates are prone to whining about the costs of such things, I simply explain that if you're worried about costs, why did you hire such poorly skilled developers in the first place? Or how about the cost of the agile infrastructure because they continue to perceive the database as an impediment? Things tend to get real quiet whenever I bring that topic up. It's as if they just don't consider a database an engine that drives the business. The level of incompetence that kind of thinking represents is extraordinary. I basically go back to "If you think professionals are expensive, wait until you hire an amateur" (Red Adair).Heh.... you beat me to it with the Red Adair quote. One of my favorite quotes is on of my own making... "If you want it real bad, you usually get it that way". 😀
Also, one of the reasons why I don't like pure Agile environments is because Project Managers are supposedly "not needed". One of the things that people don't realize is that a good Project Manager will not only help keep the current sprints on track but they also serve the huge and greatly needed functionality of being an enabler. If you need something to get the job done, a good Project Manager will go into the mode of pounding on people that can provide the correct resources, physical or otherwise, which prevents Developers from having to suffer such distractions.
I'll argue the other side here slightly. I don't think Orlando is saying that you shouldn't have competence, but that you don't necessarily need to slow down with process.
Smunson says that you need to train developers. Where in Orlando's post did he mention that developers shouldn't be trained? He said no project managers and everyone needs to work together. The proof is in the procs being used in the agile process, not in the procedures, the developers, or the DBAs.
Jeff dislikes the lack of PMs, who can save developers' time by getting things done. I'll say that I've worked in no shortage of environments where the PMs don't get things done, and really just push on everyone to do more work instead of smoothing the process. It's not Agile that's bad or PMs that are good, but you are highly dependent on the people doing a good job. It's really the people and most of you that bang on Agile or DevOps have worked with lots of unprofessional and competent people and you're blaming the system as much as the staff.
Whether you work in Agile or DevOps or Scrum or whatever, the system isn't usually the problem. Some systems hide problems more than others, but all will fail if you don't have your staff, and your management, supporting the system. DevOps, Agile, etc. are asking people to regularly and constantly improve. That doesn't just mean that work flows faster and faster, which is what developers start to want, and management does. Skills improve, we change based on feedback, and we self-manage in many ways.
Is that a good use of time? I'm torn. Certainly having a highly paid, talented Mr. Moden chase down the allocation of new resources when he could tune code (or teach/mentor others) might not be a good use of time. The thing I think about is that having a break and some changes in routine, some breaks, is a good way to ensure that talented people don't burn out. Finding a balance there is tough, and certainly in the DevOps space, this doesn't mean less staff. It means that all staff gets trained to handle a bit of everything. The HADR guy doesn't get the same coding responsibilities as the C# developer, but they ought to be able to help out in places and know a little. Just like the developer learns a bit about HADR, which might change how they build something.
It's almost always the people, not the system or platform or architecture.
I'm going to have to agree with Jeff. If devs see sprocs as an impediment in any way then there's a considerable lack of expertise in not just the devs, but also in the scrum masters and or project managers that go along with that nonsense. In my book, that rises above mere incompetence, to the level of negligence. Treating a database like it's basic structures are impediments is ludicrous, at best, and that's why my book says negligence. If the PM's are getting away with it, then the level of incompetence probably rises well up into management.
I would say that it almost always rises well up into management. Remember that there are very few (if any) management types that are one or two levels over a DBA or dev that have any depth of technical knowledge. In my wife's job she knew it was wrong to take all the sql out of procedures and move it to the application but she doesn't have the depth of knowledge to be able to explain to the development manager as to why. She asked me about it a week or so after they made that decision and I nearly fell out of my chair. They didn't treat or see the database as an impediment to progress, they foolishly saw procedures as the performance bottleneck. From a performance perspective their weeks long effort was an exercise in futility. But from a system and maintenance perspective it was huge leaps backwards. And they still have massive performance issues and she can't get the dev guy to listen to me or all the others out there that tell them they are going about is all wrong. They have queries that generate thousand and thousand of rows for grids that are paged. They aren't doing any server side paging or caching, just querying the same multiple thousand row query for every page of data.
Thanks for providing a real-world example of what happens when there's no fully qualified database professional around to provide guidance on such things. In that situation, it seems that even when confronted with information suggesting they went down the wrong road, they were so unwilling to admit they'd made a mistake, that they chose to double down on it instead. And people get their panties in a wad because I'm willing to tell the truth instead of lie to their faces... Ughhh...
Steve (aka sgmunson) 🙂 🙂 🙂
Rent Servers for Income (picks and shovels strategy)
May 8, 2018 at 6:28 am
Here’s a quote for you, Sergiy:
Amazon delivers a new feature to production every 10.6 seconds with 1000 Scrum teams. That's why they have all the money. —Jeff Sutherland
Would you consider Amazon a successful company? Apple? Google? Microsoft? They all employ Agile processes and have for many years. The myth that customers cannot successfully guide your development efforts or that you cannot build quality software in small iterations is a myth probably decided by people that are not interested in trying something different. Agile is not good or bad in and of itself. It either aligns with your optimizing goals or it doesn’t. And as Steve alluded to earlier it’s more about building a team of motivated individuals than about which process framework you choose.
There are no special teachers of virtue, because virtue is taught by the whole community.
--Plato
May 8, 2018 at 6:56 am
Sue_H - Tuesday, May 8, 2018 6:23 AMOrlando Colamatteo - Monday, May 7, 2018 11:23 PMSue_H - Monday, May 7, 2018 4:12 PMAgile is a development methodology, not a deployment methodology...Agile, according to the Agile Manifesto which marked the beginning of the movement we know as Agile (with a capital A) software development, defines a set of four values and twelve principles. Agile is not a development methodology or a methodology of any kind for that matter.
And even on the Agile Alliance site, in explaining to beginners:
What is Agile Software Development?
Agile Software Development is an umbrella term for a set of methods and practices based on the values and principles expressed in the Agile Manifesto.I could go on and on and on with more. But I'm sure everyone else wrong. So go pontificate some more. With that capital A.
Sue
Hi Sue, Sorry about the way the capital A thing came off. It helps me consider it as a proper noun as opposed to the adjective, i.e. to be agile. Agile has a ton of contexts in which it is used and carries a lot of definitions and connotations, e.g. mindset, culture, umbrella term for processes and methods, etc.
It is indeed an umbrella term but is not in and of itself a methodology. I only mention it because I think it's an important distinction and at its core. The Agile movement is actually a thoughtful departure from many of the heavyweight prescriptive methodologies like RUP and other Waterfall processes. Scrum, as an Agile process framework, itself is also not a methodology. And it's not that methodology is bad it's just that allowing it to rule how we work has been proven to be sub-optimal for many people and organizations.
There are no special teachers of virtue, because virtue is taught by the whole community.
--Plato
May 8, 2018 at 7:26 am
sgmunson - Tuesday, May 8, 2018 6:28 AMSean Lange - Monday, May 7, 2018 3:07 PMsgmunson - Monday, May 7, 2018 2:18 PMI'm going to have to agree with Jeff. If devs see sprocs as an impediment in any way then there's a considerable lack of expertise in not just the devs, but also in the scrum masters and or project managers that go along with that nonsense. In my book, that rises above mere incompetence, to the level of negligence. Treating a database like it's basic structures are impediments is ludicrous, at best, and that's why my book says negligence. If the PM's are getting away with it, then the level of incompetence probably rises well up into management.I would say that it almost always rises well up into management. Remember that there are very few (if any) management types that are one or two levels over a DBA or dev that have any depth of technical knowledge. In my wife's job she knew it was wrong to take all the sql out of procedures and move it to the application but she doesn't have the depth of knowledge to be able to explain to the development manager as to why. She asked me about it a week or so after they made that decision and I nearly fell out of my chair. They didn't treat or see the database as an impediment to progress, they foolishly saw procedures as the performance bottleneck. From a performance perspective their weeks long effort was an exercise in futility. But from a system and maintenance perspective it was huge leaps backwards. And they still have massive performance issues and she can't get the dev guy to listen to me or all the others out there that tell them they are going about is all wrong. They have queries that generate thousand and thousand of rows for grids that are paged. They aren't doing any server side paging or caching, just querying the same multiple thousand row query for every page of data.
Thanks for providing a real-world example of what happens when there's no fully qualified database professional around to provide guidance on such things. In that situation, it seems that even when confronted with information suggesting they went down the wrong road, they were so unwilling to admit they'd made a mistake, that they chose to double down on it instead. And people get their panties in a wad because I'm willing to tell the truth instead of lie to their faces... Ughhh...
Ding ding!!!! You are spot on!!! They didn't want to listen to the idea that they made a poor choice so they devoted more time and effort to going the wrong direction to save face.
_______________________________________________________________
Need help? Help us help you.
Read the article at http://www.sqlservercentral.com/articles/Best+Practices/61537/ for best practices on asking questions.
Need to split a string? Try Jeff Modens splitter http://www.sqlservercentral.com/articles/Tally+Table/72993/.
Cross Tabs and Pivots, Part 1 – Converting Rows to Columns - http://www.sqlservercentral.com/articles/T-SQL/63681/
Cross Tabs and Pivots, Part 2 - Dynamic Cross Tabs - http://www.sqlservercentral.com/articles/Crosstab/65048/
Understanding and Using APPLY (Part 1) - http://www.sqlservercentral.com/articles/APPLY/69953/
Understanding and Using APPLY (Part 2) - http://www.sqlservercentral.com/articles/APPLY/69954/
May 8, 2018 at 7:33 am
patrickmcginnis59 10839 - Monday, May 7, 2018 3:25 PMThey have queries that generate thousand and thousand of rows for grids that are paged. They aren't doing any server side paging or caching, just querying the same multiple thousand row query for every page of data.
yes, but are we saying fixing this requires a stored procedure?
edit:
let me put it another way, can we DO paging and caching without stored procedures? how about procedure cache bloat, can we avoid that without stored procedures?
I'm the type that likes to separate out concerns into discrete tidy buckets for consideration.
(another edit, I had a typo in there)
Sure you can do this without procedures. It sounds like you are on the side that doesn't want to use stored procedures. But that contradicts your statement that you like to "separate out concerns into discrete tidy buckets". One of the advantages of using stored procedures is the separation of duties and creating layers. When the application layer also implements all of the data concerns you have created a potential nightmare. For one thing you have to redeploy the entire application for a minor change in a query. And heaven forbid you need to change databases when all your queries are scattered all over the application. Now I realize that changing databases doesn't happen all that often but it does happen. I have migrated applications to sql server in the past. Some of them had code splattered throughout the code and it was a nightmare. But one client I did some work for had everything separated in layers. It was so much simpler.
_______________________________________________________________
Need help? Help us help you.
Read the article at http://www.sqlservercentral.com/articles/Best+Practices/61537/ for best practices on asking questions.
Need to split a string? Try Jeff Modens splitter http://www.sqlservercentral.com/articles/Tally+Table/72993/.
Cross Tabs and Pivots, Part 1 – Converting Rows to Columns - http://www.sqlservercentral.com/articles/T-SQL/63681/
Cross Tabs and Pivots, Part 2 - Dynamic Cross Tabs - http://www.sqlservercentral.com/articles/Crosstab/65048/
Understanding and Using APPLY (Part 1) - http://www.sqlservercentral.com/articles/APPLY/69953/
Understanding and Using APPLY (Part 2) - http://www.sqlservercentral.com/articles/APPLY/69954/
May 8, 2018 at 7:36 am
Orlando Colamatteo - Tuesday, May 8, 2018 6:56 AMHi Sue, Sorry about the way the capital A thing came off. It helps me consider it as a proper noun as opposed to the adjective, i.e. to be agile. Agile has a ton of contexts in which it is used and carries a lot of definitions and connotations, e.g. mindset, culture, umbrella term for processes and methods, etc.
It is indeed an umbrella term but is not in and of itself a methodology. I only mention it because I think it's an important distinction and at its core. The Agile movement is actually a thoughtful departure from many of the heavyweight prescriptive methodologies like RUP and other Waterfall processes. Scrum, as an Agile process framework, itself is also not a methodology. And it's not that methodology is bad it's just that allowing it to rule how we work has been proven to be sub-optimal for many people and organizations.
Per the definition of a method it is a methodology. It's not a method per your interpretation.
You can find contradictory quotes from those in the Agile world so it doesn't even matter who says what about Agile as there are different interpretations. The same is true for a lot things in the world. With Agile, there are certainly some defined principles but there are plenty of areas open to interpretation. I've seen it used in more than a dozen different places, I've seen certified scrum masters who have all had different ways of doing things, different interpretations of things in agile. Your interpretation doesn't make it THE interpretation. And your saying something doesn't make it so.
Being that pedantic, the whole Agile with a capital A, posting all over the place and the you said, he said, she said...it is just plain weird. You may have some good points but no one will ever hear them with how you present. There are probably others reading this thread who are at the point where they could give a rats behind about Agile. And if anyone in this thread didn't care for Agile, they probably like it even less.
Sue
May 8, 2018 at 7:44 am
Sean Lange - Tuesday, May 8, 2018 7:33 AMpatrickmcginnis59 10839 - Monday, May 7, 2018 3:25 PMThey have queries that generate thousand and thousand of rows for grids that are paged. They aren't doing any server side paging or caching, just querying the same multiple thousand row query for every page of data.
yes, but are we saying fixing this requires a stored procedure?
edit:
let me put it another way, can we DO paging and caching without stored procedures? how about procedure cache bloat, can we avoid that without stored procedures?
I'm the type that likes to separate out concerns into discrete tidy buckets for consideration.
(another edit, I had a typo in there)
Sure you can do this without procedures. It sounds like you are on the side that doesn't want to use stored procedures. But that contradicts your statement that you like to "separate out concerns into discrete tidy buckets". One of the advantages of using stored procedures is the separation of duties and creating layers. When the application layer also implements all of the data concerns you have created a potential nightmare. For one thing you have to redeploy the entire application for a minor change in a query. And heaven forbid you need to change databases when all your queries are scattered all over the application. Now I realize that changing databases doesn't happen all that often but it does happen. I have migrated applications to sql server in the past. Some of them had code splattered throughout the code and it was a nightmare. But one client I did some work for had everything separated in layers. It was so much simpler.
That sort of implies that dot net or other programming doesn't do modular programming and, well, this is surprising to me. Are there really no high level programming languages besides T-SQL that HAVE facilities to provide any sort of modularity for developers to use? (Sorry, typo)
May 8, 2018 at 7:56 am
sgmunson - Tuesday, May 8, 2018 6:23 AMfunbi - Monday, May 7, 2018 2:33 PMIf devs see stored procs as impediments (and I don't think that they do) it would not be because devs can't write stored procs. It would be because any DB changes mean that they then have to factor in a DBA into their deployment pipeline - who would be needed to either apply the changes or set up DB permissions so that the dev team can apply them i.e. bringing a human into a process that is automated in every other way.Heaven forbid a HUMAN gets involved to do the right thing... and the idea that anyone actually believes you can totally automate database work is so incredibly foolish that I can't even imagine going down that road at all. You pretty much make my point for me. If you had any genuinely useful experience with database work, you would know better. Clearly, you think databases are just supposed to do all the work for you. Good luck with that...
I was talking about the deployment of stored procedures; I never said "totally automate database work". In general any human involvement within a continuous integration/continuous delivery process will be seen as an impediment. I'm not sure how you came to your other conclusions about me personally but feel free to think what you like 🙂
May 8, 2018 at 8:15 am
Orlando Colamatteo - Tuesday, May 8, 2018 12:55 AMJeff Moden - Monday, May 7, 2018 11:18 AMHeh... Orlando said that they didn't want to wait on someone to write stored procedures. That strongly implies that the developers don't know how to do such things on their own which implies that they're not trained to do it... or simply don't want to do it.Goodness no. Not what I was implying at all. I was talking about the scenario where members of a Development Team (a subset of the Scrum Team) are qualified to some degree to write SQL code and would like to be able to develop their own schema and procedures however the organization pays specialists (usually called DBAs) to do that kind of work as needed and supply it to the Development Team as an input to their development process. Since these specialists have other operational responsibilities like looking after server builds, server health, environment management, backups and restores, production support, etc. etc. etc. they are too highly paid for the organization to hire one for every development team and even if the organization had that kind of cash these specialists would be too busy to be fully dedicated to the work of the Development Team.
What I actually said:
It all depends on what the optimizing goals are for the architecture. Saddling developers with external dependencies like having to wait for a proc to be developed by someone else or even send it to someone else for revisions before it can be deployed is an impediment (in Scrum terms) and as we look for ways to help the team go faster we look to remove impediments.
Logic follows:
1 Procs require specialized skills or are somehow slowing the team down
2 Some critical mass of decision makers decide Procs are an impediment
3 Decision makers decide on an architecture that does not include ProcsThe proof is not in the Procs worth it’s in their ability to be part of an agile development process. Have you looked into Visual Studio Database Projects or redgate ReadyRoll? Change or be phased out. Those are your options.
What I tried to say was that anything the Development Team has to go outside of themselves to accomplish amounts to an external dependency...even a code review. A Development Team should have all the skills on it to accomplish their work. That means if the software needs a data store and they choose SQL Server they better have some people that know or SQL Server or are getting after learning SQL Server in a hurry.
Scrum does not solve your problems but it exposes them in a hurry. If your organization does not like Scrum it probably does not like transparency and openness.
Yep... Based on what I've personally gone through with some teams, I totally misread what you intended to mean in the first quoted paragraph.
I totally agree that having to wait on external resources to pump things through their queue is the wrong thing to do. That's why I sat with the Developers (with full agreement of management) instead of some other office somewhere when I first arrived at my current company. I reviewed some of the problems they were having (10 minute timeouts 5 times a day, extremely slow multi-row and batch code, reporting, etc, etc) and saw what the previous team had done. It was pretty much a train wreck and the new Developers (and a couple of the older ones that got it) where anxious to learn better ways (especially since the Dev team was really taking it on the chin for the problems). To make a much longer story shorter, I demonstrated the power that could be had in stored procedures, what "Set Based" meant (and, no, it doesn't mean "all in one query"), a whole bunch of techniques to make their code sing, and a remarkable thing started to happen. They started taking great pride in beating the pants off the legacy code and repairing it. Their new stuff started to take on the trait of being flawless (no rework required during testing by QA and performed very well), and their throughput was absolutely incredible. I had lots of things I needed to do as a DBA but, with the understanding that production problems were always the #1 priority, I made helping them a very close second priority. I wouldn't write code for them. Instead, I encouraged them to seek me out (as in turn in their chair and ask "Hey Jeff... gotta moment?") and then I'd teach them the principle thinking required to make database code sing for their problem so they could do it on their own and for more than one thing. I even taught some "Lunch'n'Learn" sessions on some of the common problems.
Because of growth within the company, I was moved from sitting in the midst of the Developers but not more than 20 steps away and they still come to me for to bounce ideas off of me. They could have moved me to the other side of the building and it might not have mattered because they've gotten so good with stored procedures and even database code in the managed code that they not only teach each other some new tricks, they also teach any newbies to the group. Sure, they make the occasional chilling mistake (don't we all?) but those are usually flushed out in QA and UAT and we still all get together to hammer out the problem when they need the help, but they've become incredibly self sufficient and they've done so in grand style. They take great pride in writing nasty fast code that flies that almost flies though QA and UAT and then performs very well in Production.
The code is well documented within the code, as well. They quickly realized that some well placed "functional" comments make it so much easier to add improvements/required modifications for additional functionality or troubleshoot any problems that do occur even if someone hasn't actually seen the code before.
The bottom line here is that I absolutely agree with you on Developers not having to wait on someone else. The only way to do that is to enable them to not have to. And that's the thing I thought you were excluding from your comment. Sorry I misread that.
--Jeff Moden
Change is inevitable... Change for the better is not.
May 8, 2018 at 8:23 am
funbi - Tuesday, May 8, 2018 7:56 AMsgmunson - Tuesday, May 8, 2018 6:23 AMfunbi - Monday, May 7, 2018 2:33 PMIf devs see stored procs as impediments (and I don't think that they do) it would not be because devs can't write stored procs. It would be because any DB changes mean that they then have to factor in a DBA into their deployment pipeline - who would be needed to either apply the changes or set up DB permissions so that the dev team can apply them i.e. bringing a human into a process that is automated in every other way.Heaven forbid a HUMAN gets involved to do the right thing... and the idea that anyone actually believes you can totally automate database work is so incredibly foolish that I can't even imagine going down that road at all. You pretty much make my point for me. If you had any genuinely useful experience with database work, you would know better. Clearly, you think databases are just supposed to do all the work for you. Good luck with that...
I was talking about the deployment of stored procedures; I never said "totally automate database work". In general any human involvement within a continuous integration/continuous delivery process will be seen as an impediment. I'm not sure how you came to your other conclusions about me personally but feel free to think what you like 🙂
But I would ask - is it really automated in every other way? Is there an automated process to verify the changes are not causing any issues? Is there an automated process to detect exactly what those issues are and an automated process to correct those? How will the automated process address sudden blocking due to the changes? How does the automated process address performance issues that result from the changes?
And I'm not asking those to put you on the spot. Just another side of things. In those environments such as CI, there is often the assumption that everything is automated when it really isn't. And when a person gets involved, it's seen as being something negative. And whoever that person is becomes the "negative" or the bottleneck in the equation when they really are not.
Sue
May 8, 2018 at 8:39 am
patrickmcginnis59 10839 - Tuesday, May 8, 2018 7:44 AMSean Lange - Tuesday, May 8, 2018 7:33 AMpatrickmcginnis59 10839 - Monday, May 7, 2018 3:25 PMThey have queries that generate thousand and thousand of rows for grids that are paged. They aren't doing any server side paging or caching, just querying the same multiple thousand row query for every page of data.
yes, but are we saying fixing this requires a stored procedure?
edit:
let me put it another way, can we DO paging and caching without stored procedures? how about procedure cache bloat, can we avoid that without stored procedures?
I'm the type that likes to separate out concerns into discrete tidy buckets for consideration.
(another edit, I had a typo in there)
Sure you can do this without procedures. It sounds like you are on the side that doesn't want to use stored procedures. But that contradicts your statement that you like to "separate out concerns into discrete tidy buckets". One of the advantages of using stored procedures is the separation of duties and creating layers. When the application layer also implements all of the data concerns you have created a potential nightmare. For one thing you have to redeploy the entire application for a minor change in a query. And heaven forbid you need to change databases when all your queries are scattered all over the application. Now I realize that changing databases doesn't happen all that often but it does happen. I have migrated applications to sql server in the past. Some of them had code splattered throughout the code and it was a nightmare. But one client I did some work for had everything separated in layers. It was so much simpler.
That sort of implies that dot net or other programming doesn't do modular programming and, well, this is surprising to me. Are there really no high level programming languages besides T-SQL that HAVE facilities to provide any sort of modularity for developers to use? (Sorry, typo)
If you interpreted my post as implying that dotnet can't do modular programming I either need to completely rewrite what I wrote or you need to reread it. I was speaking of application layers (and the lack of utilizing them), not modularity. Sprinkling (or perhaps littering is a better term) sql statements all through code is what I was speaking of. And it is abominable.
In dotnet there are assemblies, classes, methods and many other module development opportunities. Those are all part of the application layer. In some poorly designed systems the data layer is scattered through those pieces of code which is what I was talking about.
_______________________________________________________________
Need help? Help us help you.
Read the article at http://www.sqlservercentral.com/articles/Best+Practices/61537/ for best practices on asking questions.
Need to split a string? Try Jeff Modens splitter http://www.sqlservercentral.com/articles/Tally+Table/72993/.
Cross Tabs and Pivots, Part 1 – Converting Rows to Columns - http://www.sqlservercentral.com/articles/T-SQL/63681/
Cross Tabs and Pivots, Part 2 - Dynamic Cross Tabs - http://www.sqlservercentral.com/articles/Crosstab/65048/
Understanding and Using APPLY (Part 1) - http://www.sqlservercentral.com/articles/APPLY/69953/
Understanding and Using APPLY (Part 2) - http://www.sqlservercentral.com/articles/APPLY/69954/
May 8, 2018 at 8:45 am
Sue_H - Tuesday, May 8, 2018 8:23 AM...And I'm not asking those to put you on the spot. Just another side of things. In those environments such as CI, there is often the assumption that everything is automated when it really isn't. And when a person gets involved, it's seen as being something negative. And whoever that person is becomes the "negative" or the bottleneck in the equation when they really are not.
Sue
Automation in DevOps (or other ways of describing software development) is only automated tedious, repeatable processes that humans have built. The automation is just repeating what someone would have to normally do manually. This means that automation would be:
- the process of ensuring you have the latest code
- running all the tests (or specific sets of tests)
- packaging up code
- or executing it under the right credentials to the right server
- or any other task
Most of you are programmers of some sort. You write scripts or utilities or macros or anything to reduce work and handle the tedious tasks. That's what automation in DevOps or Agile is used for. You do the creative work to build something, automation is used to repeat that over and over.
May 8, 2018 at 8:47 am
Jeff Moden - Tuesday, May 8, 2018 8:15 AMThat's why I sat with the Developers (with full agreement of management) instead of some other office somewhere when I first arrived at my current company. I reviewed some of the problems they were having (10 minute timeouts 5 times a day, extremely slow multi-row and batch code, reporting, etc, etc) and saw what the previous team had done. It was pretty much a train wreck and the new Developers (and a couple of the older ones that got it) where anxious to learn better ways (especially since the Dev team was really taking it on the chin for the problems). To make a much longer story shorter, I demonstrated the power that could be had in stored procedures, what "Set Based" meant (and, no, it doesn't mean "all in one query"), a whole bunch of techniques to make their code sing, and a remarkable thing started to happen. They started taking great pride in beating the pants off the legacy code and repairing it. Their new stuff started to take on the trait of being flawless (no rework required during testing by QA and performed very well), and their throughput was absolutely incredible. I had lots of things I needed to do as a DBA but, with the understanding that production problems were always the #1 priority, I made helping them a very close second priority. I wouldn't write code for them. Instead, I encouraged them to seek me out (as in turn in their chair and ask "Hey Jeff... gotta moment?") and then I'd teach them the principle thinking required to make database code sing for their problem so they could do it on their own and for more than one thing. I even taught some "Lunch'n'Learn" sessions on some of the common problems.
Ahem, maybe you'd like to get back to writing some of that knowledge down to share? 😉
Also, what you're describing is what DevOps is about.
May 8, 2018 at 9:07 am
Sue_H - Tuesday, May 8, 2018 7:36 AMOrlando Colamatteo - Tuesday, May 8, 2018 6:56 AMHi Sue, Sorry about the way the capital A thing came off. It helps me consider it as a proper noun as opposed to the adjective, i.e. to be agile. Agile has a ton of contexts in which it is used and carries a lot of definitions and connotations, e.g. mindset, culture, umbrella term for processes and methods, etc.
It is indeed an umbrella term but is not in and of itself a methodology. I only mention it because I think it's an important distinction and at its core. The Agile movement is actually a thoughtful departure from many of the heavyweight prescriptive methodologies like RUP and other Waterfall processes. Scrum, as an Agile process framework, itself is also not a methodology. And it's not that methodology is bad it's just that allowing it to rule how we work has been proven to be sub-optimal for many people and organizations.
Per the definition of a method it is a methodology. It's not a method per your interpretation.
You can find contradictory quotes from those in the Agile world so it doesn't even matter who says what about Agile as there are different interpretations. The same is true for a lot things in the world. With Agile, there are certainly some defined principles but there are plenty of areas open to interpretation. I've seen it used in more than a dozen different places, I've seen certified scrum masters who have all had different ways of doing things, different interpretations of things in agile. Your interpretation doesn't make it THE interpretation. And your saying something doesn't make it so.Being that pedantic, the whole Agile with a capital A, posting all over the place and the you said, he said, she said...it is just plain weird. You may have some good points but no one will ever hear them with how you present. There are probably others reading this thread who are at the point where they could give a rats behind about Agile. And if anyone in this thread didn't care for Agile, they probably like it even less.
Sue
Your logic is flawed in terms of an umbrella term for many different methods making it a methodology in and of itself but it's OK. Sorry to have disturbed you.
There are no special teachers of virtue, because virtue is taught by the whole community.
--Plato
May 8, 2018 at 9:26 am
Sean Lange - Tuesday, May 8, 2018 8:39 AMpatrickmcginnis59 10839 - Tuesday, May 8, 2018 7:44 AMSean Lange - Tuesday, May 8, 2018 7:33 AMpatrickmcginnis59 10839 - Monday, May 7, 2018 3:25 PMThey have queries that generate thousand and thousand of rows for grids that are paged. They aren't doing any server side paging or caching, just querying the same multiple thousand row query for every page of data.
yes, but are we saying fixing this requires a stored procedure?
edit:
let me put it another way, can we DO paging and caching without stored procedures? how about procedure cache bloat, can we avoid that without stored procedures?
I'm the type that likes to separate out concerns into discrete tidy buckets for consideration.
(another edit, I had a typo in there)
Sure you can do this without procedures. It sounds like you are on the side that doesn't want to use stored procedures. But that contradicts your statement that you like to "separate out concerns into discrete tidy buckets". One of the advantages of using stored procedures is the separation of duties and creating layers. When the application layer also implements all of the data concerns you have created a potential nightmare. For one thing you have to redeploy the entire application for a minor change in a query. And heaven forbid you need to change databases when all your queries are scattered all over the application. Now I realize that changing databases doesn't happen all that often but it does happen. I have migrated applications to sql server in the past. Some of them had code splattered throughout the code and it was a nightmare. But one client I did some work for had everything separated in layers. It was so much simpler.
That sort of implies that dot net or other programming doesn't do modular programming and, well, this is surprising to me. Are there really no high level programming languages besides T-SQL that HAVE facilities to provide any sort of modularity for developers to use? (Sorry, typo)
If you interpreted my post as implying that dotnet can't do modular programming I either need to completely rewrite what I wrote or you need to reread it. I was speaking of application layers (and the lack of utilizing them), not modularity. Sprinkling (or perhaps littering is a better term) sql statements all through code is what I was speaking of. And it is abominable.
In dotnet there are assemblies, classes, methods and many other module development opportunities. Those are all part of the application layer. In some poorly designed systems the data layer is scattered through those pieces of code which is what I was talking about.
I hear ya, it sounds like dot net has plenty of ways for me to "separate out concerns into discrete tidy buckets" so maybe there isn't really a contradiction of mine that I need to remediate. Are there any other potential contradictions I need to address? If you want to retype your post would you like me to remove the quotations? Especially with you saying I'm contradicting myself when I can't really spot any evidence of it?
I also get that feeling that folks are saying once developers leave stored procedures and do work in dot net, then its "all heck breaks loose" but the thing is, if you can't address the programming process with any sort of legit structure of management ensuring due diligence and care and genuinely effective programming practices, then isn't the question more like "which side of the database connection does the crap code get written to?"
If it helps any, I prefer stored procedures in many cases but as always it probably depends on the project. So if you'd like to consider that in your rewrite I'd appreciate it.
edit: I really regret feeling like I should post with the picky tone I posted with, but all too often I find that we aren't helping each other accurately hone into the differences that separate our opinions, and I guess I do get a bit frustrated, and I otherwise enjoy posting here. I know my posting style is a work in progress, if anybody would like me to remove my posts sent me a message and I'll certainly consider it, obviously those asking that I go **** ****** will have to present a much more persuasive argument in that particular case 😉
Viewing 15 posts - 121 through 135 (of 191 total)
You must be logged in to reply to this topic. Login to reply