July 14, 2021 at 12:00 am
Comments posted to this topic are about the item Improving DevOps Automation
July 14, 2021 at 12:46 pm
The adoption of DevOps, where I work, seems to experience its ups, then significant downs. And how DevOps is perceived is also mysterious.
For example, on Monday I was involved in a meeting with some people in upper management, to discuss a division of people who are requesting a Git repo for themselves to share their code. These people are not developers. Instead, they're data analysts, so they are sophisticated users. Basically, the meeting was how do we reign them in. I understand the need to provide governance. And some of this group had caused a server to fill up. I asked what happened later and learned that the disk space on the server got low. The reports that they run weren't cleaning up temporary files correctly, so it filled up the disk. The worse thing is this group of people were given administrative privileges on the server, then they installed some SAS add-ons. I don't understand what happened next, it wasn't explained to me, but some in the meeting were very concerned about this. No one in IT knows the software involved (SAS, R and I think Python), so there's a concern how to we support them. How do we review their code, etc. Since the server involved was on-prem it became a big issue, especially once the server's disk space filled up and the users demanded help from IT. This lead to a discussion of where to put Git. We used an old version TFS, so we use TFVC, but I did point out that TFS can also support Git. The discussion went to do we have them go to Azure DevOps or GitHub? Then the most senior person in the meeting expressed concern about the cost.
It was here that alarms went off for me. Our TFS server is so old that it is no longer supported. I'd like for us to move to Azure DevOps and we have the money to move IT there. But I foresee other users, like this group, requiring moving their "shadow IT" to a git server somewhere. If the only consideration is, how much is this going to cost, then we're likely to try to stop the users. I foresee those sophisticated users will find a way of getting done what they need to get done, without IT's help. After all, they have a job to do too. The horse has left the barn, trying to stop the horse after the fact isn't going to work.
As far as how DevOps is perceived at where I work, it's because people see only one person doing anything with CI/CD, rather than something all developers could do, to an extent. This may be because our TFS is so old that it doesn't support YAML. Right now, our CI is handled using PowerShell scripts, all of which were written by a PowerShell guru, who left about 2 years ago. I have some experience at YAML development, and I can tell you I prefer writing YAML over authoring PowerShell. Anyway, when I look across the organizations who are using CI/CD I see all developers involved, to some degree, in helping to author CI/CD pipelines. Where I work the perception is it is only done by one person for the whole organization. In this case that happens to be me. It surprises me the huge disconnect between what I see most companies and agencies doing verses what I see where I work. They're nothing alike.
Rod
July 15, 2021 at 8:38 am
In my organisation, regardless of whether it is web development, data engineering, DBAs, Data Analysts, Data Scientists everything gets put under a git compatible source control repositories.
Those repositories are viewable inside the organisation but not without. They are hosted externally because its a service we wish to consume rather than manage. Obviously there is some management involved, there always is.
Note that GitHub is owned by Microsoft and is hosted in Azure.
Because they are viewable internally anyone in the organisation can look at anyone's code. This is a major plus, particularly when reviewing from home and also for developer educational purposes. Transparency is all.
In terms of how you'd support SAS, R & Python as an organisation then the most you can do is spend time collaborating with your analysts/data scientists and showing them good development practices. As SAS and R are their pool of expertise their management has to make sure that they don't create single points of failure etc. Data science management may need support from software engineering management so they don't tread in the turds and bear traps that we avoid (mostly) in the software development world.
There is a different mindset in Data Science driven by experimentation and hypothesis testing. A rigid TDD approach would stifle that but at the same time showing them how to use a test framework and helping them to set it up. In my experience it is something they find uses for.
We also use source control for items relevant to the code but not the code.
It's a long hard road to walk but it does lead to a good place eventually.
July 15, 2021 at 4:19 pm
I haven't looked at pricing in a long time, but it's interesting. I don't know enough about the test plans to decide if they are worth it, but for 20 users, AzDO is $90/month. That's about $1000/year, which is a pretty low cost in most business or gov budgets. That gets you a lot, so spending more then a few minutes talking about it seems silly.
For Github, $/user/month, so 20 people using Git there would be $80. Not bad
July 16, 2021 at 2:44 am
Thank you, David. I learned a little more after making my post a couple days ago. Looks like the group that want a Git repo are doing so because whatever tool they're using, has Git built into it. Makes me wonder how experienced they are with Git? For all its shortcomings, TFVC is easy to use, compared to git. I especially mean with respect to people sharing code. Of course, I know only too well the problems multiple people using TFVC can get themselves into. There's a small, but sizeable number of people at work who have experience with Git, but I get the feeling they're experience is using a Git repo by themselves. I'd like to get the opportunity to meet with the users asking for this. At this point, I've not had that chance. And there's no guarantee that I will.
Your four areas of good code practices are worth experienced developers to know and review!
Rod
July 16, 2021 at 2:47 am
I hadn't looked up the pricing of Azure DevOps for a large-ish, small group (20 people). Thank you for letting me know. I agree that $90/month or $80/month is reasonably cheap for a company, and I hope even a government agency. And like I've pointed out to my boss yesterday, our on-prem version of TFS is so old it's already out of support. And I think that Azure DevOps services is free for groups of 5 or 6. And I believe GitHub is free for small groups of 5.
Rod
July 16, 2021 at 4:12 pm
I think both Azure DevOps and GitHub are free for 5. Certainly worth thinking about for small teams. Managing this stuff on premises has little value, and it eats up time.
Viewing 8 posts - 1 through 7 (of 7 total)
You must be logged in to reply to this topic. Login to reply