I was watching Vicky Harp in the 2020 October GroupBy conference recently talking about her challenges of working with the SQL Server platform as it's grown. It's a good keynote, with Vicky noting that she started with SQL Server 2000 and the journey to today is incredible. There's a great view of her world at the 10:00 mark (after the start).
She had this quote, which I found really thoughtful. This is something I think is true and something that many of us don't like.
"Usually growth comes at the expense of the previous comfort of safety." - Josh Waitzkin, The Art of Learning.
What's your response? Run, fight? Embrace change? Resist doing anything different? I think many of us want to think that we easily embrace change, but think about the last time someone wanted to change something at work? Reorg, new protocol, etc. Did you resist and think it was silly or go along and give it a chance? For many people, it's the former.
To be fair, it is for me as well, and I think many people that have some success in their career often want to stick with the things they are experienced in. That's not necessarily a problem, but it is worth investigating and embracing some new things, just to see if they might be better.
The data platform is certainly one of those technologies in my life that has changed dramatically, and the pace is sometimes overwhelming. At this point, it's hard to keep up, and hard to understand sometimes if new tech is better or worse. I've started to try and assume there is some good reason why Microsoft makes some changes, and then experiment, test, and evaluate the tech.
Not just once when something is released or I encounter it, but by also watching what others do and then learning where they've had success or failure. ADF is one of those areas where I initially dismissed it as a poor port of SSIS, but I've come to appreciate some of the ways in which this is an improvement to many flows, especially with hybrid workflows.
The data platform is an exciting place to work these days, and I hope you embrace some of the changes and see where a new technology might improve your environment. Of course, lots of traditional features out there work very well, and it's not worth changing just to change. Make sure there is value and an improvement in some way, beyond you just enjoying working with something shiny and new.