June 27, 2016 at 10:58 pm
Comments posted to this topic are about the item Democratic Data Science
June 28, 2016 at 12:35 am
I totally agree. This industry, like many others before it, applies tools the either automate repetitive tasks or raise the level of abstraction. More and more we are able to automate more complex, time consuming tasks.
I see this happening even in the slowest moving of businesses and IT departments.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
June 28, 2016 at 1:39 am
Data Science is just a buzz word for what used to be called Business Intelligence. I did a BI MSc and we studied Data Mining, Statistical Analysis, Optimization, Operational Research, Data Warehousing, etc. Actually, some of the previous student of the MSc I did, are now called "Data Scientists".
Microsoft with R in SQL Server seems to believe that the future is to have a devoted team of data scientists doing only statistical analysis and creating data mining algorithms and leave the data wrangling to SQL Developers. It doesn't make sense to pay people with advanced mathematical degree doing what someone with a one year experience in T-SQL can do more efficiently.
On the other hands, smalls start-up or small firms won't be able to afford a specialized "Data Scientists" and so not only the data wrangling but also the Data mining aspect will be done by the DBAs (or BI devs) using very easy to understands visual tools (such as the like of Rapidminer or Microsoft Azure machine learning). In other words, R/Python is to such tools, what t-sql is to SSIS. Learning the basic of R/Pyton is great but time wise, if you have a small firm, it makes more senses to acquire a data mining tool with strong visual component.
June 28, 2016 at 2:23 am
The big challenge with statistics is in communicating what the results actually mean. Understanding that point 51.9 vs 48.1 has no significance in a small sample but becomes immensely significant in a larger population.
Then there is http://www.tylervigen.com/spurious-correlations
Finding a high end geek with social skills who can deliver to a time and budget is a bit of a problem. I know one or two and they are truly exceptional. Seriously, seriously exceptional. That is why data scientist recruitment is referred to as recruiting the unicorn.
June 28, 2016 at 6:37 am
There will always be another cotton gin to replace what we are currently doing. But that just frees us up to do bigger and better things.
June 28, 2016 at 7:25 am
As much as I'd like to think our organization could use more data scientists, the reality is that the most accurate statistical models delivered by my colleagues were rejected by the business because they couldn't understand them. I guess if data science does become huge, it will take some time for data consumers to adjust to the trend--they still seem to be more interested in the straightforward aggregation that SQL-based reporting has always provided.
June 28, 2016 at 9:35 am
porter.james (6/28/2016)
As much as I'd like to think our organization could use more data scientists, the reality is that the most accurate statistical models delivered by my colleagues were rejected by the business because they couldn't understand them. I guess if data science does become huge, it will take some time for data consumers to adjust to the trend--they still seem to be more interested in the straightforward aggregation that SQL-based reporting has always provided.
Sadly, this rings true. It was either Netflix or Spotify that blogged on how they overcame this. They mentioned the hippo principle. Highest Paid Person's Opinion.
This is why a data science team needs an evangelist to sell the teams ideas to the powers that be. The evangelist needs to understand just enough to act of data science to act as the bridge between the two camps.
June 28, 2016 at 9:44 am
David.Poole (6/28/2016)
The big challenge with statistics is in communicating what the results actually mean. Understanding that point 51.9 vs 48.1 has no significance in a small sample but becomes immensely significant in a larger population.Then there is http://www.tylervigen.com/spurious-correlations
Finding a high end geek with social skills who can deliver to a time and budget is a bit of a problem. I know one or two and they are truly exceptional. Seriously, seriously exceptional. That is why data scientist recruitment is referred to as recruiting the unicorn.
Hear, hear! I have spent the better part of my life learning statistics the hard way. Once, I thought I had a career opportunity in developing awesome explanations of "what the results actually mean" in a clear language and using delicious visualizations.
Then I took the job, I could get...
Today I find that very few working people care about advanced data, nice visuals, or even "correct" explanations of things. Even less people wants to pay for it!!
What people want is what can help them keep their job for a little longer. If that's a lie, they'll take it! Because, when it comes to advanced statistics, the odds aren't that anyone will ever call the bluff!
So I make my money moving data around and doing all sorts of old-school data tasks. Manually. Slowly forgetting my old learning.
But for the young ones still unspoiled by reality: Go get yourself a job in one of the few companies who will try to create the sophisticated algorithms which just deliver "complex" to people for use "out of the box". That's where the money is 30 years from now, I'd say. And should the company fail, you have still got your skills at a time when the grunt work is truly automated - even in an old-school shop like mine...
June 28, 2016 at 9:45 am
David.Poole (6/28/2016)
porter.james (6/28/2016)
As much as I'd like to think our organization could use more data scientists, the reality is that the most accurate statistical models delivered by my colleagues were rejected by the business because they couldn't understand them. I guess if data science does become huge, it will take some time for data consumers to adjust to the trend--they still seem to be more interested in the straightforward aggregation that SQL-based reporting has always provided.Sadly, this rings true. It was either Netflix or Spotify that blogged on how they overcame this. They mentioned the hippo principle. Highest Paid Person's Opinion.
This is why a data science team needs an evangelist to sell the teams ideas to the powers that be. The evangelist needs to understand just enough to act of data science to act as the bridge between the two camps.
Amen!
June 28, 2016 at 9:53 am
porter.james (6/28/2016)
As much as I'd like to think our organization could use more data scientists, the reality is that the most accurate statistical models delivered by my colleagues were rejected by the business because they couldn't understand them. I guess if data science does become huge, it will take some time for data consumers to adjust to the trend--they still seem to be more interested in the straightforward aggregation that SQL-based reporting has always provided.
This is going to be something that takes time. Telecommuting couldn't catch on for a long time because too many managers couldn't understand how people would work. It is now.
Analytics in sports were dismissed by many, because they couldn't understand them and wanted an "eye test". However now most major sports, at high amateur and pro levels, use all sorts of analytics. There are a few holdouts that may lose their jobs in the next couple years for not including data analysis as part of their coaching. Not all, but part.
I think businesses will adjust, though it will be years for some, months for others.
June 28, 2016 at 9:55 am
Rod at work (6/28/2016)
Steve, your commentary makes me think that there's a high demand for data scientists. Either that or it pays well. I've not looked at the job market for it, so I don't know. Which is it?
There's demand, and the positions tend to pay well. Part of that is it's hot (hyped, talked about, etc), part is that the work is hard and supply is low.
I'd investigate it if you like stats and math. If not, stick with what you're good at.
The jobs are growing, but I think lots of companies don't quite know what to do with Data science and if you don't have lots of credentials (Experience, degrees, something), I'm not sure there's going to be a hiring boom even if Microsoft builds a MS Certified Data Scientist.
June 28, 2016 at 10:26 am
Grunt work is automated, thoughtful work is done by (well-paid) humans. If you're not doing the latter, you might not have a job with the former.
As career advice, this is absolutely true. Grunt work doesn't require much thought, makes your job performance more predictable, and is often valued at the time. It's very easy to fall into, no matter what your job. However, as your skills atrophy, it risks leaving you in the unemployment line for entry level jobs. Especially if your organization is growing and people can see what the opportunity cost of your salary is.
However, as software becomes more sophisticated, there will certainly be less grunt work in our industry...
As an industry observation, this absolutely isn't true. Never underestimate human ingenuity. There will always be new grunt work that new people will naturally fall to doing. Heck, I've even seen people make grunt work out of checking that all the automated jobs ran.
Edit: For a particular organization grunt work may be decreasing but industry-wide, it is more to do with economics. When staff counts need to be reduced, grunt work will go. When business is booming, grunt work naturally grows too.
Leonard
Madison, WI
June 28, 2016 at 10:44 am
While we're on the topic of organizations who leverage technology to automate "grunt work", this is a funny read.
How to Snare Millions of Men with Web Bots
http://gizmodo.com/ashley-madison-code-shows-more-women-and-more-bots-1727613924
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
June 28, 2016 at 11:44 am
Leonard,
You're probably right in some sense. I guess it depends on how we look at grunt work. For example, we will always need someone to rack servers and move equipment.
However, do we need more help desk people? Maybe not with newer bots that might handle lots of stuff. Do we need DBAs to manage backups? Maybe a few, or maybe it's absorbed elsewhere, but just having grunt work that doesn't require thought is going to go down for many of us.
Viewing 15 posts - 1 through 15 (of 27 total)
You must be logged in to reply to this topic. Login to reply