September 18, 2010 at 1:38 pm
Comments posted to this topic are about the item Big Data and SQL
September 20, 2010 at 8:16 am
"That is fairly telling, that training people to understand SQL, and maybe write better queries and analysis themselves can make them much more efficient."
It is high time, IMHO, for every college graduate to be introduced to set theory and how it relates to the real world. I continue to run across business people with such poor understanding of data that, at times, it seriously hampers them fulfilling the requirements of their job and definitely increases the costs to develop applications while you hold their hands through understanding their company's ERP data.
We continue to see people using Excel as a database with no relational integrity between spreadsheets and other abuses. It seems that learning the basics of relational data in something like MS Access would go a long way towards helping these people deal with the real world.
-dan
September 20, 2010 at 8:36 am
The question this article asks about developers rings true for me too (I have no formal SQL or DBMS training, rather mathematics and CS). It's amazing how many blank stares you get from other developers when you tell them even simple things like 'wrap that aggregation in a view'. It goes beyond bare SQL too when you hear things like 'this code to manage replication is getting really ugly' and 'I need to bring in some data from another db, but I can't figure out how to configure this framework plugin'. Granted there are times when you need the flexibility of coding things yourself, but the lack of knowledge regarding the DBMS platform has people the world over re-inventing the wheel...
September 20, 2010 at 8:41 am
What is the best resource for a decently experienced developer who is fairly beginner lever in SQL stuff to use to gain at least an intermediate level of SQL and SQL Server knowledge?
My studies of SQL to date have tended to be Hodge Podge and not very focused. I am hoping to find one book or online Tutorial or something of the sort that can take me to the next level in an orderly progression.
September 20, 2010 at 9:09 am
I found the line about training statisticians to think in SQL to be telling. Immediately after undergraduate college, I washed out of a graduate program in a physical science discipline, primarily because I hated, absolutely hated, using SAS and MINITAB (SPSS is much more rational, in my mind, of the major statistical packages) in a series of biometrics and statistics classes they taught all physical science grad students. I suspected the following at the time (and later as my experience with SQL and relational modelling grew, could articulate better this thought): that the statistician's mindset on proving independence is on a continuum of "orthogonal" to "180 degree reversed" to the data modeller's assumptions of relationships between entities.
Good grief, it seems the only thing physical scientists care about in their data is the p-value and the beta, because that's how you get your paper accepted in a scientific journal. Don't get me wrong, I like and understand many topics in statistics: confidence intervals, chi-square tests, using Student's T test to figure out if you can stand on a result based on a small sample: these make a lot of sense and are very useful. ANOVA and regression testing: interesting, I wish I spent more time learning these statistics topics at the time. But reducing data to only caring about the p-value and the beta? Just shoot me.
And I could never figure out how SAS Institute became a multibillion dollar software company.
If Cabela's had a staff of 7 SAS specialists who they could replace with 2 SQL specialists, that actually makes a lot of sense to me.
(edited to state: I meant the use of ANOVA and regression testing to find relationships is interesting; in the statistics course, the goal of using these tools was to prove nonrelationships among samples, so that your paper could be published. What a lot of BS busywork when there's more interesting things to do with data and your time!)
September 20, 2010 at 9:52 am
As a developer I agree that developers need to learn relationship theory, set theory and application of SQL.
Fortunately, I have covered all of these in my education (sometimes by active choice) and by commercial training. I cannot believe how many developers do not understand the need for this.
I am not, however, surprised how many managers do not see this need as essential as it truly is. (See that? That is productivity leaving through the front door!!!)
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
September 20, 2010 at 9:57 am
Gary Varga (9/20/2010)
I cannot believe how many developers do not understand the need for this.
There is just so much to learn along with there never seeming to be enough time to get our normal work done. I at least see the need to become more educated in a lot of areas but it is just so hard to find the time.
September 20, 2010 at 10:10 am
"It makes me wonder if the same thing might happen with developers. ... Do you think that building those skills is a worthwhile investment for other developers?"
Improving one's technical skills is always worthwhile. A continuous focus on writing the overall most efficient, maintainable process (not simply the most efficient SQL, routine, or program) will always pay dividends in the long run.
September 20, 2010 at 10:20 am
krowley (9/20/2010)
Gary Varga (9/20/2010)
I cannot believe how many developers do not understand the need for this.There is just so much to learn along with there never seeming to be enough time to get our normal work done. I at least see the need to become more educated in a lot of areas but it is just so hard to find the time.
I hear you but you have to fight for your training if you need to.
This is one area where your productivity will go through the roof, so will the performance of your applications.
(Also far easy to spot rubbish DBAs 😉
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
September 20, 2010 at 10:20 am
I realize every day how fortunate I am that in over 40 years of writing code for pay I have never been forced to be limited to only one role. Of course there have been times, sometimes for several years, that I have had to focus on one skill, language, or topic. But for the most part I have had the pleasure of learning multiple languages, significantly different data storage techiques, and many different computing environments.
I strongly believe that experience is the best teacher and experimentation is the path to learning. I may not know the best way to do something but I know that I can learn it and learn it quickly! (As a boy, I spent many a Saturday walking up and down streets and alleys with my firefighter father. He was a real pro - he wanted to know where he would have a challenge with his 100 foot aerial ladder before he was on his way to a fire. I spend my time "playing" with all sorts of languages and tools so that when I need to use them I have at least some sort of idea where the traps are at!)
Right now I'm working with a physcial chemist who is learning to transfer his Access experiences into the SQL Server world. I know that it's hard for him at times but he's getting it, applying all of his knowledge to it, and his applications are really getting better.
So, yes, I believe that set-based programming should be learned just as procedural, object, and functional programming should be learned. Its hard to find the time but it's well worth it!
September 20, 2010 at 10:31 am
Improving, and broadening, one's technical skills is not only worthwhile, it seems to be very necessary in today's economy, which stresses increased productivity (more output per worker). And I too find that making the time for doing so is not easy. It can be quite discouraging.
I also hope that six people did not lose their jobs at Cabela's, but somehow I doubt all six FTE's were retained -- the market doesn't reward stagnant profit margins, and reducing expenses is one way to improve margins in an economy not booming in sales.
As developers, we improve productivity and can potentially reduce the need for people (jobs). It would be nice if the savings were put towards increased innovation (and perhaps they were at Cabela's), as this is what is needed in order to create new jobs. Retraining and cross-training are great, but they aren't useful if the jobs simply aren't there.
September 20, 2010 at 10:41 am
How can you write software without some SQL knowledge?
Even gaming software requires storage, retrieval and use of data!
But alas... Each time we post ads for new developers, inevitably we get resumes where someone knows C# or VB.NET and yet has no or very limited SQL experience.
And as the CLR becomes more ubiquitous and wide spread, well, SQL folks better learn some C# / VB.NET, just as much as any coder better know enough SQL to get by.
Otherwise, you will be clapping with one hand in a world where applause is expected.
September 20, 2010 at 11:08 am
Having SQL skills, as a developer, is in my opinion essential. Well, perhaps I should say depending upon your work environment. I work in a small, departmental sized group. There are just 4 developers here, and one of them is our supervisor. We have to wear several hats, and SQL designer/writer is definitely one of them. Some years ago we had a developer here who didn't want to learn SQL. She did everything in MS Access's SQL Builder, and man, the SQL code that puts out is difficult to read and maintain when you've got to make modifications! I suppose that if one is a member of a large development team (>100 developers), then it wouldn't be necessary to be good at writing SQL, but at least in my environment, it is essential.
Kindest Regards, Rod Connect with me on LinkedIn.
September 20, 2010 at 11:36 am
Yes, it's important.
I spend a bit of tie each day consulting with the rest of the development team on SQL issues. I'm not popular as I don't just answer questions but teach principles and guide them to solve their own problems. I'm getting traction on design principles and joins.
Oh, and these principles are even more important in Mobile development. We are faced with SQL Server Compact Edition and the Compact versions of Dot Net. Still we are turning out products that blow everyone's mind.
ATBCharles Kincaid
September 20, 2010 at 12:25 pm
As a developer/Accidental DBA, I think having an in depth understanding of sql is essential. Years ago when I started here (company of 250 employees) we had a sql server 200 instance to manage all our intranet applications. It didn't really have a specific person in the role of DBA, nor did the oracle data warehouse team want anything to do with it. No indexes, primary keys, relationships nor constraints of any kind really, and many many bad queries lead to massive problems and more work overall for us devs.
Today we are in much better shape. SQL training every year is mandatory for all devs, We often work closely with reporting department (who also mentors), Our sysadmin and I both function as accidental DBA's, We're now running sql server 2008 and have set up policies, and defined a standard for the sql server development in our organization. Things run a lot of smoother and they get better by the day.
oh and I just passed 70-432 🙂
Viewing 15 posts - 1 through 15 (of 45 total)
You must be logged in to reply to this topic. Login to reply