Big Data Engineers

Question

Big Data Engineers

Phil Parkin

SSC Guru

Points: 246963
More actions
June 23, 2015 at 3:49 am

#104255

I see so many jobs for these guys.
Luckily, I am already quite big. But I feel sorry for the hobbits and small guys, who are otherwise perfectly qualified for the position.
Size-ist, that's what it is.
The absence of evidence is not evidence of absence
- Martin Rees
The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
- Phil Parkin

Viewing 5 posts - 1 through 4 (of 4 total)

You must be logged in to reply to this topic. Login to reply

TomThomson SSC Guru Points: 104773 More actions · Answer 1

I think engineering data should be a crime - it would be nice to send who engineer data to mislead into penal servitude, regardless of their size. Especially if there were any competent honest politicians left to make (and unmake) laws.

But it won't happen until after Turkeys have votes for Thanksgiving.

Tom

Steve Jones - SSC Editor SSC Guru Points: 736270 More actions · Answer 2

Big is all relative. The title here is about "big data", which is an overloaded, hyped term meaning multiple things from JSON structured data to lots of text files.

ChrisM@Work SSC Guru Points: 186127 More actions · Answer 3

Being neither big nor an engineer I'm not really qualified to play with any data, but I do, and there are plenty of tables under the roof here which are in the hundreds of millions of rows.

Should I eat more?

^{“Write the query the simplest way. If through testing it becomes clear that the performance is inadequate, consider alternative query forms.” - Gail Shaw}

For fast, accurate and documented assistance in answering your questions, please read this article.
Understanding and using APPLY, (I) and (II) Paul White
Hidden RBAR: Triangular Joins / The "Numbers" or "Tally" Table: What it is and how it replaces a loop Jeff Moden

Jeff Moden SSC Guru Points: 1004362 More actions · Answer 4

Heh... there is no such thing as "Big Data". There's only 2 categories of data insofar as size... 1) that which you can handle and 2) that which you can't. 😀

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)