Getting Big Data

Question

Getting Big Data

Steve Jones - SSC Editor

SSC Guru

Points: 736478
More actions
February 25, 2016 at 9:24 pm

#305035

Comments posted to this topic are about the item Getting Big Data

Viewing 7 posts - 1 through 6 (of 6 total)

You must be logged in to reply to this topic. Login to reply

IceDread SSCertifiable Points: 5050 More actions · Answer 1

I have actually never come across this: Big data is a broad term for data sets so large or complex that traditional data processing applications are inadequate.

I've experience from fund companies, banks and also retail and logistics.

I suppose it can happen but usually it's a badly designed system that because of design can not handle the data.

Gary Varga SSC Guru Points: 82166 More actions · Answer 2

It does appear to have become a core DBA skill. Large data set management, that is. It does not appear to have much impact on development. I am sure that some developers have felt an impact but, anecdotally, I cannot see it as being far reaching.

Gaz

-- Stop your grinnin' and drop your linen...they're everywhere!!!

xsevensinzx One Orange Chip Points: 25560 More actions · Answer 3

IceDread (2/26/2016)
I have actually never come across this: Big data is a broad term for data sets so large or complex that traditional data processing applications are inadequate.
I've experience from fund companies, banks and also retail and logistics.
I suppose it can happen but usually it's a badly designed system that because of design can not handle the data.

Big data typically refers to the 4 V's: Volume, Velocity, Variety, and Veracity.

That translates into big data being so large or complex that traditional data processing applications are inadequate.

Inadequate not the same as being unable. Most traditional RDBMS can handle large volumes at high velocities at various veracity levels. Just maybe not all at the same time in the same problem. For example, something as simple as the Variety aspect crippling traditional ETL's is a good example regardless of Volume or Velocity.

Then you may say why: "Variety should really never happen." But, if you don't control the source data, it happens all the time regardless of what you consider good or bad design.

Rod at work SSC-Dedicated Points: 34017 More actions · Answer 4

Great article, Steve! I've been thinking of download R Studio (I think that's what it's called), but without a data set to analyze it's hard to really do anything with it. Especially learn anything. I think I'll grab me one of those, download R Studio and have some fun.

Kindest Regards, Rod Connect with me on LinkedIn.

IceDread SSCertifiable Points: 5050 More actions · Answer 5

xsevensinzx (2/26/2016)
IceDread (2/26/2016)
I have actually never come across this: Big data is a broad term for data sets so large or complex that traditional data processing applications are inadequate.
I've experience from fund companies, banks and also retail and logistics.
I suppose it can happen but usually it's a badly designed system that because of design can not handle the data.
Big data typically refers to the 4 V's: Volume, Velocity, Variety, and Veracity.
That translates into big data being so large or complex that traditional data processing applications are inadequate.
Inadequate not the same as being unable. Most traditional RDBMS can handle large volumes at high velocities at various veracity levels. Just maybe not all at the same time in the same problem. For example, something as simple as the Variety aspect crippling traditional ETL's is a good example regardless of Volume or Velocity.
Then you may say why: "Variety should really never happen." But, if you don't control the source data, it happens all the time regardless of what you consider good or bad design.

Some defines big data as when it has outgrown it's system specification that then would suggest a failure of architectural design.

Complexity I find usually is a time factor to divide and conquer, not that it's actually is very complex just a lot of it. But of course I belie it exists in several areas like cancer research and lots of analytical world research situations and probably other suggestions as well. I just have not encountered it.

I do however think you have a point, but I feel the term big data might be overused in some areas when I read about it in ordinary daily news webs.

akljfhnlaflkj SSC Guru Points: 76202 More actions · Answer 6

I guess I've never really dealt with big data then.