Generating Test Data: Part 1 - Generating Random Integers and Floats

Question

Generating Test Data: Part 1 - Generating Random Integers and Floats

Viewing 15 posts - 46 through 60 (of 60 total)

You must be logged in to reply to this topic. Login to reply

Jeff Moden SSC Guru Points: 1004151 More actions · Answer 1

Spam reported. And, no... $33 for a pair of jeans is no way to thank anyone.

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

Evil Kraig F SSC Guru Points: 100851 More actions · Answer 2

Playing catchup on being underwater in a project for the last few weeks. Article looks great Jeff, and I'm sure will be very valuable. Nice Work!

- Craig Farrell

Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.

For better assistance in answering your questions[/url] | Forum Netiquette
For index/tuning help, follow these directions.[/url] |Tally Tables[/url]
Twitter: @AnyWayDBA

Jeff Moden SSC Guru Points: 1004151 More actions · Answer 3

Cool... 5 birds with 1 stone. SPAM reported. 😉

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

paul.knibbs SSCoach Points: 15320 More actions · Answer 4

Jeff Moden (3/30/2012)
Cool... 5 birds with 1 stone. SPAM reported. 😉

Well, strictly speaking it's the same bird 5 times...and what is it with this thread and spam, anyway? You must be particularly attractive to them, Jeff. 😀

Misha_SQL SSCertifiable Points: 5407 More actions · Answer 5

Another brilliant article from Jeff. Easy and fun read (not very often you can say this about a technical text). Can't wait for Part 2.

Jeff Moden SSC Guru Points: 1004151 More actions · Answer 6

mishaluba (4/21/2012)
Another brilliant article from Jeff. Easy and fun read (not very often you can say this about a technical text). Can't wait for Part 2.

Thanks, Mishaluba. I had gotten into a bit of a rut on this one and the folks I asked to review it for me (I knew I was in a rut) gave me some excellent suggestions to keep it from being monotonous. It's a tough thing to write something so technical and not be totally boring.

Thank you for the nice feedback on the article.

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

Devendrakumar SSC-Forever Points: 42493 More actions · Answer 7

I was almost disconnected with SSC for couple of months and got a chance to read the article today. Nice work!!!

I remember when I read one of your posts (1 million rows test) in a discussion; I found it difficult to understand how these 4 lines of code actually work. I followed classic divide & conquer technique to understand it (abs | checksum | newid | top | cross join). I am sure many of the readers might have lost their interest there as long as it served their purpose. This explanation will keep them connected to your articles.

Thanks for the article!!!

Jeff Moden SSC Guru Points: 1004151 More actions · Answer 8

Well done and thanks for the feedback, Dev.

Yeah... this article will give me a place to point to if someone comes up with the question on one of my posts. Hopefully, that's not all it's good for. Hopefully more people will do their own testing before they say such things as "In my experience, the best method is..." or "I've been at this for 25 years and the fastest method is..." or things like "Recursion rocks for everything!" because I've found that they're frequently wrong. Help destroy SQL Myths! Performance test some code today!!! 😉

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

Dwain Camps SSC Guru Points: 86938 More actions · Answer 9

Hey Jeff!

Using your test harness yesterday, I ran into some unexpected (for me) results. Let me explain:

--===== Declare some obviously named variables

DECLARE @NumberOfRows INT,

@StartValue INT,

@EndValue INT,

@Range INT

;

--===== Preset the variables to known values

SELECT @NumberOfRows = 1000000,

@StartValue = 400,

@EndValue = 500,

@Range = @EndValue - @StartValue + 1

;

--===== Conditionally drop the test table to make reruns easier in SSMS

IF OBJECT_ID('tempdb..#SomeTestTable','U') IS NOT NULL

DROP TABLE #SomeTestTable

;

--===== Create the test table with "random constrained" integers and floats

-- within the parameters identified in the variables above.

SELECT TOP (@NumberOfRows)

SomeRandomInteger = ABS(CHECKSUM(NEWID())) % @Range + @StartValue,

SomeRandomFloat = RAND(CHECKSUM(NEWID())) * @Range + @StartValue

INTO #SomeTestTable

FROM sys.all_columns ac1

CROSS JOIN sys.all_columns ac2

SELECT MinFloat=MIN(SomeRandomFloat), MaxFloat=MAX(SomeRandomFloat)

FROM #SomeTestTable

DROP TABLE #SomeTestTable

This returns the following:

MinFloat MaxFloat

400.000012322329 500.999933381006

My surprise was due to the fact that RAND() returns a random floating point number on the closed interval {0,1}, whereas this approach yielded some random floats outside the interval {400,500}. Clearly this approach is suitable when you are interested in applying the FLOAT to a date range, such as the following:

--===== Declare some obviously named variables

DECLARE @NumberOfRows INT,

@StartValue INT,

@EndValue INT,

@Range INT,

@FStartValue INT,

@FEndValue INT,

@FRange INT

;

--===== Preset the variables to known values

SELECT @NumberOfRows = 1000000,

@StartValue = 400,

@EndValue = 500,

@Range = @EndValue - @StartValue + 1,

@FStartValue = 400,

@FEndValue = 500,

@FRange = @EndValue - @StartValue

;

--===== Conditionally drop the test table to make reruns easier in SSMS

IF OBJECT_ID('tempdb..#SomeTestTable','U') IS NOT NULL

DROP TABLE #SomeTestTable

;

--===== Create the test table with "random constrained" integers and floats

-- within the parameters identified in the variables above.

SELECT TOP (@NumberOfRows)

SomeRandomInteger = ABS(CHECKSUM(NEWID())) % @Range + @StartValue,

SomeRandomFloat = RAND(CHECKSUM(NEWID())) * @FRange + @FStartValue,

SomeRandomDate = RAND(CHECKSUM(NEWID())) * @Range +

DATEADD(day, -@StartValue, DATEADD(day, DATEDIFF(day, 0, GETDATE()), 0))

INTO #SomeTestTable

FROM sys.all_columns ac1

CROSS JOIN sys.all_columns ac2

SELECT MinFloat=MIN(SomeRandomFloat), MaxFloat=MAX(SomeRandomFloat)

,MinDate=MIN(SomeRandomDate), MaxDate=MAX(SomeRandomDate)

FROM #SomeTestTable

DROP TABLE #SomeTestTable

Which delivers these results:

MinFloat MaxFloat MinDate MaxDate

400.000069523232 499.999990152787 2011-05-04 00:00:16.187 2011-08-12 23:59:58.753

I have taken the liberty to change the formula for SomeRandomFloat to deliver results on the closed interval {400,500}, which is what I was expecting. Date results end up being in the range of GETDATE() - 400 days and spans 100 days.

Mind you, I'm not reporting a bug. I'd prefer to think of it as an unexplained feature. 🙂

Let me know if I've misinterpreted something.

My mantra: No loops! No CURSORs! No RBAR! Hoo-uh![/I]
My thought question: Have you ever been told that your query runs too fast?

My advice:
INDEXing a poor-performing query is like putting sugar on cat food. Yeah, it probably tastes better but are you sure you want to eat it?
The path of least resistance can be a slippery slope. Take care that fixing your fixes of fixes doesn't snowball and end up costing you more than fixing the root cause would have in the first place.

Need to UNPIVOT? Why not CROSS APPLY VALUES instead?[/url]
Since random numbers are too important to be left to chance, let's generate some![/url]
Learn to understand recursive CTEs by example.[/url]
[url url=http://www.sqlservercentral.com/articles/St

paul.knibbs SSCoach Points: 15320 More actions · Answer 10

dwain, Jeff said this was the expected behaviour in the article:

The following code generates a million rows of random integers (the SomeRandomInteger column) with possible values from 400 to 500 and float values (the SomeRandomFloat column) from 400 to 500.999999999999 in just over 4 seconds on my 10 year old, single CPU computer.

Dwain Camps SSC Guru Points: 86938 More actions · Answer 11

paul.knibbs (6/7/2012)
dwain, Jeff said this was the expected behaviour in the article:
The following code generates a million rows of random integers (the SomeRandomInteger column) with possible values from 400 to 500 and float values (the SomeRandomFloat column) from 400 to 500.999999999999 in just over 4 seconds on my 10 year old, single CPU computer.

Ooops. Must'a missed it. Anyway it is clear now. Thanks.

My mantra: No loops! No CURSORs! No RBAR! Hoo-uh![/I]
My thought question: Have you ever been told that your query runs too fast?

My advice:
INDEXing a poor-performing query is like putting sugar on cat food. Yeah, it probably tastes better but are you sure you want to eat it?
The path of least resistance can be a slippery slope. Take care that fixing your fixes of fixes doesn't snowball and end up costing you more than fixing the root cause would have in the first place.

Need to UNPIVOT? Why not CROSS APPLY VALUES instead?[/url]
Since random numbers are too important to be left to chance, let's generate some![/url]
Learn to understand recursive CTEs by example.[/url]
[url url=http://www.sqlservercentral.com/articles/St

Jeff Moden SSC Guru Points: 1004151 More actions · Answer 12

And, just to be sure... RAND does NOT return the closed interval of {0,1}. It's a closed-open where 0 materializes but 1 never does. As Dwain has shown there are a couple of ways to handle it depending on what you want to do. The key in the article was to make every keenly aware that while constraining such numbers by range and domain is quite similar to the integer method, you have to account for the whole number-line between the integers instead of just the integers.

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

Dwain Camps SSC Guru Points: 86938 More actions · Answer 13

Jeff Moden (6/7/2012)
And, just to be sure... RAND does NOT return the closed interval of {0,1}. It's a closed-open where 0 materializes but 1 never does. As Dwain has shown there are a couple of ways to handle it depending on what you want to do. The key in the article was to make every keenly aware that while constraining such numbers by range and domain is quite similar to the integer method, you have to account for the whole number-line between the integers instead of just the integers.

Guess I need to spend more time with my nose firmly planted in BOL.

Thanks Jeff, yet again, for teaching me something!

My mantra: No loops! No CURSORs! No RBAR! Hoo-uh![/I]
My thought question: Have you ever been told that your query runs too fast?

My advice:
INDEXing a poor-performing query is like putting sugar on cat food. Yeah, it probably tastes better but are you sure you want to eat it?
The path of least resistance can be a slippery slope. Take care that fixing your fixes of fixes doesn't snowball and end up costing you more than fixing the root cause would have in the first place.

Need to UNPIVOT? Why not CROSS APPLY VALUES instead?[/url]
Since random numbers are too important to be left to chance, let's generate some![/url]
Learn to understand recursive CTEs by example.[/url]
[url url=http://www.sqlservercentral.com/articles/St

Jeff Moden SSC Guru Points: 1004151 More actions · Answer 14

My pleasure and thank you for the kind feedback, Dwain.

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

Steen Dybboe Valued Member Points: 56 More actions · Answer 15

Meant to rate it 5 Stars. Pushed one by accident - sorry

Great article Jeff

/Steen