December 21, 2016 at 8:01 am
whenriksen (12/21/2016)
CELKO (12/19/2016)
The best teachers I ever had were the ones that humiliated me when I was blindly stupid and could not get the basic concepts.Here you just scare people away. How many times does the OP come back after you have posted one of your rants? This is not a classroom and you are not our teacher. Most people are not here to improve their grades, and the ones that are may end up learning more than their current assignment. Learning to ask for help is critical to success. I would much rather people come here to get good advice than try to build it on their own when they are struggling with the fundamentals.
I understand you think you are improving the SQL Server community, and you certainly have a lot of knowledge that could benefit people here, but I think your arrogant, vitriolic response to people asking for help does more harm than good.
If the content of your responses was good and helpful, you wouldn't have to constantly remind people how long you've been doing this or how many books you've written.
+ a googol to the googolth power.
Steve (aka sgmunson) 🙂 🙂 🙂
Rent Servers for Income (picks and shovels strategy)
December 21, 2016 at 2:03 pm
sgmunson (12/21/2016)
whenriksen (12/21/2016)
CELKO (12/19/2016)
The best teachers I ever had were the ones that humiliated me when I was blindly stupid and could not get the basic concepts.Here you just scare people away. How many times does the OP come back after you have posted one of your rants? This is not a classroom and you are not our teacher. Most people are not here to improve their grades, and the ones that are may end up learning more than their current assignment. Learning to ask for help is critical to success. I would much rather people come here to get good advice than try to build it on their own when they are struggling with the fundamentals.
I understand you think you are improving the SQL Server community, and you certainly have a lot of knowledge that could benefit people here, but I think your arrogant, vitriolic response to people asking for help does more harm than good.
If the content of your responses was good and helpful, you wouldn't have to constantly remind people how long you've been doing this or how many books you've written.
+ a googol to the googolth power.
Thats a DAMN LOT OF ZEROS!!! :w00t:
Best,
Kevin G. Boles
SQL Server Consultant
SQL MVP 2007-2012
TheSQLGuru on googles mail service
December 21, 2016 at 3:03 pm
TheSQLGuru (12/21/2016)
sgmunson (12/21/2016)
whenriksen (12/21/2016)
CELKO (12/19/2016)
The best teachers I ever had were the ones that humiliated me when I was blindly stupid and could not get the basic concepts.Here you just scare people away. How many times does the OP come back after you have posted one of your rants? This is not a classroom and you are not our teacher. Most people are not here to improve their grades, and the ones that are may end up learning more than their current assignment. Learning to ask for help is critical to success. I would much rather people come here to get good advice than try to build it on their own when they are struggling with the fundamentals.
I understand you think you are improving the SQL Server community, and you certainly have a lot of knowledge that could benefit people here, but I think your arrogant, vitriolic response to people asking for help does more harm than good.
If the content of your responses was good and helpful, you wouldn't have to constantly remind people how long you've been doing this or how many books you've written.
+ a googol to the googolth power.
Thats a DAMN LOT OF ZEROS!!! :w00t:
Yep... more like a googol universes full... a mere googol is more than the number of subatomic particles in the known universe by a rather ginormous quantity. A googol to it's own power is called a "googolplex". Learned about the number googol (10100) in grade school, then about taking it to it's own power (googolplex) in high school. Also the kind of number one might use to describe the quantity of uselessness in some loser whose values just don't match up with even "remotely resembling normal"...
Steve (aka sgmunson) 🙂 🙂 🙂
Rent Servers for Income (picks and shovels strategy)
December 21, 2016 at 7:11 pm
whenriksen (12/21/2016)
CELKO (12/19/2016)
The best teachers I ever had were the ones that humiliated me when I was blindly stupid and could not get the basic concepts.Here you just scare people away. How many times does the OP come back after you have posted one of your rants? This is not a classroom and you are not our teacher. Most people are not here to improve their grades, and the ones that are may end up learning more than their current assignment. Learning to ask for help is critical to success. I would much rather people come here to get good advice than try to build it on their own when they are struggling with the fundamentals.
I understand you think you are improving the SQL Server community, and you certainly have a lot of knowledge that could benefit people here, but I think your arrogant, vitriolic response to people asking for help does more harm than good.
If the content of your responses was good and helpful, you wouldn't have to constantly remind people how long you've been doing this or how many books you've written.
There is an expression in Russian: "A bear's service".
Which means - a service which would better never been given.
That could be said about the vast majority of those "good advises" you referring to.
Ready-made solutions which an OP can use but not understand are more about bragging about your skills rather than actually helping.
It's the same old dilemma of fish vs. rod.
A kid cannot tie the shoe lace - how would you help?
You obviously advocating for doing the laces yourself - quick and easy, and the kid is happy off to the street.
Everyone is happy!
But would it be actually a good service for the kid?
Look at the forum - same members ask basic level questions week after week, month after month.
Which means - your answers did not actually help them.
They remain as illiterate in SQL as they've been before.
You help was exactly what they name "a bear's service".
Which means - you're doing wrong, and Joe may have a good point.
_____________
Code for TallyGenerator
December 21, 2016 at 8:44 pm
CELKO (12/14/2016)
The order by clause converts the set into a cursor. A cursor is a sequential file structure, which does have an ordering.
No, Sir. I know what you're trying to say but you're only half correct. Adding an ORDER BY to a SELECT does NOT necessarily convert the "set" into a cursor because all "sets" in SQL Server are created by the underlying nested machine language level cursors in every SELECT whether there's an ORDER BY or not. Although very sophisticated, SQL Server is nothing more than a file server. "Sets" are a logical thing in SQL Server sometimes matching an underlying physical set of rows, which must be read one at a time just as is true in almost every file system.
Let's have a look at the ever so humble Tally Table. For those that don't know how to create one, here's the code to do so.
SELECT TOP 1000000
N = IDENTITY(INT,1,1)
INTO dbo.Tally
FROM sys.all_columns ac1
CROSS JOIN sys.all_columns ac2
;
ALTER TABLE dbo.Tally
ADD CONSTRAINT PK_Tally PRIMARY KEY CLUSTERED (N)
;
Now, let's run a simple query without an ORDER BY against that Tally Table.
SELECT N
FROM dbo.Tally
WHERE N >= 500000
AND N < 501000
;
Here's the execution plan for that.
Now, let's run the same query with an ORDER BY.
SELECT N
FROM dbo.Tally
WHERE N >= 500000
AND N < 501000
ORDER BY N
;
And here's the execution plan for that code.
Most immediately pick up on the fact that both execution plans are identical. Some miss the fact that THERE IS NO SORT OPERATION in the second execution plan even though THERE IS AN ORDER BY IN THE CODE.
How is that possible? The answer is in the file structure, which is a doubly-linked-list set of pages, which may or may not be in a physical order nor do the pages have to be physically adjacent to each other. To read the rows, SQL Server must use a cursor to step through the pages. Once it reads a page, it must step through the row offsets stored on each page to read each row. So it's not just a cursor behind the scenes, it's a nested cursor. If the data is contained in more than 8 pages, uniform extents will eventually enter the picture, which SQL Server must also step though adding yet another nested cursor to every SELECT, with an ORDER BY or not.
If the rows are stored in the correct logical order needed for the ordered output stipulated by an ORDER BY, SQL Server doesn't need to add yet another cursor to the nest of cursors. In other words, SQL Server understands the concepts of both "toset" and "poset" and will not necessarily create an additional cursor to support an ORDER BY.
So, ORDER BY doesn't necessarily do anything but guarantee the order and it certainly doesn't guarantee the SELECT will become a cursor because the SELECT is already a cursor... with or without an ORDER BY. At the worst, an ORDER BY will simply add another cursor.
--Jeff Moden
Change is inevitable... Change for the better is not.
December 21, 2016 at 9:27 pm
Jeff Moden (12/21/2016)
So, ORDER BY doesn't necessarily do anything but guarantee the order and it certainly doesn't guarantee the SELECT will become a cursor because the SELECT is already a cursor... with or without an ORDER BY. At the worst, an ORDER BY will simply add another cursor.
Well, saying that - there is nothing in the world which is in chaos.
Everything is in order.
What we name "chaos" is an order different from the one we desire.
:hehe:
_____________
Code for TallyGenerator
December 21, 2016 at 10:29 pm
Jeff Moden (12/21/2016)
CELKO (12/14/2016)
Let's have a look at the ever so humble Tally Table. For those that don't know how to create one, here's the code to do so.
SELECT TOP 1000000
N = IDENTITY(INT,1,1)
INTO dbo.Tally
FROM sys.all_columns ac1
CROSS JOIN sys.all_columns ac2
;
ALTER TABLE dbo.Tally
ADD CONSTRAINT PK_Tally PRIMARY KEY CLUSTERED (N)
;
Now, let's run a simple query without an ORDER BY against that Tally Table.
SELECT N
FROM dbo.Tally
WHERE N >= 500000
AND N < 501000
;
Here's the execution plan for that.
Now, let's run the same query with an ORDER BY.
SELECT N
FROM dbo.Tally
WHERE N >= 500000
AND N < 501000
ORDER BY N
;
And here's the execution plan for that code.
Most immediately pick up on the fact that both execution plans are identical. Some miss the fact that THERE IS NO SORT OPERATION in the second execution plan even though THERE IS AN ORDER BY IN THE CODE.
Sorry Jeff, but this is actually a bad example.
I caught Paul White on this kind of mistake (what he and his admirers can't forgive me till now), and here I can see it again.
The source of the error is here:
EXEC sys.sp_spaceused @objname = N'dbo.Tally'
3.6 kb - single page.
Which means there is no actual Index seek, it's full table scan for any query you can run against this table.
Because SQL server will read the whole page into memory every time.
That's the first lie from the SQL Server in the execution plans you posted.
The second lie will be reviled by the following test.
Firsr, create a slightly bigger Tally table.
SELECT tg.N, CONVERT(VARCHAR(50), NEWID ()) Value
INTO #Tally
FROM dbo.TallyGenerator(1,1000000,NULL,1) tg
"TallyGenerator" is my inline function based on the well known code.
Then mess it up a bit:
ALTER TABLE #Tally ALTER COLUMN N BIGINT NOT NULL
ALTER TABLE #Tally ADD PRIMARY KEY CLUSTERED (N)
And now check what we've got:
EXEC sys.sp_spaceused @objname = N'tempdb..#Tally'
namerowsreserveddata index_sizeunused
Tally100000056744 KB56352 KB208 KB 184 KB
Sweet.
Now, to make things worse, choose some horrible selection criteria:
SET STATISTICS PROFILE ON
SELECT Value
FROM dbo.Tally
WHERE Value >= 'D'
AND Value < 'E'
SELECT Value
FROM dbo.Tally
WHERE Value >= 'D'
AND Value < 'E'
ORDER BY N
SET STATISTICS PROFILE OFF
If you look at the graphic execution plans, you'll find them identical: Clustered Index Scan, Parallelism, and then SELECT. No ordering on any step.
But on the text plans you can spot the difference:
SELECT [Value] FROM [dbo].[Tally] WHERE [Value]>=@1 AND [Value]<@2
|--Parallelism(Gather Streams)
|--Clustered Index Scan(OBJECT: ([tempdb].[dbo].[Tally].[PK__Tally__3BD019B31F12741A]), WHERE: ([tempdb].[dbo].[Tally].[Value]>=[@1] AND [tempdb].[dbo].[Tally].[Value]<[@2]))
SELECT [Value] FROM [dbo].[Tally] WHERE [Value]>=@1 AND [Value]<@2 ORDER BY [N] ASC
|--Parallelism(Gather Streams, ORDER BY: ([tempdb].[dbo].[Tally].[N] ASC))
|--Clustered Index Scan(OBJECT: ([tempdb].[dbo].[Tally].[PK__Tally__3BD019B31F12741A]), WHERE: ([tempdb].[dbo].[Tally].[Value]>=[@1] AND [tempdb].[dbo].[Tally].[Value]<[@2]) ORDERED FORWARD)
This difference is actually present on the graphic plans as well, on the "details" pane for Clustered Index Scan you may see "Ordered" item is "False" for query 1 and "true" for Query 2.
And you may see (or may not - because broken watches indicate correct time twice a day) the difference in the returned recordsets.
Now, if you go back to your original queries:
SET STATISTICS PROFILE ON
SELECT Value
FROM dbo.Tally
WHERE N >= 500000
AND N < 510000
SELECT Value
FROM dbo.Tally
WHERE N >= 500000
AND N < 510000
ORDER BY N
SET STATISTICS PROFILE OFF
You'll find that both execution plans actually have ORDER BY statement included:
SELECT [Value] FROM [dbo].[Tally] WHERE [N]>=@1 AND [N]<@2
|--Clustered Index Seek(OBJECT:([tempdb].[dbo].[Tally].[PK__Tally__3BD019B31F12741A]), SEEK:([tempdb].[dbo].[Tally].[N] >= CONVERT_IMPLICIT(bigint,[@1],0) AND [tempdb].[dbo].[Tally].[N] < CONVERT_IMPLICIT(bigint,[@2],0)) ORDERED FORWARD)
SELECT [Value] FROM [dbo].[Tally] WHERE [N]>=@1 AND [N]<@2 ORDER BY [N] ASC
|--Clustered Index Seek(OBJECT:([tempdb].[dbo].[Tally].[PK__Tally__3BD019B31F12741A]), SEEK:([tempdb].[dbo].[Tally].[N] >= CONVERT_IMPLICIT(bigint,[@1],0) AND [tempdb].[dbo].[Tally].[N] < CONVERT_IMPLICIT(bigint,[@2],0)) ORDERED FORWARD)
SQL Server needs to put the records in order for the index seek.
So, it actually executed ORDER BY in both queries.
It just did not tell you about that.
Which is its second lie in the plans.
_____________
Code for TallyGenerator
December 22, 2016 at 2:32 am
TomThomson (12/17/2016)
CELKO (12/14/2016)
its working ok with ORDER BY and the same results are been returned...
what does that mean though? that you always have to insert ORDER BY ?
You are missing some fundamental concepts in RDBMS. The first is that tables are sets, and sets have no ordering by their nature. They are completed wholes. Without seeing anything because you did not post it, I will guess both result sets from both queries were actually identical. The order by clause converts the set into a cursor. A cursor is a sequential file structure, which does have an ordering.
There are many reasons this happens. Different indexing, parallelism, or totally different database architectures. For example, in the old days the way we did groupings was to first sort the result set and then scan it. However, we got hashing later. In products that make heavy use of this, like Teradata, the smaller hash buckets tend to be returned first from the SQL engine. In products that have parallelism, you get the same sort of affect, but it is unpredictable.
I am going to guess your mindset is still locked in sequential files, and not in RDBMS yet. It is kind of like learning a new language with totally different grammar.
Joe, I (and many others) get really pissed off when people make illiterate/ignorant claims like "an order clause converts a set into a cursor". I've got used to you shouting unhelpfully, but for heaven's sake stop it - I know you're better than that because I haven't seen any of that sort of crap on the odd occasions when we have communicated one to one. A lot of your work has earned respect, but posting piles of nonsense (often offensive and unacceptable nonsense) makes people think much less of you than your numerous helpful contributions to discussion of issues in RDBMS understanding and relational algebra deserve.
Joe, I and many others are very pleased to read your posts, as you obviously know what you are talking about. We wouldn't have such amazingly good RDBMS products available if it wasn't for people like you.
Please keep trying to educate the masses.
There are people on this site who claim to be working with SQL Server but ask such basic questions that it scares me. They need to have their way of thinking challenged, rather than simply being given answers to their questions.
December 22, 2016 at 7:36 am
Sergiy (12/21/2016)
The source of the error is here:
EXEC sys.sp_spaceused @objname = N'dbo.Tally'
3.6 kb - single page.
Which means there is no actual Index seek, it's full table scan for any query you can run against this table.
Because SQL server will read the whole page into memory every time.
Correct... the source of the error is there but the error is not mine, ol' friend. Do the math in your head. Even if each row were just one byte, a million of them is much more than 8,192 bytes and a page has space reserved for header and row offset information, at the very least. Here's what I get for sp_SpaceUsed on the million row Tally Table created by the code I posted.
name rows reserved data index_size unused
----- ------- -------- -------- ---------- ------
Tally 1000000 13360 KB 12872 KB 80 KB 408 KB
There are 128 pages per megabyte so, according to the data amount above, there should be more than 1 page even just for the data. To prove that out, run the following SELECT.
SELECT *
FROM sys.dm_db_index_physical_stats(DB_ID(),OBJECT_ID('dbo.Tally'),NULL,NULL,'DETAILED')
;
index_depth index_level fragment_count avg_fragment_size_in_pages page_count record_count
----------- ----------- -------------- -------------------------- ---------- ------------
3 0 29 55.4827586206897 1609 1000000
3 1 4 1 4 1609
3 2 1 1 1 4
The output shows that even the second level of the B-Tree contains more than 1 page and the avg_fragment_size_in_pages suggests that the pages aren't contiguous nor necessarily physically ordered in relation to each other. 😉
There was also an index seek on both queries. Here's the related XML common to both the unordered SELECT and the SELECT with the ORDER BY in my examples.
<RelOp AvgRowSize="11" EstimateCPU="0.0014061" EstimateIO="0.00386574" EstimateRebinds="0" EstimateRewinds="0" EstimateRows="1135.55" LogicalOp="Clustered Index Seek" NodeId="0" Parallel="false"
[font="Arial Black"]PhysicalOp="Clustered Index Seek" [/font]EstimatedTotalSubtreeCost="0.00527184" TableCardinality="1000000">
<OutputList>
<ColumnReference Database="[Test]" Schema="[dbo]" Table="[Tally]" Column="N" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="1000" ActualEndOfScans="1" ActualExecutions="1" />
</RunTimeInformation>
[font="Arial Black"]<IndexScan Ordered="true" ScanDirection="FORWARD" [/font]ForcedIndex="false" ForceSeek="false" NoExpandHint="false">
------------------------------------------------------------------------------------
<RelOp AvgRowSize="11" EstimateCPU="0.0014061" EstimateIO="0.00386574" EstimateRebinds="0" EstimateRewinds="0" EstimateRows="1135.55" LogicalOp="Clustered Index Seek" NodeId="0" Parallel="false"
[font="Arial Black"]PhysicalOp="Clustered Index Seek" [/font]EstimatedTotalSubtreeCost="0.00527184" TableCardinality="1000000">
<OutputList>
<ColumnReference Database="[Test]" Schema="[dbo]" Table="[Tally]" Column="N" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="1000" ActualEndOfScans="1" ActualExecutions="1" />
</RunTimeInformation>
[font="Arial Black"]<IndexScan Ordered="true" ScanDirection="FORWARD"[/font] ForcedIndex="false" ForceSeek="false" NoExpandHint="false">
Once the first row page was found, then the expected ordered scan to read the data took over on both.
I caught Paul White on this kind of mistake (what he and his admirers can't forgive me till now), and here I can see it again.
Heh... nah. They'll never forgive you... It's not their nature. I don't need to forgive you because there's nothing to forgive. I always appreciate your feedback and the discussions we have.
--Jeff Moden
Change is inevitable... Change for the better is not.
December 22, 2016 at 10:49 am
Jeff Moden (12/21/2016)
CELKO (12/14/2016)
The order by clause converts the set into a cursor. A cursor is a sequential file structure, which does have an ordering.No, Sir. I know what you're trying to say but you're only half correct. Adding an ORDER BY to a SELECT does NOT necessarily convert the "set" into a cursor because all "sets" in SQL Server are created by the underlying nested machine language level cursors in every SELECT whether there's an ORDER BY or not.
Really, when we're talking databases, cursors to me are very specific constructs, they're declared with a specific keyword and in SQL Server's case come with a fairly significant set of concerns which I think we should be aware of. The low level programming constructs used by SQL Server really don't share any of our usual concerns with actual SQL cursors and as many times as I've read records from storage in a loop in any programming language I can't recall ever calling one a cursor.
I think its very helpful to go with the common terminology when used in a particular domain, ie., we should all agree on what a cursor is in the context of routine usage of the term when discussing today's SQL database technology.
Although very sophisticated, SQL Server is nothing more than a file server.
LOL never mind
December 22, 2016 at 12:50 pm
Sergiy (12/21/2016)
That could be said about the vast majority of those "good advises" you referring to.
Advise does not necessarily equal a solution.
Ready-made solutions which an OP can use but not understand are more about bragging about your skills rather than actually helping.
Some people learn by putting things together, others by taking things apart. Sometimes I can figure things out on my own, other times I need to copy someone else's solution and break it apart to understand why it works. Posting a solution here does not equal bragging. It adds to the conversation. I learn a lot from the solutions others have posted as well as the feedback I get from solutions I post.
It's the same old dilemma of fish vs. rod.
A kid cannot tie the shoe lace - how would you help?
Depends. Is this kid late for school or spending the afternoon in front of the TV? Is the OP in crisis or doing general development?
But would it be actually a good service for the kid?
Would berating the kid for doing it wrong and telling them they don't understand the fundamental purpose of laces and how laces got where they are today be considered a good service?
Look at the forum - same members ask basic level questions week after week, month after month.
Which means - your answers did not actually help them.
They remain as illiterate in SQL as they've been before.
Agreed, but no answer is going to make that person learn. When they can't cut it, they will eventually get fired and someone else will take over. If questions by a lazy, incompetent OP frustrate you, don't reply. Let someone else handle it. A less capable member can use that opportunity to help someone else and maybe learn something as well. In the meantime, many others can learn from the conversation that results.
In my opinion, people post questions here to get something done. If they can learn something in the process, that is a bonus.
For my part now, I do apologize
- to zouzou for not contributing anything of value to his question
- to the forum members for being off-topic and sending the thread on a tangent
- to Joe for the aggressive, attacking tone of my response
Happy Holidays!
Respectfully,
Wes
Wes
(A solid design is always preferable to a creative workaround)
December 22, 2016 at 1:48 pm
patrickmcginnis59 10839 (12/22/2016)
Jeff Moden (12/21/2016)
CELKO (12/14/2016)
The order by clause converts the set into a cursor. A cursor is a sequential file structure, which does have an ordering.No, Sir. I know what you're trying to say but you're only half correct. Adding an ORDER BY to a SELECT does NOT necessarily convert the "set" into a cursor because all "sets" in SQL Server are created by the underlying nested machine language level cursors in every SELECT whether there's an ORDER BY or not.
Really, when we're talking databases, cursors to me are very specific constructs, they're declared with a specific keyword and in SQL Server's case come with a fairly significant set of concerns which I think we should be aware of. The low level programming constructs used by SQL Server really don't share any of our usual concerns with actual SQL cursors and as many times as I've read records from storage in a loop in any programming language I can't recall ever calling one a cursor.
I think its very helpful to go with the common terminology when used in a particular domain, ie., we should all agree on what a cursor is in the context of routine usage of the term when discussing today's SQL database technology.
Although very sophisticated, SQL Server is nothing more than a file server.
LOL never mind
Agreed. I normally refer to such cursors as "pseudo-cursors" (Phrase coined by R.Barry Young) to make the distinction but didn't want to confuse Celko with a term that would be new to him. It's also a shame that most people don't think of SQL Server as a file server... they'd be able to grasp certain performance concepts a whole lot easier.
--Jeff Moden
Change is inevitable... Change for the better is not.
Viewing 12 posts - 31 through 41 (of 41 total)
You must be logged in to reply to this topic. Login to reply