September 24, 2018 at 10:13 am
Eirikur Eiriksson - Monday, September 24, 2018 10:07 AMDuration, data length and cardinality, will see if I can publish this as an article here, lots of work gone into this research.
😎
In short, 4440 items+ will bomb the performance.
Thanks Eirikur! Article would be great to see. I will admit, however, if someone has 4,400 items in a delimited list in SQL Server, they are really misusing their DBMS :hehe:
Thom~
Excuse my typos and sometimes awful grammar. My fingers work faster than my brain does.
Larnu.uk
September 24, 2018 at 10:14 am
Thom A - Monday, September 24, 2018 10:13 AMEirikur Eiriksson - Monday, September 24, 2018 10:07 AMDuration, data length and cardinality, will see if I can publish this as an article here, lots of work gone into this research.
😎
In short, 4440 items+ will bomb the performance.Thanks Eirikur! Article would be great to see. I will admit, however, if someone has 4,400 items in a delimited list in SQL Server, they are really misusing their DBMS :hehe:
Abusing may be a better term.
Michael L John
If you assassinate a DBA, would you pull a trigger?
To properly post on a forum:
http://www.sqlservercentral.com/articles/61537/
September 24, 2018 at 10:17 am
Lynn Pettis - Monday, September 24, 2018 10:13 AMThere is an OPENJSON method? I must be under a rock.
Haha, this is the version I have on my Sandbox server at home, and I take no credit for design, however, looks along the lines:
ALTER FUNCTION [dbo].[DelimitedSplitJSON] (@String varchar(MAX), @Delimiter varchar(10))
RETURNS TABLE
AS RETURN
SELECT [value] AS Item,
AS Position
FROM OPENJSON('["' + REPLACE(@String,@Delimiter,'","') + '"]"');
Thom~
Excuse my typos and sometimes awful grammar. My fingers work faster than my brain does.
Larnu.uk
September 24, 2018 at 10:33 am
Thom A - Monday, September 24, 2018 10:17 AMHaha, this is the version I have on my Sandbox server at home, and I take no credit for design, however, looks along the lines:
ALTER FUNCTION [dbo].[DelimitedSplitJSON] (@String varchar(MAX), @Delimiter varchar(10))
RETURNS TABLE
AS RETURN
SELECT [value] AS Item,
AS Position
FROM OPENJSON('["' + REPLACE(@String,@Delimiter,'","') + '"]"');
Here is a safer version for varchar(8000),
😎
USE TEEST;
GO
SET
NOCOUNT
ON
;
GO
ALTER
FUNCTION
dbo.ITVFN_SPLIT_8K_STRING_JSON
(
@INPUT_STRING
VARCHAR
(8000)
,@DELIMITER
VARCHAR
(250)
)
RETURNS
TABLE
WITH
SCHEMABINDING
AS
RETURN
(
SELECT
Y
AS
Item
from
OPENJSON
(
'[{"Y":"'
+
REPLACE
(STRING_ESCAPE(@INPUT_STRING,
'json'
),@DELIMITER,
'"},{"Y":"'
) +
'"}]'
)
WITH
(Y
VARCHAR
(8000)
'$.Y'
)
);
GO
I'll try to convert the document into an article on the flights to an from Pittsburgh next weekend when I'm meeting up with some of the regulars here 😉
September 24, 2018 at 11:40 am
Lynn Pettis - Thursday, September 20, 2018 9:56 AMOkay, not really sure where to post this so I think the water cooler is as good a place as any. My daughter needs ideas for a Big Data Project this semester and I am at a loss. Any ideas are welcome and apparently there is no other information regarding requirements at this time, so high level ideas with perhaps some links to potential data, i don't know (that's why i am asking for help).
Ask her to go here and find a dataset: https://cloud.google.com/bigquery/public-data/
Once she's interested in data, there could be lots of project items to tackle.
September 24, 2018 at 12:45 pm
And there it is. Azure Data Platform. It's really cool. I've done a quick unboxing here.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
September 24, 2018 at 1:51 pm
Eirikur Eiriksson - Monday, September 24, 2018 10:33 AMHere is a safer version for varchar(8000),
😎
USE TEEST;
GO
SET
NOCOUNT
ON
;
GO
ALTER
FUNCTION
dbo.ITVFN_SPLIT_8K_STRING_JSON
(
@INPUT_STRING
VARCHAR
(8000)
,@DELIMITER
VARCHAR
(250)
)
RETURNS
TABLE
WITH
SCHEMABINDING
AS
RETURN
(
SELECT
Y
AS
Item
from
OPENJSON
(
'[{"Y":"'
+
REPLACE
(STRING_ESCAPE(@INPUT_STRING,
'json'
),@DELIMITER,
'"},{"Y":"'
) +
'"}]'
)
WITH
(Y
VARCHAR
(8000)
'$.Y'
)
);
GO
I'll try to convert the document into an article on the flights to an from Pittsburgh next weekend when I'm meeting up with some of the regulars here 😉
Check for performance. I've seen several such JSON splitters and then someone compares against DelimitedSplit8K and the new built in splitter function and the JSON is reportedly a whole lot slower.
[EDIT] Might have even been you. I just noticed the post you made on the subject above. :blush:
--Jeff Moden
Change is inevitable... Change for the better is not.
September 24, 2018 at 1:53 pm
Thom A - Monday, September 24, 2018 10:13 AMEirikur Eiriksson - Monday, September 24, 2018 10:07 AMDuration, data length and cardinality, will see if I can publish this as an article here, lots of work gone into this research.
😎
In short, 4440 items+ will bomb the performance.Thanks Eirikur! Article would be great to see. I will admit, however, if someone has 4,400 items in a delimited list in SQL Server, they are really misusing their DBMS :hehe:
Heh... maybe not... it might be a rows from a file filled with telephone CDRs (Call Detail Records).
--Jeff Moden
Change is inevitable... Change for the better is not.
September 24, 2018 at 4:13 pm
patrickmcginnis59 10839 - Monday, September 24, 2018 8:33 AMI've had pretty good luck in getting people to think declaratively. SQL helps here with its select and update syntax, plus many people start off by doing queries with designers. Isolation modes seem to be more difficult.
I would be interested in why queries written in some lower isolation modes would not be declarative. Would that be because "imperative" implications are leaking into the query, like subqueries being run in a particular order as compared to the parent query?
Single queries are always declarative.
The issue is sequences of queries - if a sequence is declarative, all reads are repeatable within the sequence, there can be no query which delivers a phantom read within the sequence, and if identical read queries occur in the sequence each one of them delivers the same results. If we take a transaction as a the unit at whose end there can be a globally visible state-change, then declarativeness (or its absence) is a property of the sequence of queries that occurr within the transaction, and to enforce repeatable reads and the absence of non-repeatable reads without having extremely complex rules about what sequences of queries are allowed within a transaction one has to use either snapshot isolation level or sequential isolation level. I'm not 100% sure that either of those two isolation levels is enough to guarantee that sequences within a transaction are always strictly declarative, but it seems to me that they come close enough for me to accept that SQL, when using either of those isolation levels, is effectively declarative for most purposes.
Thirty years ago when I was chief architect of project involving building an RDBMS in a (declarative) functional language and providing a fully declarative functional interface (instead of SQL) as the interface to the RDBMS, I realised that there has to be an element of non-declarativeness within a transaction if one want extreme error management (including paranoid error detection) because it's neccessary to check before commiting the transaction that the new visible database state that will be delivered by commit is actually correct - so some error detection code has to see that state before commitment to check if the changes from the previous state are those required. So maybe paranoid error detection and a requirement for clean and tidy declarativeness are not easy to fit together without having some extra features (two stage commit is one option - stage one makes the new state visible locally and that allows paranoid error detection to run, stage two makes it visible globally if the error detection detects nothing wrong, and logs diagnostics and rolls back the transaction if something is wrong) which might be performance-expensive.
Tom
September 25, 2018 at 4:05 am
Lynn Pettis - Thursday, September 20, 2018 9:56 AMOkay, not really sure where to post this so I think the water cooler is as good a place as any. My daughter needs ideas for a Big Data Project this semester and I am at a loss. Any ideas are welcome and apparently there is no other information regarding requirements at this time, so high level ideas with perhaps some links to potential data, i don't know (that's why i am asking for help).
What sort of things is she interested in? What sort of hobbies does she have? Surely there's a big data type something hiding in there.
If not, what about voter trends / demographics / issues? Or if she wants something less political, what about determining the heuristics involved with getting an A.I. programmed or perhaps use S.E.T.I. data or go to NASA's website to help with the search for new planets?
September 25, 2018 at 4:09 am
Jeff Moden - Monday, September 24, 2018 6:43 AMThe real fact of the matter is that whether or not T-SQL is actually declarative or not, doesn't actually matter. It is what it is. The key is to get people to understand what it does and get them to make the paradigm shift from procedural (meaning row based in this context) programming to the way T-SQL and SQL Server work. If people want to call that way "Declarative", I'm ok with that even if it may not meet all of the requirements of a pure "Declarative" language, which I'm not actually going to take the time to explore further. 😀
"It is what it is" is probably the best quote I can think of. I've been teaching server administrators how to troubleshoot SQL Server and T-SQL and they absolutely hate that it's not "Problem is A, therefore Fix is B or C" kind of logic.
September 25, 2018 at 4:23 am
My "Joy" job of the month. Building a SQL Server Jobs Dictionary in MS Word for our team to know what jobs we have and what they do. If nothing else, it'll help us determine what is still needed post-migration and what we can get rid of.
Each job has its own page. In case anyone is interested, here's my "template."
Job Name:
Job Frequency:
# of Job Steps:
Job Definition:
Important Job Notes:
Step 1: <name>
Type:
Definition:
Special Execution Options:
Log File:
Log File Properties: Append output to existing file
Include step output in history
<copy step 1 template for all steps)Package Details
Package Name:
Configuration File Name:
Important Package Notes:
Package Server Location: <servername>, \Stored Packages\MSDB\<foldername>
Config Server Location:
SourceControl Location:
Config local box Location (what's in the package):
Files Involved
Names:
Data Source:
Data Destination:
NAS Inbound:
NAS Inbound Archive:
NAS Destination:Process Owners:
Contacts:
September 25, 2018 at 6:41 am
The trouble with that is, it'll get lost.
Send that template out to people that own the jobs, have them fill it in, and then copy that information into the description of the job so that it'll always be available. It's like documenting code.
--Jeff Moden
Change is inevitable... Change for the better is not.
September 25, 2018 at 6:50 am
Jeff Moden - Tuesday, September 25, 2018 6:41 AMThe trouble with that is, it'll get lost.
Send that template out to people that own the jobs, have them fill it in, and then copy that information into the description of the job so that it'll always be available. It's like documenting code.
My team owns most of the jobs. And I will be farming it out to other teams once I've filled in all my knowledge.
The job description thing? That is an excellent idea. I didn't think about that.
September 25, 2018 at 7:09 am
Jeff Moden - Monday, September 24, 2018 1:51 PMEirikur Eiriksson - Monday, September 24, 2018 10:33 AMHere is a safer version for varchar(8000),
😎
USE TEEST;
GO
SET
NOCOUNT
ON
;
GO
ALTER
FUNCTION
dbo.ITVFN_SPLIT_8K_STRING_JSON
(
@INPUT_STRING
VARCHAR
(8000)
,@DELIMITER
VARCHAR
(250)
)
RETURNS
TABLE
WITH
SCHEMABINDING
AS
RETURN
(
SELECT
Y
AS
Item
from
OPENJSON
(
'[{"Y":"'
+
REPLACE
(STRING_ESCAPE(@INPUT_STRING,
'json'
),@DELIMITER,
'"},{"Y":"'
) +
'"}]'
)
WITH
(Y
VARCHAR
(8000)
'$.Y'
)
);
GO
I'll try to convert the document into an article on the flights to an from Pittsburgh next weekend when I'm meeting up with some of the regulars here 😉
Check for performance. I've seen several such JSON splitters and then someone compares against DelimitedSplit8K and the new built in splitter function and the JSON is reportedly a whole lot slower.
[EDIT] Might have even been you. I just noticed the post you made on the subject above. :blush:
I have posted few warnings on different splitters' issues in the past, JSON being one of those 😉
😎
There is a threshold, which when reached, CPU goes through the roof and performance hides in the basement, typical definition of a code constructed time bomb.
Viewing 15 posts - 62,326 through 62,340 (of 66,712 total)
You must be logged in to reply to this topic. Login to reply