Check duplicates records while bulk inserting in to Table

Question

Post reply

Check duplicates records while bulk inserting in to Table

mohanaraj

SSC-Addicted

Points: 406
More actions
June 28, 2010 at 1:48 am

#145746

Hello,
I want to insert the record in to a table using bulk insert method fro that I use the below
EXEC sp_xml_preparedocument @index OUTPUT, @xmlReviewComments
INSERT INTO PC_Tbl_FooterTemplate_Position
(
TemplateId,
Position,
[Content]
)
SELECT @FooterId ,Position,Content
FROM OPENXML (@index, '/NewDataSet/Table1')
WITH (
Templateid bigint 'Templateid',
Position nvarchar(50) 'Position',
Content nvarchar(500) 'Content'
)
EXEC sp_xml_removedocument @index
above is an Example
Before insert How can I check for a duplication whether the insertion record already present in the table.
I don't want to insert duplicate records to table.
Please help me to avoid duplicate insert while bulk upload.
Thanks
Regards,
Mohanaraj.S

Viewing 5 posts - 1 through 4 (of 4 total)

You must be logged in to reply to this topic. Login to reply

Eugene Elutin SSC Guru Points: 59322 More actions · Answer 1

Try this:

INSERT INTO PC_Tbl_FooterTemplate_Position

(

TemplateId,

Position,

[Content]

)

SELECT @FooterId, Position, Content

FROM OPENXML (@index, '/NewDataSet/Table1')

WITH (

Templateid bigint 'Templateid',

Position nvarchar(50) 'Position',

Content nvarchar(500) 'Content'

) indx

LEFT JOIN PC_Tbl_FooterTemplate_Position pp

ON pp.TemplateId = indx.Templateid

-- ? what are the keys of PC_Tbl_FooterTemplate_Position table?

-- add them all into join here! (don't need to change where clause, it is enough to check one key column for null)

WHERE pp.TemplateId IS NULL

_____________________________________________
"The only true wisdom is in knowing you know nothing"
"O skol'ko nam otkrytiy chudnyh prevnosit microsofta duh!":-D
(So many miracle inventions provided by MS to us...)

How to post your question to get the best and quick help[/url]

ChrisM@Work SSC Guru Points: 186127 More actions · Answer 2

mohanaraj (6/28/2010)
...I want to insert the record in to a table using bulk insert method fro that I use the below...

It's good practice to use a staging table when importing data. Specify column types to maximise data capture. Run TSQL validation / dedupe scripts against it. Flag up rows for eyeball validation.

Everything in your database is valuable, everything outside is garbage. A staging table serves as a firewall between the two.

^{“Write the query the simplest way. If through testing it becomes clear that the performance is inadequate, consider alternative query forms.” - Gail Shaw}

For fast, accurate and documented assistance in answering your questions, please read this article.
Understanding and using APPLY, (I) and (II) Paul White
Hidden RBAR: Triangular Joins / The "Numbers" or "Tally" Table: What it is and how it replaces a loop Jeff Moden

Jeff Moden SSC Guru Points: 1004366 More actions · Answer 3

Chris Morris-439714 (6/28/2010)
Everything in your database is valuable, everything outside is garbage. A staging table serves as a firewall between the two.

VERY well stated, Mr Morris! 🙂

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

ChrisM@Work SSC Guru Points: 186127 More actions · Answer 4

Jeff Moden (6/28/2010)
Chris Morris-439714 (6/28/2010)
Everything in your database is valuable, everything outside is garbage. A staging table serves as a firewall between the two.
VERY well stated, Mr Morris! 🙂

Thanks Jeff! It never ceases to amaze me how resistant some folks can be to something so obvious.

^{“Write the query the simplest way. If through testing it becomes clear that the performance is inadequate, consider alternative query forms.” - Gail Shaw}

For fast, accurate and documented assistance in answering your questions, please read this article.
Understanding and using APPLY, (I) and (II) Paul White
Hidden RBAR: Triangular Joins / The "Numbers" or "Tally" Table: What it is and how it replaces a loop Jeff Moden