speed up deleting million of records from a table with one filter

Question

Post reply

speed up deleting million of records from a table with one filter

kuppurajdpm

SSC Veteran

Points: 231
More actions
November 8, 2012 at 12:14 am

#393556

hi all,
I want to delete million of records from a table with only one filter of varchar(1) column
for that which one is best option.....

Viewing 8 posts - 1 through 7 (of 7 total)

You must be logged in to reply to this topic. Login to reply

Koen Verbeeck SSC Guru Points: 259105 More actions · Answer 1

What do you mean with filter?

Something like this?

DELETE FROM myTable WHERE columnA = 'B'

ps: why define a VARCHAR(1) column? What is wrong with CHAR(1). If you do have millions of rows, you're literally throwing storage away!

Need an answer? No, you need a question
My blog at https://sqlkover.com.
MCSE Business Intelligence - Microsoft Data Platform MVP

kuppurajdpm SSC Veteran Points: 231 More actions · Answer 2

hi

It's actually char(1) column only.I am not asking abt that.

I am asking batch processing is better or any other option will be better.

Because I am going to delete million of records it's time consuming process.so that I am asking...

Koen Verbeeck SSC Guru Points: 259105 More actions · Answer 3

kuppurajdpm (11/8/2012)
hi
It's actually char(1) column only.I am not asking abt that.
I am asking batch processing is better or any other option will be better.
Because I am going to delete million of records it's time consuming process.so that I am asking...

And I asked something about the filter. It's important that we know exactly what you're trying to achieve.

Need an answer? No, you need a question
My blog at https://sqlkover.com.
MCSE Business Intelligence - Microsoft Data Platform MVP

peterdru401 Mr or Mrs. 500 Points: 535 More actions · Answer 4

Simplest solution is to use a WITH. e.g.

WITH results ( <select> ) DELETE FROM results –

Sean Lange SSC Guru Points: 286590 More actions · Answer 5

If you want to prevent locking your table for an extended period of time you can do this in batches.

--This will ensure that the value of @@ROWCOUNT for the loop will be > 0

select 1

while @@ROWCOUNT > 0

begin

with DeleteList as (select top 1000 columnA from MyTable order by [Clustered Index])

--order by clustered index so the engine should not have to revisit any given page more than neccesary.

delete DeleteList

end

_______________________________________________________________

Need help? Help us help you.

Read the article at http://www.sqlservercentral.com/articles/Best+Practices/61537/ for best practices on asking questions.

Need to split a string? Try Jeff Modens splitter http://www.sqlservercentral.com/articles/Tally+Table/72993/.

Cross Tabs and Pivots, Part 1 – Converting Rows to Columns - http://www.sqlservercentral.com/articles/T-SQL/63681/
Cross Tabs and Pivots, Part 2 - Dynamic Cross Tabs - http://www.sqlservercentral.com/articles/Crosstab/65048/
Understanding and Using APPLY (Part 1) - http://www.sqlservercentral.com/articles/APPLY/69953/
Understanding and Using APPLY (Part 2) - http://www.sqlservercentral.com/articles/APPLY/69954/

ScottPletcher SSC Guru Points: 101093 More actions · Answer 6

It depends. Often it's faster just to keep the desired rows rather than delete all the undesired. That is, copy the good rows to another table, trunc the table, copy the good rows back in.

Otherwise you can DELETE in batches as Sean suggests ... but don't forget the filter on the DELETE statement itself :-).

SQL DBA,SQL Server MVP(07, 08, 09) "It's a dog-eat-dog world, and I'm wearing Milk-Bone underwear." "Norm", on "Cheers". Also from "Cheers", from "Carla": "You need to know 3 things about Tortelli men: Tortelli men draw women like flies; Tortelli men treat women like flies; Tortelli men's brains are in their flies".

CapnHector SSCoach Points: 16143 More actions · Answer 7

ScottPletcher (11/8/2012)
It depends. Often it's faster just to keep the desired rows rather than delete all the undesired. That is, copy the good rows to another table, trunc the table, copy the good rows back in.
Otherwise you can DELETE in batches as Sean suggests ... but don't forget the filter on the DELETE statement itself :-).

If im only keeping a small number of rows i will insert the rows into a new table but instead of truncating and reinserting the rows i will rename the tables. this will have much less down time as i dont have to reinsert the records into the original table after the truncate. once the tables are renamed i can keep the old table around to make sure every thing is working fine then just drop the table. Personally i think this method has a little more redundancy as the data is never gone until im sure i have every thing i need.

For faster help in answering any problems Please read How to post data/code on a forum to get the best help - Jeff Moden[/url] for the best way to ask your question.

For performance Issues see how we like them posted here: How to Post Performance Problems - Gail Shaw[/url]

Need to Split some strings? Jeff Moden's DelimitedSplit8K[/url]
Jeff Moden's Cross tab and Pivots Part 1[/url]
Jeff Moden's Cross tab and Pivots Part 2[/url]