Delete 5 lac records from table

Question

Delete 5 lac records from table

VSSGeorge

SSCrazy Eights

Points: 8151
More actions
February 9, 2018 at 1:06 am

#410227

I have a table where there is a column called nationality having possible values "US" and "Non -US'.
I need to delete the "Non -US' records from my table which amounts to more than 5 lac records, in the best possible way considering the database performance.
Kindly suggest the ways to do that.

Viewing 8 posts - 1 through 7 (of 7 total)

You must be logged in to reply to this topic. Login to reply

John Mitchell-245523 SSC Guru Points: 148809 More actions · Answer 1

-- Subsititute x for the number of rows you're happy to -- delete at a time in order to limit transaction log growth DELETE TOP(x) FROM MyTable WHERE MyColumn = 'Non -US' -- Rinse and repeat

John

Conficker SSCommitted Points: 1567 More actions · Answer 2

Agree with the DELETE statement, would suggest to use BEGIN TRAN to be on a safe side and then COMMIT

John Mitchell-245523 SSC Guru Points: 148809 More actions · Answer 3

Conficker - Friday, February 9, 2018 4:14 AM
Agree with the DELETE statement, would suggest to use BEGIN TRAN to be on a safe side and then COMMIT

Doesn't make much difference for a single statement. If you're doing this in a lot of batches then you don't want to be doing an explicit COMMIT after each one.

John

andrew gothard SSChampion Points: 12314 More actions · Answer 4

Hi there,
            in terms of performance, one thing you'll need - in order to prevent issues with other processes, even when they will not be touching the records in question is lock escalation.
    Very briefly:
        Initially, your SQL Server engine will look at either taking out row or page locks on the table concerned.
        However, locks have a cost in terms of management and resource allocation - they're, to a degree, expensive.
        Therefore, after a certain threshold, where the Engine considers locking the individual records / pages in question too expensive, it will escalate the lock to a table lock. This prevents other processes accessing those records, due to an inability to take out their own locks.
        The approximate threshold per operation is c. 5000 locks (3000 pre 2005).

So, something based around this;

DECLARE @Rows INT = 1
WHILE @Rows > 0
BEGIN
    DELETE TOP <number>
    FROM <table>
    WHERE <your filter>
    SELECT @Rows = @@ROWCOUNT
END
PRINT 'Done'

I'm a DBA.
I'm not paid to solve problems. I'm paid to prevent them.

Gail Shaw SSC Guru Points: 1004504 More actions · Answer 5

If there aren't too many foreign keys and the amount you're deleting is more than the amount of rows you want to keep, it may well be faster to insert the rows you want to keep into a new table, drop the old table and then rename the new one (and put all the keys and indexes back afterwards)

Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability

We walk in the dark places no others will enter
We stand on the bridge and no one may pass

VSSGeorge SSCrazy Eights Points: 8151 More actions · Answer 6

GilaMonster - Friday, February 9, 2018 5:29 AM
If there aren't too many foreign keys and the amount you're deleting is more than the amount of rows you want to keep, it may well be faster to insert the rows you want to keep into a new table, drop the old table and then rename the new one (and put all the keys and indexes back afterwards)

That's a good idea. But should I include it in a transaction? If you can post a sample code, it will be ideal. Thanks

Jeff Moden SSC Guru Points: 1004313 More actions · Answer 7

VSSGeorge - Saturday, February 24, 2018 10:57 PM
GilaMonster - Friday, February 9, 2018 5:29 AM
If there aren't too many foreign keys and the amount you're deleting is more than the amount of rows you want to keep, it may well be faster to insert the rows you want to keep into a new table, drop the old table and then rename the new one (and put all the keys and indexes back afterwards)
That's a good idea. But should I include it in a transaction? If you can post a sample code, it will be ideal. Thanks

Can you describe the table a bit more? For example, what are the total number of rows that it has? How many foreign keys does it have pointing at other tables and how many foreign keys point at the table being deleted from? How much total disk space does the table occupy?

All these questions will help tell me if you can do the ol' trick of creating a temporary file group to isolate the good rows on, truncate the original table, and copy the rows back in a minimally logged fashion (for performance and to keep from blowing the log file out), and then dropping the temporary file group to keep from unnecessarily increasing the size of the MDF file. It's nearly the same as what Gail mentioned but has the advantage of not expanding the MDF file any further.

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)