Procedures are timing out because of Index Defragmentation Job ...
-
Guys,
We have INDEXDEFRAG job running every few days. On our heaviest environment, it takes about 30 minutes to run. During the time it is running, we have many timeouts of stored procedures (a timeout occurs when a stored procedure takes > 30 seconds to complete). All of our stored procedures take under a second to complete during normal times. And when defragmentation job is off, we have no timeouts.
I know that we need to use defragmentation as our tables are constantly being deleted to and inserted from, thereby fragmenting the indexes. However, we do not want to have timeouts either.
Can anyone clue me in as to why timeouts occur during this process and is there any way to modify it to reduce the number of timeouts. The script I use to perform INDEXDEFRAG is shown below (I got it from some website).
/*Perform a 'USE <database name>' to select the database in which to run the script.*/
-- Declare variables
SET NOCOUNT ON
DECLARE @tablename VARCHAR (128)
DECLARE @execstr VARCHAR (255)
DECLARE @objectid INT
DECLARE @indexid INT
DECLARE @frag DECIMAL
DECLARE @maxfrag DECIMAL
-- Decide on the maximum fragmentation to allow
SELECT @maxfrag = 10.0
-- Declare cursor
DECLARE tables CURSOR FOR
SELECT TABLE_NAME
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE = 'BASE TABLE'
-- Create the table
CREATE TABLE #fraglist (
ObjectName CHAR (255),
ObjectId INT,
IndexName CHAR (255),
IndexId INT,
Lvl INT,
CountPages INT,
CountRows INT,
MinRecSize INT,
MaxRecSize INT,
AvgRecSize INT,
ForRecCount INT,
Extents INT,
ExtentSwitches INT,
AvgFreeBytes INT,
AvgPageDensity INT,
ScanDensity DECIMAL,
BestCount INT,
ActualCount INT,
LogicalFrag DECIMAL,
ExtentFrag DECIMAL)
-- Open the cursor
OPEN tables
-- Loop through all the tables in the database
FETCH NEXT
FROM tables
INTO @tablename
WHILE @@FETCH_STATUS = 0
BEGIN
-- Do the showcontig of all indexes of the table
INSERT INTO #fraglist
EXEC ('DBCC SHOWCONTIG (''' + @tablename + ''')
WITH FAST, TABLERESULTS, ALL_INDEXES, NO_INFOMSGS')
FETCH NEXT
FROM tables
INTO @tablename
END
-- Close and deallocate the cursor
CLOSE tables
DEALLOCATE tables
-- Declare cursor for list of indexes to be defragged
DECLARE indexes CURSOR FOR
SELECT ObjectName, ObjectId, IndexId, LogicalFrag
FROM #fraglist
WHERE LogicalFrag >= @maxfrag
AND INDEXPROPERTY (ObjectId, IndexName, 'IndexDepth') > 0
-- Open the cursor
OPEN indexes
-- loop through the indexes
FETCH NEXT
FROM indexes
INTO @tablename, @objectid, @indexid, @frag
WHILE @@FETCH_STATUS = 0
BEGIN
PRINT 'Executing DBCC INDEXDEFRAG (0, ' + RTRIM(@tablename) + ',' + RTRIM(@indexid) + ') - fragmentation currently '
+ RTRIM(CONVERT(varchar(15),@frag)) + '%'
SELECT @execstr = 'DBCC INDEXDEFRAG (0, ' + RTRIM(@objectid) + ',' + RTRIM(@indexid) + ')'
EXEC (@execstr)
FETCH NEXT
FROM indexes
INTO @tablename, @objectid, @indexid, @frag
END
-- Close and deallocate the cursor
CLOSE indexes
DEALLOCATE indexes
-- Delete the temporary table
DROP TABLE #fraglist
Thanks a lot!
-
You could check if the defrag actually helps (fragmentation before/after) with DBCC SHOWCONTIG.
Perhaps you should iterate on index position rather than tablename
(order by IndexId for the indexes cursor), so sql won't be working at one table for a longer period.
-
Based on the SQL posted, non-clustered indicies are being defragemented twice - one when the clustered index is defragemented, which also rebuilds the non-clustered indicies, and then again if it was originally highly fragmented.
Below are two stored procedures for defragementation.
Known problems:
Restart has not been thougly tested.
The check for the transaction log backup job already running does not work always due to slow posting by the SQL Server Agent, so occassionally the job will fail becuase the job is already running.
Good Luck.
if exists (select * from dbo.sysobjects where id = object_id(N'[dbo].[DBM_DBREINDEX_By_OutOfOrderPagePct]') and OBJECTPROPERTY(id, N'IsProcedure') = 1)
drop procedure [dbo].[DBM_DBREINDEX_By_OutOfOrderPagePct]
GO
if exists (select * from dbo.sysobjects where id = object_id(N'[dbo].[DBM_Index_get_FragmentionStats]') and OBJECTPROPERTY(id, N'IsProcedure') = 1)
drop procedure [dbo].[DBM_Index_get_FragmentionStats]
GO
CREATE procedure DBM_DBREINDEX_By_OutOfOrderPagePct
( @RunMinutes integer = null
, @StopTime varchar(8) = null
, @DefragOption varchar(12) = 'DBREINDEX'
, @LogBackupJobName sysname
, @RestartInd char(1) = 'N'
, @DebugInd char(1) = 'N'
, @DisplayProgressInd char(1) = 'N'
)
as
/*
Description: Defragments each table index in the database from most fragmented to least fragemented.
Privileges: System Adminstrator only (DB Owner is insufficient)
Run Time Stops when the specified end time is reached but after the current index rebuild completes.
Locking: When 'DBREINDEX' is specified, locks the table being defragemented
When "INDEXDEFRAG" is specified, locks some rows for the table being defragemented
Dependencies: Table TableIndexFragementation must be populated by executing stored procedure DBM_Index_get_FragmentionStats
Table TableIndexDefragmenting is used for restarting and after an index is rebuilt, the index row is removed from the table and on successful completion, drops the restart table.
Install: Into each database
Inputs RunMinutes Optional The number of minutes, after which, that procedure is to stop running.
StopTime Optional The time that procedure is to stop running.
Note that either RunMinutes or StopTime must be provided, but not both.
DefragOption Required 'DBREINDEX' or 'INDEXDEFRAG'
LogBackupJobName Optional Name of the transction log backup job and is optional if the database is in simple recovery
mode.
DebugInd Optional When "y", prints diagnose without defragementing the index. Defaults to no debug.
RestartInd Optional Indicates if the defragmentation is to restart at the next table index or start over.
Outputs None
Logic:
Validates all input parameters.
Checks that there is a transction log backup job of the given name if the database is not in simple recovery mode.
Ranks indexes based on Out of Order page percentage.
Since when a clustered index is rebuilt, the table's non-clustered indexes are also rebuild, removes non-clustered indexes when the clustered index is expected to be rebuilt first.
For each table index in order by most fragmented and higher row count:
Runs the index defragmentation
Removes the specific index row from the Table TableIndexDefragmenting to allow restartability.
If the clustered index was rebuilt, removes all rows for the table from the Table TableIndexDefragmenting to allow restartability.
the database is not in simple recovery mode and the transaction log backup job is not running, starts the job
Future Enhancements:
Currently, the transaction log is backed up after each index is rebuilt but a check for the usage of the transaction log
and then only performing that backup when the log is filled over a current percentage may be usefull.
DBCC SQLPERF(logsize) provides the percent of log space used.
*/
set nocount on
--set xact_abort on
IF COALESCE ( IS_SRVROLEMEMBER(N'sysadmin'), 0) <> 1
BEGIN
raiserror('This procedure requires system adminstrator priveleges', 16, 1 )
return +1
END
if @RunMinutes is null
and @StopTime is null
BEGIN
raiserror('Either @RunMinutes or Stop Time must be provided', 16, 1 )
return +1
END
if @RunMinutes is not null
and @StopTime is not null
BEGIN
raiserror('@RunMinutes and @StopTime cannot both be provided', 16, 1 )
return +1
END
if @StopTime is not null
and @StopTime not like '[0-2][0-9]:[0-6][0-9]:[0-6][0-9]'
BEGIN
raiserror('@StopTime must in format HH:MM:SS with a leading zero', 16, 1 )
return +1
END
if @RunMinutes is not null
and @RunMinutes > ( 60 * 8 )
BEGIN
raiserror('@RunMinutes must be less than 8 hours', 16, 1 )
return +1
END
IF @DefragOption is null
BEGIN
raiserror('@DefragOption must be provided', 16, 1 )
return +1
END
IF @DefragOption not in ('DBREINDEX','INDEXDEFRAG' )
BEGIN
raiserror('@DefragOption must be DBREINDEX or INDEXDEFRAG', 16, 1 )
return +1
END
IF @RestartInd not in ('Y','N')
BEGIN
raiserror('@RestartInd must be Y or N', 16, 1 )
return +1
END
Declare @RecoveryMode varchar(255)
set @RecoveryMode = CAST( databasepropertyex(db_name() , 'recovery') as varchar(255) )
IF @RecoveryMode != 'simple'
BEGIN
IF not exists
(SELECT 1
from msdb.dbo.sysjobs
where name = @LogBackupJobName
 
BEGIN
raiserror('@LogBackupJobName provided does not exist.', 16, 1 )
return +1
END
END
IF object_id('TableIndexFragementation') is null
BEGIN
raiserror('Stored Procedure DBM_Index_get_FragmentionStats must be run first.', 16, 1 )
return +1
END
if object_id('tempdb..#JobStatus') is not null
drop table #JobStatus
CREATE TABLE #JobStatus
( job_id uniqueidentifier not null
, last_run_date integer not null
, last_run_time integer not null
, next_run_date integer not null
, next_run_time integer not null
, next_run_schedule_id integer not null
, requested_to_run integer not null
, request_source integer not null
, request_source_id sysname COLLATE database_default NULL
, running integer not null
, current_step integer not null
, current_retry_attempt integer not null
, job_state integer not null
)
if object_id('TableIndexDefragment_JobStatus') is not null
drop table TableIndexDefragment_JobStatus
if @DisplayProgressInd = 'Y'
select current_timestamp as JobStatusTs, *
into TableIndexDefragment_JobStatus
from #JobStatus
DECLARE @is_sysadmin integer
, @job_owner sysname
set @is_sysadmin = ISNULL(IS_SRVROLEMEMBER(N'sysadmin'), 0)
set @job_owner = SUSER_SNAME()
declare @Job_id uniqueidentifier
select @Job_id = job_id
from msdb.dbo.sysjobs
where name = @LogBackupJobName
DECLARE @StopTs datetime
if @RunMinutes is not null
BEGIN
set @StopTs = dateadd(mi, @RunMinutes , current_timestamp)
END
ELSE
BEGIN
-- Uses feature of <datetime> + integer , where the integer is days.
-- For example, to add one hour, convert to days my dividing the number of hours in a day
set @StopTs
= cast( CAST( current_timestamp - .5 as integer) as datetime )
+ ( cast( substring(@StopTime,1,2) as tinyint ) / ( 24.0 ) )
+ ( cast( substring(@StopTime,4,2) as tinyint ) / ( 24.0 * 60 ) )
+ ( cast( substring(@StopTime,7,2) as tinyint ) / ( 24.0 * 60 * 60 ) )
IF @StopTs <= current_timestamp
set @StopTs = @StopTs + 1
END
IF @DisplayProgressInd = 'Y'
BEGIN
print cast( @StopTs as varchar) + ' is stopping time'
print cast(CURRENT_TIMESTAMP as varchar) + ' is CURRENT time'
END
IF @RestartInd = 'N'
BEGIN
IF object_id('TableIndexDefragmenting') is NOT null
drop table TableIndexDefragmenting
IF object_id('TableIndexDefragmented') is NOT null
drop table TableIndexDefragmented
if object_id('tempdb..#IndexDefragOptions') is not null
drop table #IndexDefragOptions
Create table #IndexDefragOptions
( Priority tinyint
, OutOfOrderPagePct_low decimal(18) not null unique
, OutOfOrderPagePct_high decimal(18)
-- Note: named constraints not allowed on temporary tables
--, constraint indexDefragOptions_C_LH check ( outOfOrderPagePct_low < OutOfOrderPagePct_high )
, check ( outOfOrderPagePct_low < OutOfOrderPagePct_high )
 
insert into #IndexDefragOptions
(OutOfOrderPagePct_low)
SELECT 50 union all
SELECT 30 union all
SELECT 40 union all
SELECT 20 union all
SELECT 10 union all
SELECT 0
update #IndexDefragOptions
set Priority =
(select count(*)
from #IndexDefragOptions as HigherPrioritys
where HigherPrioritys.OutOfOrderPagePct_low >= #IndexDefragOptions.OutOfOrderPagePct_low
 
, OutOfOrderPagePct_high =
COALESCE(
(SELECT MIN(OutOfOrderPagePct_low)
from #IndexDefragOptions as L
where L.OutOfOrderPagePct_low > #IndexDefragOptions.OutOfOrderPagePct_low
 
, 101 ) - 1IF @DebugInd = 'Y'
select * from #IndexDefragOptions
-- Create table of index rebuild completed (where 1 = 2 insures that there are no rows)
select current_timestamp as IndexRebuildTs
, *
into TableIndexDefragmented
from #IndexDefragOptions
, TableIndexFragementation
WHERE 1 = 2
-- Create table of index rebuild to be performed
SELECT *
into TableIndexDefragmenting
from #IndexDefragOptions
join TableIndexFragementation
on TableIndexFragementation.OutOfOrderPagePct
between OutOfOrderPagePct_low
and OutOfOrderPagePct_high
where IndexInd between 1 and 254 -- 0 is heap and 255 is lob
and TableIndexFragementation.OutOfOrderPagePct > 0
create unique clustered index TableIndexDefragmenting_P
on TableIndexDefragmenting
(TableId, IndexInd)
-- Remove all non-clustered indicies where:
-- the clustered index is in a higher, equal or next lower fragmention priority
delete from TableIndexDefragmenting
where IndexInd between 2 and 255
and exists
(SELECT *
from TableIndexDefragmenting as ClusteredInd
where ClusteredInd.TableId = TableIndexDefragmenting.TableId
and ClusteredInd.Priority >= TableIndexDefragmenting.Priority - 1
and ClusteredInd.IndexInd = 1
 
END
IF @DisplayProgressInd = 'Y'
BEGIN
SELECT TableName
, IndexName
, TableId
, IndexInd
from TableIndexDefragmenting
WHERE PageCnt > 2
order by Priority ASC, rowCnt DESC
END
declare @TableName sysname
, @IndexName sysname
, @TableId integer
, @IndexInd integer
, @DBReindexTemplate nvarchar(4000)
, @DBReindexSQL nvarchar(4000)
, @rc_sp_start_job INTEGER
set @DBReindexTemplate = 'DBCC &DefragOption ( &tableName , &index_name ) WITH NO_INFOMSGS'
set @DBReindexTemplate
= REPLACE(@DBReindexTemplate , '&DefragOption' , @DefragOption)
WHILE @StopTs > current_timestamp
BEGIN
set rowcount 1
SELECT @TableName = TableName
, @IndexName = IndexName
, @TableId = TableId
, @IndexInd = IndexInd
from TableIndexDefragmenting
WHERE PageCnt > 2
order by Priority ASC, rowCnt DESC
IF @@ROWCOUNT = 0 BREAK
set rowcount 0
set @DBReindexSQL
= REPLACE(@DBReindexTemplate,'&tableName',@TableName)
set @DBReindexSQL
= REPLACE(@DBReindexSQL,'&index_name',@IndexName)
IF @DisplayProgressInd = 'Y'
BEGIN
PRINT 'PROCESSING'
PRINT '@TableName ' + @TableName
PRINT '@IndexName ' + @IndexName
PRINT ' '
END
if @DebugInd = 'Y'
BEGIN
SELECT @DBReindexSQL
END
ELSE
BEGIN
exec ( @DBReindexSQL )
END
IF @DisplayProgressInd = 'Y'
BEGIN
print 'insert into TableIndexDefragmented'
END
insert into TableIndexDefragmented
select current_timestamp, *
from TableIndexDefragmenting
where TableId = @TableId
and ( IndexInd = @IndexInd
or @IndexInd = 1
 
-- remove the specific index OR
-- remove non-clustered indexes after the clustered index is rebuilt
IF @DisplayProgressInd = 'Y'
BEGIN
print 'deleting from table TableIndexDefragmenting'
END
delete from TableIndexDefragmenting
where TableId = @TableId
and ( IndexInd = @IndexInd
or @IndexInd = 1
 
IF @RecoveryMode != 'simple'
BEGIN
IF @DisplayProgressInd = 'Y'
BEGIN
print 'Getting job status'
END
insert into #JobStatus
EXECUTE master.dbo.xp_sqlagent_enum_jobs @is_sysadmin, @job_owner, @job_id
IF NOT EXISTS
(select 1
from #JobStatus
where job_state <> 4
 
BEGIN
IF @DisplayProgressInd = 'Y'
BEGIN
print 'Starting transaction log backup job'
END
exec @rc_sp_start_job = msdb.dbo.sp_start_job
@job_name = @LogBackupJobName
, @output_flag = 0
if @rc_sp_start_job <> 0
BEGIN
if @DebugInd = 'Y'
insert into TableIndexDefragment_JobStatus
select current_timestamp, *
from #JobStatus
IF @DisplayProgressInd = 'Y'
BEGIN
print 'FAILED - Starting transaction log backup job'
END
END
else
BEGIN
IF @DisplayProgressInd = 'Y'
BEGIN
print 'Successfull - Starting transaction log backup job'
END
-- Wait for the job to start
waitfor delay '00:00:05'
end
END
END
END
set rowcount 0
GO
CREATE procedure DBM_Index_get_FragmentionStats
as
/*
Description: Runs "DBCC Showcontig" for each table in the database
Privileges: DB Owner or System Adminstrator
Run Time Versus LCC preprod database, 7 minutes
Locking: No
Dependencies: None
Install: Into each database
Inputs None
Outputs table TableIndexFragementation is populated
Problems: None
Future Enhancements:
None needed.
*/
SET NOCOUNT ON
SET Xact_Abort ON
IF EXISTS
(SELECT 1
FROM INFORMATION_SCHEMA.TABLES
WHERE table_name = 'TableIndexFragementation'
AND TABLE_TYPE = 'BASE TABLE'
 
BEGIN
truncate table dbo.TableIndexFragementation
END
ELSE
BEGIN
CREATE TABLE dbo.TableIndexFragementation (
TableName varchar (255) NULL ,
TableId int NULL ,
IndexName varchar (255) NULL ,
IndexInd int NULL ,
IndexLevel int NULL ,
PageCnt int NULL ,
RowCnt int NULL ,
MinRecSize int NULL ,
MaxRecSize int NULL ,
AvgRecSize int NULL ,
ForwardedRecordCnt int NULL ,
ExtentCnt int NULL ,
ExtentSwitchCnt int NULL ,
AvgFreeBytes int NULL ,
AvgPageFullPct int NULL ,
ScanDensity decimal(18, 0) NULL ,
ExtentIdealCnt int NULL ,
ExtentActualCnt int NULL ,
OutOfOrderPagePct decimal(18, 0) NULL ,
OutOfOrderExtentPct decimal(18, 0) NULL
 
END
DECLARE @tablename VARCHAR (128)
, @execstr VARCHAR (255)
, @objectid INT
, @indexid INT
, @frag DECIMAL
, @maxfrag DECIMAL
, @TAbleCnt integer
-- Decide on the maximum fragmentation to allow
SET @maxfrag = 30.0
SET @TAbleCnt = 0
-- Declare cursor
DECLARE tables_CSR CURSOR FOR
SELECT TABLE_NAME
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE = 'BASE TABLE'
IF object_id('tempdb..#fraglist') is not null
drop table #fraglist
-- Create the table
CREATE TABLE #fraglist
( TableName varchar (255)
, TableId integer
, IndexName varchar (255)
, IndexInd integer
, IndexLevel integer
, PageCnt integer
, RowCnt integer
, MinRecSize integer
, MaxRecSize integer
, AvgRecSize integer
, ForwardedRecordCnt integer
, ExtentCnt integer
, ExtentSwitchCnt integer
, AvgFreeBytes integer
, AvgPageFullPct integer
, ScanDensity decimal
, ExtentIdealCnt integer
, ExtentActualCnt integer
, OutOfOrderPagePct decimal
, OutOfOrderExtentPct decimal
 
-- Open the cursor
OPEN tables_CSR
WHILE 1 = 1
BEGIN
FETCH NEXT FROM tables_CSR
INTO @tablename
IF @@FETCH_STATUS != 0 BREAK
set @TAbleCnt = @TAbleCnt + 1
-- select 'processing ' , @tablename , @TAbleCnt
-- Get Table Index physical information including fragementation
INSERT INTO #fraglist
EXEC ('DBCC SHOWCONTIG (''' + @tablename + ''') WITH TABLERESULTS, ALL_INDEXES, NO_INFOMSGS')
END
-- Close and deallocate the cursor
CLOSE tables_CSR
DEALLOCATE tables_CSR
Insert into dbo.TableIndexFragementation
(
TableName, TableId, IndexName, IndexInd, IndexLevel
, PageCnt, RowCnt
, MinRecSize, MaxRecSize, AvgRecSize
, ForwardedRecordCnt, ExtentCnt
, ExtentSwitchCnt, AvgFreeBytes, AvgPageFullPct, ScanDensity
, ExtentIdealCnt, ExtentActualCnt
, OutOfOrderPagePct, OutOfOrderExtentPct
 
select TableName, TableId, IndexName, IndexInd
, IndexLevel, PageCnt, RowCnt, MinRecSize, MaxRecSize, AvgRecSize
, ForwardedRecordCnt, ExtentCnt, ExtentSwitchCnt
, AvgFreeBytes, AvgPageFullPct, ScanDensity, ExtentIdealCnt, ExtentActualCnt
, OutOfOrderPagePct, OutOfOrderExtentPct
from #fraglist
GO
SQL = Scarcely Qualifies as a Language
-
But why do u do these kind of job in production hours plan these activities in non-production hous or time when there is less activity in sql server.
Cheers,
Sugeshkumar Rajendran
SQL Server MVP
http://sugeshkr.blogspot.com -
Hi Carl. Can you cite any supporting documentation?
I know that rebuilding a clustered index will cause the non-clustered indexes to be rebuilt, according to the BOL, but I don't see anything about that under INDEXDEFRAG.
Thanks,
JG
-
Actually, I stand corrected. There is a MS blog on the SQL Storage Engine that has some insight and part of the blog is posted below. Looks like the logic of my defrag routine needs to change.
- Rebuilding a unique clustered index: This operation has no effect on the cluster key, so there is no need for the non-clustered indexes to be rebuilt. Various early Service Packs of SQL Server 2000 had bugs that caused this behavior to be broken - and this is the cause of much of the confusion around this behavior.
- Rebuilding a non-unique clustered index: In SQL Server 2000, when a non-unique clustered index (which contains uniquifier columns) is rebuilt, all the uniquifier values are regenerated. This essentially means that the cluster keys have changed and so all the non-clustered indexes are rebuilt after the clustered index rebuild has completed.
SQL = Scarcely Qualifies as a Language
-
Note:
DEPENDING on the *strength* of your IO subsystem you may still recive timeouts eventhough defrag is an online operation still requires a transaction log space.
The best bet is as it was suggested above do it a off peak hours.
* Noel -
One thing we do is run the defrags outside peak hours and limit the run time. So it builds a list of fragmented indexes, as per your job, then defrags them in order of fragmentation then exits from the job after it has been running for more than 45 minutes.
We then monitor the fragmentation levels overtime to determine wether fragmentation is being managed trend wise...
-
I like the idea of changing the loop so that you don't process two indexes against the same table. No guarantee that it will help, but I think its worth trying. One easy way would be to order by newid() to get random order (which wouldn't necessarily guarantee you don't hit the same table twice in a row, but it is easy to implement).
You could also try adding a WAITFOR inside the loop, give the server a break of x seconds after every index rebuild so that if you have developed a disk queue, maybe it will sort things out.
I suppose you could also find a wait to run the defrag for about 25 seconds and then stop, then try again, and again.
All of those are bandaids not sure to help much.
-
I don't think we have any off peak hours. We have some low usage hours, and this is when I scheduled to run INDEXDEFRAG.
I have noted all the suggestions above. For now, I implemented the following 2:
1. ORDER BY INDEXID - This, I believe, processes 1 index for each table at a time
2. WAITFOR DELAY 1 SECOND - This, I believe, gives some breathing time to the SQL server
The jobs are scheduled to run tonight. I'll see what happens tomorrow.
I also plan to use PERFMON and PROFILER to do more detailed exploration ...
Thanks for all the suggestions!
-
Here is another, if you do defrag don't forget to update statistics when you are done
* Noel -
Thanks Noel. I usually do that as well!
Viewing 12 posts - 1 through 11 (of 11 total)
You must be logged in to reply to this topic. Login to reply