| Author |
Topic |
|
mike123
Master Smack Fu Yak Hacker
1462 Posts |
Posted - 2007-05-02 : 23:20:41
|
Hi, credits to great members at SQLTEAM ;) I have the following delete statement that I am looking to automate a bit better.This statement works perfectly, but my goal is to put it into a nightly task that runs on itself. The problem is I don't know exactly how many loops to run it for. I can set the loop count really high, and let it loop thru deletes of 0 rows, but I'd like to do a better solution.I think it would be much better to determine with a count of how many records we want to delete first, then loop thru an appropriate amount of loops. Any thoughts on the best way to do this? I'm thinking I'll probably end up with something like 50-100 loops daily of 500-600 deletes. This is a number I have found that doesnt slow the DB server down.How can I incorporate the count of the total number rows we want to delete into the amount of loops to do ? This just doesnt have to be EXACT, i just don't want to loop thru multiple loops of deleting 0 rows.(Here is code for determining how many rows are to be deleted: )SELECT count(*)FROM tblinstantmessage WHERE (DateDiff(dd, date, GetDate()) > 60 AND checked = 1 )OR (deletedbysender =1 and deletedbyrecipient = 1 AND DateDiff(dd, date, GetDate()) > 14) Thanks very much!mike123DECLARE @intRowCount bigint,@intBatchSize bigint,@intRowTotal bigint,@dtStart datetime,@intElapsedS bigint,@intTotalMax bigintSELECT @intRowCount = 1, -- Force first loop iteration@intBatchSize = 500, -- Records to delete in each batch@intRowTotal = 0,@intTotalMax = 0WHILE @intRowCount >= 1 or @intTotalMax <= 50BEGINSET ROWCOUNT @intBatchSize -- Restrict ALL subsequent operations to batch sizeBEGIN TRANSACTIONSELECT @dtStart = GetDate() SELECT TOP 600 instantMessageIDINTO #TEMP_IMFROM tblinstantmessage WHERE (DateDiff(dd, date, GetDate()) > 60 AND checked = 1 )OR deletedbysender =1 and deletedbyrecipient = 1 AND DateDiff(dd, date, GetDate()) > 14DELETE DFROM #TEMP_IM AS T JOIN tblinstantmessage AS D ON D.instantMessageID = T.instantMessageID-- WHERE deletedbysender =1 and deletedbyrecipient =1 -- Belt and braces!drop table #TEMP_IMSELECT @intRowCount = @@ROWCOUNT, -- Rows actually deleted@intRowTotal = @intRowTotal + @intRowCount -- Total rowsCOMMITSET ROWCOUNT 0 -- Remove the restrictionSELECT @intElapsedS = DATEDIFF(Second, @dtStart, GetDate())IF @intElapsedS > 120BEGINSELECT @intBatchSize = @intBatchSize / 2 -- Reduce batch sizeENDELSEIF @intElapsedS < 30BEGINSELECT @intBatchSize = @intBatchSize + (@intBatchSize / 4) -- Increase batch sizeENDSELECT @intTotalMax = @intTotalMax + 1-- ... Optionally backup TLog if @intRowTotal > 1,000,000 and reset @intRowTotal to 0 ...WAITFOR DELAY '000:00:06' -- Allow other queries to runEND |
|
|
DonAtWork
Master Smack Fu Yak Hacker
2167 Posts |
Posted - 2007-05-03 : 08:29:26
|
Why tell it how many loops at all?declare @LeftToDelete intselect @LeftToDelete = count(*)FROM tblinstantmessage WHERE (DateDiff(dd, date, GetDate()) > 60 AND checked = 1 )OR deletedbysender =1 and deletedbyrecipient = 1 AND DateDiff(dd, date, GetDate()) > 14while @LeftToDelete > 0BEGIN-- Start a transaction-- delete the top 600-- commit the transaction-- get a new countEND Or am I missing some step in the logic here? I still think like a programmer and not set based.[Signature]For fast help, follow this link:http://weblogs.sqlteam.com/brettk/archive/2005/05/25.aspxLearn SQLhttp://www.sql-tutorial.net/ http://www.firstsql.com/tutor.htm http://www.w3schools.com/sql/default.asp |
 |
|
|
nr
SQLTeam MVY
12543 Posts |
Posted - 2007-05-03 : 08:36:53
|
| can dowhile @@rowcount > 0begin delete ...endordelete ....while @@rowcount = @batchsizebegin delete ...end==========================================Cursors are useful if you don't know sql.DTS can be used in a similar way.Beer is not cold and it isn't fizzy. |
 |
|
|
SwePeso
Patron Saint of Lost Yaks
30421 Posts |
Posted - 2007-05-03 : 08:40:20
|
With SQL Server 2005, you can use the TOP keyword DELETE TOP (600)FROM Table1WHERE Date >= DATEADD(DAY, -60, CURRENT_TIMESTAMP) AND checked = 1 OR DeletedBySender = 1 AND DeletedByRecipient = 1 AND DATEADD(DAY, -14, CURRENT_TIMESTAMP) Peter LarssonHelsingborg, Sweden |
 |
|
|
SwePeso
Patron Saint of Lost Yaks
30421 Posts |
Posted - 2007-05-03 : 08:41:24
|
| [code]DECLARE @Dummy INTSELECT @Dummy = 1WHILE @@ROWCOUNT > 0 DELETE TOP (600) FROM Table1 WHERE Date >= DATEADD(DAY, -60, CURRENT_TIMESTAMP) AND checked = 1 OR DeletedBySender = 1 AND DeletedByRecipient = 1 AND DATEADD(DAY, -14, CURRENT_TIMESTAMP)[/code]Peter LarssonHelsingborg, Sweden |
 |
|
|
jshepler
Yak Posting Veteran
60 Posts |
Posted - 2007-05-03 : 08:45:29
|
You could keep looping until rowcount = 0. Something likeWHILE 1=1BEGIN -- do delete IF @@ROWCOUNT = 0 BREAK -- process new batch size and do delayEND No need to try and figure out how many iterations you need. You do, at most, 1 iteration with 0 deletes./jeff |
 |
|
|
harsh_athalye
Master Smack Fu Yak Hacker
5581 Posts |
Posted - 2007-05-03 : 08:47:20
|
quote: Originally posted by jshepler You could keep looping until rowcount = 0. Something likeWHILE 1=1BEGIN -- do delete IF @@ROWCOUNT = 0 BREAK -- process new batch size and do delayEND No need to try and figure out how many iterations you need. You do, at most, 1 iteration with 0 deletes./jeff
I am sure Brett would love to see that.... Brett, Where are you??Harsh AthalyeIndia."The IMPOSSIBLE is often UNTRIED" |
 |
|
|
SwePeso
Patron Saint of Lost Yaks
30421 Posts |
Posted - 2007-05-03 : 08:47:39
|
| Harsh, if you try that, you will see that @@ROWCOUNT is equal to Zero on first check and thus never looping.The "SELECT @Dummy" ensures @@ROWCOUNT to be equal to 1 on first check.Peter LarssonHelsingborg, Sweden |
 |
|
|
harsh_athalye
Master Smack Fu Yak Hacker
5581 Posts |
Posted - 2007-05-03 : 08:51:14
|
| Oh..I see.I thought it was just a dead variable sitting there. Missed the @@rowcount.Harsh AthalyeIndia."The IMPOSSIBLE is often UNTRIED" |
 |
|
|
jshepler
Yak Posting Veteran
60 Posts |
Posted - 2007-05-03 : 08:55:08
|
| What's wrong with "WHILE 1=1"?/jeff |
 |
|
|
spirit1
Cybernetic Yak Master
11752 Posts |
Posted - 2007-05-03 : 08:57:54
|
| i'd go with while 1=1 too... maybe add a variable for batch size_______________________________________________Causing trouble since 1980blog: http://weblogs.sqlteam.com/mladenp |
 |
|
|
harsh_athalye
Master Smack Fu Yak Hacker
5581 Posts |
Posted - 2007-05-03 : 08:58:21
|
| jshepler,There is nothing wrong with your code. But recently there was discussion about how silly or how useful it was to use 1=1 as a condition in while loop and Brett was against using it.Harsh AthalyeIndia."The IMPOSSIBLE is often UNTRIED" |
 |
|
|
jshepler
Yak Posting Veteran
60 Posts |
Posted - 2007-05-03 : 09:12:09
|
| Ah, ok... I thought there was something I didn't know that made using something like that a "bad thing"./jeff |
 |
|
|
|