Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2005 Forums
 Transact-SQL (2005)
 help with query (mass deletes)

Author  Topic 

mike123
Master Smack Fu Yak Hacker

1462 Posts

Posted - 2007-05-02 : 23:20:41

Hi, credits to great members at SQLTEAM ;) I have the following delete statement that I am looking to automate a bit better.

This statement works perfectly, but my goal is to put it into a nightly task that runs on itself. The problem is I don't know exactly how many loops to run it for. I can set the loop count really high, and let it loop thru deletes of 0 rows, but I'd like to do a better solution.

I think it would be much better to determine with a count of how many records we want to delete first, then loop thru an appropriate amount of loops.

Any thoughts on the best way to do this? I'm thinking I'll probably end up with something like 50-100 loops daily of 500-600 deletes. This is a number I have found that doesnt slow the DB server down.

How can I incorporate the count of the total number rows we want to delete into the amount of loops to do ? This just doesnt have to be EXACT, i just don't want to loop thru multiple loops of deleting 0 rows.

(Here is code for determining how many rows are to be deleted: )


SELECT count(*)
FROM tblinstantmessage
WHERE (DateDiff(dd, date, GetDate()) > 60 AND checked = 1 )
OR (deletedbysender =1 and deletedbyrecipient = 1 AND DateDiff(dd, date, GetDate()) > 14)



Thanks very much!
mike123




DECLARE @intRowCount bigint,
@intBatchSize bigint,
@intRowTotal bigint,
@dtStart datetime,
@intElapsedS bigint,
@intTotalMax bigint

SELECT @intRowCount = 1, -- Force first loop iteration
@intBatchSize = 500, -- Records to delete in each batch
@intRowTotal = 0,
@intTotalMax = 0

WHILE @intRowCount >= 1 or @intTotalMax <= 50
BEGIN

SET ROWCOUNT @intBatchSize -- Restrict ALL subsequent operations to batch size

BEGIN TRANSACTION
SELECT @dtStart = GetDate()


SELECT TOP 600 instantMessageID
INTO #TEMP_IM
FROM tblinstantmessage

WHERE (DateDiff(dd, date, GetDate()) > 60 AND checked = 1 )

OR deletedbysender =1 and deletedbyrecipient = 1 AND DateDiff(dd, date, GetDate()) > 14


DELETE D
FROM #TEMP_IM AS T
JOIN tblinstantmessage AS D
ON D.instantMessageID = T.instantMessageID
-- WHERE deletedbysender =1 and deletedbyrecipient =1 -- Belt and braces!

drop table #TEMP_IM

SELECT @intRowCount = @@ROWCOUNT, -- Rows actually deleted
@intRowTotal = @intRowTotal + @intRowCount -- Total rows
COMMIT
SET ROWCOUNT 0 -- Remove the restriction
SELECT @intElapsedS = DATEDIFF(Second, @dtStart, GetDate())
IF @intElapsedS > 120
BEGIN
SELECT @intBatchSize = @intBatchSize / 2 -- Reduce batch size
END
ELSE
IF @intElapsedS < 30
BEGIN
SELECT @intBatchSize = @intBatchSize + (@intBatchSize / 4) -- Increase batch size
END

SELECT @intTotalMax = @intTotalMax + 1

-- ... Optionally backup TLog if @intRowTotal > 1,000,000 and reset @intRowTotal to 0 ...
WAITFOR DELAY '000:00:06' -- Allow other queries to run
END

DonAtWork
Master Smack Fu Yak Hacker

2167 Posts

Posted - 2007-05-03 : 08:29:26
Why tell it how many loops at all?


declare @LeftToDelete int

select @LeftToDelete =
count(*)
FROM tblinstantmessage

WHERE (DateDiff(dd, date, GetDate()) > 60 AND checked = 1 )

OR deletedbysender =1 and deletedbyrecipient = 1 AND DateDiff(dd, date, GetDate()) > 14

while @LeftToDelete > 0

BEGIN

-- Start a transaction
-- delete the top 600
-- commit the transaction
-- get a new count

END


Or am I missing some step in the logic here? I still think like a programmer and not set based.




[Signature]For fast help, follow this link:
http://weblogs.sqlteam.com/brettk/archive/2005/05/25.aspx
Learn SQL
http://www.sql-tutorial.net/
http://www.firstsql.com/tutor.htm
http://www.w3schools.com/sql/default.asp
Go to Top of Page

nr
SQLTeam MVY

12543 Posts

Posted - 2007-05-03 : 08:36:53
can do

while @@rowcount > 0
begin
delete ...
end

or

delete ....
while @@rowcount = @batchsize
begin
delete ...
end



==========================================
Cursors are useful if you don't know sql.
DTS can be used in a similar way.
Beer is not cold and it isn't fizzy.
Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2007-05-03 : 08:40:20
With SQL Server 2005, you can use the TOP keyword

DELETE TOP (600)
FROM Table1
WHERE Date >= DATEADD(DAY, -60, CURRENT_TIMESTAMP) AND checked = 1
OR DeletedBySender = 1 AND DeletedByRecipient = 1 AND DATEADD(DAY, -14, CURRENT_TIMESTAMP)


Peter Larsson
Helsingborg, Sweden
Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2007-05-03 : 08:41:24
[code]DECLARE @Dummy INT

SELECT @Dummy = 1

WHILE @@ROWCOUNT > 0
DELETE TOP (600)
FROM Table1
WHERE Date >= DATEADD(DAY, -60, CURRENT_TIMESTAMP) AND checked = 1
OR DeletedBySender = 1 AND DeletedByRecipient = 1 AND DATEADD(DAY, -14, CURRENT_TIMESTAMP)[/code]

Peter Larsson
Helsingborg, Sweden
Go to Top of Page

jshepler
Yak Posting Veteran

60 Posts

Posted - 2007-05-03 : 08:45:29
You could keep looping until rowcount = 0. Something like

WHILE 1=1
BEGIN
-- do delete
IF @@ROWCOUNT = 0 BREAK
-- process new batch size and do delay
END

No need to try and figure out how many iterations you need. You do, at most, 1 iteration with 0 deletes.


/jeff
Go to Top of Page

harsh_athalye
Master Smack Fu Yak Hacker

5581 Posts

Posted - 2007-05-03 : 08:47:20
quote:
Originally posted by jshepler

You could keep looping until rowcount = 0. Something like

WHILE 1=1
BEGIN
-- do delete
IF @@ROWCOUNT = 0 BREAK
-- process new batch size and do delay
END

No need to try and figure out how many iterations you need. You do, at most, 1 iteration with 0 deletes.


/jeff




I am sure Brett would love to see that....

Brett, Where are you??

Harsh Athalye
India.
"The IMPOSSIBLE is often UNTRIED"
Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2007-05-03 : 08:47:39
Harsh, if you try that, you will see that @@ROWCOUNT is equal to Zero on first check and thus never looping.
The "SELECT @Dummy" ensures @@ROWCOUNT to be equal to 1 on first check.


Peter Larsson
Helsingborg, Sweden
Go to Top of Page

harsh_athalye
Master Smack Fu Yak Hacker

5581 Posts

Posted - 2007-05-03 : 08:51:14
Oh..I see.

I thought it was just a dead variable sitting there. Missed the @@rowcount.

Harsh Athalye
India.
"The IMPOSSIBLE is often UNTRIED"
Go to Top of Page

jshepler
Yak Posting Veteran

60 Posts

Posted - 2007-05-03 : 08:55:08
What's wrong with "WHILE 1=1"?

/jeff
Go to Top of Page

spirit1
Cybernetic Yak Master

11752 Posts

Posted - 2007-05-03 : 08:57:54
i'd go with while 1=1 too...
maybe add a variable for batch size

_______________________________________________
Causing trouble since 1980
blog: http://weblogs.sqlteam.com/mladenp
Go to Top of Page

harsh_athalye
Master Smack Fu Yak Hacker

5581 Posts

Posted - 2007-05-03 : 08:58:21
jshepler,

There is nothing wrong with your code. But recently there was discussion about how silly or how useful it was to use 1=1 as a condition in while loop and Brett was against using it.


Harsh Athalye
India.
"The IMPOSSIBLE is often UNTRIED"
Go to Top of Page

jshepler
Yak Posting Veteran

60 Posts

Posted - 2007-05-03 : 09:12:09
Ah, ok... I thought there was something I didn't know that made using something like that a "bad thing".


/jeff
Go to Top of Page
   

- Advertisement -