| Author |
Topic |
|
Transact Charlie
Master Smack Fu Yak Hacker
3451 Posts |
Posted - 2009-02-11 : 05:53:10
|
Hi all,My company is currently going through a round of Load testing where the front end is hammered by Load Runner to simulate a lot of concurrent users.As part of this I've been running SQL profiler to trace the transactions. One object sticks out a mile -- it required a ton of plan recompilations and seemed to hardly ever reused a cached plan.Management have decided that they don't want us to rewrite the stored proc totally -- it needs to present the same data in the same way to go with the business rules - don't want to change the basic way the logic works. They do want us to make things faster however :(There's a lot of IF statements that then do a block of SQL -- the If's are triggered by paramaters passed into the proc. I'm guessing that the fact that different SQL is executed depending on passed params is causing the cache plan recompile.Would it make a difference to move the sql into another stored prc and then call that stored proc from inside the IF statemens?:EXAMPLEIF( @value IS NULL ) BEGIN INSERT INTO #getEffectiveInitiatedValueTEMP SELECT TOP 1 ae.id, aer.newValue , aer.oldValue, ae.initiatedDate FROM #tempApprovalEventRow aer, #tempApprovalEvent ae WHERE ae.id = aer.eventId AND DATEDIFF(dd, @significantEffectiveDate, ae.effectiveDate ) > 0 AND ( (@repriceApprovalEventId IS NULL ) OR (@repriceApprovalEventId <= 0 ) OR ( @repriceApprovalEventId > ae.id ) ) AND DATEDIFF( dd, ae.effectiveDate, ae.initiatedDate ) >= 0 AND aer.tableName = @tableName AND aer.columnName = @columnName AND aer.idValue = @idValue AND ae.status <> 'D' ORDER BY ae.initiatedDate ASC, ae.id ASC INSERT INTO #getEffectiveInitiatedValueTEMP SELECT TOP 1 ae.id, aer.newValue, aer.oldValue, ae.effectiveDate FROM #tempApprovalEventRow aer, #tempApprovalEvent ae WHERE ae.id = aer.eventId AND DATEDIFF(dd, @significantEffectiveDate, ae.effectiveDate ) > 0 AND ( (@repriceApprovalEventId IS NULL ) OR (@repriceApprovalEventId <= 0 ) OR ( @repriceApprovalEventId > ae.id ) ) AND DATEDIFF( dd, ae.initiatedDate, ae.effectiveDate ) >= 0 AND aer.tableName = @tableName AND aer.columnName = @columnName AND aer.idValue = @idValue AND ae.status <> 'D' ORDER BY ae.effectiveDate ASC, ae.id ASC SELECT TOP 1 @value = oldValue FROM #getEffectiveInitiatedValueTEMP ORDER BY significantDate ASC, aeId ASC END Move all that SQL into another stored proc that just did that work. The if statement would become:IF( @value IS NULL ) BEGIN EXEC new_StoredProc @value OUTEND Will that improve matters as the child stored proc would always have a valid plan in cache to do the work?I've got some indexes to add as well but I'm comfortable with that.charlie.Charlie===============================================================Msg 3903, Level 16, State 1, Line 1736The ROLLBACK TRANSACTION request has no corresponding BEGIN TRANSACTION |
|
|
GilaMonster
Master Smack Fu Yak Hacker
4507 Posts |
Posted - 2009-02-11 : 09:12:50
|
| What you're probably seeing is a combination of object change recompiles (caused by the creation of a temp table) and statistics change recompiles (caused by adding data to the temp tables)Creating or referencing a previously created temp table is near certain to cause a recompile, and temp tables are more likely to trigger statistics-based recompiles.An if statement will never cause a recompile by itself, rather it causes parameter sniffing due to poor plan selection.--Gail ShawSQL Server MVP |
 |
|
|
sodeep
Master Smack Fu Yak Hacker
7174 Posts |
Posted - 2009-02-11 : 09:18:05
|
| You could use KEEPPLAN option to minimize recompiles for Temp Table and declare with local variables to reduce parameter sniffing. |
 |
|
|
Transact Charlie
Master Smack Fu Yak Hacker
3451 Posts |
Posted - 2009-02-11 : 10:19:47
|
| Thanks to Both of you. Very interesting.The temp tables referenced are kind of like a cache == I realised that multiple similar operations were being performed (could be up to 30) in a kind of cascade in this sp -- effectively if a value wasn't found then another search was initiated. etc. until a value was found.That used to reference the whole base table (30 times) (which contains entries for every employee). However, whenever this sp is called it gets passed 1 employee.Id who's info is the only one we care about.I made a change so that:1) first of all it fetched all the relevant info possible into the temp table for that employee so that the sp could do searches only on a small dataset2) Checked that the worst case scenario didn't exist (no info relevant-- if so just return NULL immediately)3) do the queries only on the small dataset (temp table) (maybe around 50 - 300 rows max) instead of against the whole base table (could be many millions of rows)I didn't bother with any indices on the temp tables as I though a table scan would probably be more efficient in most cases on a data set that small. However I didn't realise that the temp table would cause cache recompiles.If the structure of the temp table is never going to change -- only the data in it, will I see a benefit with the query hint KEEP PLAN?If so, what's the best way to use the hint -- when you first call the proc? or some more selective way?Was the cache table even a good idea in the first place? it seems to speed things up drastically for single runs but the cache recompiles are killing the concurrent load test.Charlie===============================================================Msg 3903, Level 16, State 1, Line 1736The ROLLBACK TRANSACTION request has no corresponding BEGIN TRANSACTION |
 |
|
|
Transact Charlie
Master Smack Fu Yak Hacker
3451 Posts |
Posted - 2009-02-11 : 10:23:02
|
Would a processed key cache table be a better way rather than the temp table? the aim is to be able to sustain a 1000 concurrent instances.Stupid plan! -- if I'm going to do that I may as well just reference the base table direct I guess.Charlie===============================================================Msg 3903, Level 16, State 1, Line 1736The ROLLBACK TRANSACTION request has no corresponding BEGIN TRANSACTION |
 |
|
|
Transact Charlie
Master Smack Fu Yak Hacker
3451 Posts |
Posted - 2009-02-11 : 10:38:42
|
| Do table variables require recompiles?Charlie===============================================================Msg 3903, Level 16, State 1, Line 1736The ROLLBACK TRANSACTION request has no corresponding BEGIN TRANSACTION |
 |
|
|
sodeep
Master Smack Fu Yak Hacker
7174 Posts |
Posted - 2009-02-11 : 14:08:11
|
| I would definitely use KEEP PLAN to reduce recompiles in this scenerio? Also take advantages of index in Temp table for faster retrieval. |
 |
|
|
sodeep
Master Smack Fu Yak Hacker
7174 Posts |
Posted - 2009-02-11 : 14:13:54
|
quote: Originally posted by Transact Charlie Do table variables require recompiles?
Normally I don't use Table variable when you are looking for more than 100 records? Infact it uses temp table after certain point and gives really bad performance when you have to deal with many records? Yes, Table Variable will reduce recompiles to some extent. |
 |
|
|
yosiasz
Master Smack Fu Yak Hacker
1635 Posts |
Posted - 2009-02-11 : 14:19:18
|
| I don't know about table variable but have you looked into common tables...I have definitely seen performance increase when using common tables instead of rable variables. I do not know how common tables will perform with such a huge load..2 cents |
 |
|
|
sodeep
Master Smack Fu Yak Hacker
7174 Posts |
Posted - 2009-02-11 : 14:34:51
|
quote: Originally posted by yosiasz I don't know about table variable but have you looked into common tables...I have definitely seen performance increase when using common tables instead of rable variables. I do not know how common tables will perform with such a huge load..2 cents
Are you referring Common table as TempTable or..? |
 |
|
|
yosiasz
Master Smack Fu Yak Hacker
1635 Posts |
Posted - 2009-02-11 : 15:10:51
|
| I mean CTE Common Table Expressions WITH CtempApprovalEventRow(Select * from X),WITH CtempApprovalEvent(Select * from X),With CgetEffectiveInitiatedValueTEMP()SELECT TOP 1 @value = oldValue FROM #getEffectiveInitiatedValueTEMP ORDER BY significantDate ASC, aeId ASCI was thinking something like this. Is that not possible? |
 |
|
|
GilaMonster
Master Smack Fu Yak Hacker
4507 Posts |
Posted - 2009-02-11 : 15:17:59
|
quote: Originally posted by Transact Charlie Do table variables require recompiles?
No, but they can cause some amusing performance problems with lots of rows.Why not reference the base table? Have you identified that doing slow is a performance problems and that querying a temp table is faster?yosiasz: A common table expression, despite its name, is just a temporary view and gives no performance benefits over querying the table direct. It's mainly for readability and easier queries, not faster ones.--Gail ShawSQL Server MVP |
 |
|
|
yosiasz
Master Smack Fu Yak Hacker
1635 Posts |
Posted - 2009-02-11 : 16:04:22
|
| Thanks for the insight Gail! |
 |
|
|
sodeep
Master Smack Fu Yak Hacker
7174 Posts |
Posted - 2009-02-11 : 16:33:12
|
| From my experience,I would say Table variable reduces Recompilation as statistics,parallel plan are not maintained in it. |
 |
|
|
yosiasz
Master Smack Fu Yak Hacker
1635 Posts |
Posted - 2009-02-11 : 17:41:07
|
| this is great thank you all. This thread helped me fix a certain problem with a query that was taking too long. |
 |
|
|
SwePeso
Patron Saint of Lost Yaks
30421 Posts |
Posted - 2009-02-11 : 18:15:11
|
The Query Optimizer always assumes there is one and only one record in a table variable because table variables does not have statistics.That is why Query Optimizer sometimes choosed an incorrect plan.My rule of thumb is that when the chance is high the table variable will hold more than one page of data, I change to temp table instead.One page is the minimum amount of data read at any given time by SQL Server. E 12°55'05.63"N 56°04'39.26" |
 |
|
|
Transact Charlie
Master Smack Fu Yak Hacker
3451 Posts |
Posted - 2009-02-12 : 05:02:29
|
| Thanks for all the tips:Gail: "Why not reference the base table? Have you identified that doing slow is a performance problems and that querying a temp table is faster?"The base table is an audit table for 2d time. It keeps track of *everything* for all employees. The way that the stored proc is written is that only a few rows in the base table are pertinent. The stored proc can make 15 separate selects on the base table each time it's called and it gets called a lot.Because the table is constantly getting written to / updated and (occasionally) deleted from then the indices seem to become out of date really quickly. This can mean that sometimes the original stored proc takes *minutes* to finish when it should under a second.I suggested that we should reindex the table more often but it was decided that we shouldn't do reindex except in the 1 hour downtime window available each day. I came up with the cache table idea to cut down the number of physical reads from the audit table (now only one read from the base table is required). That seems to solve the original problem well. But obviously has introduced this recompilation issue.Peso : "The Query Optimizer always assumes there is one and only one record in a table variable because table variables does not have statistics.That is why Query Optimizer sometimes choosed an incorrect plan.My rule of thumb is that when the chance is high the table variable will hold more than one page of data, I change to temp table instead.One page is the minimum amount of data read at any given time by SQL Server."I think for this stored proc I can narrow down the data needed in the cache table some more. I think that for the vast majority of the time an employee will have under 10 rows that are pertinent to the task in hand (working out holiday entitlements at a point in time). I really only need 4 pieces of data for each line 2 DATETIME values and 2 FLOATS. If I manage to get down to this do you think a table variable is a good idea?Sodeep : "I would definitely use KEEP PLAN to reduce recompiles in this scenerio? Also take advantages of index in Temp table for faster retrieval."Added (KEEP PLAN) hint to the sp call -- will see what impact that has. Is it really worth adding index to temp table for around 20 rows (now that I'm narrowing down data even more). At the moment the temp table is a HEAP -- will it improve matters if I add a surrogate KEY to get a clustered index? for that many rows will the time taken making the index hurt more than help?update :: I've added a check to find out if employee was in the worst case scenario (no relevant info) and if so not perform any more that the initial select. (and not bother making the temp tables) I think that will speed things up drastically for the load testing. I'd still like to get recommendations for further improvements though. A lot of our code is written like this.NB : Also -- I managed to eliminate on of the temp tables completely and instead of INSERTING twice and SELECTING once now perform a SELECT from a derived table using UNIONS to join the different queries)Charlie===============================================================Msg 3903, Level 16, State 1, Line 1736The ROLLBACK TRANSACTION request has no corresponding BEGIN TRANSACTION |
 |
|
|
sodeep
Master Smack Fu Yak Hacker
7174 Posts |
Posted - 2009-02-12 : 10:06:31
|
quote: Originally posted by Transact Charlie Thanks for all the tips:Gail: "Why not reference the base table? Have you identified that doing slow is a performance problems and that querying a temp table is faster?"The base table is an audit table for 2d time. It keeps track of *everything* for all employees. The way that the stored proc is written is that only a few rows in the base table are pertinent. The stored proc can make 15 separate selects on the base table each time it's called and it gets called a lot.Because the table is constantly getting written to / updated and (occasionally) deleted from then the indices seem to become out of date really quickly. This can mean that sometimes the original stored proc takes *minutes* to finish when it should under a second.I suggested that we should reindex the table more often but it was decided that we shouldn't do reindex except in the 1 hour downtime window available each day. I came up with the cache table idea to cut down the number of physical reads from the audit table (now only one read from the base table is required). That seems to solve the original problem well. But obviously has introduced this recompilation issue.Peso : "The Query Optimizer always assumes there is one and only one record in a table variable because table variables does not have statistics.That is why Query Optimizer sometimes choosed an incorrect plan.My rule of thumb is that when the chance is high the table variable will hold more than one page of data, I change to temp table instead.One page is the minimum amount of data read at any given time by SQL Server."I think for this stored proc I can narrow down the data needed in the cache table some more. I think that for the vast majority of the time an employee will have under 10 rows that are pertinent to the task in hand (working out holiday entitlements at a point in time). I really only need 4 pieces of data for each line 2 DATETIME values and 2 FLOATS. If I manage to get down to this do you think a table variable is a good idea?Sodeep : "I would definitely use KEEP PLAN to reduce recompiles in this scenerio? Also take advantages of index in Temp table for faster retrieval."Added (KEEP PLAN) hint to the sp call -- will see what impact that has. Is it really worth adding index to temp table for around 20 rows (now that I'm narrowing down data even more). At the moment the temp table is a HEAP -- will it improve matters if I add a surrogate KEY to get a clustered index? for that many rows will the time taken making the index hurt more than help?It will force to use same cache plan .Then you don't need to index at all. Why not try with Table variables if you are dealing with 20 rows? Compare performance with Temp tables. Also are your TEMPDB Data files spread accross number of cores in your server? Is it on RAID 10 disk? This makes difference as well.update :: I've added a check to find out if employee was in the worst case scenario (no relevant info) and if so not perform any more that the initial select. (and not bother making the temp tables) I think that will speed things up drastically for the load testing. I'd still like to get recommendations for further improvements though. A lot of our code is written like this.NB : Also -- I managed to eliminate on of the temp tables completely and instead of INSERTING twice and SELECTING once now perform a SELECT from a derived table using UNIONS to join the different queries)Charlie===============================================================Msg 3903, Level 16, State 1, Line 1736The ROLLBACK TRANSACTION request has no corresponding BEGIN TRANSACTION
|
 |
|
|
Transact Charlie
Master Smack Fu Yak Hacker
3451 Posts |
Posted - 2009-02-12 : 10:32:47
|
| Thanks Sodeep,Going to try the changes in a sustained test later. Apparently the people doing the load testing need to buy licences for LoadRunner in 2 hour slots and they are very expensive.Our dev testing so far with 9KEEPPLAN and by eliminating one of the temp tables entirely and using derived tables seems to be vastly reducing the number of recompiles needed so fingers crossed.Thanks for all the advice -- I'll post some results when we next do the load test.Charlie===============================================================Msg 3903, Level 16, State 1, Line 1736The ROLLBACK TRANSACTION request has no corresponding BEGIN TRANSACTION |
 |
|
|
|