Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
| Author |
Topic |
|
noonway
Starting Member
2 Posts |
Posted - 2008-10-28 : 16:54:09
|
| I have a stored procedure that runs on my Production database in 0.394 seconds. However, when I try to run the SP on a DEV box it takes well over 15 minutes to run *every* time.I have checked indexes and everything seems to be identical (I did a backup/restore to the Dev box in the first place). The only major difference (besides the computer hardware) is that the Production DB is SQL Server 2000 and DEV is SQL Server 2005.Production environment hardware:Xeon 1.6GHZ (x2)8GB RAMDev environment hardware:P4 3GHZ (x1)4GB RAMI can't imagine the hardware difference is causing that kind of slow down, but I didn't want to rule it out. And I really don't think that is the problem because other queries seem to run ok (I haven't tested all 175 of them yet though).Also, just so you know, the # of records in the tables is 1399, 3528, 28442, 8230, 100 and 19 so we're not talking about loads of data. Like I said before, the query runs in less than a second in production. Any suggestions as to what might be causing this in my DEV environment?Here is the query from the stored procedure that is taking so long to run... Keep in mind that I have replaced declared and temporary variables with hard-coded variables in several places for simplicity sake. SELECT DISTINCT(rfq.rfqid), rfq.startdateFROM dbo.rfqs AS rfqLEFT JOIN dbo.organizations AS org ON rfq.orgid = org.orgidLEFT JOIN dbo.bidsubmissions AS bs ON rfq.rfqid = bs.rfqidLEFT JOIN dbo.rfqtimext AS rfqet ON bs.subid = rfqet.subidLEFT JOIN dbo.timezones AS tz ON rfq.tzid = tz.tzidWHERE ((org.spid IN (1) AND rfqet.newendtime IS NULL AND rfq.enddate > DATEADD(hh, tz.offset, GETUTCDATE()) AND rfq.enddate < '12/31/3000 23:59:59.999' AND org.[name] LIKE '') OR(org.spid IN (1) AND rfqet.newendtime IS NOT NULL AND rfqet.timeextID = (SELECT MAX(timeextID) FROM dbo.rfqtimext AS rte LEFT JOIN bidsubmissions AS bs ON rte.subid = bs.subid LEFT JOIN dbo.rfqpause AS rp ON rte.pauseid = rp.pauseid WHERE bs.rfqid = rfq.rfqid OR rp.rfqid = rfq.rfqid) AND rfqet.newendtime > DATEADD(hh, tz.offset, GETUTCDATE())) AND rfqet.newendtime < '12/31/3000 23:59:59.999' AND org.[name] LIKE '' OR rfq.ispaused = 1) |
|
|
sodeep
Master Smack Fu Yak Hacker
7174 Posts |
Posted - 2008-10-28 : 17:04:15
|
| 1)compare the execution plan in both environment 2) Did you rebuild fragmented indexes and update stats after restoring in DEV? |
 |
|
|
hanbingl
Aged Yak Warrior
652 Posts |
Posted - 2008-10-28 : 17:05:24
|
| Deskspace and where do you store your log files?? |
 |
|
|
sodeep
Master Smack Fu Yak Hacker
7174 Posts |
Posted - 2008-10-28 : 17:24:58
|
quote: Originally posted by hanbingl Deskspace and where do you store your log files??
How does it make query slow? Do you mean log file is full or I/O contention issues? |
 |
|
|
hanbingl
Aged Yak Warrior
652 Posts |
Posted - 2008-10-28 : 17:53:55
|
| if log disk write latency is too high, there's nothing he can do about the query but put the log on its own disk. |
 |
|
|
Lamprey
Master Smack Fu Yak Hacker
4614 Posts |
Posted - 2008-10-28 : 18:50:31
|
| Would clearing out the cache's help? Perhaps you have a parameter sniffing issue with bad executions plans..?Just a thought. |
 |
|
|
noonway
Starting Member
2 Posts |
Posted - 2008-10-28 : 19:55:03
|
| Thanks for the tips so far. I do have a weekly maintenance plan running on that database and it has run since I loaded it, but I had not done some of what you guys suggested. I did the following:* tested all indexes for fragmentation and did not find any* updated stats* cleared the proc cacheNothing sped up the query in the DEV environment.Logs are located on same hard disk (40GB free). The MDF is only 90MB. The LDF is 50MB. I really don't think disk I/O is the issue here but... Is there a way to test for that?I did run an execution plan on both environments and came up with some differences however I don't know enough about execution plans to know where to go from here. Maybe one of you could make a suggestion based on these... I also have statistics on the execution plan for each step if those would help let me know?PRODUCTIONPlan1 Scanning the entire clustered index .2 Scanning the entire non-clustered index .3 Scanning the entire non-clustered index .4 Sorting the input from steps 3 .5 Matching rows from two suitably sorted input tables exploiting their sort order.6 Scanning the entire clustered index .7 Scanning the entire non-clustered index .8 Matching rows from two suitably sorted input tables exploiting their sort order.9 Use each row from the top input step 5 to build a hash table, and each row from the bottom input step 8 to probe into the hash table, outputting all matching rows.10 Use each row from the top input step 1 to build a hash table, and each row from the bottom input step 9 to probe into the hash table, outputting all matching rows.11 Scanning an internal table of constants.12 Restricting the set of rows based on predicate .13 Scanning the entire clustered index .14 Scanning the entire clustered index .15 Scanning the entire clustered index .16 Use each row from the top input step 14 to build a hash table, and each row from the bottom input step 15 to probe into the hash table, outputting all matching rows.17 Use each row from the top input step 13 to build a hash table, and each row from the bottom input step 16 to probe into the hash table, outputting all matching rows.18 Restricting the set of rows based on predicate .19 Computing summary values for groups of rows in a suitably sorted stream.20 Restricting the set of rows based on predicate .21 Appending multiple input tables to form the output table.22 For each row in the top (outer) input (step 10), scan the bottom (inner) input (step 21), and output matching rows.23 Use each row from the top input step 22 to build a hash table, and each row from the bottom input step UNKNOWN to probe into the hash table, outputting all matching rows.24 Rows were returned by the SELECT statement.DEVPlan1 Scanning the entire clustered index .2 Perform the Distribute Streams, Gather Streams, and Repartition Streams logical operations.3 Scanning the entire clustered index .4 Scanning a particular range of rows from the clustered index .5 For each row in the top (outer) input (step 3), scan the bottom (inner) input (step 4), and output matching rows.6 Use each row from the top input step 2 to build a hash table, and each row from the bottom input step 5 to probe into the hash table, outputting all matching rows.7 Perform the Distribute Streams, Gather Streams, and Repartition Streams logical operations.8 Scanning the entire clustered index .9 Perform the Distribute Streams, Gather Streams, and Repartition Streams logical operations.10 Scanning the entire non-clustered index .11 Perform the Distribute Streams, Gather Streams, and Repartition Streams logical operations.12 Use each row from the top input step 9 to build a hash table, and each row from the bottom input step 11 to probe into the hash table, outputting all matching rows.13 Perform the Distribute Streams, Gather Streams, and Repartition Streams logical operations.14 Use each row from the top input step 7 to build a hash table, and each row from the bottom input step 13 to probe into the hash table, outputting all matching rows.15 Scanning the entire clustered index .16 Scanning the entire clustered index .17 Scanning the entire non-clustered index .18 Use each row from the top input step 16 to build a hash table, and each row from the bottom input step 17 to probe into the hash table, outputting all matching rows.19 Use each row from the top input step 15 to build a hash table, and each row from the bottom input step 18 to probe into the hash table, outputting all matching rows.20 Restricting the set of rows based on predicate .21 Computing summary values for groups of rows in a suitably sorted stream.22 Use each row from the top input step 21 to build a hash table, and each row from the bottom input step UNKNOWN to probe into the hash table, outputting all matching rows.23 For each row in the top (outer) input (step 14), scan the bottom (inner) input (step 22), and output matching rows.24 Restricting the set of rows based on predicate .25 Perform the Distribute Streams, Gather Streams, and Repartition Streams logical operations.26 Sorting the input from steps 25 .27 Rows were returned by the SELECT statement. |
 |
|
|
sodeep
Master Smack Fu Yak Hacker
7174 Posts |
Posted - 2008-10-29 : 00:07:10
|
quote: Originally posted by noonway Thanks for the tips so far. I do have a weekly maintenance plan running on that database and it has run since I loaded it, but I had not done some of what you guys suggested. I did the following:* tested all indexes for fragmentation and did not find any* updated stats* cleared the proc cacheNothing sped up the query in the DEV environment.Logs are located on same hard disk (40GB free). The MDF is only 90MB. The LDF is 50MB. I really don't think disk I/O is the issue here but... Is there a way to test for that?I did run an execution plan on both environments and came up with some differences however I don't know enough about execution plans to know where to go from here. Maybe one of you could make a suggestion based on these... I also have statistics on the execution plan for each step if those would help let me know?PRODUCTIONPlan1 Scanning the entire clustered index .2 Scanning the entire non-clustered index .3 Scanning the entire non-clustered index .4 Sorting the input from steps 3 .5 Matching rows from two suitably sorted input tables exploiting their sort order.6 Scanning the entire clustered index .7 Scanning the entire non-clustered index .8 Matching rows from two suitably sorted input tables exploiting their sort order.9 Use each row from the top input step 5 to build a hash table, and each row from the bottom input step 8 to probe into the hash table, outputting all matching rows.10 Use each row from the top input step 1 to build a hash table, and each row from the bottom input step 9 to probe into the hash table, outputting all matching rows.11 Scanning an internal table of constants.12 Restricting the set of rows based on predicate .13 Scanning the entire clustered index .14 Scanning the entire clustered index .15 Scanning the entire clustered index .16 Use each row from the top input step 14 to build a hash table, and each row from the bottom input step 15 to probe into the hash table, outputting all matching rows.17 Use each row from the top input step 13 to build a hash table, and each row from the bottom input step 16 to probe into the hash table, outputting all matching rows.18 Restricting the set of rows based on predicate .19 Computing summary values for groups of rows in a suitably sorted stream.20 Restricting the set of rows based on predicate .21 Appending multiple input tables to form the output table.22 For each row in the top (outer) input (step 10), scan the bottom (inner) input (step 21), and output matching rows.23 Use each row from the top input step 22 to build a hash table, and each row from the bottom input step UNKNOWN to probe into the hash table, outputting all matching rows.24 Rows were returned by the SELECT statement.DEVPlan1 Scanning the entire clustered index .2 Perform the Distribute Streams, Gather Streams, and Repartition Streams logical operations.3 Scanning the entire clustered index .4 Scanning a particular range of rows from the clustered index .5 For each row in the top (outer) input (step 3), scan the bottom (inner) input (step 4), and output matching rows.6 Use each row from the top input step 2 to build a hash table, and each row from the bottom input step 5 to probe into the hash table, outputting all matching rows.7 Perform the Distribute Streams, Gather Streams, and Repartition Streams logical operations.8 Scanning the entire clustered index .9 Perform the Distribute Streams, Gather Streams, and Repartition Streams logical operations.10 Scanning the entire non-clustered index .11 Perform the Distribute Streams, Gather Streams, and Repartition Streams logical operations.12 Use each row from the top input step 9 to build a hash table, and each row from the bottom input step 11 to probe into the hash table, outputting all matching rows.13 Perform the Distribute Streams, Gather Streams, and Repartition Streams logical operations.14 Use each row from the top input step 7 to build a hash table, and each row from the bottom input step 13 to probe into the hash table, outputting all matching rows.15 Scanning the entire clustered index .16 Scanning the entire clustered index .17 Scanning the entire non-clustered index .18 Use each row from the top input step 16 to build a hash table, and each row from the bottom input step 17 to probe into the hash table, outputting all matching rows.19 Use each row from the top input step 15 to build a hash table, and each row from the bottom input step 18 to probe into the hash table, outputting all matching rows.20 Restricting the set of rows based on predicate .21 Computing summary values for groups of rows in a suitably sorted stream.22 Use each row from the top input step 21 to build a hash table, and each row from the bottom input step UNKNOWN to probe into the hash table, outputting all matching rows.23 For each row in the top (outer) input (step 14), scan the bottom (inner) input (step 22), and output matching rows.24 Restricting the set of rows based on predicate .25 Perform the Distribute Streams, Gather Streams, and Repartition Streams logical operations.26 Sorting the input from steps 25 .27 Rows were returned by the SELECT statement.
Sometime query uses multiple processors doing parallel execution and it slows down query. Use Option(MAXDOP 1). |
 |
|
|
sodeep
Master Smack Fu Yak Hacker
7174 Posts |
Posted - 2008-10-29 : 00:08:02
|
| We use (n-1) formula for processors. |
 |
|
|
AndrewMurphy
Master Smack Fu Yak Hacker
2916 Posts |
Posted - 2008-10-29 : 06:20:14
|
| Note OP's original post suggested that Prod had more processors than Dev....so parallel execution should not be in force for the Dev machine.Regardless, there is also a lot of index scanning in progress....can these be converted to "index seeks"? |
 |
|
|
sodeep
Master Smack Fu Yak Hacker
7174 Posts |
Posted - 2008-10-29 : 09:43:46
|
quote: Originally posted by AndrewMurphy Note OP's original post suggested that Prod had more processors than Dev....so parallel execution should not be in force for the Dev machine.Regardless, there is also a lot of index scanning in progress....can these be converted to "index seeks"?
I am not talking about more or less processors.The fact is sometimes when query uses more processors it can slow down query but not in case of production in this case.Please see the difference in Execution plan as I indicated in green for difference. Also Why would you change indexing strategy? As Everything is from production. |
 |
|
|
AndrewMurphy
Master Smack Fu Yak Hacker
2916 Posts |
Posted - 2008-10-29 : 09:53:33
|
| 1. you pointed out something on the DEV execution plan which you seemed to be relating to multiple processors, when the DEV environment only had 1 processor.2. index seeking usually is a lot more efficient than index scanning, so rather than just look to bring the 2 environments back into sync performance-wise, I was suggesting that maybe bringing one or both a step further/better by working towards index-seeks would produce a far more useful payoff. |
 |
|
|
hanbingl
Aged Yak Warrior
652 Posts |
Posted - 2008-10-29 : 10:54:31
|
| See if Microsoft SQL Server 2005 Upgrade Advisor can help you to identify some hidden issues:http://www.microsoft.com/downloads/details.aspx?familyid=1470E86B-7E05-4322-A677-95AB44F12D75&displaylang=en |
 |
|
|
|
|
|
|
|