Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2000 Forums
 Transact-SQL (2000)
 Creating Multiple Execution Plans - Search Proc??

Author  Topic 

mike123
Master Smack Fu Yak Hacker

1462 Posts

Posted - 2004-10-28 : 23:53:35
Hi,

I've been having some problems with my sqlserver slowing down. I recently ran a trace, and as expected I knew what my slowest running queriy by far was. I did not know how drastic the increase in performance would be when I disabled it. My server usually sits between 80-100% CPU usage at peak times, after I disable this SPROC its down at an estimated 5-25%.

My experience and knowledge of optimizing queries like this is pretty limited so I'm hoping some of the SQLTeam Guru's can help me out here I'm willing to do anything to speed this up to its greatest potential.

There are 28 parameters, 2 of which are for paging, and 1 for orderby. This leaves 25 parameters to be searched on, however many searches may only pass 3 or 4 values.

Since its getting so many parameters I was wondering if I could create multiple search SPROCS depending on the criteria that was being searched, and execute the appropriate one. Would this help? Is this ever done in practice?

I don't care if I have to make 200 SPROCS to run this query faster, I need to give this thing the biggest boost I can. Am I barking up the wrong tree ? ANY tips on what I can do here is greatly appreciated.

Thanks alot once again
mike123

mike123
Master Smack Fu Yak Hacker

1462 Posts

Posted - 2004-10-28 : 23:56:45
[code]
CREATE Procedure dbo.UserSearch

@Page int,
@RecsPerPage int,
@GenderID tinyint = NULL,
@NameOnline varchar(15) = NULL, --LIKE ?
@sexualityID tinyint = NULL,
@minAge tinyint = NULL,
@maxAge tinyint = NULL,
@PostalCode varchar(7) = NULL, --LIKE ?
@City varchar(25) = NULL,
@StateProvID tinyint = NULL,
@CountryID tinyint = NULL,
@statusID tinyint = NULL,
@bodyTypeID tinyint = NULL,
@hairColorID tinyint = NULL,
@eyeColorID tinyint = NULL,
@HeightFeet tinyint = NULL,
@HeightInches tinyint = NULL,
@educationID tinyint = NULL,
@employmentID tinyint = NULL,
@drinkID tinyint = NULL,
@smokeID tinyint = NULL,
@memberSinceDays tinyint = NULL,
@qsVideo char(2) = NULL,
@qsOnline char(2) = NULL,
@qsSuperPhoto char(2) = NULL,
@orderBy tinyint

AS

Set Nocount on


Declare @Rows int

Create Table #Paging (
--Declare @Paging Table (
RowID int IDENTITY,
userID int,
NameOnline varchar(15),
age tinyint,
statusID tinyint,
Points int,
Votes int,
userNote varchar(35),
GenderID tinyint,
[date] smalldatetime,
thumb_Count tinyint,
mainPhotoID int,
vidExist char,
SuperPhoto char
)


INSERT INTO #Paging (
userID, NameOnline, age, statusID, Points,
Votes, userNote, GenderID, date, thumb_Count, mainPhotoID, vidExist, SuperPhoto)
SELECT
TOP 50
tblUserDetails.userID, tblUserDetails.nameOnline, age, statusID, points,
votes, userNote, genderID, date,
(
select count(userID) as tblthumb_count
from tblthumbs where status = 1 AND
tblUserDetails.userid = tblthumbs.userid
) as thumb_count,

(select counterID FROM tblExtraPhotos where photoID = '0' AND tblUserDetails.userid = tblExtraPhotos.userid) as mainPhotoID,

case
when exists (
select userID
from tblvideo
where tblUserDetails.userid = tblvideo.userid AND active ='1' )
then 'Y' Else 'N' end as vidExist,
case
when exists (
select userID from tblExtraPhotos where tblUserDetails.userid = tblExtraPhotos.userID AND active ='1' AND photoID ='99'
--select userID from tblThumbs where tblUserDetails.userid = tblThumbs.userid AND active ='1' AND thumbID ='99'
) then 'Y' Else 'N' end as SuperPhoto

FROM
tblUserDetails
WHERE
tblUserDetails.active = '1' and
GenderID = isNull(@GenderID, GenderID) and
NameOnline like '%' + isNull(@NameOnline, '') + '%' and
sexualityID = isNull(@sexualityID , sexualityID) and
(
age >= @minAge OR @minAge IS NULL
) and
(
age <= @maxAge OR @maxAge IS NULL
) and
PostalCode like '%' + isNull(@PostalCode, '') + '%' and
City like '%' + isNull(@City, '') + '%' and
StateProvID = isNull(@StateProvID, StateProvID) and
CountryID = isNull(@CountryID, CountryID) and

statusID = isNull(@statusID, statusID) and
bodyTypeID = isNull(@bodyTypeID, bodyTypeID) and
hairColorID = isNull(@hairColorID, hairColorID) and
eyeColorID = isNull(@eyeColorID, eyeColorID) and
HeightFeet = isNull(@HeightFeet, HeightFeet) and
HeightInches = isNull(@HeightInches, HeightInches) and
educationID = isNull(@educationID, educationID) and
employmentID = isNull(@employmentID, employmentID) and
drinkID = isNull(@drinkID, drinkID) and
smokeID = isNull(@smokeID, smokeID) and
(
@memberSinceDays is NULL OR
DateDiff(d, date, GetDate()) <= @memberSinceDays
) and
(
@qsVideo != 'on' OR
EXISTS (
select userID from
tblvideo where userId = tblUserDetails.userID and tblvideo.active ='1')
) and
(
@qsOnline != 'on' OR
EXISTS (
select userID from
tblActive_Users where userId = tblUserDetails.userID)
) and
(
@qsSuperPhoto != 'on' OR
EXISTS (
select userID from
tblExtraPhotos where userId = tblUserDetails.userID AND photoID ='99')
)



ORDER BY
Case @OrderBy
WHEN 1 THEN date
WHEN 3 THEN lastLoggedIn
ELSE NULL END DESC,
Case WHEN @OrderBy = 2 and votes > 0 THEN (points / votes) END DESC


SET @Rows = @@Rowcount


Declare @FirstRow int,
@LastRow int,
@TotalPages int

SET @FirstRow = ((@Page - 1) * @RecsPerPage) + 1
SET @LastRow = @FirstRow + (@RecsPerPage - 1)
SET @TotalPages = ceiling(Cast(@rows as decimal)/ @RecsPerPage )



SELECT *, @Rows TotalRows, @TotalPages TotalPages
FROM #Paging
WHERE rowid BETWEEN @FirstRow and @LastRow

Drop Table #Paging

Set Nocount off
GO
[/code]
Go to Top of Page

spirit1
Cybernetic Yak Master

11752 Posts

Posted - 2004-10-29 : 04:49:42
i had a similar problem:
i got a 50% speed up by doing this for each parameter:
...AND (statusID = @statusID or @statusID is null) AND....

but the catch here is that the execution plan can change dramatically how you put it:
i use simple trial and error and i got:
...(col1 = @col1 or @col1 is null) AND
(@col2 is null or col2 = @col2) AND
(col2 = @col2 or @col2 is null) AND ...

it depends on how many nulls are in the specific column.

hope it helps a little.


Go with the flow & have fun! Else fight the flow
Go to Top of Page

mike123
Master Smack Fu Yak Hacker

1462 Posts

Posted - 2004-10-29 : 16:22:03

Ok that gives me something to try. Do you think I can get more than 50% increase on this ? Is 300% dreaming?

What do you think of having multiple SPROCS?


Thanks again

mike123
Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2004-10-29 : 19:30:22
If you can live with the permissions issues you could use dynamic SQL

DECLARE @strSQL varchar(8000)

SELECT @strSQL = 'SELECT ...
FROM tblUserDetails
WHERE tblUserDetails.active = ''1'''

IF @GenderID IS NOT NULL
SELECT @strSQL = @strSQL + ' AND GenderID = @GenderID'

IF @GenderID IS NOT NULL
SELECT @strSQL = @strSQL + ' AND NameOnline like ''%'' + @NameOnline + ''%'' '

...

EXEC sp_excuteSQL @strSQL, @GenderID tinyint, @NameOnline varchar(15), ...

This will mean that the frequently occuring combinations of parameters will be in the query plan, and will be reused.

Note that its OK to have a stack of parameters to sp_excuteSQL - even if they are not referenced in @strSQL :-)

If the dynamic SQL is a no-no then I've got a couple of suggestions:

NameOnline like '%' + isNull(@NameOnline, '') + '%'

this is a performance killer and will cause a table scan every time. If instead you do:

IF @NameOnline IS NOT NULL THEN SELECT @NameOnline = '%' + @NameOnline + '%'

then in your WHERE clause you can put

(@NameOnline IS NULL OR NameOnline like @NameOnline)

which may optimise better.

As an alternative you could pull data into a temporary table using the more popular criteria - in a query that was tested to give good performance (all columns indexed etc.)

And then progressively delete records [from the temporary table] based on the other criteria (you need to add columns to #Paging for all the criteria you want to test separately) using statements like:

IF @NameOnline IS NOT NULL
DELETE #Paging
WHERE NameOnline NOT like '%' + @NameOnline + '%'

Finally, beware that your current statement

NameOnline like '%' + isNull(@NameOnline, '') + '%'

will EXCLUDE all rows in the table where NameOnline IS NULL :-(

Hopefully all your LIKE tests are on NOT NULL columns :-)

Kristen
Go to Top of Page

mike123
Master Smack Fu Yak Hacker

1462 Posts

Posted - 2004-10-29 : 22:14:21
Hi Kristen,

thanks for the pointers, I can't live with dynamic-SQL so I'll try your second recommendation for now and see what happens.

What do you think of the general idea of having different SPROCS for different search criteria resultion in different execution plans for each SPROC? This SPROC is killing me and I really need to find a way to boost performance ALOT not just a little.

More opinions welcome if anyone else has thoughts.

Thanks again

mike123
Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2004-10-29 : 22:47:19
"What do you think of the general idea of having different SPROCS"

As an off-hand-remark I'd say you have too many parameters to pull that trick.

Kristen
Go to Top of Page

hgorijal
Constraint Violating Yak Guru

277 Posts

Posted - 2004-10-30 : 00:25:52
try this, ..

1. Remove these Subqueries in SELECT clause and use regular JOINS instead.

(
select count(userID) as tblthumb_count
from tblthumbs where status = 1 AND
tblUserDetails.userid = tblthumbs.userid
) as thumb_count,

(select counterID FROM tblExtraPhotos where photoID = '0' AND tblUserDetails.userid = tblExtraPhotos.userid) as mainPhotoID,

case
when exists (
select userID
from tblvideo
where tblUserDetails.userid = tblvideo.userid AND active ='1' )
then 'Y' Else 'N' end as vidExist,
case
when exists (
select userID from tblExtraPhotos where tblUserDetails.userid = tblExtraPhotos.userID AND active ='1' AND photoID ='99'
--select userID from tblThumbs where tblUserDetails.userid = tblThumbs.userid AND active ='1' AND thumbID ='99'
) then 'Y' Else 'N' end as SuperPhoto


2. Split the Query into multiple queries...
[code]
(
@qsVideo != 'on' OR
EXISTS (
select userID from
tblvideo where userId = tblUserDetails.userID and tblvideo.active ='1')
) and
(
@qsOnline != 'on' OR
EXISTS (
select userID from
tblActive_Users where userId = tblUserDetails.userID)
) and
(
@qsSuperPhoto != 'on' OR
EXISTS (
select userID from
tblExtraPhotos where userId = tblUserDetails.userID AND photoID ='99')
)
[code]

Take the above code out of this query. Then individually check the variables (@qsVideo, @qsOnline...) and delete from #Paging if required.


Also, seems to me that you are using this proc to browse, calling it one time for each page. If performance is really the highest priority, you might want to look into incorporating caching techniques, so that the first time you run, you load the data into a permanent or permanent-temporary(##) table and you read this table for the subsequent pages, instead of rerunning the entire proc.

hope it helps...



Hemanth Gorijala
BI Architect / DBA...(yuhoo!!! and now, an "Yak Master")

Exchange a Dollar, we still have ONE each.
Exchange an Idea, we have TWO each.
Go to Top of Page

mike123
Master Smack Fu Yak Hacker

1462 Posts

Posted - 2004-10-30 : 01:47:53
quote:
Originally posted by Kristen

"What do you think of the general idea of having different SPROCS"

As an off-hand-remark I'd say you have too many parameters to pull that trick.

Kristen



Yeah I estimate it at abou 400 diff sprocs for each specific combination, although i haven't double checked my math. If I could get a 300% increase in speed I will gladly, and painfully write these 400 SPROCS :) just wondering if its the most ridiculous thing you've ever seen attempted lol

mike123
Go to Top of Page

mike123
Master Smack Fu Yak Hacker

1462 Posts

Posted - 2004-10-30 : 01:48:32
quote:
Originally posted by hgorijal

try this, ..

1. Remove these Subqueries in SELECT clause and use regular JOINS instead.

(
select count(userID) as tblthumb_count
from tblthumbs where status = 1 AND
tblUserDetails.userid = tblthumbs.userid
) as thumb_count,

(select counterID FROM tblExtraPhotos where photoID = '0' AND tblUserDetails.userid = tblExtraPhotos.userid) as mainPhotoID,

case
when exists (
select userID
from tblvideo
where tblUserDetails.userid = tblvideo.userid AND active ='1' )
then 'Y' Else 'N' end as vidExist,
case
when exists (
select userID from tblExtraPhotos where tblUserDetails.userid = tblExtraPhotos.userID AND active ='1' AND photoID ='99'
--select userID from tblThumbs where tblUserDetails.userid = tblThumbs.userid AND active ='1' AND thumbID ='99'
) then 'Y' Else 'N' end as SuperPhoto


2. Split the Query into multiple queries...
[code]
(
@qsVideo != 'on' OR
EXISTS (
select userID from
tblvideo where userId = tblUserDetails.userID and tblvideo.active ='1')
) and
(
@qsOnline != 'on' OR
EXISTS (
select userID from
tblActive_Users where userId = tblUserDetails.userID)
) and
(
@qsSuperPhoto != 'on' OR
EXISTS (
select userID from
tblExtraPhotos where userId = tblUserDetails.userID AND photoID ='99')
)
[code]

Take the above code out of this query. Then individually check the variables (@qsVideo, @qsOnline...) and delete from #Paging if required.


Also, seems to me that you are using this proc to browse, calling it one time for each page. If performance is really the highest priority, you might want to look into incorporating caching techniques, so that the first time you run, you load the data into a permanent or permanent-temporary(##) table and you read this table for the subsequent pages, instead of rerunning the entire proc.

hope it helps...



Hemanth Gorijala
BI Architect / DBA...(yuhoo!!! and now, an "Yak Master")

Exchange a Dollar, we still have ONE each.
Exchange an Idea, we have TWO each.





thanks hgorijal I'll play around with this and see what happens :)

cheers

mike123
Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2004-10-30 : 07:39:55
"If I could get a 300% increase in speed"

I would think you should get a 10 fold improvement in speed, maybe more. At least for some of the combinations of queries.

But I don't think you need to do loads of SProcst ... I think pulling a "first cut" dataset into a temporary table and progresively deleting is the best strategy ... unless you are going to pull gazillions of rows from which you then delete 90%

I think my first approach would be to create a "log" table and have the existing SProc record the parameters it is given. Then you could use those "real world" examples to run some benchmarks on how many rows a "first cut" would grab, compared to the actual number in the final query.

You might very well have a handful of SProcs for that job - "IF @minAge IS NOT NULL AND @maxAge IS NOT NULL EXEC SProc_FirstCut_A ELSE ..." so that your "first cut" is biased towards a criteria which is provided by the user AND you know has good query performance AND is likely to produce a modest-ish "first cut"

I'll be interested to know how you get on please

Kristen
Go to Top of Page

spirit1
Cybernetic Yak Master

11752 Posts

Posted - 2004-10-30 : 12:16:38
when i was speeding up my search (remember those joins kristen )
i thought about an sproc but in the "we" decided we're going to build it directly in the asp and execute the query directly because the user could also chose the operator to use in data compare.
so we had a bunch of search conditions enabled (about 25) and we built the query for those that were chosen to search for.
so the more conditions were chosen the resultset was smaller so that was no problem.
i used 2 conditions that returned more than 20K rows and optimized that.
i guess i'm probably saying the same as Kristen
optimize the hungriest SQL and put in the first IF ... ELSE ... than do the in the descending order of search conditions.

Go with the flow & have fun! Else fight the flow
Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2004-10-30 : 12:47:11
Do you remember what sort of speed improvement you got? Maybe you told me at the time, at my age I forget! But its always nice to reminisce about the performance gains I had when I was younger - by a week or two!

Kristen
Go to Top of Page

spirit1
Cybernetic Yak Master

11752 Posts

Posted - 2004-10-30 : 14:27:50
heh....
i got 65% speed increase in average...

i also changed the asp page a bit and got it to load 10 times faster so i guess everyone's happy now . me too as i got a raise

i did forgot to look up that bit slice thing you told me about. or was that just a joke on your part?

Go with the flow & have fun! Else fight the flow
Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2004-10-31 : 02:15:41
http://www.cs.caltech.edu/courses/cs184/winter2003/lectures/Day3.ppt

Kristen
Go to Top of Page

Arnold Fribble
Yak-finder General

1961 Posts

Posted - 2004-10-31 : 05:59:52
Or rather more abstractly,
http://www.informatik.uni-bonn.de/~ralf/publications/MPC2004.pdf
Go to Top of Page

spirit1
Cybernetic Yak Master

11752 Posts

Posted - 2004-10-31 : 07:02:09
damn... i guess you weren't joking...

that's quite a reading... so does anyone acctually use this practically???

Go with the flow & have fun! Else fight the flow
Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2004-10-31 : 12:23:50
I'm pretty sure they used to - having two parallel tasks doing 4-bits each meant the job was done in half the time, I suppose. And back then that was a neat trick. Now you can probably do the whole job in 1 cycle, they ahve so many acres of silicon to play with, so its probably no longer relevant. But hey! I don't do no sticking hardware, so who knows!

Kristen
Go to Top of Page
   

- Advertisement -