Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2008 Forums
 Transact-SQL (2008)
 How to Optimize the Use of the “OR” Clause

Author  Topic 

cmdcenter
Starting Member

15 Posts

Posted - 2010-01-29 : 14:09:15
I wonder if there is any wise way to rewrite the following query so that the indexes on columns get used by optimizer?


Create Procedure select_Proc1
@Key1 int=0,
@Key2 int=0
As
BEGIN
Select key3
From Or_Table
Where (@key1 =0 OR Key1 =@Key1) AND
(@key2 =0 OR Key2 =@Key2)
END
GO


Even though columns in the WHERE clauses are covered by indexes, SQL Server is unable to use these indexes. This raises the question as to whether anything is “blocking” the use of the indexes. The answer to this question is yes -- the culprits are the parameters and the “OR” condition. The parameters are not covered by indexes, which means SQL Server cannot use any of the indexes to evaluate “@key1=0” (a condition which also applies to @key2=0). Effectively, this means SQL Server cannot use indexes to evaluate the clause “@key1=0 OR Key1= @key1” (as the “OR” clause is the union of rows covered by both conditions). The same principle applies to the other clause (re. key2) as well. This leads SQL Server to conclude that no indexes can be used to extract the rows, leaving SQL Server to utilize the next best approach -- a clustered index scan

As you see, the SQL optimizer will not use indexes on columns if the predicates are "OR"ed in the WHERE clause. One solution for this problem, is to separate queries with IF clause for all possible combination of parameters.

Please read this short article to get a better view of the problem: http://www.sql-server-performance.com/articles/per/optimize_or_clause_p1.aspx

Now my question is, what should we do if the possible combinations are more that just three or four? Writing a separate query for each combination does not seem a rational solution. Is there any other workaround for this problem?

Lamprey
Master Smack Fu Yak Hacker

4614 Posts

Posted - 2010-01-29 : 15:36:44
Although I'm not a fan of Dynamic SQL, it tends to perform the best for these "catch-all queries."
Go to Top of Page

Bustaz Kool
Master Smack Fu Yak Hacker

1834 Posts

Posted - 2010-01-29 : 19:29:56
[CODE]Create Procedure select_Proc1
@Key1 int=0,
@Key2 int=0
As
BEGIN
Select key3
From Or_Table
Where (@key1 =0 OR Key1 =@Key1) AND
(@key2 =0 OR Key2 =@Key2)
OPTION (RECOMPILE)
END[/CODE]
This will force the query engine to look explicitly at your parameters and generate an optimized execution plan. For instance, in the case where @key = 1, it won't bother to see if key1 = @key1 since it will already know that the OR logic will pass.


=======================================
Few things are harder to put up with than the annoyance of a good example. (Mark Twain)
Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2010-01-30 : 04:38:33
Might SQL not take longer to make the Query Plan than to do the query (using an existing cache plan)? (perhaps not in this query, but in a more complex one)

I've not used RECOMPILE for that reason, but maybe I should be using it?!
Go to Top of Page

Lamprey
Master Smack Fu Yak Hacker

4614 Posts

Posted - 2010-02-02 : 11:20:55
quote:
Originally posted by Bustaz Kool

[CODE]Create Procedure select_Proc1
@Key1 int=0,
@Key2 int=0
As
BEGIN
Select key3
From Or_Table
Where (@key1 =0 OR Key1 =@Key1) AND
(@key2 =0 OR Key2 =@Key2)
OPTION (RECOMPILE)
END[/CODE]
This will force the query engine to look explicitly at your parameters and generate an optimized execution plan. For instance, in the case where @key = 1, it won't bother to see if key1 = @key1 since it will already know that the OR logic will pass.

That will cause the stored procedure to recompile, to be sure. But, that won't affect how SQL handles OR clauses.
Go to Top of Page

harsh_athalye
Master Smack Fu Yak Hacker

5581 Posts

Posted - 2010-02-02 : 11:29:23
You can try this approach, not tested though:


Select key3
From Or_Table
Where
Key1 = isnull(nullif(@Key1,0), Key1) AND
Key2 = isnull(nullif(@Key2,0), Key2)


Harsh Athalye
http://www.letsgeek.net/
Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2010-02-02 : 12:58:45
If you set @Key1=0 (i.e. "Don't Care) you won't get any row where KEY1is NULL using that method, will you?

Also even more Index unfriendly I think?
Go to Top of Page

harsh_athalye
Master Smack Fu Yak Hacker

5581 Posts

Posted - 2010-02-02 : 13:18:55
True about first part, but then OP's approach also doesn't take into account NULL values either.

About use of index, I guess if the columns are defined as NOT NULL and has index on them, the index will still be used despite columns being wrapped inside ISNULL function. But then I might be wrong.

Harsh Athalye
http://www.letsgeek.net/
Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2010-02-03 : 01:17:55
Not sure I agree on first part Harsh, what do you think?
[code]
Where (@key1 =0 OR Key1 =@Key1) AND
(@key2 =0 OR Key2 =@Key2)
END
[code]
if @key1 = 0 then Key1 can be any value (including NULL), whereas
[CODE]
where Key1 = isnull(nullif(@Key1,0), Key1) AND
Key2 = isnull(nullif(@Key2,0), Key2)
[/CODE]
If @Key1 = 0 then the test is "Key1 = Key1" (TRUE for all cases except Key1 IS NULL) otherwise Key1 is compared to @Key1
Go to Top of Page

harsh_athalye
Master Smack Fu Yak Hacker

5581 Posts

Posted - 2010-02-03 : 01:53:53
You are right, Kristen. I missed the part where NULL != NULL.
Funny how easy it is to forget this important rule.

Harsh Athalye
http://www.letsgeek.net/
Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2010-02-03 : 02:01:40
Hehehe ... well there is always ANSI_NULLS
Go to Top of Page

Lumbago
Norsk Yak Master

3271 Posts

Posted - 2010-02-03 : 10:26:56
I have to admit I didn't read the entire article that you linked but have you considered UNION ALL? I know it's not overly practical for a changing amount of predicates but it might be worth considering. I changed an OR query to a UNION ALL and the query was trimmed from 20 seconds to less than a second
Select key3
From Or_Table
Where (@key1 =0)
AND (@key2 =0)
UNION ALL
Select key3
From Or_Table
Where (Key1 =@Key1)
AND (Key2 =@Key2)
Now the question remains: did I get the selection logic right? (I probably didn't...hmm)

- Lumbago
If the facts don't fit the theory, change the facts. Albert Einstein
Go to Top of Page

Transact Charlie
Master Smack Fu Yak Hacker

3451 Posts

Posted - 2010-02-03 : 12:33:54
A few different ways. Edit -- I don't think showplan shwos you dynamic sql. Therefore you'll just have to use the show actual execution plan which does.


SET SHOWPLAN_ALL OFF
GO

IF OBJECT_ID('tempdb..#or_table') IS NOT NULL DROP TABLE #or_table
GO

CREATE TABLE #or_table (
[ID] INT IDENTITY (1,1) PRIMARY KEY
, [key1] INT
, [key2] INT
, [key3] INT
)
CREATE INDEX IX_OR_KEY1_KEY2 ON #or_table ([key1], [key2])
GO

INSERT #or_table ([key1], [key2], [key3])
SELECT
CAST(CAST(NEWID() AS VARBINARY(64)) AS INT)
, CAST(CAST(NEWID() AS VARBINARY(64)) AS INT)
, CAST(CAST(NEWID() AS VARBINARY(64)) AS INT)
FROM
(SELECT 1 AS [1] UNION SELECT 2 AS [2] UNION SELECT 3 AS [3] UNION SELECT 4 AS [4]) a
CROSS JOIN (SELECT 1 AS [1] UNION SELECT 2 AS [2] UNION SELECT 3 AS [3] UNION SELECT 4 AS [4]) b
CROSS JOIN (SELECT 1 AS [1] UNION SELECT 2 AS [2] UNION SELECT 3 AS [3] UNION SELECT 4 AS [4]) c
CROSS JOIN (SELECT 1 AS [1] UNION SELECT 2 AS [2] UNION SELECT 3 AS [3] UNION SELECT 4 AS [4]) d
CROSS JOIN (SELECT 1 AS [1] UNION SELECT 2 AS [2] UNION SELECT 3 AS [3] UNION SELECT 4 AS [4]) e
CROSS JOIN (SELECT 1 AS [1] UNION SELECT 2 AS [2] UNION SELECT 3 AS [3] UNION SELECT 4 AS [4]) f
CROSS JOIN (SELECT 1 AS [1] UNION SELECT 2 AS [2] UNION SELECT 3 AS [3] UNION SELECT 4 AS [4]) g
GO

SELECT COUNT(*) FROM #or_table AS [Records]
GO



DECLARE @key1 INT
DECLARE @key2 INT
SET @key1 = 3
SET @key2 = 4
Select key3
From #Or_Table
Where (@key1 =0 OR Key1 =@Key1) AND
(@key2 =0 OR Key2 =@Key2)

GO
DECLARE @key1 INT
DECLARE @key2 INT
SET @key1 = 3
SET @key2 = 4

SELECT key3
FROM
#or_table
WHERE
key1 = CASE @key1 WHEN 0 THEN key1 ELSE @key1 END
AND key2 = CASE @key2 WHEN 0 THEN key2 ELSE @key2 END

GO

DECLARE @sql NVARCHAR(4000)
DECLARE @key1 INT
DECLARE @key2 INT

SET @key1 = 3
SET @key2 = 4

SET @sql = N'
SELECT
[key3]
FROM
#or_table
WHERE
1=1'

IF @key1 > 0 SET @sql = @sql + N'
AND key1 = @key1'

IF @key2 > 0 SET @sql = @sql + N'
AND key2 = @key2'

EXEC sp_ExecuteSql
@sql
, N'@key1 INT
, @key2 INT'
, @key1
, @key2
GO

DECLARE @key1 INT
DECLARE @key2 INT

SET @key1 = 3
SET @key2 = 4
Select key3
From #Or_Table
Where (@key1 =0)
AND (@key2 =0)
UNION ALL
Select key3
From #Or_Table
Where (Key1 =@Key1)
AND (Key2 =@Key2)
Go to Top of Page

Transact Charlie
Master Smack Fu Yak Hacker

3451 Posts

Posted - 2010-02-03 : 12:37:08
Lumbago --

You union all seems to do both an index scan and then an index seek

probably not that performant!


Charlie
===============================================================
Msg 3903, Level 16, State 1, Line 1736
The ROLLBACK TRANSACTION request has no corresponding BEGIN TRANSACTION
Go to Top of Page

Lumbago
Norsk Yak Master

3271 Posts

Posted - 2010-02-03 : 14:50:16
I didn't even consider the performance of this particular query, I was simply adding UNION ALL as a consideration. If there was an index on both Key1 and Key2 and the query was SELECT * FROM table WHERE Key1 = 1 OR Key2 = 5 then the UNION ALL would perform quite well in comparison.

- Lumbago
If the facts don't fit the theory, change the facts. Albert Einstein
Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2010-02-03 : 14:53:50
"Now the question remains: did I get the selection logic right?["

I fear not ...

I don't think it caters for @Key1=0 + @Key2='Something'

I think there are four use-cases:

@Key1=0, @Key2=0
@Key1=0, @Key2=Something
@Key1=Something, @Key2=0
@Key1=Something, @Key2=Something
Go to Top of Page
   

- Advertisement -