| Author |
Topic |
|
cmdcenter
Starting Member
15 Posts |
Posted - 2010-01-29 : 14:09:15
|
I wonder if there is any wise way to rewrite the following query so that the indexes on columns get used by optimizer?Create Procedure select_Proc1 @Key1 int=0, @Key2 int=0AsBEGIN Select key3 From Or_Table Where (@key1 =0 OR Key1 =@Key1) AND (@key2 =0 OR Key2 =@Key2)ENDGO Even though columns in the WHERE clauses are covered by indexes, SQL Server is unable to use these indexes. This raises the question as to whether anything is “blocking” the use of the indexes. The answer to this question is yes -- the culprits are the parameters and the “OR” condition. The parameters are not covered by indexes, which means SQL Server cannot use any of the indexes to evaluate “@key1=0” (a condition which also applies to @key2=0). Effectively, this means SQL Server cannot use indexes to evaluate the clause “@key1=0 OR Key1= @key1” (as the “OR” clause is the union of rows covered by both conditions). The same principle applies to the other clause (re. key2) as well. This leads SQL Server to conclude that no indexes can be used to extract the rows, leaving SQL Server to utilize the next best approach -- a clustered index scanAs you see, the SQL optimizer will not use indexes on columns if the predicates are "OR"ed in the WHERE clause. One solution for this problem, is to separate queries with IF clause for all possible combination of parameters.Please read this short article to get a better view of the problem: http://www.sql-server-performance.com/articles/per/optimize_or_clause_p1.aspxNow my question is, what should we do if the possible combinations are more that just three or four? Writing a separate query for each combination does not seem a rational solution. Is there any other workaround for this problem? |
|
|
Lamprey
Master Smack Fu Yak Hacker
4614 Posts |
Posted - 2010-01-29 : 15:36:44
|
| Although I'm not a fan of Dynamic SQL, it tends to perform the best for these "catch-all queries." |
 |
|
|
Bustaz Kool
Master Smack Fu Yak Hacker
1834 Posts |
Posted - 2010-01-29 : 19:29:56
|
| [CODE]Create Procedure select_Proc1 @Key1 int=0, @Key2 int=0AsBEGIN Select key3 From Or_Table Where (@key1 =0 OR Key1 =@Key1) AND (@key2 =0 OR Key2 =@Key2)OPTION (RECOMPILE)END[/CODE]This will force the query engine to look explicitly at your parameters and generate an optimized execution plan. For instance, in the case where @key = 1, it won't bother to see if key1 = @key1 since it will already know that the OR logic will pass.=======================================Few things are harder to put up with than the annoyance of a good example. (Mark Twain) |
 |
|
|
Kristen
Test
22859 Posts |
Posted - 2010-01-30 : 04:38:33
|
| Might SQL not take longer to make the Query Plan than to do the query (using an existing cache plan)? (perhaps not in this query, but in a more complex one)I've not used RECOMPILE for that reason, but maybe I should be using it?! |
 |
|
|
Lamprey
Master Smack Fu Yak Hacker
4614 Posts |
Posted - 2010-02-02 : 11:20:55
|
quote: Originally posted by Bustaz Kool [CODE]Create Procedure select_Proc1 @Key1 int=0, @Key2 int=0AsBEGIN Select key3 From Or_Table Where (@key1 =0 OR Key1 =@Key1) AND (@key2 =0 OR Key2 =@Key2)OPTION (RECOMPILE)END[/CODE]This will force the query engine to look explicitly at your parameters and generate an optimized execution plan. For instance, in the case where @key = 1, it won't bother to see if key1 = @key1 since it will already know that the OR logic will pass.
That will cause the stored procedure to recompile, to be sure. But, that won't affect how SQL handles OR clauses. |
 |
|
|
harsh_athalye
Master Smack Fu Yak Hacker
5581 Posts |
Posted - 2010-02-02 : 11:29:23
|
You can try this approach, not tested though:Select key3From Or_TableWhere Key1 = isnull(nullif(@Key1,0), Key1) ANDKey2 = isnull(nullif(@Key2,0), Key2) Harsh Athalyehttp://www.letsgeek.net/ |
 |
|
|
Kristen
Test
22859 Posts |
Posted - 2010-02-02 : 12:58:45
|
If you set @Key1=0 (i.e. "Don't Care) you won't get any row where KEY1is NULL using that method, will you?Also even more Index unfriendly I think? |
 |
|
|
harsh_athalye
Master Smack Fu Yak Hacker
5581 Posts |
Posted - 2010-02-02 : 13:18:55
|
True about first part, but then OP's approach also doesn't take into account NULL values either.About use of index, I guess if the columns are defined as NOT NULL and has index on them, the index will still be used despite columns being wrapped inside ISNULL function. But then I might be wrong. Harsh Athalyehttp://www.letsgeek.net/ |
 |
|
|
Kristen
Test
22859 Posts |
Posted - 2010-02-03 : 01:17:55
|
| Not sure I agree on first part Harsh, what do you think?[code]Where (@key1 =0 OR Key1 =@Key1) AND (@key2 =0 OR Key2 =@Key2)END[code]if @key1 = 0 then Key1 can be any value (including NULL), whereas[CODE]where Key1 = isnull(nullif(@Key1,0), Key1) AND Key2 = isnull(nullif(@Key2,0), Key2)[/CODE]If @Key1 = 0 then the test is "Key1 = Key1" (TRUE for all cases except Key1 IS NULL) otherwise Key1 is compared to @Key1 |
 |
|
|
harsh_athalye
Master Smack Fu Yak Hacker
5581 Posts |
Posted - 2010-02-03 : 01:53:53
|
| You are right, Kristen. I missed the part where NULL != NULL.Funny how easy it is to forget this important rule.Harsh Athalyehttp://www.letsgeek.net/ |
 |
|
|
Kristen
Test
22859 Posts |
Posted - 2010-02-03 : 02:01:40
|
Hehehe ... well there is always ANSI_NULLS |
 |
|
|
Lumbago
Norsk Yak Master
3271 Posts |
Posted - 2010-02-03 : 10:26:56
|
I have to admit I didn't read the entire article that you linked but have you considered UNION ALL? I know it's not overly practical for a changing amount of predicates but it might be worth considering. I changed an OR query to a UNION ALL and the query was trimmed from 20 seconds to less than a second Select key3From Or_TableWhere (@key1 =0) AND (@key2 =0)UNION ALLSelect key3From Or_TableWhere (Key1 =@Key1) AND (Key2 =@Key2) Now the question remains: did I get the selection logic right? (I probably didn't...hmm)- LumbagoIf the facts don't fit the theory, change the facts. Albert Einstein |
 |
|
|
Transact Charlie
Master Smack Fu Yak Hacker
3451 Posts |
Posted - 2010-02-03 : 12:33:54
|
A few different ways. Edit -- I don't think showplan shwos you dynamic sql. Therefore you'll just have to use the show actual execution plan which does.SET SHOWPLAN_ALL OFFGOIF OBJECT_ID('tempdb..#or_table') IS NOT NULL DROP TABLE #or_tableGOCREATE TABLE #or_table ( [ID] INT IDENTITY (1,1) PRIMARY KEY , [key1] INT , [key2] INT , [key3] INT )CREATE INDEX IX_OR_KEY1_KEY2 ON #or_table ([key1], [key2])GOINSERT #or_table ([key1], [key2], [key3])SELECT CAST(CAST(NEWID() AS VARBINARY(64)) AS INT) , CAST(CAST(NEWID() AS VARBINARY(64)) AS INT) , CAST(CAST(NEWID() AS VARBINARY(64)) AS INT)FROM (SELECT 1 AS [1] UNION SELECT 2 AS [2] UNION SELECT 3 AS [3] UNION SELECT 4 AS [4]) a CROSS JOIN (SELECT 1 AS [1] UNION SELECT 2 AS [2] UNION SELECT 3 AS [3] UNION SELECT 4 AS [4]) b CROSS JOIN (SELECT 1 AS [1] UNION SELECT 2 AS [2] UNION SELECT 3 AS [3] UNION SELECT 4 AS [4]) c CROSS JOIN (SELECT 1 AS [1] UNION SELECT 2 AS [2] UNION SELECT 3 AS [3] UNION SELECT 4 AS [4]) d CROSS JOIN (SELECT 1 AS [1] UNION SELECT 2 AS [2] UNION SELECT 3 AS [3] UNION SELECT 4 AS [4]) e CROSS JOIN (SELECT 1 AS [1] UNION SELECT 2 AS [2] UNION SELECT 3 AS [3] UNION SELECT 4 AS [4]) f CROSS JOIN (SELECT 1 AS [1] UNION SELECT 2 AS [2] UNION SELECT 3 AS [3] UNION SELECT 4 AS [4]) gGOSELECT COUNT(*) FROM #or_table AS [Records]GODECLARE @key1 INTDECLARE @key2 INTSET @key1 = 3SET @key2 = 4Select key3 From #Or_Table Where (@key1 =0 OR Key1 =@Key1) AND (@key2 =0 OR Key2 =@Key2)GODECLARE @key1 INTDECLARE @key2 INTSET @key1 = 3SET @key2 = 4SELECT key3FROM #or_tableWHERE key1 = CASE @key1 WHEN 0 THEN key1 ELSE @key1 END AND key2 = CASE @key2 WHEN 0 THEN key2 ELSE @key2 ENDGODECLARE @sql NVARCHAR(4000)DECLARE @key1 INTDECLARE @key2 INTSET @key1 = 3SET @key2 = 4SET @sql = N'SELECT [key3]FROM #or_tableWHERE 1=1'IF @key1 > 0 SET @sql = @sql + N' AND key1 = @key1'IF @key2 > 0 SET @sql = @sql + N' AND key2 = @key2'EXEC sp_ExecuteSql @sql , N'@key1 INT , @key2 INT' , @key1 , @key2GODECLARE @key1 INTDECLARE @key2 INTSET @key1 = 3SET @key2 = 4Select key3From #Or_TableWhere (@key1 =0) AND (@key2 =0)UNION ALLSelect key3From #Or_TableWhere (Key1 =@Key1) AND (Key2 =@Key2) |
 |
|
|
Transact Charlie
Master Smack Fu Yak Hacker
3451 Posts |
Posted - 2010-02-03 : 12:37:08
|
| Lumbago --You union all seems to do both an index scan and then an index seekprobably not that performant!Charlie===============================================================Msg 3903, Level 16, State 1, Line 1736The ROLLBACK TRANSACTION request has no corresponding BEGIN TRANSACTION |
 |
|
|
Lumbago
Norsk Yak Master
3271 Posts |
Posted - 2010-02-03 : 14:50:16
|
| I didn't even consider the performance of this particular query, I was simply adding UNION ALL as a consideration. If there was an index on both Key1 and Key2 and the query was SELECT * FROM table WHERE Key1 = 1 OR Key2 = 5 then the UNION ALL would perform quite well in comparison.- LumbagoIf the facts don't fit the theory, change the facts. Albert Einstein |
 |
|
|
Kristen
Test
22859 Posts |
Posted - 2010-02-03 : 14:53:50
|
"Now the question remains: did I get the selection logic right?["I fear not ... I don't think it caters for @Key1=0 + @Key2='Something'I think there are four use-cases:@Key1=0, @Key2=0@Key1=0, @Key2=Something@Key1=Something, @Key2=0@Key1=Something, @Key2=Something |
 |
|
|
|