Author |
Topic |
Lumbago
Norsk Yak Master
3271 Posts |
Posted - 2006-09-06 : 04:49:15
|
I have created this UDF that will be hit on a massive scale during some data load operations and I need to find out if I can do some performance tuning on it. *Anything* that will increase performance on this baby is appreciated. What it basically does is take a number in varchar format and convert it to a decimal(18, 2). One thing though, if the last character in the passed variable is either } or between J and R it means that the number is negative and the caracter has to be replaced with a number equivalent and multiplied by -1! Please don't ask why...CREATE FUNCTION [dbo].[FindNegative] (@Number varchar(30)) RETURNS decimal(18, 2)ASBEGINIF @Number = '' SET @Number = NULLELSE BEGIN DECLARE @Letter varchar(1) DECLARE @NewNumber varchar(1) SET @Letter = RIGHT(@Number, 1) SELECT @NewNumber = CASE WHEN @Letter = '}' THEN '0' WHEN @Letter = 'J' THEN '1' WHEN @Letter = 'K' THEN '2' WHEN @Letter = 'L' THEN '3' WHEN @Letter = 'M' THEN '4' WHEN @Letter = 'N' THEN '5' WHEN @Letter = 'O' THEN '6' WHEN @Letter = 'P' THEN '7' WHEN @Letter = 'Q' THEN '8' WHEN @Letter = 'R' THEN '9' END IF @Letter <> @NewNumber BEGIN SET @Number = LEFT(@Number, LEN(@Number)-1) SET @Number = @Number + @NewNumber SET @Number = CONVERT(decimal(18, 2), @Number) * -1 END ELSE SET @Number = CONVERT(decimal(18, 2), @Number) ENDRETURN @NumberEND--> For testingDECLARE @table table (Col varchar(30))INSERT INTO @tableSELECT '000035}' UNION ALL SELECT '00045874' UNION ALL SELECT '00659871R'SELECT Col, dbo.FindNegative(Col)FROM @tableExpected output:Col NewCol------------------------------ ---------------------------------------000035} -35000045874 4587400659871R -6598719 --Lumbago"Real programmers don't document, if it was hard to write it should be hard to understand" |
|
SwePeso
Patron Saint of Lost Yaks
30421 Posts |
Posted - 2006-09-06 : 04:58:21
|
Why DECIMAL? Will INT not be enough?Peter LarssonHelsingborg, Sweden |
|
|
Lumbago
Norsk Yak Master
3271 Posts |
Posted - 2006-09-06 : 05:19:22
|
Problem is that some of the data coming in are defined as "numbers" in the source system, but can typically be bank account numbers and such and int just isn't big enough. I might be able to determine this on account of the record lengt but I have to think a bit about it...--Lumbago"Real programmers don't document, if it was hard to write it should be hard to understand" |
|
|
chiragkhabaria
Master Smack Fu Yak Hacker
1907 Posts |
Posted - 2006-09-06 : 05:25:46
|
How about this.. ??CREATE FUNCTION [dbo].[FindNegative] (@Number varchar(30)) RETURNS decimal(18, 2)ASBEGINIF @Number = '' SET @Number = NULLELSE BEGIN DECLARE @Letter varchar(1) DECLARE @NewNumber varchar(1) SET @Letter = RIGHT(@Number, 1) SELECT @Number = CASE WHEN @Letter = '}' THEN Convert(Decimal(18,2),Replace(@Number,'}','0')) * -1 WHEN ASCII(@LETTER) BETWEEN ASCII('J') AND ASCII('R') THEN Convert(Decimal(18,2),Replace(@Number,@Letter,Convert(char(1),1 + (ASCII(@Letter) - ASCII('J'))))) * -1 ELSE CONVERT(decimal(18, 2), @Number) End ENDRETURN @NumberENDGODECLARE @table table (Col varchar(30))INSERT INTO @tableSELECT '000035}' UNION ALL SELECT '00045874' UNION ALL SELECT '00659871R'SELECT Col, dbo.FindNegative(Col)FROM @table--Output.. Col ------------------------------ -------------------- 000035} -350.0000045874 45874.0000659871R -6598719.00 Chirag |
|
|
SwePeso
Patron Saint of Lost Yaks
30421 Posts |
Posted - 2006-09-06 : 05:35:46
|
Here is a version with only one cast/convert.CREATE FUNCTION dbo.FindNegative( @Number VARCHAR(30))RETURNS DECIMAL(18, 2)ASBEGIN IF LEN(@Number) = 0 OR @Number IS NULL RETURN NULL IF RIGHT(@Number, 1) IN ('}', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R') SELECT @Number = '-' + LEFT(@Number, LEN(@Number) - 1) + CASE WHEN RIGHT(@Number, 1) = '}' THEN '0' ELSE CHAR(ASCII(RIGHT(@Number, 1)) - 25) END RETURN CAST(@Number AS DECIMAL(18,2))END Peter LarssonHelsingborg, Sweden |
|
|
SwePeso
Patron Saint of Lost Yaks
30421 Posts |
Posted - 2006-09-06 : 05:47:10
|
Look! No CASE either...ALTER FUNCTION dbo.FindNegative( @Number VARCHAR(30))RETURNS DECIMAL(18, 2)ASBEGIN IF LEN(@Number) = 0 OR @Number IS NULL RETURN NULL IF RIGHT(@Number, 1) IN ('}', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R') SELECT @Number = '-' + LEFT(@Number, LEN(@Number) - 1) + CHAR(ASCII(RIGHT(@Number, 1)) % 50 + 25) RETURN CAST(@Number AS DECIMAL(18,2))END Peter LarssonHelsingborg, Sweden |
|
|
harsh_athalye
Master Smack Fu Yak Hacker
5581 Posts |
Posted - 2006-09-06 : 06:29:42
|
Does T-SQL supports Short-Circuit evaluation? I mean will the following two be equivalent in terms of performance:1. IF LEN(@Number) = 0 OR @Number IS NULL RETURN NULL2. IF LEN(IsNull(@Number,'')) = 0 RETURN NULLHarsh AthalyeIndia."Nothing is Impossible" |
|
|
SwePeso
Patron Saint of Lost Yaks
30421 Posts |
Posted - 2006-09-06 : 06:40:07
|
I don't know if SQL Server parses complete comparison before returning true, for either value is true.Peter LarssonHelsingborg, Sweden |
|
|
pootle_flump
1064 Posts |
Posted - 2006-09-06 : 06:53:44
|
quote: Originally posted by Lumbago *Anything* that will increase performance on this baby is appreciated.
Well - since you say *Anything* - SET is quicker than SELECT when only assigning a value to a single variable (SELECT @Var1 = 'Something', @Var2 = 'Something Else' is quicker than two SETs though). |
|
|
harsh_athalye
Master Smack Fu Yak Hacker
5581 Posts |
Posted - 2006-09-06 : 07:14:11
|
One more thing...Since these two function calls,RIGHT(@Number, 1) and LEN(@Number)are used twice, it is better to assign them to two local variable and use those variables instead of repeating function calls like below:ALTER FUNCTION dbo.FindNegative( @Number VARCHAR(30))RETURNS DECIMAL(18, 2)ASBEGIN DECLARE @SUFFIX VARCHAR(1) DECLARE @LEN INT SET @LEN = LEN(@Number) IF @LEN = 0 OR @Number IS NULL RETURN NULL SET @SUFFIX = RIGHT(@Number, 1) IF @SUFFIX IN ('}', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R') SELECT @Number = '-' + LEFT(@Number, @LEN - 1) + CHAR(ASCII(@SUFFIX) % 50 + 25) RETURN CAST(@Number AS DECIMAL(18,2))END Harsh AthalyeIndia."Nothing is Impossible" |
|
|
byrmol
Shed Building SQL Farmer
1591 Posts |
Posted - 2006-09-06 : 07:53:08
|
>> Does T-SQL supports Short-Circuit evaluation? No it does not..DavidMProduction is just another testing cycle |
|
|
harsh_athalye
Master Smack Fu Yak Hacker
5581 Posts |
Posted - 2006-09-06 : 07:57:19
|
quote: Originally posted by byrmol >> Does T-SQL supports Short-Circuit evaluation? No it does not..DavidMProduction is just another testing cycle
May I know how did you tested it or how did you determined it?Harsh AthalyeIndia."Nothing is Impossible" |
|
|
SwePeso
Patron Saint of Lost Yaks
30421 Posts |
Posted - 2006-09-06 : 08:05:12
|
Does this count?if 1 = 1 or (select avg(number) from f_table_number_range(1, 16777215) where number % 7 = 0) > 0 select 'true'else select 'false' Last statement takes several seconds to evaluate, but the code executes instantly.Peter LarssonHelsingborg, Sweden |
|
|
byrmol
Shed Building SQL Farmer
1591 Posts |
Posted - 2006-09-06 : 08:10:44
|
>>May I know how did you tested it or how did you determined it?use Northwindgodeclare @d intset @d = 1if 1=@d or exists (Select * from dbo.Employees where EmployeeID = 2) print 'Check the Execution Plan' The exists clause (second comparison) will always get evaluated...DavidMProduction is just another testing cycle |
|
|
byrmol
Shed Building SQL Farmer
1591 Posts |
Posted - 2006-09-06 : 08:15:07
|
>> if 1 = 1 The compiler spots the constant comparison, removes it and then evalutes the plan. Completely different from variable comparions.DavidMProduction is just another testing cycle |
|
|
Arnold Fribble
Yak-finder General
1961 Posts |
Posted - 2006-09-06 : 08:25:04
|
SET STATISTICS IO ONRun the Byrmol query...Table 'Employees'. Scan count 0, logical reads 0, physical reads 0, read-ahead reads 0.Don't look like it hit Employees to me. |
|
|
SwePeso
Patron Saint of Lost Yaks
30421 Posts |
Posted - 2006-09-06 : 08:26:20
|
[code]declare @test table (i int)insert @testselect 1 union allselect 2 union allselect 3 union allselect 4 union allselect 5select t.*from @test twhere t.i in (1,2,3,4,5) or (select avg(1.0 * q.number) from f_table_number_range(1, 16777215) q where q.number % t.i = 0) > 0[/code]Executes instantly! Run the subquery separately and time it. My time is 13 seconds for the subquery.And for 5 rows my test query should have taken over a minute to run, but it doesn't. It takes less than 0.1 second.Peter LarssonHelsingborg, Sweden |
|
|
chiragkhabaria
Master Smack Fu Yak Hacker
1907 Posts |
Posted - 2006-09-06 : 08:33:34
|
quote: Originally posted by Peso
declare @test table (i int)insert @testselect 1 union allselect 2 union allselect 3 union allselect 4 union allselect 5select t.*from @test twhere t.i in (1,2,3,4,5) or (select avg(1.0 * q.number) from f_table_number_range(1, 16777215) q where q.number % 7 = 0) > 0 Executes instantly! Run the subquery separately and time it. My time is 13 seconds for the subquery. And for 5 rows my test query should have taken over a minute to run, but it doesn't. It takes less than 0.1 second.Peter LarssonHelsingborg, Sweden
]I ran the subquery separetly after 11 sec its gave me select avg(q.number) from f_table_number_range(1, 16777215) q where q.number % 7 = 0 AFter 11 secs i got this error. quote: Arithmetic overflow error converting expression to data type int.
Chirag |
|
|
SwePeso
Patron Saint of Lost Yaks
30421 Posts |
Posted - 2006-09-06 : 08:35:14
|
Yes. I changed the AVG to include "1.0 *" too.And I changed the MOD operation to be not a constant but the table value for not excused with same argument that the subquery is being cached.But in fact, your finding is even better! That proofs that the query is never run since the complete query does not return an error.Peter LarssonHelsingborg, Sweden |
|
|
harsh_athalye
Master Smack Fu Yak Hacker
5581 Posts |
Posted - 2006-09-06 : 08:43:49
|
I think it does Short-Circuit evaluation as evident from the execution plans of following queries:use Northwindgodeclare @d int-- This will both conditions to be evaluatedif 1=1 and exists(select * from orders) print 'Check the Execution Plan'-- Short-Circuit evaluationif 1=0 and exists(select * from orders) print 'Check the Execution Plan'-- Short-Circuit evaluationif 1 = 1 or exists(select * from orders) print 'Check the Execution Plan'However, same is not the case when variable is used in the condition (at least not in execution plan)However, as Arnold pointed out, even then it doesn't actually hit the table, just show up in the execution plan.Harsh AthalyeIndia."Nothing is Impossible" |
|
|
Michael Valentine Jones
Yak DBA Kernel (pronounced Colonel)
7020 Posts |
Posted - 2006-09-06 : 08:50:17
|
quote: Originally posted by Lumbago...if the last character in the passed variable is either } or between J and R it means that the number is negative and the caracter has to be replaced with a number equivalent and multiplied by -1!...
So you decided to go with your own custom datatype conversation for the COBOL data?I hope you don’t run into any numbers in any of the various COMP formats.CODO ERGO SUM |
|
|
Next Page
|