Formulas & precision - SQL Server Forums

Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

All Forums

General SQL Server Forums

New to SQL Server Programming

Formulas & precision

Author

Topic

Bluespud
Starting Member

5 Posts

Posted - 2006-09-21 : 12:58:53

I have a table with a 'quantity' column (decimal 9:3) and a 'price' column (9:3). I have a third column 'amount' with a formula of (price * quantity). The formula gives the correct answer, but the precision is automatically set to 5. Is there any way to set the precision of the result to 2?

Thanks.

AndrewMurphy
Master Smack Fu Yak Hacker

2916 Posts

Posted - 2006-09-21 : 13:26:19

implement a "convert" on the formula to get the right scale.

snSQL
Master Smack Fu Yak Hacker

1837 Posts

Posted - 2006-09-21 : 14:19:46

Here's an example

declare @price decimal(9,3)
declare @qty decimal(9,3)
set @price = 12.5
set @qty = 15.25
select @price * @qty, cast(@price * @qty as decimal(9,2))

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2006-09-21 : 18:23:11

Why not a ROUND(@price * @qty, 2) ?

Peter Larsson
Helsingborg, Sweden

Bluespud
Starting Member

5 Posts

Posted - 2006-09-25 : 03:13:23

Thanks for the replies, all suggestions worked just fine.

Subscribe to SQLTeam.com

SQLTeam.com Articles via RSS

SQLTeam.com Weblog via RSS

- Advertisement -

Resources