Value not expected, not understanding

Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

All Forums

SQL Server 2005 Forums

Transact-SQL (2005)

Value not expected, not understanding

Author

Topic

dpcolgan
Starting Member

4 Posts

Posted - 2008-10-29 : 20:20:51

Help!
I have a stored procedure line that is driving me batty.

DECLARE @TOTALMONTHSEMPLOYED FLOAT
DECLARE @CALCMONTHSSERVICE FLOAT

@TOTALMONTHSEMPLOYED = 0.0
@CALCMONTHSSERVICE = 2.0

SET @TOTALMONTHSEMPLOYED = (ISNULL(@TOTALMONTHSEMPLOYED,0) + ISNULL(@CALCMONTHSSERVICE,0))
SELECT @TOTALMONTHSEMPLOYED
SET @TOTALMONTHSEMPLOYED = CONVERT(FLOAT,(@TOTALMONTHSEMPLOYED) /30)
SELECT @TOTALMONTHSEMPLOYED

The return is giving me 6.6666666666666666E-2

what is going on?

sodeep
Master Smack Fu Yak Hacker

7174 Posts

Posted - 2008-10-29 : 21:15:20

quote:
Originally posted by dpcolgan

Help!
I have a stored procedure line that is driving me batty.

DECLARE @TOTALMONTHSEMPLOYED FLOAT
DECLARE @CALCMONTHSSERVICE FLOAT

SET @TOTALMONTHSEMPLOYED = 0.0
SET @CALCMONTHSSERVICE = 2.0

SET @TOTALMONTHSEMPLOYED = (ISNULL(@TOTALMONTHSEMPLOYED,0) + ISNULL(@CALCMONTHSSERVICE,0))
SELECT @TOTALMONTHSEMPLOYED
SET @TOTALMONTHSEMPLOYED = CONVERT(FLOAT,(@TOTALMONTHSEMPLOYED) /30)
SELECT @TOTALMONTHSEMPLOYED

The return is giving me 6.6666666666666666E-2

what is going on?

why are converting it to float again? I got:0.0666666666666667

visakh16
Very Important crosS Applying yaK Herder

52326 Posts

Posted - 2008-10-30 : 04:00:21

if you're looking for accurate decimal result convert it to decimal rather than float.

Subscribe to SQLTeam.com

SQLTeam.com Articles via RSS

SQLTeam.com Weblog via RSS

- Advertisement -

Resources