Please help urgent !!! Problem Data Type Float.

Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

All Forums

SQL Server 2005 Forums

Transact-SQL (2005)

Please help urgent !!! Problem Data Type Float.

Author

Topic

s0ftw4r32002
Starting Member

6 Posts

Posted - 2008-08-27 : 01:27:29

DECLARE @qty float
DECLARE @value float
SET @qty = 65300000.00
SET @value = 26491627.7334113
SELECT @value / @qty

result : 0.40569108320691

SELECT 26491627.7334113/65300000.00

result = 0.405691083206911

Why the result not same?
please help me urgent!!!

khtan
In (Som, Ni, Yak)

17689 Posts

Posted - 2008-08-27 : 01:30:43

because FLOAT is approximate value

You will get same result if you convert the value in 2nd query to float
SELECT convert(float, 26491627.7334113) / convert(float, 65300000.00)

KH
[spoiler]Time is always against us[/spoiler]

visakh16
Very Important crosS Applying yaK Herder

52326 Posts

Posted - 2008-08-27 : 01:34:31

quote:
Originally posted by s0ftw4r32002

DECLARE @qty float
DECLARE @value float
SET @qty = 65300000.00
SET @value = 26491627.7334113
SELECT @value / @qty

result : 0.40569108320691

SELECT 26491627.7334113/65300000.00

result = 0.405691083206911

Why the result not same?
please help me urgent!!!

if you want accurate results use Numeric or Decimal where you can specify scale and precision explicitly

s0ftw4r32002
Starting Member

6 Posts

Posted - 2008-08-27 : 01:35:47

quote:
Originally posted by khtan

because FLOAT is approximate value

You will get same result if you convert the value in 2nd query to float
SELECT convert(float, 26491627.7334113) / convert(float, 65300000.00)

KH
[spoiler]Time is always against us[/spoiler]

Thank you,

I want from the code :

DECLARE @qty float
DECLARE @value float
SET @qty = 65300000.00
SET @value = 26491627.7334113
SELECT @value / @qty

the result : 0.405691083206911

khtan
In (Som, Ni, Yak)

17689 Posts

Posted - 2008-08-27 : 01:38:08

use decimal or numeric data type as advice by visakh

KH
[spoiler]Time is always against us[/spoiler]

s0ftw4r32002
Starting Member

6 Posts

Posted - 2008-08-27 : 01:46:39

quote:
Originally posted by khtan

use decimal or numeric data type as advice by visakh

KH
[spoiler]Time is always against us[/spoiler]

if i use decimal data type. please tell me about settings scale and precision for result 0.405691083206911

s0ftw4r32002
Starting Member

6 Posts

Posted - 2008-08-27 : 01:52:23

DECLARE @qty float
DECLARE @value float
DECLARE @cogs float
SET @qty = 65300000.00
SET @value = 26491627.7334113 ---> original value
set @cogs = @value / @qty

select @qty * @cogs

result : 26491627.7334112

why the result not back orginal value?

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2008-08-27 : 02:04:09

because FLOAT is approximate value

E 12°55'05.25"
N 56°04'39.16"

khtan
In (Som, Ni, Yak)

17689 Posts

Posted - 2008-08-27 : 02:38:52

[code]DECLARE @qty float
DECLARE @value float
SET @qty = 65300000.00
SET @value = 26491627.7334113
select 0.405691083206911 as [desired], round(@value / @qty, 15) as [calculated]
[/code]

KH
[spoiler]Time is always against us[/spoiler]

Subscribe to SQLTeam.com

SQLTeam.com Articles via RSS

SQLTeam.com Weblog via RSS

- Advertisement -

Resources