Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 General SQL Server Forums
 Database Design and Application Architecture
 Numeric Data Type Precision (Rhetorical Question?)

Author  Topic 

readysetstop
Posting Yak Master

123 Posts

Posted - 2007-04-12 : 08:25:11
I'm reading the MS Training Kit for SQL 2005 Admin, Exam 70-431, (no debating certs, please), and I come across this gem, discussing numeric data types such as bigint, int, smallint, decimal, numeric, etc...

"The storage is also precisely defined, so any data stored in these data types returns and calculates to the same value on either an Intel or an AMD processor and architecture."

So exactly how precise would you have to be to notice a difference between architectures?

AndrewMurphy
Master Smack Fu Yak Hacker

2916 Posts

Posted - 2007-04-12 : 09:41:54
hmmmm....vague alright. would be useful to have an example to go with that statement.
Only thing I can think of is that it may be inferring that how "Floats" get evaluated, may be different at the microcode level???
Go to Top of Page

readysetstop
Posting Yak Master

123 Posts

Posted - 2007-04-12 : 10:35:29
That's quite possible. The current specification for SQL 2005 is that the 'decimal' type, which SQL considers an "Exact Numeric" data type, goes out to 38 places. 'Float' on the other hand, is considered an "Approximate Numeric" type, and goes out to 308 places. Aside: 'Float' replaced 'real' in order to conform to the SQL-92 standard. 'Real' goes out to 38 places. 'Float' is the only numeric type in SQL with that large a significance. Everything else is 38 places or less.

I could easily see where an accuracy of a couple hundred decimal places could have errors at the micro-level.

Go to Top of Page
   

- Advertisement -