Double Precision is rounding at the 5th digit?
4 visualizzazioni (ultimi 30 giorni)
Mostra commenti meno recenti
(Newbie question)
I am trying to do some basic math and I am running into what appears to be a rounding or precision problem. MATLAB uses doubles, which should be good out to 15 digits. It appears that MATLAB is rounding at less than 4. What am I missing? Note: I do not have access to the Symbolic toolkit, and frankly can't understand why any additional code would be needed for this basic equaiton.
value1 = double(0.999969482421875)
eps(value1)
value2 = double(32767) ./ double(32768)
Output is
value1 =
1.0000
ans =
1.1102e-16
value2 =
1.0000
0 Commenti
Risposta accettata
Più risposte (0)
Vedere anche
Categorie
Scopri di più su Numeric Types in Help Center e File Exchange
Prodotti
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!