round-off issue?

1 visualizzazione (ultimi 30 giorni)
Leor Greenberger
Leor Greenberger il 20 Set 2011
I can't figure out why the floor wont come to 1752?
>> 3.501/40*20*10^3+1.5
ans =
1.752000000000000e+003
>> floor(ans)
ans =
1751

Accedi per rispondere a questa domanda.

Risposta accettata

Fangjun Jiang
Fangjun Jiang il 20 Set 2011
Rounding error caused by floating point calculation.
>> 3.501/40*20*10
ans =
17.504999999999995
  2 Commenti
Jan
Jan il 20 Set 2011
sprintf('%.13f', 3.501/40*20*10^3+1.5)
>> 1751.9999999999998
sprintf('%.12f', 3.501/40*20*10^3+1.5)
>> 1752.000000000000
Fangjun Jiang
Fangjun Jiang il 20 Set 2011
Thanks, Jan! That's even better to explain the problem!

Accedi per commentare.

Più risposte (0)

Categorie

Scopri di più su Logical in Help Center e File Exchange

Tag

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by