Get rid of far right decimals for comparison

Started by twgonder, November 23, 2021, 18:39:34

Previous topic - Next topic

twgonder

I'm working with fractions multiplied against big numbers.
Sometimes I get a comparison something like this:
10 <> 10.0000000539... (is true, they are not equal)
Is there an easy way to stop the comparison at maybe the 6th digit of the decimal, so that the comparison would then be false?
Thanks

TomToad

Easiest way would be to check the difference of the numbers and see if it is less than a small amount.
If Abs(A-B) < .0000001 Then Print "Equal" Else Print "Not Equal"
------------------------------------------------
8 rabbits equals 1 rabbyte.

twgonder

#2
I can live with that solution. I must be too used to functions.  ::)
Thanks

Baggey

#3
Im not any good. But maybe you need to use Byte, Short etc,,!  :o

Baggey
Running a PC that just Aint fast enough!? i7 4Ghz Quad core 24GB ram 1TB SSD and NVIDIA Quadro K620 . DID Technology stop! Or have we been assimulated!

ZX Spectrum 48k, C64, ORIC Atmos 48K, Enterprise 128K, The SID chip. Im Misunderstood!

twgonder

Byte and short aren't keywords in the documentation.

dawlane

Most languages in this day and age have math functions to round floating point numbers either up or down.
These would be something like Round(number) to the nearest whole number either up or down, Floor(number) to round to the lowest whole number, or Ceiling to round up to the nearest whole number.