Jump to content

Strange Number Conversion Algorithm

Go to solution Solved by igormp,

Convert from hex to decimal, then divide by 10 (as said by 0.1º/dig).

Ex.: 0xEE -> 238 -> (divided by 10) 23.8

0x5DC -> 1500 -> (divided by 10) 150

Hey guys,

 

I am converting a temperature from a piece of equipment and the given unit conversion is very strange. I have sliced this problem up a couple of different ways but I am not getting anywhere. The given information is (see attached)
data type: unsigned 16 bit word

data range: -110.0degC to 150.0degC

input range: 0xFBB4 to 0x5DC

 

There is one example in the manual, 0x00EE == 23.8degC.

 

Any idea how to convert this to a C++ function to convert between the two?

 

Thanks.

Legend.PNG

Example.PNG

Link to comment
https://linustechtips.com/topic/1253175-strange-number-conversion-algorithm/
Share on other sites

Link to post
Share on other sites

Convert from hex to decimal, then divide by 10 (as said by 0.1º/dig).

Ex.: 0xEE -> 238 -> (divided by 10) 23.8

0x5DC -> 1500 -> (divided by 10) 150

FX6300 @ 4.2GHz | Gigabyte GA-78LMT-USB3 R2 | Hyper 212x | 3x 8GB + 1x 4GB @ 1600MHz | Gigabyte 2060 Super | Corsair CX650M | LG 43UK6520PSA
ASUS X550LN | i5 4210u | 12GB
Lenovo N23 Yoga

Link to post
Share on other sites

8 minutes ago, Pinguinsan said:

Thanks for your quick response! What about the negative range?
0xFBB4 -> 64436 -> 644.36?

Maybe take the sign bit out?

0x7BB4 -> 31668 -> 316.68?

Subtract 0xFBB4 with 0xFFFF (-0x44B), convert to decimal (-1099) then divide by 10 again (-109.9)

FX6300 @ 4.2GHz | Gigabyte GA-78LMT-USB3 R2 | Hyper 212x | 3x 8GB + 1x 4GB @ 1600MHz | Gigabyte 2060 Super | Corsair CX650M | LG 43UK6520PSA
ASUS X550LN | i5 4210u | 12GB
Lenovo N23 Yoga

Link to post
Share on other sites

So basically it's returning in tenths of a degree. If you're going to print that on a display or something you don't even need to convert anything, just drop a decimal dot in there. That's often done with microcontrollers to avoid floating point.

Link to post
Share on other sites

Agree with what @igormp solution.  One question though... @Pinguinsan  was your example in the manual correct?

 

Quote

0x00EE == 22.3degC.

Was it meant to be 23.8C?  If it is 22.3C, then there might be something else going on, aside from a linear relationship...but then again I'm inclined to believe it was a mistake and what ignormp's answer is correct.

 

For reference as well, they are storing the number as a 16 bit 2's complement [which pretty much every system under the planet uses...love the math behind two's complement number system].  In C++ terms that means, int16_t should actually match the value...so you could just convert the 4 bytes to an int16_t and then when you need the final number it's just a division by 10 (converting the type of course)

 

3735928559 - Beware of the dead beef

Link to post
Share on other sites

6 hours ago, wanderingfool2 said:

Agree with what @igormp solution.  One question though... @Pinguinsan  was your example in the manual correct?

 

Was it meant to be 23.8C?  If it is 22.3C, then there might be something else going on, aside from a linear relationship...but then again I'm inclined to believe it was a mistake and what ignormp's answer is correct.

 

For reference as well, they are storing the number as a 16 bit 2's complement [which pretty much every system under the planet uses...love the math behind two's complement number system].  In C++ terms that means, int16_t should actually match the value...so you could just convert the 4 bytes to an int16_t and then when you need the final number it's just a division by 10 (converting the type of course)

 

You are exactly right, thanks for pointing that out. I typed it out from memory without referring to the image. Updated.

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×