Jump to content

Strange Number Conversion Algorithm

Pinguinsan
Go to solution Solved by igormp,

Convert from hex to decimal, then divide by 10 (as said by 0.1º/dig).

Ex.: 0xEE -> 238 -> (divided by 10) 23.8

0x5DC -> 1500 -> (divided by 10) 150

Hey guys,

 

I am converting a temperature from a piece of equipment and the given unit conversion is very strange. I have sliced this problem up a couple of different ways but I am not getting anywhere. The given information is (see attached)
data type: unsigned 16 bit word

data range: -110.0degC to 150.0degC

input range: 0xFBB4 to 0x5DC

 

There is one example in the manual, 0x00EE == 23.8degC.

 

Any idea how to convert this to a C++ function to convert between the two?

 

Thanks.

Legend.PNG

Example.PNG

Link to comment
Share on other sites

Link to post
Share on other sites

Convert from hex to decimal, then divide by 10 (as said by 0.1º/dig).

Ex.: 0xEE -> 238 -> (divided by 10) 23.8

0x5DC -> 1500 -> (divided by 10) 150

FX6300 @ 4.2GHz | Gigabyte GA-78LMT-USB3 R2 | Hyper 212x | 3x 8GB + 1x 4GB @ 1600MHz | Gigabyte 2060 Super | Corsair CX650M | LG 43UK6520PSA
ASUS X550LN | i5 4210u | 12GB
Lenovo N23 Yoga

Link to comment
Share on other sites

Link to post
Share on other sites

Thanks for your quick response! What about the negative range?
0xFBB4 -> 64436 -> 644.36?

Maybe take the sign bit out?

0x7BB4 -> 31668 -> 316.68?

Link to comment
Share on other sites

Link to post
Share on other sites

8 minutes ago, Pinguinsan said:

Thanks for your quick response! What about the negative range?
0xFBB4 -> 64436 -> 644.36?

Maybe take the sign bit out?

0x7BB4 -> 31668 -> 316.68?

Subtract 0xFBB4 with 0xFFFF (-0x44B), convert to decimal (-1099) then divide by 10 again (-109.9)

FX6300 @ 4.2GHz | Gigabyte GA-78LMT-USB3 R2 | Hyper 212x | 3x 8GB + 1x 4GB @ 1600MHz | Gigabyte 2060 Super | Corsair CX650M | LG 43UK6520PSA
ASUS X550LN | i5 4210u | 12GB
Lenovo N23 Yoga

Link to comment
Share on other sites

Link to post
Share on other sites

So basically it's returning in tenths of a degree. If you're going to print that on a display or something you don't even need to convert anything, just drop a decimal dot in there. That's often done with microcontrollers to avoid floating point.

Link to comment
Share on other sites

Link to post
Share on other sites

Agree with what @igormp solution.  One question though... @Pinguinsan  was your example in the manual correct?

 

Quote

0x00EE == 22.3degC.

Was it meant to be 23.8C?  If it is 22.3C, then there might be something else going on, aside from a linear relationship...but then again I'm inclined to believe it was a mistake and what ignormp's answer is correct.

 

For reference as well, they are storing the number as a 16 bit 2's complement [which pretty much every system under the planet uses...love the math behind two's complement number system].  In C++ terms that means, int16_t should actually match the value...so you could just convert the 4 bytes to an int16_t and then when you need the final number it's just a division by 10 (converting the type of course)

 

3735928559 - Beware of the dead beef

Link to comment
Share on other sites

Link to post
Share on other sites

6 hours ago, wanderingfool2 said:

Agree with what @igormp solution.  One question though... @Pinguinsan  was your example in the manual correct?

 

Was it meant to be 23.8C?  If it is 22.3C, then there might be something else going on, aside from a linear relationship...but then again I'm inclined to believe it was a mistake and what ignormp's answer is correct.

 

For reference as well, they are storing the number as a 16 bit 2's complement [which pretty much every system under the planet uses...love the math behind two's complement number system].  In C++ terms that means, int16_t should actually match the value...so you could just convert the 4 bytes to an int16_t and then when you need the final number it's just a division by 10 (converting the type of course)

 

You are exactly right, thanks for pointing that out. I typed it out from memory without referring to the image. Updated.

Link to comment
Share on other sites

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×