Resolution of INT

Any general or miscellaneous queries that do not fit into the other forum catagories

Moderators: Benj, Mods

Post Reply
medelec35
Matrix Staff
Posts: 9520
Joined: Sat May 05, 2007 2:27 pm
Location: Northamptonshire, UK
Has thanked: 2585 times
Been thanked: 3815 times
Contact:

Resolution of INT

Post by medelec35 »

I am after a simple formula to calculate resolution of INT at 5V

E.G. if 8 bit Byte type, Resolution = 5/255 = 0.196
So If ADC I/P = 3V then Value = 255/5 x 3 = 153

So what number do replace 255 with if using INT instead of BYTE please?
Is it 16 Bit 65536 which = 5/65536 = 76 Microvolts?

So if 3V is used again. Value = 65536/5 x 3 =39321

If this is correct, is 39321 going to be actual value assigned if ADC I/P is 3V?

Update: I have just rembered: If using ADC within Flowcode then max bits = 2y10 = 1024
therefore 3V would give 1024/5 x 3 = 614.
Am I correct?
Martin

User avatar
Steve
Matrix Staff
Posts: 3422
Joined: Tue Jan 03, 2006 3:59 pm
Has thanked: 114 times
Been thanked: 422 times
Contact:

Re: Resolution of INT

Post by Steve »

almost - 1023

Post Reply