Digital Data, Arithmetic, and Processor Fundamentals

Introduction

A microprocessor is fundamentally a machine for representing, moving, transforming, and interpreting patterns of bits. Instructions, addresses, sensor readings, program variables, text, and control information are all encoded as binary data. The quality of a processor-based design therefore depends on understanding not only logic circuits and instruction sets, but also the numerical limits and representation choices that govern computation.

This chapter develops the foundations required to analyze digital data in microprocessors and microcontrollers. It explains binary and hexadecimal notation, Boolean operations, unsigned and signed integers, fixed- and floating-point arithmetic, character codes, byte ordering, and arithmetic status flags. Particular attention is given to overflow, underflow, truncation, rounding, and loss of precision—failures that can produce serious engineering errors even when the hardware operates exactly as designed.

Learning Objectives

After studying this chapter, you should be able to:

• convert among binary, hexadecimal, decimal, and positional representations;
• apply Boolean operations to individual bits and bit fields;
• interpret unsigned, sign-magnitude, one’s-complement, and two’s-complement numbers;
• perform and analyze fixed-point and floating-point arithmetic;
• distinguish carry, borrow, overflow, zero, negative, and related status flags;
• explain ASCII, Unicode, and encoded character storage;
• determine the effect of little-endian and big-endian byte ordering;
• identify numerical overflow, underflow, rounding, and precision limitations;
• select suitable representations for practical embedded-system applications.

1. Data as Bit Patterns

A binary digit, or bit, has two logical states, conventionally written 0 and 1. A group of eight bits is a byte. A processor may manipulate 8-bit, 16-bit, 32-bit, or 64-bit quantities, although the internal datapath width and the memory access width need not always be identical.

For a positional number with digits dₙ₋₁ … d₁d₀ in radix r, the value is:

Here, r is the radix, n is the number of digits, and each digit satisfies 0 ≤ dᵢ < r. The formula applies to unsigned positional notation. A fractional point extends the same principle to negative powers of r.

Binary and hexadecimal notation

Binary uses radix 2 and the digits 0 and 1. Hexadecimal uses radix 16 and the symbols 0–9 and A–F, where A represents 10 and F represents 15. One hexadecimal digit corresponds exactly to four binary bits, called a nibble.

For example:

• 10110110₂ = 128 + 32 + 16 + 4 + 2 = 182₁₀;
• B6₁₆ = 11 × 16 + 6 = 182₁₀;
• 0xB6 is a common programming-language notation for the same hexadecimal value.

Binary-to-hexadecimal conversion is performed by grouping bits from the radix point outward in groups of four. Thus 110101011011₂ becomes D5B₁₆. Leading zeros may be added without changing the value, so an 8-bit quantity is commonly written with exactly two hexadecimal digits.

For an n-bit unsigned quantity, the representable range is:

The number of distinct patterns is 2ⁿ. For example, an 8-bit unsigned value ranges from 0 to 255, while a 16-bit value ranges from 0 to 65,535.

Binary arithmetic

Binary addition follows the same positional rules as decimal addition:

• 0 + 0 = 0;
• 0 + 1 = 1;
• 1 + 0 = 1;
• 1 + 1 = 10₂, producing a sum bit of 0 and a carry of 1.

Subtraction may be implemented directly or transformed into addition using two’s complement. Multiplication by 2 shifts an unsigned binary value one place to the left, provided significant bits are not discarded. Division by 2 shifts right, but the interpretation of a right shift depends on whether the number is unsigned or signed.

2. Boolean Operations and Bit Manipulation

Boolean algebra operates on variables that have values 0 or 1. The principal operations are AND, OR, NOT, and exclusive OR (XOR).

NOT inverts one input: NOT 0 = 1 and NOT 1 = 0. AND is useful for masking selected bits, OR for setting bits, XOR for toggling bits, and NOT for forming complements.

For example, suppose an 8-bit control register contains 10100110₂ and a mask is 00001111₂. AND gives 00000110₂, isolating the low nibble. OR with 00001000₂ sets bit 3. XOR with the same mask toggles the selected positions.

A left shift by k positions is commonly written x << k. For unsigned arithmetic, it is equivalent to multiplication by 2^k only when no significant bit is lost. A logical right shift inserts zeros at the left. An arithmetic right shift replicates the sign bit and is commonly used for signed two’s-complement values.

Boolean operations are central to peripheral control. A microcontroller may use one bit to enable an interrupt, another to indicate data readiness, and several bits to select an operating mode. Correct masking is essential: writing an entire register when only one control bit should change can unintentionally alter unrelated hardware functions.

3. Signed Integer Representations

Unsigned notation cannot represent negative values. Several signed representations have been used historically and remain important for understanding processor behavior.

Sign-magnitude representation

The most significant bit indicates the sign: 0 for positive and 1 for negative. The remaining bits represent magnitude. In an 8-bit sign-magnitude system, +13 is 00001101₂ and −13 is 10001101₂. This representation has both +0 (00000000) and −0 (10000000), which complicates arithmetic and comparison.

One’s complement

A negative number is formed by inverting every bit of its positive counterpart. For example, +13 is 00001101, so −13 is 11110010. One’s complement also has two zeros and requires end-around carry in some additions.

Two’s complement

Modern processors overwhelmingly use two’s complement. A negative value is obtained by inverting all bits and adding one. For 8 bits:

+13 = 00001101

invert → 11110010

add 1 → −13 = 11110011

The range for an n-bit two’s-complement integer is:

Thus, 8-bit signed integers range from −128 to +127. There is one zero, and addition and subtraction use the same binary adder as unsigned arithmetic. This hardware simplicity is a major reason for the dominance of two’s complement.

To interpret an n-bit pattern, if its most significant bit is 0, interpret it as unsigned. If that bit is 1, subtract 2ⁿ from its unsigned value. For example, 11111010₂ is 250 unsigned; 250 − 256 = −6 signed.

Sign extension and truncation

When a signed value is widened, its sign bit is copied into the new high-order positions. For example, 8-bit −6, 11111010, becomes 16-bit 1111111111111010. This preserves value. Zero extension must be used for unsigned values.

Reducing width discards high-order bits. The result is correct only if the discarded bits are consistent with the intended representation. Truncating a 16-bit integer to 8 bits may produce wraparound, loss of sign, or an entirely different value.

4. Integer Arithmetic and Status Flags

Processor arithmetic usually updates a status or condition-code register. Common flags include:

Flags are architecture-specific. In particular, subtraction carry/borrow conventions differ, so programmers must consult the processor manual rather than assume a universal definition.

Carry versus signed overflow

Carry and overflow answer different questions. Carry concerns an unsigned interpretation; overflow concerns a signed interpretation.

Example 1: 8-bit addition 250 + 10. The binary operation produces 4 with a carry out, so unsigned arithmetic wraps modulo 256. The signed interpretations are −6 + 10 = 4, which is representable; therefore signed overflow is not indicated.

Example 2: 127 + 1 in 8-bit two’s complement produces 10000000, interpreted as −128. There is no meaningful signed result in the range −128 to 127, so overflow is set. The unsigned operation is 127 + 1 = 128 and does not carry out of the eighth bit.

For addition of two signed two’s-complement operands, overflow occurs when the operands have the same sign and the result has the opposite sign. An equivalent bit-level condition is:

Here A, B, and R are the sign bits of the two operands and result; ¬ denotes NOT and ⊕ denotes XOR. This expression applies to fixed-width two’s-complement addition.

For subtraction, signed overflow occurs when the operands have different signs and the result’s sign differs from the first operand’s sign. Comparisons and multiword arithmetic depend on the processor’s precise flag definitions.

5. Fixed-Point Representation

Fixed-point notation allocates a predetermined number of bits to the fractional part. If an unsigned integer field stores I and the design specifies F fractional bits, the represented value is:

The scaling factor is 2^F. The formula assumes the stored field is interpreted as an unsigned fixed-point value; signed formats require a specified signed interpretation.

For an 8-bit unsigned format with four integer bits and four fractional bits, often called Q4.4, the stored pattern 01011010₂ has integer value 90 and represents 90/16 = 5.625. Its resolution is 2⁻⁴ = 0.0625.

In a signed two’s-complement Qm.n format, n bits are fractional and the remaining bits, including the sign bit, determine the signed range. The stored integer is scaled by 2⁻ⁿ. Addition is straightforward when operands use the same scaling. Multiplication requires rescaling because the product contains twice as many fractional bits.

Fixed point is attractive in microcontrollers because it uses predictable memory, deterministic execution time, and often less hardware than floating point. It is widely used in motor control, digital filters, audio processing, and sensor algorithms. Its main challenges are selecting sufficient range and resolution and preventing intermediate overflow.

6. Floating-Point Representation

Floating-point represents a value approximately as:

Here s is the sign bit, m is the significand (or mantissa), b is the base, and e is the exponent. In common binary formats, the significand is normalized and the base is 2. IEEE 754 formats define binary32 and binary64, commonly called single and double precision.

A normalized binary floating-point number has a sign, biased exponent, and fraction field. For normal values in binary32, the effective significand is 1.fraction₂, and the exponent is obtained by subtracting the bias 127 from the stored exponent. Special encodings represent zero, subnormal numbers, infinity, and NaN (not-a-number).

Floating point provides a very large dynamic range, but most decimal fractions cannot be represented exactly in binary. For example, 0.1₁₀ has a repeating binary expansion. Consequently, a calculation such as repeated addition of 0.1 may yield a value close to, but not exactly, the expected decimal result.

The spacing between representable floating-point numbers grows as magnitude grows. If a format has p significant binary digits, the relative rounding error for correctly rounded basic operations is commonly bounded on the order of one half of one unit in the last place, subject to the operation, rounding mode, and exceptional conditions. This is not a guarantee that arbitrary algorithms have small total error: errors can accumulate or be amplified by subtraction of nearly equal quantities.

Floating-point arithmetic can produce overflow, underflow, loss of significance, cancellation, and comparison problems. Equality tests should generally use a tolerance appropriate to the scale and conditioning of the calculation, rather than assuming exact equality.

7. Character Codes and Encoded Text

A character code maps symbols to numeric code points or byte sequences. ASCII assigns 7-bit codes to English letters, decimal digits, punctuation, and control characters. For example, uppercase A is hexadecimal 41 and lowercase a is hexadecimal 61; the difference is 0x20 in ASCII.

Unicode provides a universal code space for scripts and symbols. UTF-8 encodes Unicode code points using one to four bytes and preserves ASCII as a subset. UTF-16 uses one or two 16-bit code units, while UTF-32 uses one 32-bit code unit per code point. A user-perceived character may consist of multiple code points, so “number of characters” is not always the same as number of bytes or code units.

Embedded systems must distinguish character encoding from numeric value. The byte 0x31 represents the character ‘1’ in ASCII, not the integer value 1. Converting between text and numbers requires explicit parsing or formatting. Incorrect assumptions about encoding can cause corrupted messages, security vulnerabilities, or interoperability failures.

8. Endianness and Memory Organization

A multi-byte value must be arranged in memory in some byte order. In little-endian order, the least significant byte is stored at the lowest address. In big-endian order, the most significant byte is stored at the lowest address.

Suppose the 32-bit value is 0x12345678 and it begins at address 0x1000:

Endianness affects memory inspection, communication protocols, file formats, DMA buffers, and type-punning. It does not normally change the abstract numerical value when a processor loads a correctly aligned value using its own rules. Network protocols often specify network byte order, traditionally big endian, even when communicating hosts use different native orders.

A robust interface specifies field widths, byte order, alignment, signedness, and encoding. Serialization and deserialization code should perform explicit conversions rather than relying on compiler layout or machine architecture.

9. Arithmetic Limitations and Reliable Design

Overflow and wraparound

Unsigned fixed-width arithmetic is usually modular. For n bits:

This statement applies when the processor discards the carry out and the operation is interpreted as unsigned. Modular behavior is useful for timers and circular buffers, but dangerous for array sizes, financial quantities, physical limits, and security checks.

Signed overflow in languages such as C and C++ may have language-level consequences different from hardware wraparound. Embedded developers must distinguish the processor instruction’s behavior from the programming language’s rules and compiler assumptions.

Precision and rounding

Integer division discards or rounds the fractional component according to the language and signedness rules. Fixed-point calculations may need guard bits, saturation, or carefully ordered operations. Floating-point calculations require attention to units, scale, rounding mode, and numerical conditioning.

Saturation arithmetic clamps a result to the largest or smallest representable value instead of wrapping. It is often safer in audio and control systems. However, saturation can hide an out-of-range condition unless a diagnostic flag or error path is also provided.

Practical best practices

1. Specify width, signedness, units, scaling, and valid range for every externally visible quantity.
2. Use wider intermediate types when multiplying or accumulating.
3. Check overflow before narrowing conversions.
4. Use explicit masks and shifts for register fields.
5. Document endianness in every binary interface.
6. Test boundary values: zero, maximum, minimum, sign transitions, and carry boundaries.
7. Avoid comparing floating-point values for exact equality unless exactness is guaranteed.
8. Use static analysis, compiler warnings, assertions, and property-based tests where appropriate.
9. Treat malformed encodings and out-of-range inputs as expected fault cases.

Applied Case: An Embedded Temperature Controller

Background

Consider an illustrative 16-bit microcontroller controlling a heating element. A sensor produces temperature in hundredths of a degree Celsius. The controller stores this value as a signed integer with a scale of 0.01 °C. Thus, 25.37 °C is stored as 2537.

Problem

The controller must compare temperature with a set point, transmit measurements over a serial link, and prevent the heater from operating when the sensor reading is invalid. Engineers observe occasional incorrect readings during communication and an unsafe response near the maximum permitted temperature.

Analysis

The sensor value is signed because diagnostic codes may be negative. A 16-bit signed integer ranges only from −32,768 to 32,767, corresponding to −327.68 °C through 327.67 °C at the selected scale. If an intermediate calculation multiplies temperature by a calibration gain before division, it may exceed this range even when the final result would not.

The serial protocol transmits the 16-bit value as two bytes, least significant byte first. A receiver that assumes big-endian order will interpret 0x09E9 (2537) as 0xE909, a negative and entirely different value under two’s complement.

Approach and solution

The design specifies the signed Q-style scale explicitly, uses a 32-bit intermediate for calibration, checks the valid sensor range before conversion, and serializes with named little-endian conversion functions. The heater command uses saturation and an independent fault flag. Boundary tests include −327.68 °C, 327.67 °C, 0 °C, 25.37 °C, and values just outside the permitted range.

Outcome and learning points

The communication error is eliminated because byte order is defined at the interface. The calibration operation no longer wraps because intermediate width is sufficient. Saturation prevents an unsafe control command, while the fault flag ensures that saturation is not mistaken for a valid measurement. The case demonstrates that representation, arithmetic width, encoding, and interface design are inseparable parts of reliable embedded engineering.

Contemporary Relevance

Modern microcontrollers increasingly combine integer cores with hardware floating-point units, digital signal-processing instructions, cryptographic accelerators, and machine-learning extensions. These features improve performance but do not remove representation risks. AI and signal-processing workloads may use reduced-precision formats such as 16-bit floating point or integer quantization to reduce memory and energy consumption. Such methods require calibration, error analysis, and validation against application-level safety requirements.

In safety-critical systems, numerical assumptions should be traceable to requirements. Standards and organizational processes may require range analysis, defensive coding, independent review, and evidence that arithmetic faults are detected. The exact applicable standard depends on the application domain, such as automotive, medical, industrial, or aerospace systems.

Chapter Summary

Digital systems represent all information as bit patterns, but interpretation depends on radix, width, signedness, scaling, encoding, and byte order. Binary and hexadecimal notation provide compact ways to inspect processor data. Boolean operations support masking, setting, clearing, and testing register fields.

Unsigned n-bit values range from 0 to 2ⁿ − 1, while n-bit two’s-complement values range from −2ⁿ⁻¹ to 2ⁿ⁻¹ − 1. Carry indicates an unsigned boundary condition; signed overflow indicates that a mathematical result lies outside the signed range. Fixed-point arithmetic offers deterministic scale and efficient implementation, while floating point provides broad dynamic range at the cost of rounding and precision concerns.

ASCII and Unicode encode characters rather than ordinary numeric quantities. Endianness determines byte order in memory and communication. Reliable processor design requires explicit specifications, adequate intermediate width, range checks, correct serialization, and tests at representation boundaries.

Key Terms

Bit: A binary digit with value 0 or 1.

Byte: A group of eight bits.

Radix: The base of a positional number system.

Nibble: A group of four bits.

Mask: A bit pattern used to select or modify particular bits.

Unsigned integer: A nonnegative fixed-width binary value.

Two’s complement: The dominant signed-integer representation formed by bit inversion followed by addition of one.

Sign extension: Copying a signed value’s sign bit into added high-order positions.

Carry: A carry out of a fixed-width addition or an architecture-defined subtraction condition.

Overflow: A condition in which a signed mathematical result cannot be represented at the selected width.

Fixed point: A representation with a predetermined fractional-bit position.

Floating point: A representation using sign, significand, and exponent fields.

Quantization: Mapping a continuous or higher-precision value to discrete representable levels.

Rounding: Selecting a representable value near an exact result.

ASCII: A 7-bit character code for basic English text and control characters.

Unicode: A universal character-code standard.

Endianness: The ordering of bytes within a multi-byte value.

Saturation: Clamping a result to the representable minimum or maximum.

Significance: The meaningful precision carried by a numerical representation.

Review Questions

5 Short Answer Questions

1. Why does one hexadecimal digit correspond to four binary bits?
2. State the range of an 8-bit signed two’s-complement integer.
3. Distinguish carry from signed overflow.
4. What is the purpose of a bit mask?
5. Explain the difference between little-endian and big-endian storage.

5 Long Answer Questions

1. Explain two’s-complement formation, interpretation, sign extension, and its advantages over sign-magnitude representation.
2. Compare fixed-point and floating-point arithmetic in terms of range, precision, hardware cost, and embedded applications.
3. Describe common processor status flags and explain how they support conditional branching.
4. Discuss ASCII, Unicode, UTF-8, and the distinction between a character, a code point, and a byte.
5. Explain the main sources of numerical failure in embedded systems and propose engineering practices to reduce them.

3 Analytical / Application Questions

1. An 8-bit register contains 0xF2. Interpret it as both unsigned and signed two’s complement. Then explain what happens if it is sign-extended to 16 bits.
2. A controller stores voltage in unsigned Q4.4 format. Determine the represented voltage and resolution for the bit pattern 01101001₂. State whether a measured 16.2 V can be represented exactly.
3. A protocol transmits the 32-bit value 0x12345678. Show the byte sequence for each common endian order and explain how a receiver using the wrong order interprets the data.

Multiple Choice Questions with Answers

Question 1

What is the decimal value of 10110110₂?

A. 166

B. 182

C. 190

D. 214

Correct Answer: B. 182

Question 2

What is the range of an 8-bit signed two’s-complement integer?

A. −127 to 127

B. −128 to 127

C. 0 to 255

D. −128 to 128

Correct Answer: B. −128 to 127

Question 3

Which operation is normally used to clear selected bits while leaving other bits unchanged?

A. OR with a mask

B. XOR with a mask

C. AND with an appropriate mask

D. Addition of a mask

Correct Answer: C. AND with an appropriate mask

Question 4

In 8-bit two’s-complement arithmetic, what is the result pattern of 127 + 1?

A. 00000000

B. 01111111

C. 10000000

D. 11111111

Correct Answer: C. 10000000

Question 5

Which condition specifically indicates signed overflow during addition?

A. A carry occurs in every addition

B. Two same-sign operands produce a result with the opposite sign

C. The result contains at least one zero bit

D. The least significant bit is one

Correct Answer: B. Two same-sign operands produce a result with the opposite sign

Question 6

An unsigned Q4.4 value stores the integer bit pattern 90. What value does it represent?

A. 4.625

B. 5.625

C. 5.90625

D. 90.4

Correct Answer: B. 5.625

Question 7

Which statement about binary floating-point is correct?

A. Every finite decimal fraction is represented exactly

B. The spacing between values is constant at all magnitudes

C. Many decimal fractions require approximation

D. Floating point cannot represent zero

Correct Answer: C. Many decimal fractions require approximation

Question 8

In little-endian storage, which byte is placed at the lowest memory address?

A. The most significant byte

B. The least significant byte

C. The sign byte only

D. The parity byte

Correct Answer: B. The least significant byte

Question 9

What does the ASCII byte 0x31 represent?

A. The integer value 1 in binary storage

B. The character ‘1’

C. The character ‘A’

D. A Unicode replacement character

Correct Answer: B. The character ‘1’

Question 10

Why might a 32-bit intermediate be used when calculating with a 16-bit sensor value?

A. To change the byte order automatically

B. To guarantee floating-point precision

C. To reduce the chance of intermediate overflow

D. To eliminate the need for range checks

Correct Answer: C. To reduce the chance of intermediate overflow