Computer Architecture: CPUs -- Memory and Storage

1. Introduction

Previous sections examine one of the major components used in computer systems: processors. The sections review processor architectures, including instruction sets, operands, and the structure of complex CPUs.

This section introduces the second major component used in computer systems:

memories. Successive sections explore the basic forms of memory: physical memory, virtual memory, and caches. Later sections examine I/O, and show how I/O devices use memory.

2. Definition

When programmers think of memory, they usually focus on the main memory found in a conventional computer. From the programmer's point of view, the main memory holds running programs as well as the data the programs use. In a broader sense, computer systems use a storage hierarchy that includes general-purpose registers, main memory, and secondary storage (e.g., a disk or flash storage). Throughout this text, we will use the term memory to refer specifically to main memory, and generally use the term storage for the broader hierarchy and the abstractions programmers use with the hierarchy.

An architect views a memory as a solid-state digital device that provides storage for data values. The next sections clarify the concept by examining the variety of possibilities.

3. The Key Aspects of Memory

When an architect begins to design a memory system, two key choices arise:

-- Technology

-- Organization

Technology refers to the properties of the underlying hardware mechanisms used to construct the memory system. We will learn that many technologies are available, and see examples of their properties. We will also learn how basic technologies operate, and understand when each technology is appropriate.

Organization refers to the way the underlying technology is used to form a working system. We will see that there are many choices about how to combine a one-bit memory cell into multibit memory cells, and we will learn that there are multiple ways to map a memory address into the underlying units.

In essence, memory technology refers to the lowest-level hardware pieces (i.e., individual chips), and memory organization refers to how those pieces are combined to create meaningful storage systems. We will see that both aspects contribute to the cost and performance of a memory system.

4. Characteristics of Memory Technologies

Memory technology is not easy to define because a wide range of technologies has been invented. To help clarify the broad purpose and intent of a given type of memory, engineers use several characteristics:

-- Volatile or nonvolatile

-- Random or sequential access

-- Read-write or read-only

-- Primary or secondary

[†An emerging technology known as Nonvolatile RAM (NVRAM) operates like a traditional main memory, but retains values when the power is removed.]

4.1 Memory Volatility

A memory is classified as volatile if the contents of the memory disappear when power is removed. The main memory used in most computers (RAM) is volatile - when the computer is shut down, the running applications and data stored in the main memory vanish.

In contrast, memory is known as nonvolatile if the contents survive even after power is removed†. For example, the flash memory used in digital cameras and Solid State Disks (SSDs) is nonvolatile - the data stored in the camera or on the disk

remains intact when the power is turned off. In fact, data remains even if the storage device is removed from the camera or computer.

4.2 Memory Access Paradigm

The most common forms of memory are classified as random access, which means that any value in the memory can be accessed in a fixed amount of time independent of its location or of the sequence of locations accessed. The term Random Access Memory (RAM) is so common that consumers look for RAM when they purchase a computer.

The alternative to random access is sequential access in which the time to access a given value depends on the location of that value in the memory and the location of the previously accessed value (typically, accessing the next sequential location in memory is much faster than accessing any other location). For example, one type of sequential access memory consists of a FIFO† queue implemented in hardware.

4.3 Permanence Of Values

Memory is characterized by whether values can be extracted, updated, or both.

The primary form of memory used in a conventional computer system permits an arbitrary value in memory to be accessed (read) or updated (written) at any time. Other forms of memory provide more permanence. For example, some memory is characterized as Read Only Memory (ROM) because the memory contains data values that can be accessed, but cannot be changed.

A form of ROM, Programmable Read Only Memory (PROM), is designed to allow data values to be stored in the memory and then accessed many times. In the extreme case, a PROM can only be written once - high voltage is used to alter the chip permanently.

Intermediate forms of permanence also exist. For example, the flash memory commonly used in smart phones and solid state disks represents a compromise between permanent ROM and technologies with little permanence - although it retains data when power is removed, the items in flash memory do not last forever. An exercise asks the reader to research flash technologies to discover how long data will last if a flash device sits idle.

4.4 Primary And Secondary Memory

The terms primary memory and secondary memory are qualitative. Originally, the terms were used to distinguish between the fast, volatile, internal main memory of a computer and the slower, nonvolatile storage provided by an external electromechanical device such as a hard disk. However, many computer systems now use solid-state memory technologies for both primary and secondary storage. In particular, Solid State Disks (SSDs) are used for secondary storage.

5. The Important Concept of a Memory Hierarchy

The notions of primary and secondary memory arise as part of the memory hierarchy in a computer system. To understand the hierarchy, we must consider both performance and cost: memory that has the highest performance characteristics is also the most expensive. Thus, an architect must choose memory that satisfies cost constraints.

Research on memory use has led to an interesting principle: for a given cost, optimal performance is not achieved by using one type of memory throughout a computer.

Instead, a set of technologies should be arranged in a conceptual memory hierarchy.

The hierarchy has a small amount of the highest performance memory, a slightly larger amount of slightly slower memory, and so on. For example, an architect selects a small number of general-purpose registers, a larger amount of primary memory, and an even larger amount of secondary memory. We can summarize the principle:

To optimize memory performance for a given cost, a set of technologies are arranged in a hierarchy that contains a relatively small amount of fast memory and larger amounts of less expensive, but slower memory.

Section 12 further examines the concept of a memory hierarchy. The section presents the scientific principle behind a hierarchical structure, and explains how a memory mechanism known as a cache uses the principle to achieve higher performance without high cost.

6. Instruction and Data Store

Recall that some of the earliest computer systems used a Harvard Architecture with separate memories for programs and data. Later, most architects adopted a Von Neumann Architecture in which a single memory holds both programs and data.

Interestingly, the advent of specialized solid state memory technologies has reintroduced the separation of program and data memory -- special-purpose systems some times use separate memories. Memory used to hold a program is known as instruction store, and memory used to hold data is known as data store.

One of the motivations for a separate instruction store comes from the notion of memory hierarchy: on many systems, overall performance can be increased by increasing the speed of the instruction store. To understand why, observe that high-speed instructions are designed to operate on values in general-purpose registers rather than values in memory. Thus, to optimize speed, data is kept in registers whenever possible.

However, an instruction must be accessed on each iteration of the fetch-execute cycle.

Thus, the instruction store experiences more activity than the data store. More important, although data accesses tend to follow a random pattern of accessing a variable and then another variable, a processor typically accesses an instruction store sequentially.

That is, instructions are placed one after another in memory, and the processor moves from one to the next unless a branch occurs. Separating the two memories allows a designer to optimize the instruction store for sequential access.

We can summarize:

Although most modern computer systems place programs and data in a single memory, it is possible to separate the instruction store from the data store. Doing so allows an architect to select memory performance appropriate for each activity.

7. The Fetch-Store Paradigm

As we will see, all memory technologies use a single paradigm that is known as fetch-store. For now, it is only important to understand that there are two basic operations associated with the paradigm: fetching a value from the memory or storing a value into the memory. The fetch operation is sometimes called read or load. If we think of memory as an array of locations, we can view reading a value from memory as an array index operation in which a memory address is used:

value --> memory[ address ]

The analogy also holds for store operations, which are sometimes called write operations. That is, we can view storing a value into memory as storing a value into an array:

memory[ address ] --> value

The next section explains the idea in more detail. Later sections on I/O explain how the fetch-store paradigm is used for input and output devices, and how the underlying memory access relates to I/O.

8. Summary

The two key aspects of memory are the underlying technology and the organization. A variety of technologies exist; they can be characterized as volatile or nonvolatile, random or sequential access, permanent or nonpermanent (read-only or read-write), and primary or secondary.

To achieve maximal performance at a given cost, an architect organizes memory into a conceptual hierarchy. The hierarchy contains a small amount of high performance memory and a large amount of lower performance memory.

Memory systems use a fetch-store paradigm. The memory hardware only supports two operations: one that retrieves a value from memory and another that stores a value into memory.

EXERCISES

1. Define storage hierarchy and give an example.

2. What are the two key choices an architect makes when designing a memory system?

3. Read about RAM and SSD technologies for a typical computer. What is the approximate financial cost per byte of each type of memory?

4. Extend the previous exercise by finding the speed (access times) of the memories and comparing the financial cost of the performance.

5. Which type of memory is more secure (i.e., less susceptible to someone trying to change the contents), flash memory or ROM?

6 If data is stored in a volatile memory, what happens to the data when power is removed?

7 Suppose NVRAM were to replace DRAM. What characteristic of memory technologies becomes less important (or even disappears)?

8. Compare the performance of an NVRAM to traditional RAM. How much slower is NVRAM?

9. Research the flash technology used in typical USB flash drives which are also called jump drives or thumb drives. If a flash drive is left unused, how long will the data persist? Are you surprised by the answer?

10. Registers are much faster than main memory, which means a program could run much faster if all the data were kept in registers instead of main memory. Why do designers create processors with so few registers?

11. If a computer follows a Harvard Architecture, do you expect to find two identical memories, one for instructions and one for data? Why or why not?

12 Do the terms fetch-execute and fetch-store refer to the same concept? Explain.

PREV. | NEXT

Related Articles -- Top of Page -- Home