Intro to Computer Systems

Chapter 5: Memory & Hamming/SECDED Codes

Memory Systems and Technologies

Computers need memory as "working space": a quickly-accessible area where they can store the programs and data that they need to process. Modern computer systems run so quickly that their CPUs often run many times faster than the rest of the system (including memory).

A computer system includes a special type of memory called processor cache; this is a small amount of ultra-fast memory that helps alleviate any bottlenecks in the flow of information.

Memory Construction

There are essentially two different types of core (main) memory technology:

Cache memory is implemented with static memory.

Main system memory is implemented with dynamic memory.

Cache

As the processor is often many times faster than the memory system, if left to its own the processor would need to wait a (relatively) long time for data to be retrieved. This is obviously quite inefficient: why bother having a fast processor if nothing can keep up with it? A processing system can only be as fast as its slowest component.

The concept of processor cache memory is that it is a small, ultra-fast pool of memory that sits between the processor and the standard memory system. Some support circuitry monitors how the processor is using the memory, and attempts to fill the cache memory up with information that the processor is most likely to need.

Fast memory is expensive; that's why the entire memory system isn't made up of cache.

Such predictions are never perfect, but most computing tasks have a distinctive pattern in how they use memory. A correct predition is called a cache "hit", an incorrect prediction is a "miss". Depending on the task, modern cache control systems can achieve cache hit rates exceeding 90%.

The disparity betwen processor and memory speed has become so great that modern computer systems include a number of cache "levels" of increasing size and decreasing speed. The idea behind this is that the most frequently-accessed data will end up in a faster, higher-level cache pool, while still providing much of the benefit of large cache sizes.

Level 1 Cache

"Level 1" cache memory is a very small data cache that exists right on the processor itself, running at the same break-neck speed as the processor's execution units. (Some designs have level-1 cache that runs at half speed.)

As it is running at the same speed as the processor, there is no delay when data is fetched from the cache.

Level 2 Cache

Due to the cost constraints of high-speed memory, the level 1 cache is often backed up by a larger "level 2" cache: this is a slower bank of memory, still residing on the CPU itself.

Level 3 Cache

(Can you see where this is heading?) modern desktops, servers and high-end workstations, include a third level of cache. This is an even larger, slower than Level 2 (but still faster than main memory) cache pool. The size of these caches can dictate that it is on a separate chip from the processor.

Miniaturisation of transistors on silicon has now got to the point where Level 3 caches can reside on the CPU itself.

Main Memory

The cache memory is a fast, convenient temporary storage area for frequently-accessed data; however it is usually tiny compared to the amount of memory required to fit complete program and data sets. Let's take into consideration a typical Intel Core i5-based desktop computer system:

Memory Type Size Proportion of Total
Main Memory
8 GB
100%
Level 3 Cache
8 MB
0.1%
Level 2 Cache
256 KB
0.364%
Level 1 Cache
64 KB
7.6x10-6 %

Despite the cache memory making up for the performance shortfalls of main memory, constant developments are being made to its speed, in order to keep up with ever increasing processor performance.

Memory Bandwidth

The primary factor in desktop computing performance is memory bandwidth, a measure of how quickly data can be transferred: all else being equal, the faster the memory can deliver data, the faster the computer will perform.

Memory Latency

Memory latency refers to how long it takes from a request for data being issued, and that data finally being retrieved from memory. For almost all desktop computing applications, memory latency does not have a significant effect on performance. (It is, however, more critical in server systems.)

Memory Technologies

Here is a summary of common memory technologies in use, and their peak bandwidth abilities of their fastest speed ratings (some historical processor requirements listed in italics for comparison)

Technology Speed Rating Peak Bandwidth
Intel Pentium 4
1066MHz FSB
8.5 GB/sec
Intel Core 2 series
1600MHz FSB
12.8 GB/sec
AMD Athlon64
HyperTransport
14.4 GB/sec
SDRAM
PC133
1.06 GB/sec
DDR SDRAM
DDR-400
3.2 GB/sec
DDR2 SDRAM
DDR2-800
6.4 GB/sec
DDR3 SDRAM
DDR3-1600
12.8 GB/sec

Some memory technologies can be run in "dual channel" mode; this means that two sets of memory are interleaved together, effectively doubling the peak bandwidth. For example, "dual channel DDR-400" is capable of 6.4GB/sec, twice the 3.2GB/sec of standard "single channel" DDR-400.

A note about these large numbers; they are peak memory bandwidth requirements. That is, the highest theoretical requirement. In normal use, the processors are much less demanding.

The knife cuts both ways, though; memory systems often can only work at a fraction of their peak bandwidth efficiency.

SDRAM

The oldest machines still capable of running modern software use memory known as SDRAM (Synchronous Dynamic Random Access Memory). This memory was introduced in 1997 and often used with Intel Pentium II and III and early AMD Athlon-based computers. Most of Apple's "G3" and "G4" systems also use this memory.

DDR SDRAM

DDR (Double Data Rate) SDRAM is a development above normal SDRAM which allowed for the memory bandwidth to be doubled, by allowing it to perform twice the amount of data transfer per clock tick -- hence the name "double data rate".

This memory technology was cheap to manufacture and was massively popular amongst all desktop computing systems in the early-to-mid 2000s.

DDR2 SDRAM

Standard DDR memory chips encountered a design limit as to how fast they could be run; this prompted the development of the DDR2 SDRAM standard, which allowed for faster clock speeds. Another side-benefit of the design was that with the use of a lower voltage, power consumption was also reduced.

DDR3 SDRAM

DDR3 is yet faster and more power efficient. Nearly all modern desktop and notebook computers use DDR3 for their main system memory.

DDR3 vs. DDR2: an article on anandtech.com comparing DDR3 and DDR2 memory.

RDRAM (Rambus)

RDRAM (Rambus Dynamic Random Access Memory, named after "Rambus", the company that invented it) was a memory technology designed in the late 1990s with a focus on high memory bandwidth, at the expense of latency. This happened to suit well with Intel's then-new Pentium 4 processor, which had memory bandwidth requirements that were unheard of at the time.

However, due to licencing and manufacturing-difficulty issues, RDRAM remained an expensive and unpopular choice in the general computing market, and has since been left behind by cheaper DDR SDRAM memory (explained below). It has, however, found a niche in some specialist processing markets: many game consoles, such as the Sony Playstation 2 and 3, use Rambus memory.

Non-Volatile Memory

Most memory in a computer is volatile: when the power is turned off, the memory is erased.

Although appropriate for most processing situations, it does not lend well to permanent storage. All computers contain a small core program, called the BIOS (Basic Input/Output System), which provides a very primitive way for software to access various parts of the hardware.

The BIOS must stay present at all times, and cannot be erased. A memory type known as ROM (Read Only Memory) is used for this purpose. Some forms of ROM can be reprogrammed by the computer, known as PROM (Programmable Read Only Memory).

An EPROM chip.
An EPROM chip.

Legacy PROMs were only erasable using ultraviolet light; these are known as EPROM (Erasable-Programmable Read Only Memory). These chips had a little window on the top onto which UV light had to be shined to erase. This meant that:

  1. the entire chip had to be erased; it wasn't possible to erase a portion
  2. it was necessary to put a sticker over the window, lest UV light from the sun slowly erase the chip.

Nowadays, the most common form in industry is called Flash EEPROM (the EE stands for "Electrically Erasable"), as it allows the contents of the memory chip to be changed relatively quickly and conveniently. Almost all computers have a BIOS chip made of Flash EEPROM, however the most common consumer use of this type of memory is in portable devices such as music players, smartphones and tablets. As memory chips are solid-state (no moving parts), they are much more reliable and abuse-resistant than other storage technologies.