Multiprocessors

Growth in data-intensive applications

• Data bases, file servers, …
  Growing interest in servers, server perf.
  Improved understanding in how to use multiprocessors effectively
• Especially server where significant natural TLP
  Advantage of leveraging design investment by replication
• Rather than unique design

“A parallel computer is a collection of processing elements that cooperate and communicate to solve large problems fast.”

Parallel Architecture = Computer Architecture + Communication Architecture

2 classes of multiprocessors WRT memory:

1. Centralized Memory Multiprocessor
• < few dozen processor chips (and < 100 cores)
• Small enough to share single, centralized memory

2. Distributed Memory multiprocessor
• Larger number chips and cores than 1.
• BW demands -> Memory distributed among processors

Centralized vs. Distributed Memory

Centralized Memory Multiprocessor

Also called symmetric multiprocessors (SMPs) because single main memory has a symmetric relationship to all processors

Can scale to a few dozen processors by using a switch and by using many memory banks

Although scaling beyond that is technically conceivable, it becomes less attractive as the number of processors sharing centralized memory increases

Distributed Memory Multiprocessor

Pro: Cost-effective way to scale memory bandwidth
• If most accesses are to local memory
Pro: Reduces latency of local memory accesses

Con: Communicating data between processors more complex