Data is Key for AI, ML, Genomics, ...
- Important workloads are all data intensive
- They require rapid and efficient processing of large amounts of data
- Data is increasing
-- We can generate more than we can process
Data is Key for Future Workloads
In-memory Databases
Graph/Tree Processing
In-Memory Data Analytics
Datacenter Workloads
Data -> performance & energy bottleneck
Data Movement Overwhelms Modern Machines
62.7% of total system energy is spent on data movement
'>'90% of the total system energy is spent on memory in large ML models
Google Workloads for Consumer Devices:
Mitigating Data Movement Bottlenecks
Data Movement vs. Computation Energy
Communication Dominates Arithmetic
A memory access consumes ~100-1000X the energy of a complex addition.
A memory acesss consumes 6400X the energy of a simple integer addition.
Many Novel Concepts Investigated Today
- New Computing Paradigms (Rethinking the Full Stack)
-- Processing in Memory, Processing Near Data
-- Neuromorphic Computing, Quantum Computing
-- Fundamentally Secure and Dependable Computers - New Accelerators & Systems (Algorithm-Hardware Co-Designs)
-- Artificial Intelligence & Machine Learning
-- Graph & Data Analytics, Vision, Video
-- Genome Analysis - New Memories, Storage Systems, Interconnects, Devices
-- Non-Volatile Main Memory, Intelligent Memory Systems, Quantum
-- High-Speed Interconnects, Disaggregated Systems
Increasingly Complex Systems
Past systems
Microprocessor <-> Main Memory <-> Storage (SSD/HDD)
Modern systems
FPGAs
Heterogeneous Processors and Accelerators
(General Purpose) GPUs
<->
Hybrid Main Memory
<->
Persistent Memory/Storage
Increasingly Complex Systems on Chip
8-Core GPU
SLC Cache
4 Firestorm Perf Cores +12MB L2
16-Core Neural Engine
4 Icestorm Efficiency Cores +4MB L2
8x 16b LPDDR4X Channels
Computer Architecture Today
- Computing landscape is very different from 10-20 years ago
- Applications and technology both demand novel architectures
Every component and its interfaces, as well as entire system designs are being re-examined
Computer Architecture Today (2)
- You can revolutionize the way computers are built, if you understand both the hardware and the software (and change each accordingly)
- You can invent new paradigms for computation, communication, and storage
- Recommended book: Thomas Kuhn, "The Structure of Scientific Revolutions" (1962)
-- Pre-paradigm science: no clear consensus in the field
-- Normal science: dominant theory used to explain/improve things (business as usual); exceptions considered anomalies
-- Revolutionary science: underlying assumptions re-examined
Takeaways
- It is an exciting time to be understanding and designing computing architectures
- Many challenging and exciting problems
-- That no one has tackled (or thought about) before
-- That can have huge impact on the world's future - Driven by explosion of data, new applications (ML/AI, graph analytics, genomics), ever-greater realism, ...
-- We can easily collect more data than we can analyze/understand - Driven by significant difficulties in keeping up with that hunger at the technology layer
-- Five walls: Energy, reliability, complexity, security, scalability
Takeaways
- It all starts from the basic building blocks and design principles
- And, knowledge of how to use, apply, enhance them
- Underlying technology might change(e.g., steel vs. wood)
-- but methods of taking advantage of technology bear resemblance
-- methods used for design depend on the principles employed
The Same Applies to Processor Chips
- There are basic building blocks and design principles
AMD Barcelona 4 cores
Intel Core i7 8 cores
IBM Cell BE 8+1 cores
IBM POWER7 8 cores
Sun Niagara 2 8 cores
Nvidia Fermi 448 "cores"
Intel SCC 48 cores, networked
Tilera TILE Gx 100 cores, networked
TESLA Full Self-Driving Computer (2019)
- ML accelerator: 260 mm2, 6 billion transistors, 600 GFLOPS GPU, 12 ARM 2.2 GHz CPUs.
- Two redundant chips for better safety.
Basic Building Blocks
- Electrons
- Transistors
- Logic Gates
- Combinational Logic Circuits
- Sequential Logic Circuits
-- Storage Elements and Memory - ...
- Cores
- Caches
- interconnect
- Memories
- ...
Recall: High-Level Goals of This Course
- In Digital Design & Computer Architecture
- Understand the basic
- Understand the principles (of design)
- Understand the precedents
- Based on such understanding:
-- learn how a modern computer works underneath
-- evaluate tradeoffs of different designs and ideas
-- implement a principled design (a simple microprocessor)
-- learn to systematically debug increasingly complex systems
-- Hopefully enable you to develop novel, out-of-the-box designs - The focus is on basics, principles, precedents, and how to use them to create/implement good designs
Recall: Why These Goals?
- Because you are here for a Computer Science degree!
- Regardless of your future direction, learning the principles of digital design & computer architecture will be useful to
-- design better hardware
-- design better software
-- design better systems
-- make better tradeoffs in design
-- understand why computers behave the way they do
-- solve problems better
-- think "in parallel"
-- think ciritically
-- ...
