CS 475/575 -- Spring Quarter 2024

Test #1 Review

This page was last updated: April 16, 2024

Under Construction:
This content is not final until you see this message go away!

Test #1 will be taken using the Canvas Quiz mechanism. It will consist of 40 multiple choice questions to be done in 60 minutes. It will be Open Notes.

Warning! "Open Notes" is not the same as "I don't need to study for it"! You will run out of time if you have to lookup in the notes every one of the questions.

Test date and time range:

Test Information:

• This will be a multiple choice test cast as a Canvas "Quiz".
• There will be 40 questions, worth 2.5 points each.
• You will have 60 minutes to complete it. Once you start, you need to finish. Canvas does not allow you to pause, leave, then come back and resume.
• The test is open notes and closed friends.
  Warning! "Open Notes" is not the same as "I don't need to study for it"! You will run out of time if you have to look up in the notes every one of the questions.
• Clearly, I cannot stop you from accessing information on the Internet. However, the test has been written against our class notes. If you miss a particular question, any protest of the form "But somethingsomething.com said that..." will be ignored.
• You are responsible for
  1. what is in the handouts
  2. what was said in class and the videos, including the Live Lectures
  3. what was covered on the quizzes
  4. what you have done in the projects

The test can potentially cover any of the following topics:

• Class Introduction and Class Resources Page:
  Bonus Days
  Grade-cutoff table

• Project Notes, Scripting, Pivot Tables, and Graphing: drawing the performance graphs correctly, turning in the projects correctly.

• Three reasons to study parallel programming.
  1. Make existing problems compute faster
  2. Make larger problems compute in the same time
  3. Programming convenience

• Kinds of Parallelism:
  Data Level Parallelism
  Thread Level Parallelism

• Von Neumann architecture: memory, control unit, arithmetic logic unit, accumulator.

• Timing:
  Speedup: S_n = T₁/T_n = P_n/P₁
  Speedup Efficiency = S_n/n
  Amdahl's Law to compute S_n, given F_p and n
  [ You do need to know the Amdahl's Law formula. ]
  What does Parallel Fraction mean?
  Inverse Amdahl's law to compute F_p, given S_n and n
  [ You don't need to know the inverse formula. ]
  Using the Parallel Fraction to compute the maximum speedup possible, no matter how many cores you use
  Gustafson's Observation

• Moore's Law:
  Transistor density (which really is Moore's Law)
  Clock speed (which really is not Moore's law)
  Power consumption
  Heat dissipation
  Multicore (hardware)
  Multithreading (software)
  Multicore without multithreading can happen -- what does that look like?
  Multithreading without multicore can happen -- what does that look like?

• What is a thread?
  An independent path through your executable
  Thread State: program counter, stack pointer, registers.
  Each thread has its own stack space
  volatile keyword.

• Definitions:
  atomic,
  barrier,
  coarse-grain parallelism,
  deterministic,
  dynamic scheduling
  fine-grain parallelism,
  private variable,
  reduction,
  shared variable,
  static scheduling

• Things to Watch Out For:
  Deadlock
  Race conditions
  Keeping internal state: what that means, why not to do it, how to avoid doing it
  Difference between strtok( ) function and strtok_r( ) function

• OpenMP:
  #pragma omp parallel ...
  The parts of the architecture that are shared among threads (heap, executable, globals)
  The parts of the architecture that aren't shared among threads (stack, stack pointer, program counter, registers)
  What's stored on the stack (local variables, function call arguments, function call return addresses)
  OpenMP requirements for for-loops
  What is an interloop dependency?
  The for-loop collapse clause
  Problems with interloop dependencies
  Declaring shared vs. private variables
  default(none), and why it is recommended
  Declaring a variable inside the for-loop automatically makes it private
  Setting the scheduling (static vs. dynamic)
  Setting the chunksize ("dealing cards")
  Reduction: why it is necessary to do something different instead of just adding into a shared variable
  Reduction, Atomic, Critical: what they are, which is fastest and why
  Synchronization: mutexes, barriers, atomic, critical, single
  Sections: what they are, asking for them, the fact that the number of sections is static
  Tasks: what they are, asking for them, the fact that the number of tasks is dynamic

• Hyperthreading:
  What it is
  What it potentially gets you
  In what circumstances is it an advantage?

• Cache:
  L1, L2, L3
  Two types of L1 cache: Instruction and Data
  Each core has its own L1 and L2 cache
  Cache hits and misses
  Coherence: spatial, temporal
  Cache lines: what they are, how large they are (64 bytes = 16 floats = 8 doubles)
  [ you don't have to be able to replicate the computation showing what cache line a particular memory location will end up in ]
  Linked list cache strategy
  Modified, Exclusive, Shared, Invalid (MESI) cache states
  False sharing: what it is, why it happens, two ways of fixing it

• Functional Decomposition:
  Done for convenience, not for performance
  The 3 barriers: why does each one need to be there?
  What the Watcher thread does

• Data Decomposition:
  [ You don't need to understand the heat transfer differential equation (whew)! ]
  You do need to understand what is happening after that equation becomes discretized
  Know what the DoAllWork( me ) function does

• Single Instruction Multiple Data (SIMD):
  Four different kinds of CPU SIMD: MMX, SSE, AVX, AVX-512
  Why use SIMD?
  How SSE works
  Each core has its own SIMD unit
  Various ways of asking for SIMD: Assembly language, OpenMP (#pragma omp simd), Intel intrinsics
  Combining SIMD with multicore -- what happens?

• Prefetching:
  What it is, why do it
  "Prefetch Distance" (PD)
  [ You will not be asked what the prefetching code looks like for different compilers ]

• No GPU material will be on the test.

• There could be questions about the Projects:
  Project 0: Simple OpenMP Experiment
  Project 1: OpenMP: Monte Carlo Simulation
  Project 2: OpenMP: Functional Decomposition
  Because Project 3 (K-Means) won't have been due yet, there will be no questions about it

• Hint #1: You won't have to write any code.

• Hint #2: I might give you code and ask what it does.

• Hint #3: I might give you code and ask what is wrong with it and how to fix it.

• Hint #4: Any arithmetic on the test will be things that you can do in your head, but you can have a calculator handy if you want.