CS 475 / 575 -- Parallel Programming

Spring Quarter 2024

http://cs.oregonstate.edu/~mjb/cs575

Class Resources Page


Projects Handouts Bailey Zoom


This page was last updated: March 28, 2024


Health Information

Your well-being is our #1 concern! I want all of you to have ready-access to the latest information. Find it here at the Beaver Healthy site.

Attention Ecampusers!

A special welcome to all you Ecampusers! It is good to have a cohort here that comes from all over the world. That makes you very special!

You will experience the same course that the on-campus students experience. Same material, same notes, same projects, same quizzes, same tests, and same online Office Hours. There are recorded videos that go along with the notes (they are called "LV" for "Lecture Videos" and you will see them in the same table that has links to the notes).

Like I do whenever I have Ecampus students, I will setup a time every week to do a Live Lecture so that you can hear a discussion of the topics with the chance to ask live questions. During the LL, you can also ask questions in the Zoom Chat, and I will answer them during the LL and in a document that I will post in the same place I post the recorded LL videos. On-campus Parallelers are welcome to come as well.

These Live Lectures will be recorded so that if you miss them, you can catch them later.

Thanks, Ecampusers, for being here!

Stuff You Should Be Interested In:


Graduation! Graduation! Graduation!

  1. On Friday, June 14 (the Friday of Finals Week), EECS will hold its annual Graduation Celebration. The event runs from 3:00-5:30 at Gill Coliseum and there is a reception afterwards in the plaza just outside. Invite your friends and families to join us to celebrate your extraordinary achievement! Ecampusers: this includes you too! Go here for more information.

    BTW, this will all be livestreamed for your far-flung friends and relatives! The link will be posted here.

  2. The all-university graduation is the next day, Saturday, June 15. Ecampusers: this includes you too! Go here for more information.

  3. And, as if that isn't enough, Ecampus is holding a special reception for all online graduates, Saturday morning, June 15, 7:30-9:00 AM, leaving you plenty of time to lineup for the all-university graduation. Go here to see last year's information.


Job-Hunting Information:


What We Will Be Doing

The goals of this course are to leave you "career-ready" (i.e., both work-ready and research-ready) for tasks that require you to speed the execution of programs using parallelism.

CS 475/575 topics include:


Prerequisites and Course Incoming Expectations

This course will use C/C++ for most of its programming. You should be comfortable with the concepts of function calls, arrays, for-loops, structures, arrays of structures, structures of arrays, pointers, stacks, queues, trees, and linked lists. It is strongly suggested that you not use CS 475/575 as an opportunity to learn programming for the first time.

Many of the assignments can be done on the OSU Linux systems, which you will have ready-access to.

It would be good if you already know how to use the Linux command line and tools such as: ls, mv, cp, mkdir, cd, pwd, rm, echo, gcc/g++, and diversion to a file. It will also help if you know at least one Linux-based editor (vim is good).

This class does a lot of graphing of performance data. You will need access to a program that will let you enter data into a 2D table and graph it. (Excel is good, but there are others). You will need to be able to copy-and-paste those tables and graphs into a word processing document, add your own text around them, and then produce a PDF file from it.


Professor

The class is being taught by Professor Mike Bailey.

Office: Kelley 2117 (2nd floor, south side)
Email: mjb@cs.oregonstate.edu
Phone: 541-737-2542

TAs

The TAs for this course are: Ninad Anklesaria, Naren Digambar Khake, Anton Nitkin, Nirmit Patel, Hojun Shin, Kaung Myat Htay Win, Grace Todd, Bhagyashree Basudkar, Uddyan Sinha, and Jing Wang. All have taken this course before. We are lucky to have them!

Office Hours

TBD.


Live Lectures

I will be holding Live Lectures sometime during the week. We will go over some current materials, some upcoming materials, and maybe some extra materials. Most of it will be a repeat of what was done in the on-campus class that same week. But, if you couldn't attend the on-campus class (or even if you could), this will be a time for you to hear about the material and to ask live questions.

Attendance at the Live Lectures is optional, but the material we cover there could be on quizzes and tests. The material will definitely help you on the projects. These sessions will be recorded (see below) in case you missed them.

Here is my Zoom-room.

LLV = Live Lecture Video


Notes in place of a textbook

Course material will consist of my notes and web pages.
All required course materials for this class will cost you $0.00 (i.e., free).

On-campus Section of the Class

Our on-campus class time is: Monday and Wednesday, 12:00 - 1:50. All of our classes will be in the Learning Innovation Center (LiNC), room 128.

Online Section of the Class

The lectures for the Ecampus version of the class will be asynchronous, that is, you are free to review the notes and watch the videos whenever it suits you. But, be careful not to fall behind! The programming projects, and their due dates, will assume that you have been keeping up with the material.

Both Sections of the Class

Everyone learns in different ways. As such, feel free to attend/use resources that have been setup for the other class modality.
Ecampus students, if you are in the area, feel free to attend the on-campus class.
On-campus students, feel free to watch the videos and attend the Live Lectures.
Everyone should be studying the notes!

Note Handouts and Recorded Videos

To see an Academic Year calendar, click here.
LV = Lecture Video

WeekTopic
1 Class Resources Page HTML      
1 Course Introduction 1pp 2pp 4pp 6pp LV
1 Project Notes 1pp 2pp 4pp 6pp LV
1 Scripting 1pp 2pp 4pp 6pp LV
1 Graphing 1pp 2pp 4pp 6pp LV
1 Pivot Tables 1pp 2pp 4pp 6pp LV
1 Windows Powershell Scripting 1pp 2pp 4pp 6pp LV
1 Simple OpenMP 1pp 2pp 4pp 6pp LV
1 Parallel Programming: Background Information 1pp 2pp 4pp 6pp LV
2 OpenMP 1pp 2pp 4pp 6pp LV-1 LV-2 LV-3
2 OpenMP 4.0 Quick Reference Card PDF
2 Trapezoid Integration with OpenMP 1pp 2pp 4pp 6pp LV
2 Speedups and Amdahl's Law 1pp 2pp 4pp 6pp LV
2 Moore's Law and Multicore 1pp 2pp 4pp 6pp LV
2 Functional Decomposition 1pp 2pp 4pp 6pp LV
2 A Tale of Two Assembly Codes 1pp 2pp 4pp 6pp LV
3 Hyperthreading and "Almost Amdahl" 1pp 2pp 4pp 6pp LV
3 Caching Issues in Multicore Performance 1pp 2pp 4pp 6pp LV-1 LV-2
3 OpenMP Tasks 1pp 2pp 4pp 6pp LV
4 Data Decomposition 1pp 2pp 4pp 6pp LV
4 SIMD Vector Parallel Programming 1pp 2pp 4pp 6pp
4 Prefetching 1pp 2pp 4pp 6pp LV
5 GPU 101 1pp 2pp 4pp 6pp LV
5 Test #1 Review HTML      
6 CUDA 1pp 2pp 4pp 6pp LV
6 CUDA Quick Reference Card PDF
6 DGX System 1pp 2pp 4pp 6pp LV
6 CUDA Array Multiplication 1pp 2pp 4pp 6pp LV
6 CUDA Matrix Multiplication 1pp 2pp 4pp 6pp
7 Transition CUDA ↔ OpenCL 1pp 2pp 4pp 6pp LV
7 OpenCL 1pp 2pp 4pp 6pp LV-1 LV-2 LV-3 LV-4
7 OpenCL 2.0 Quick Reference Card: PDF
7 OpenCL Matrix Multiplication 1pp 2pp 4pp 6pp
7 OpenCL Events 1pp 2pp 4pp 6pp LV
8 OpenCL Array Multiplication TXT        
8 OpenCL Reduction 1pp 2pp 4pp 6pp LV
8 Looking At OpenCL Assembly language 1pp 2pp 4pp 6pp LV
8 OpenCL / OpenGL Vertex Buffer Interoperability 1pp 2pp 4pp 6pp LV-1 LV-2
9 Message Passing Interface (MPI) 1pp 2pp 4pp 6pp LV-1 LV-2 LV-3
9 Compute:Communicate Ratio 1pp 2pp 4pp 6pp LV
10 Parallelism Jeopardy 1pp 2pp 4pp 6pp LV
10 Finding More Information PDF
10 Test #2 Review HTML      


Downloadable Files

CUDA ArrayMult Visual Studio Solution Folder NewCudaArrayMul2019.zip Un-zip and double-click on the .sln file, then select Build→Clean Solution, then select Build→Build Sample, then select Debug→Start Without Debugging
CUDA ArrayMult .cu program arrayMul.cu  
CUDA Linux Makefile (works on the DGX systems, but not on flip) Makefile  
Just the CUDA part of the array multiplication code JustTheCuda.txt  
Just the CUDA part of the array multiplication + reduction code JustTheReduceCuda.txt  
Sample OpenCL C++ source first.cpp  
Sample OpenCL CL source first.cl.txt  
OpenCL Code to Select the Best (imho...) OpenCL Platform/Device selectopencldevice.cpp


Projects

Note: The flip machines do not have GPU cards in them, so CUDA and OpenCL will not run there. (You can compile there, you just can't run.) If your own system has a GPU, you can try using that. You can also use rabbit or the DGX machine, but please be good about sharing them.

Project # Points Title Due Date Lecture Video
0 30 Simple OpenMP Experiment April 8 LV
1 100 OpenMP: Monte Carlo Simulation April 16 LV
2 100 Functional Decomposition April 28
3 100 Mutex Stack Challenge May 8
4 60 Vectorized Array Multiplication and Reduction using SSE May 15
5 100 CUDA: Monte Carlo Simulation May 23
6 100 OpenCL June 2
7 120 MPI June 11 -- No BDs
575-only paper 100 Paper Analysis Project June 11 -- No BDs LV

Note: The flip machines do not have GPU cards in them, so CUDA and OpenCL will not run there. (You can compile there, you just can't run.) If your own system has a GPU, you can try using that. You can also use rabbit or the DGX machine, but please be good about sharing them.

Project Turn-In Procedures

Bonus Days and Late Assignments

Projects are due at 23:59:59 on the listed due date, with the following exception:

Each of you has been granted 5 Bonus Days, which are no-questions-asked one-day extensions which may be applied to any project, subject to the following rules:

  1. No more than 2 Bonus Days may be applied to any one project
  2. Bonus Days cannot be applied to quizzes
  3. Bonus Days cannot be applied to tests
  4. Bonus Days cannot be applied to Project #7 or the 575 Paper Project

Bonus Days are to enhance your success!
They are not there to enable your procrastination.
Save them for when you really need them.

If you turn in a project three or more days late, the score is a zero.

If you turn in a project late and don't have enough Bonus Days left to spend on it, the score is a zero.

You don't need to ask me, or even tell me. that you are using Bonus Days. Just turn your project in late. I have a script that will check your turn-in date and deduct the proper number of Bonus Days.

It is your job to track your Bonus Day usage. But, if you lose track, feel free to email me and ask me to look it up.


Grading

Your scores will be posted on Canvas

CS 475/575 will be graded on a fill-the-bucket basis. There will be 8 programming projects, 10 quizzes, and two tests. You get to keep all the points you earn.

Those taking the class as CS 575 will also do a 100-point essay summarizing a research paper. (Those taking the class as CS 475 will get a free 100 points for this assignment.)

All programming projects will be turned in on Teach, http://teach.engr.oregonstate.edu .

The quizzes will be done on Canvas. They will open each Friday at 12:01 AM Pacific Time and will close Sunday night at 23:59 (=11:59 PM) Pacific Time. Canvas is very unforgiving about due times -- don't push it.

Your final grade will be based on your overall class point total. Based on an available point total of 1110, grade cutoffs will be no higher than:

Points Grade
1060
1040 A- 
1020 B+
1000
980 B- 
960 C+
940
920 C- 
900 D+
880
860 D- 

Notice that this grade scale is not 90%-80%-70%-60%. That is because I do such a soft grade on the projects.


Class Rules


Other Notes You Might Enjoy!

 University   
Intro to Computer Graphics
Computer Graphics Shaders
CS Skills for Simulation and Game Programming
Paraview
Scientific Visualization
Vulkan

K-12 Outreach 
Blender
Processing
Scratch
SketchUp
TinkerCad


Other Useful Parallel Programming Information

  • vi Quick Reference Card

  • Association of Computing Machinery http://www.acm.org

  • C++ Style Guide used by Google: https://google.github.io/styleguide/cppguide.html

  • Interesting collection of information about the history of the CPU: CPU-DB.pdf


    Student Resources:

    Academic Calendar

    All students are subject to the registration and refund deadlines as stated in the Academic Calendar: https://registrar.oregonstate.edu/osu-academic-calendar

    Counseling and Psychological Services

    Oregon State University's Counseling and Psychological Services (CAPS) provides OSU students with individual, relationship, and group counseling. I have met some of these people, and they are really good. If you are in any type of emotional difficulty, don't hesitate to contact them. If it makes it easier for you, I will walk over with you.

    MySSP: Mental Health Support for Students

    MySSP @ OSU is an app that gives all OSU students, including Ecampus students, 24/7 access by text or phone with a licensed mental health counselor (MySSP stands for “My Student Support Program”). The app makes it easy to schedule short-term counseling appointments with the same ongoing counselor and provides educational materials covering mental health topics. Students can communicate with a counselor in five different languages (Mandarin, Cantonese, French, Spanish or English); additional language options are available upon request. Students can download the Anytime Anywhere: MySSP @ OSU and learn more about the service at http://beav.es/anytimeanywhere. For questions, please contact Bonnie Hemrick ( bonnie.hemrick@oregonstate.edu ), Director of Mental Health Promotion.

    Establishing a Positive Community

    It is important you feel safe and welcome in this course. If somebody is making discriminatory comments against you, sexually harassing you, or excluding you in other ways, contact the professor, your academic advisor, and/or report what happened at https://studentlife.oregonstate.edu/studentconduct/ reporting so we can connect you with resources.

    Academic Dishonesty

    You are expected to do your own work. Helping each other, with explanations or clarifications, is OK. Sharing code, however, is considered cheating. Anyone caught cheating will fail this class, and the matter will be turned over to the Dean's Office.

    You are expected to read and understand Oregon State University's Statement of Expectations for Student Conduct, found here: https://beav.es/codeofconduct . If there is any parts of this document that you don't understand, ask me!

    Students With Disabilities

    Accommodations for students with disabilities are determined and approved by Disability Access Services (DAS). If you, as a student, believe you are eligible for accommodations but have not obtained approval please contact DAS immediately at 541-737-4098 or at http://ds.oregonstate.edu. DAS notifies students and faculty members of approved academic accommodations and coordinates implementation of those accommodations. While not required, students and faculty members are encouraged to discuss details of the implementation of individual accommodations.

    Student Bill of Rights

    OSU has twelve established student rights. They include due process in all university disciplinary processes, an equal opportunity to learn, and grading in accordance with the course syllabus. See: https://asosu.oregonstate.edu/advocacy/rights

    Religious Holidays

    Oregon State University strives to respect all religious practices. If you have religious holidays that are in conflict with any of the requirements of this class, please see me immediately so that we can make alternative arrangements.

    Life Events

    As {John Lennon? Allen Saunders?} has said: "Life is what happens to you while you're busy making other plans". I care about you as a person. When life happens to you, send me an email and come see me. I might be able to help. But I surely can listen. You are not alone.

    Reach Out for Success

    University students encounter setbacks from time to time. If you encounter difficulties and need assistance, it's important to reach out. Consider discussing the situation with me or an academic advisor. Learn about resources that assist with wellness and academic success at http://oregonstate.edu/ReachOut. If you are in immediate crisis, please contact the Crisis Text Line by texting OREGON to 741-741 or call the National Suicide Prevention Lifeline at 1-800-273-TALK (8255)

    Basic Needs

    Success at OSU means knowing and using your resources. One helpful resource is the community of staff available at the Basic Needs Center (BNC) for support (bnc@oregonstate.edu, 541-737-3747, https://studentlife.oregonstate.edu/bnc). Students can drop in during open hours and talk with a BNC student leader for resources, ideas and strategies connected to basic needs challenges. The BNC is often known for its food pantry but there are other resources connected to groceries and affording food and staff who can help you work through housing stressors. Undergraduate students, living in Oregon, are especially encouraged to explore SNAP (up to $236 in grocery money each month for eligible students) as a resource. Domestic undergraduate students living in Oregon are more likely than not to be eligible for SNAP. BNC staff are skilled with helping students navigate this process.

    Additionally, the BNC Textbook Lending Program offers students the opportunity to check out required textbooks for the academic term.

    Furthermore, if you are comfortable doing so, please talk with Professor Bailey. He will do everything he can to help you.