Last Updated: 2016-02-02 Tue 11:50

CS 499: Parallel Computing

George Mason University
Spring 2016
3 Credits

1 Basic Course Information

1.1 Prerequisite

Grade of C or better in CS 211.

1.2 Lecture Meeting Time and Location

Section Days Time Professor Location
002 T/R 12:00-1:15 Kauffman Arts and Design L008

1.3 Instructors

Name Chris Kauffman
Office Engineering 5341
Phone 703-993-5194

1.4 Teaching Assistants

Name Email
Indranil Banerjee

1.5 Course Materials

Required Text: Introduction to Parallel Computing by Grama, Gupta, Karypis, and Kumar. 2nd Edition. Addison Wesley 2003.

An electronic version of the text is available online through the GMU library.

Computing: It is assumed you will have access to a computer with the ability to edit, compile, and run C programs. Examples and assignments will be run in a unix environment so familiarity with Unix is a de facto requirement. Some university computing resources will be made available to students to execute programs. In order to access these resources, students should become acquainted with with the unix shell, secure shell communication, and virtual private networking if off-campus access is desired.

1.6 Communication

  • Piazza is the central site for our announcements, documents repository, and discussion board. The announcements and discussion board are part of the required reading for the course.
    • All instructors and TAs can view all material on Piazza
    • Do not e-mail course staff about programming problems; use the discussion board.
    • Use public posts on Piazza to discuss programming project requirements, labs, and other material related to the course.
    • When prompted by a TA, use private posts on Piazza to share portions of your code pertaining to your questions.
    • Refer to the Piazza main page for etiquette on what should be posted publicly versus privately
    • Email course staff only for logistical issues such as meeting outside of office hours, missing lab/lecture, grading disputes, medical situations, etc. Email addresses are listed on Piazza (Resources/Staff).
  • BlackBoard is used for project submission and to post grades
  • Office Hours are listed for all staff on Piazza under (Resources/Staff). No appointments are required to attend office hours: staff will be at the listed location at the given times with doors open ready to help. In addition to attending the regular meeting times, you are strongly encouraged to visit course staff during office hours to further your understanding of the material: we are here to help you learn.

2 Course Description

2.1 Goals

This is a special topics course on parallel computing, the art of using multiple processing units to solve problems. The primary goal of this effort is either to solve a problem in a shorter time than is possible with a single computer or use the same amount of time to solve a larger problem. The course will primarily focus on tightly coupled parallel computers which have a relatively low overhead to communicate in contrast to networked computers in which communication time is high. We will discuss a variety of parallel computing technologies including the following topics.

  • Basic parallel computing architecture
  • Shared and distributed memory computers
  • POSIX Threads and OpenMP for shared memory computers
  • Java Threads for shared memory architectures
  • MPI: Message Passing Interface for distributed memory computers
  • Common communication patterns in parallel programs
  • Parallel algorithms for matrix operations, sorting, graphs, and discrete optimization
  • Evaluation metrics for parallel computing including speedup, efficiency, and isoefficiency
  • Emerging technologies such as parallel programming languages, GPU computing, and supercomputing architectures

2.2 Learning Outcomes

By the end of the semester, a passing student will be able to carry out the following types of activities:

  • Describe the differences, advantages, and limitations of various parallel computer hardware architectures.
  • Convert an existing serial program into a parallel version accounting for coordination, load balancing, and efficient communication.
  • Implement a parallel programs using shared memory and distributed memory paradigms.
  • Evaluate the theoretical and actual efficiency of parallel programs using standard metrics.

3 Coursework

3.1 Lectures

During lectures we will discuss programming concepts and instructors will provide demos of programming relevant to assignments. In addition to attending the regular meeting times, you are strongly encouraged to visit the professor and teaching assistant(s) during office hours to further your understanding of the material: we are here to help you learn.

3.2 Reading

Readings from the textbook relevant to each lecture are listed in the schedule. You will increase your understanding of lectures by reading associated textbook sections ahead of time, though this is not assumed. Additional reading material may be provided to supplement the textbook which will be posted on the course web page.

3.3 Written and Programming Assignments

Several problem sets will be assigned during the semester. These will involve solving problems and discussing trade-offs relevant to course topics. Problem set assignments should be typed and submitted electronically to blackboard in standard format such as TXT, DOC, or PDF.

There will also be several programming assignments which will involve writing programs and answering questions about them to illustrate an understanding of course material. The programming assignments will be electronically submitted to Blackboard on dates specified in the assignment by 11:59 p.m. Students may submit assignments as many times as needed up to the deadline.

3.4 Exams

There will be a series of 30-minute Mini-exams of course topics. These will occur at the end of the lecture on the dates they occur. Dates for the Mini-exams appear in the schedule.

There will also be a comprehensive final exam at the end semester. Refer to the schedule for dates of the exams.

3.5 Open Resource Exams

Unless otherwise specified, exams will be open resource: physical or electronic forms of notes, textbook, lecture slides, editor, compiler, and any code the student may be used during the exam. No communication is allowed during the exam (no email/texting/chat), no searching the internet is allowed, and visiting unauthorized web sites is prohibited. If in doubt, ask about specifics before or during the exam.

3.6 Bonus Credit

Bonus credit will be awarded based on participation in class discussions in lecture. Students may elect to sit in the first few rows of the room ("hot seats") and answer questions. Reasonable effort on answering questions in class will garner class participation credit. Participation points may also earned for involvement in the class discussion board such as giving suggestions to students with questions (but not revealing answers wholesale). The highest point winner at the end of the semester will receive a 3% bonus to their overall score in the course. All other students will receive a bonus proportional to the highest point winner. For example, someone tied with the highest point scorer will also receive a 3% bonus while someone with half the participation points will receive a 1.5% bonus.

4 Grading Policies

4.1 Components

Final grades will be determined by scores obtained on homework assignments, quizzes, projects, the midterm, and final exams. If circumstances require it, the grading scale may be adjusted, generally in the students' favor.

Component Weight
Assignments 50%
Mini-Exams 25%
Final Exam 25%

4.2 Final Grade Determination

Final grades will be assigned without rounding according to the following criteria. It is a 10-point scale per letter grade, with the upper and lower 2% of each 10% earning a + or -.

Percent Grade Percent Grade Percent Grade Percent Grade
>= 98 A+ 89-88 B+ 79-78 C+ 69-60 D
97-92 A 87-82 B 77-72 C <60 F
91-90 A- 81-80 B- 71-70 C-    

4.3 Homework Grading and Disputes

Programming homework grading will usually be divided into two portions weighted 50% for each portion.

  1. Automatic Testing (50%): Public tests will be used to assess the correctness of programs. Running these tests while constructing your program will indicate exactly what your score on this portion will be.
  2. Manual Inspection (50%): Each project will include a checklist of features your programs should exhibit. These usually comprise things that cannot be easily checked via unit tests such as good variable name selection, proper decomposition of a problem into multiple functions or cooperating objects, overall design elegance, and proper asymptotic complexity. These features will be checked by graders and assigned credit based on level of compliance.

Address grading issues with the grader first, usually a GTA that is listed in the grading feedback. This should be done respectfully either in person or via e-mail. If it is not possible to reach a resolution, the professor may be contacted by the grader to resolve the dispute.

If you have not initiated contact within 1 week after receiving a grade, the chance to contest the grade has closed.

4.4 Assignment Deadlines, Late Submission, Late Tokens

  • Assignments are due via electronic submission by 11:59 p.m. on the dates specified. You can submit work to BlackBoard as many times as desired. Only the last submission will be graded.
  • Work will not be accepted more than 48 hours after the assigned deadline
  • Late work will be accepted with a -25% penalty per 24-hour period late within the acceptance deadline. The penalty is a ceiling penalty which lowers the maximum score that can be received.

    Examples: HW is due on 10/10 at 11:59pm and Jafar hands in his assignment on 10/11 at 1:00pm (13 hours late). Jafar would have scored 83 but instead gets min(83, 100-1*25) = 75.

    Iago hands his assignment in on 10/12 at 10:00pm (46 hours late); his un-adjusted score is 43 and with the late penalties his score is still 43: min(43, 100-2*25) = 43.

    Cassim submits his assignment on 10/13 at 3:00am (51 hours late) but receives a 0 because his submission is beyond the final cutoff..

  • Each student starts the semester with three Day-Late Tokens. Whenever a student turns in a project late, tokens are automatically applied to the assignment. One token removes the penalty for submitting one day late. Assignments will not be accepted more than 48 hours late regardless of late tokens.

    Examples: HW is due on 10/10 at 11:59pm and Jafar hands in his assignment on 10/11 at 1:00pm (13 hours late). The TAs automatically deduct 1 late token from Jafar's stock so he receives no late penalty and gets an 83 on the HW.

    Iago hands his assignment in on 10/12 at 10:00pm (46 hours late); his un-adjusted score is 79. He has only 1 late token which is automatically deducted to remove one day of late penalties. His final score after penalties are applied is min(79, 100-1*25) = 75.

    Cassim submits his assignment on 10/13 at 3:00am (51 hours late). Despite having 3 late tokens, Cassim receives a 0 because his submission is beyond the final cutoff.

  • Each late token remaining at the end of the semester are worth a 0.25% bonus to a student's overall grade.
  • On-time submissions will generally be graded and available a week from submission, though exceptional circumstances may cause delays. Late submissions will be graded in as timely a fashion as schedules allow.
  • Contact the professor as soon as possible if health or family emergencies occur that would interfere with completing course work.

4.5 Exam Policies

  • Your Mason ID is required for the midterm and final exams.
  • Missing an exam results in a zero score and make-up exams will be considered only in situations involving death and near death. Proof of such circumstances will be required for a make-up to be considered.
  • Failing the final exam will result in an F in the entire course. Failing is defined as receiving less than 50% of the available points on the exam.

5 Academic Honesty and Collaboration


PRIME DIRECTIVE: Be able to explain your own work including homework code and exam solutions.

Nearly all cheating in programming can be averted by adhering to the PRIME DIRECTIVE. Students may be asked at any time to explain code or exam solutions they submit. Inability to do so will be construed as evidence of misconduct. More specific guidelines are given below.

5.2 Thou Shalt Not

For the purposes of this course, the following actions constitute scholastic misconduct (cheating):

  • Directly copying someone else's solution to a homework problem, including student solutions from a previous semester
  • Directly copying an answer from some outside source such as the Internet or friend for a homework problem
  • Making use of an Instructor Solution manual to complete homework problems
  • Paying someone for a homework solution or submitting someone else's work as your own
  • Posting solutions to any web site including our course web site
  • Collaborating or copying someone else's answer during an exam
  • Aiding or abetting any of the above
  • Witnessing any of the above and failing to report it the instructor immediately

Refer to the following links for additional information.

5.3 Penalties

Any instance of misconduct that is detected will be referred to the honor board and will likely result in failing the course. Be advised that the teaching team will be employing electronic means to detect plagiarism. This is extremely easy with computer code so keep your nose clean.

5.4 Fair Collaboration

The purpose of this course is to learn about programming and learning from one another is a great help. To that end, the following actions will NOT be considered cheating in this course.

  • Talking to other students in the course about HW problems and informally describing how a problem may be solved.
    • Be very careful as you do this that you do not share any sort of code as this will be detected.
  • Getting or giving help fixing a small bug or two: a second set of eyes is a great boon to finding that misplaced semicolon that is preventing your code from compiling.
  • Searching the Internet for alternative presentations of a programming concept.
  • When unsure whether collaboration is fair or not, stop the activity until it can be cleared with instructor.

At all times keep the PRIME DIRECTIVE in mind when studying with another student. The above collaborations should be limited to getting someone over a hurdle, not carrying them across the finish line. Doing programming projects individually prepares you for the exams in which no collaboration of any kind is allowed.

6 Additional Policies

6.1 Programming Assignments

This course involves a significant amount of programming leading to some special policies.

  • Programming assignments will be submitted to Blackboard at the specified dates. You may submit assignments to Blackboard as many times as you like: only the most recent submission will be graded (up to the deadline).
  • Back up your program regularly. Keeping backup copies of your work is your responsibility. Losing work due to a disk failure, accidental deletion, etc., is not a justification for late submission. Technologies like Dropbox automate the task of backing up which can make life much easier and safer. Submitting occasional early versions to Blackboard is also wise. Should last-minute problems happen such as accidental deletion, you will at least have some of your work to show. No code to submit means no credit.
  • Submitted code that does not compile will receive little to no credit.
  • When test cases are provided by the instructor use them and make sure your code passes all tests.
  • Familiarize yourself with how plagiarism works with programming as described in other parts of the syllabus.
  • Keep an untouched copy of your final code submission. It is important that you not touch your programs once you have made your final submission. If there are any submission problems, consideration for credit will only be given if it can be verified that the programs were not changed after being submitted.

6.2 Behavior and Accommodations

Students are expected to maintain a high level of civility for all participants in and out of class meetings. This includes respecting the beliefs of participants of all genders, ethnicities, and social backgrounds. Harassment of any type will not be tolerated and failure to behave in a respectful manner will result in referrals to University Counseling or the Office of Student Judicial Affairs. Any instances of sexual harassment will be reported to the Office of Equal Opportunity according the following policy:

Observance of religious events will be accommodated for students of any faith.

All possible accommodations will be made for students with disabilities. Please contact Disability Services ( and the instructor for further information.

7 Guidelines for Success

  • Most homework problems have analogs that are (1) discussed in lecture or (2) described in an example problem in the textbook. This should encourage you to
    • Attend lecture and take notes
    • Read the content of each section of the textbook carefully
    • Analyze the example code distributed by the instructor

    Most successful students do all of these.

  • Seek help from the instructor or teaching assistant when a concept is unclear. If you blow a problem on a HW or exam and don't see why, attend office hours to ask about it. We are here to aid your learning process.
  • Form good habits early on. We have a large amount of material to cover which means you will need to spend time outside of lecture reading and experimenting. Things will move fast so don't fall behind.
  • There is a wealth of information on data structures online. Don't be afraid to consult online tutorials if a particular topic is troublesome for you. However, do not misuse to the Internet to obtain solutions that you don't understand. Follow the PRIME DIRECTIVE at all times.

Author: Chris Kauffman (
Date: 2016-02-02 Tue 11:50