CSCE 790: Parallel Programming Models for Multicore and Manycore Processors, Fall 2017

Department of Computer Science and Engineering, University of South Carolina

Basic Information

• Website: https://passlab.github.io/CSCE790/
• Meeting Time: 12:00 - 1:15PM, Monday and Wednesday;
• Class Room: Swearingen 2A19
• Instructor: Yonghong Yan
  • Email: yanyh@cse.sc.edu
  • Office: 3A48, Swearingen Engineering Center
  • Office Hours: 10:00 - 11:00AM, Monday and Wednesday, or by appointment
• CRN number for course registration: 23224

Textbooks

• Recommended: Barbara Chapman, Gabriele Jost, and Ruud van der Pas, Using OpenMP: Portable Shared Memory Parallel Programming, 2007
• Recommended: John Cheng, Max Grossman, and Ty McKercher, Professional CUDA C Programming, 1st Edition 2014
• Recommended: James Jeffers, James Reinders, Avinash Sodani, Intel Xeon Phi Processor High Performance Programming: Knights Landing Edition, 2nd Edition 2016

Course Description

Outline

Fundamentals and parallel programming practices for multicore and manycore processors including general purpose multicore CPU, Graphic Processing Unit (GPU) manycore accelerator and Intel Xeon Phi manycore processor. Compiler and runtime techniques for mapping high-level parallel programming models to the three machine architectures mentioned before. Topics include: processor architecture of multicore CPU, NVIDIA GPU manycore accelerator, and Intel Xeon Phi manycore; parallel programming using OpenMP and CUDA threading models for the three types of processors; compilation and runtime techniques based on LLVM compiler; and performance optimization techniques and application study using computation kernels for imaging processing, deep learning and scientific simulations. The course include a significant portion of programming exercises to create software solutions for ongoing research problems that use CPU, GPU and Xeon Phi systems in heterogeneous systems.

Overview

Multicore CPU, NVIDIA GPU and Intel Xeon Phi manycore are the three major processor architectures used in today's parallel and high performance computing system, and OpenMP and CUDA are the top two programming models for these three architectures for computational and data science applications. This course will introduce the architectures of the three types of processors, the two programming models and how they are used for high performance computing applications. The course will then shift to the topics covering the implementation challenges in compiler and runtime systems for mapping high-level programming models to low-level hardware capabilities and features, including runtime task scheduling, memory management and data movement for using GPU accelerators, vectorization, synchronization, etc. The course will be organized as half-lecture and half-project. Course projects will be based on real research problem and will give students opportunities for hands-on development of software tools and applications for using the three architectures.

Prerequisites:

This is an advanced graduate level course. Preliminary courses and experiences of computer architecture, programming languages and compiler, system or parallel programming, data structure and operating systems, fluent programming using C and self-motivated learning attitude are essential for students’ success in this course.

Course Objectives

Following completion of the course, students should be able to:

• Describe the architecture, execution model and memory system of modern multicore CPU, NVIDIA GPU and Intel Xeon Phi processors
• Program with OpenMP and CUDA for multicore CPU, NVIDIA GPU and Intel Xeon Phi for high performance computing applications.
• Understand in-depth how high-level programming models are implemented by the compiler and runtime systems
• Identify and solve a research problem in the area related to the course content, including related work study, design of methodologies and techniques, implementation and evaluation, and presentation as both talks and technical publication.

Course Delivery Structure

There will be two 75-minute lectures per week, which constitutes 100% of the course delivery. There will be no separate labs, and there are no plans for distance learning.

Course Assignments and Projects

There will be two regular homework assignments for each student in the first part of the course, about four to five weeks. The class will then be organized into multiple groups, each of which has two or three students working on a research problem as the course project focusing one of the three architectures. Each group will be responsible for collecting materials, presenting the project topic, work on the project and creating project report. The outcome of each group includes documents and materials related to the topic, software implementation of the project, project presentation, and a project report. The work of each project will be split into three to four tasks; each has a deadline to finish. All the class will participate in the lecture, presentation and discussion of the course topics and projects. Accesses to multicore CPU, NVIDIA GPU and Intel Xeon Phi will be provided for students during the semester.

Projects

Grading Scheme

• Two homework: 30%
• Project: 70%

Letter Grade Correspond to Score Percentage as Follows:

Schedule

Policies and Procedures

Homework and Project

The homework and project exercises are chiefly for your own benefit. You may collaborate and consult outside sources freely when doing the homework, but you must tell me who you are collaborating with. Please remember, however, that the best way to master the material is to try the exercises on your own, then submit your own work to get feedback on it. (You will get credit even if you do not solve the problem, as long as you make an honest effort.)

Due Date and Late Policy

Each assignment has a due date and a cutoff date. Assignment due 11:55PM on the date, cut-off date is 48 hours after the due date/time. You have a total of 7 "slip days" throughout the semester that you can use at your discretion to turn in programming assignments past the posted due date. Slip days are used in whole day increments. Once your slip days are consumed, late programming assignments will be penalized at 10% per day. Assignment submission will not be accepted after the cutoff date.

Class Policies

Reading and lectures: The students are expected to read all assigned material before the lecture begins and review the material after the lecture.

Attendance Policy:

There is no required attendance policy, but if for some reason you cannot attend class, you are responsible for any material covered during your absence. Late arrivals must enter the classroom quietly and discreetly.

Exams and Test:

Exam and tests, if there are, are given in class and are open-book/open-notes. You may use any printed materials you wish during the test, but you may not use electronic devices except for use as timepieces or legitimate use by disabled students with prior notice. No make-up exams will be given except under extreme circumstances (such as severe illness or death in the immediate family) in which case you must give me notice well before the exam if at all possible.

Academic Honesty:

Examination work is expected to be the sole effort of the student submitting the work. If a student collaborates with anyone on the homework (whether or not a student), all collaborators must be listed at the top of the first page. Students are expected to follow the Code of Student Academic Responsibility. Every instance of a suspected violation will be reported. Students found guilty of violations of the Code will receive the grade of F for the course in addition to whatever disciplinary sanctions are applied by the University.

Proper Use of Computing Resources:

Students are expected to be aware of the university policy on use of computing resources, including the Student Guidelines for Responsible Computing, as well as the college and departmental policies on proper use of computing resources. Every instance of a suspected violation will be reported.

Students With Disabilities:

Any student with a documented disability should contact the Office of Student Disability Services at 803-777-6142 to make arrangements for appropriate accommodations.