Information Package / Course Catalogue
Parallel Systems
Course Code: CSE407
Course Type: Area Elective
Couse Group: First Cycle (Bachelor's Degree)
Education Language: English
Work Placement: N/A
Theory: 2
Prt.: 2
Credit: 3
Lab: 0
ECTS: 6
Objectives of the Course

In this course, the intent is to familiarize students with parallel computer systems and their programming models. In addition to emphasizing the benefits of parallel systems and parallel programming, the challenges in writing a correct parallel code are mentioned. We also discuss the technology barriers (power wall and ILP wall) in single core processors (superscalar), which make the parallel computers a necessity to ensure the sustainability of Amdahl's Law.

Course Content

The coherence issue arising from the presence of private caches in parallel systems is introduced, and cache coherence protocols for overcoming this problem are covered in detail. Cache coherent nonuniform memory access (ccNUMA) systems are introduced, and their performance benefits are mentioned as well. As a programming model for shared memory based parallel machines, OpenMP is covered extensively with a focus on its features that enable work to be shared among multiple threads. MPI programming paradigm for distributed memory systems is also briefly discussed.

Name of Lecturer(s)
Learning Outcomes
1.Classify the parallel architectures.
2.Appraise the benefits of parallel machines as well as recognize the challenges arising from writing correct parallel codes to obtain the best performance from such computers.
3.Asses if parallel computing is inevitable.
4.Explain what the cache coherence is and enumerate the protocols for solving this problem.
5.Interpret the software demands and technological pushes that led to the emergence of chip multiprocessors.
6.Justify ccNUMA systems and their performance benefits as well.
7.Write programs in the OpenMP and MPI programming models.
8.Enumerate the different forms of parallelism in programs and tell what kind of parallel machine is required to exploit a particular type.
Recommended or Required Reading
1.Using OpenMP, Portable Shared Memory Parallel Programming by B. Chapman, G. Jost, R. V. D. Pas, The MIT Press Cam-bridge, Massachusetts, 2008.
2.Parallel Programming in OpenMP by R. Chandra, L. Dagum, D. Kohr, D. Maydan, J. McDonald, R. Menon, Morgan Kaufmann Publishers, 2001
Weekly Detailed Course Contents
Week 1 - Theoretical
Introduction to parallel computers
Week 2 - Theoretical
Different types of parallelism existing in software: instruction level, task level, and program level; The hardware support to take advantage of such forms of parallelism
Week 3 - Theoretical
Types of parallel machines: superscalar, simultaneous multi threading, distributed memory, shared memory, and chip multiprocessors
Week 4 - Theoretical
Coherency and consistency problems in shared memory parallel machines; Cache coherence protocols.
Week 5 - Theoretical
Chip multiprocessors (CMP) and their on-chip interconnects
Week 6 - Theoretical
On-chip interconnects (cont)
Week 7 - Theoretical
Simultaneous multithreading (hyper-threading)
Week 8 - Theoretical
Parallel programming models: OpenMP (shared memory) and MPI (distributed memory); An overview of OpenMP; Sharing work among threads
Week 9 - Theoretical
OpenMP synchronization constructs (barriers, ordered contruct, critical construct, atomic construct, locks, etc.).
Week 10 - Theoretical
Tuning the number of threads in OpenMP; Software and hardware techniques in order to obtain good performance from parallel programs (shared cache, TLB, loop optimizations, etc.)
Week 11 - Theoretical
Measuring OpenMP performance; Avoiding pitfalls that limit scalable performance, such as false sharing, wrong choice on private/shared data choice, and working with an unnecessarily large number of threads, etc
Week 12 - Theoretical
Using OpenMp in the real world and scalability challenges for OpenMP
Week 13 - Theoretical
Using OpenMP and MPI together to extract maximum performance for distributed shared memory architectures
Week 14 - Theoretical
Using OpenMP and MPI together to extract maximum performance for distributed shared memory architectures
Assessment Methods and Criteria
Type of AssessmentCountPercent
Laboratory1%5
Assignment5%5
Quiz4%15
Midterm Examination1%15
Final Examination1%60
Workload Calculation
ActivitiesCountPreparationTimeTotal Work Load (hours)
Lecture - Theory140228
Lecture - Practice140228
Assignment52115
Project18715
Quiz45126
Midterm Examination116218
Final Examination116218
TOTAL WORKLOAD (hours)148
Contribution of Learning Outcomes to Programme Outcomes
PÇ-1
PÇ-2
PÇ-3
PÇ-4
PÇ-5
PÇ-6
PÇ-7
PÇ-8
PÇ-9
PÇ-10
PÇ-11
OÇ-1
5
4
4
4
OÇ-2
5
4
5
4
4
4
OÇ-3
4
4
OÇ-4
5
4
OÇ-5
4
5
4
4
4
4
OÇ-6
4
4
4
4
OÇ-7
5
4
5
5
OÇ-8
4
4
4
4
4
4
Adnan Menderes University - Information Package / Course Catalogue
2026