ECE 4822: Engineering Computation IV

Syllabus

Contact Information:

Lecture MWF: 10:00 AM - 10:50 AM (ENGR 301A / Online)
Lecturer Joseph Picone, Professor
Office: ENGR 718
Office Hours: (MWF) 08:00 AM - 10:00 AM, other times by appointment
Phone: 215-204-4841 (desk); 215-954-7076 (cell - preferred)
Email: picone@temple.edu
Zoom: picone@temple.edu or joseph.picone@gmail.com
Email Google Group URL: https://groups.google.com/forum/#!forum/temple_engineering_ece4822
Google Group Email: temple-engineering-ece4822@googlegroups.com
Website http://www.isip.piconepress.com/courses/temple/ece_4822
Textbook / Online Resources Like many computer-related disciplines, there are an infinite number of textbooks and web sites devoted to this topic. Some of the resources we will make use of in this course are:
Reference
Textbooks
This is one of the better books on parallel processing:

D.B. Kirk and W.W. Hwu
Programming Massively Parallel Processors: A Hands-On Approach
Morgan Kaufmann; Third Edition
December 21, 2016, 576 pages
ISBN: 978-0128119860
URL: Morgan Kaufmann; Amazon Link

This book presents an excellent programmer's view of GPU programming:

J. Han and B. Sharma
Learn CUDA Programming: A beginner's guide to GPU programming and parallel computing with CUDA 10.x and C/C++
Packt Publishing; First Edition (September 27, 2019)
508 pages
ISBN: 978-1788996242
URL: Packt Publishing Link; Amazon Link

Other
Resources
Internet-based resources play a major role in this course. We will make extensive use of the Linux operating system, but will only have time to scratch the surface on this topic. An excellent in-depth training course can be found here:

Free Linux Online Training: a wide range of Linux tutorials are available.
LearnPython.org: many excellent interactive tutorials.

Learning how to use the Internet to problem solve is another very important skill you will learn in this course. We often describe this as "learning how to learn." An amazing resource that contains an answer to just about any computer question you can imagine is:

Stack Overflow: where you can find answers to almost any programming question.

Prerequisites Minimum grade of C- in (ECE 3822 or CIS 2168)

Course Description: Please see the university bulletin for a description of the course.

ABET Syllabus:: Please see click here to view the ABET syllabus for this course.

University Policy Statements:: Please refer to the College of Engineering Policies and Procedures web site regarding the policies and procedures you are responsible for. The source location for this information is in the TU Portal: College of Engineering -> Advising -> Temple University Syllabus Policy.

Course Overview:

This course is organized into four sections: This course will be very hands-on. Students will use the Neuronix Cluster to develop and test code. This cluster gives you access to several generations of GPU chips.

Course grades will be based on a series of programming assignments and a class final project that involves a competition. All assignments will be submitted electronically by placing files in the following directories: $HOME/ece_4822/homework/hw_xx/.

Grading Policies:

Item
Weight
Programming Assignments (10) 75%
Final Exam 25%
TOTAL: 100%


Lecture Schedule:

The lecture component of ECE 4822 meets three times a week and will cover the following topics:

Class
Date
Topic(s)
Online Materials
01
08/26
  General Computing: A Brief Introduction to Linux and Linux Clusters     slides | aux | code | lecture
02
08/28
  General Computing: A Brief Introduction into Supercomputing   slides | aux | code | lecture
03
08/30
  C/C++ Programming: Overview of HW #1   slides | aux | code | lecture
04
09/04
  General Computing: A Brief History of Coprocessors   slides | aux | code | lecture
05
09/06
  C/C++ Programming: Pointers, Array Indexing and Compiler Optimizations   slides | aux | code | lecture
06
09/09
  Basic GPU Programming: The GPU Economy   slides | aux | code | lecture
07
09/11
  Basic GPU Programming: GPU Architectures   slides | aux | code | lecture
08
09/13
  DSP Review: Digital Filters and Factoring Computations   slides | aux | code | lecture
09
09/16
  DSP Review: Real-Time Systems   slides | aux | code | lecture
10
09/18
  DSP Review: Linear System Theory   slides | aux | code | lecture
11
09/20
  Basic GPU Programming: Host and Coprocessor Dynamics   slides | aux | code | lecture
12
09/23
  Basic GPU Programming: Blocks and Threads   slides | aux | code | lecture
13
09/25
  Basic GPU Programming: Cooperating Threads     slides | aux | code | lecture
14
09/27
  Basic GPU Programming: GPU Memory   slides | aux | code | lecture
15
09/30
  Basic CPU Programming: Virtual Memory   slides | aux | code | lecture
16
10/02
  Basic CPU Programming: Cache, Multithreading and Hyperthreading   slides | aux | code | lecture
17
10/04
  Basic GPU Programming: Using Multiple GPUs (Manual)   slides | aux | code | lecture
18
10/07
  Parallel Computing: Foundations   slides | aux | code | lecture
19
10/09
  Parallel Computing: Basic Parallel Programming Using OpenMP   slides | aux | code | lecture
20
10/14
  Parallel Computing: Shared Memory and Multithreading in OPenMP   slides | aux | code | lecture
21
10/16
  Python Programming: Interfaces to CUDA   slides | aux | code | lecture
22
10/18
  Python Programming: Creating Efficient Functions in Numba   slides | aux | code | lecture
23
10/21
  Parallel Computing: Alternatives to CUDA   slides | aux | code | lecture
24
10/23
  Advanced GPU Programming: Streams and Concurrency   slides | aux | code | lecture
25
10/25
  Advanced GPU Programming: CUDA BLAS   slides | aux | code | lecture
26
10/28
  Parallel Computing: Message Passing     slides | aux | code | lecture
27
10/30
  Parallel Computing: MapReduce and Data Parallelism   slides | aux | code | lecture
28
11/01
  General Computing: Error Correction Coding   slides | aux | code | lecture
29
11/04
  General Computing: Disks, File Systems and I/O   slides | aux | code | lecture
30
11/06
  Parallel Computing: Hadoop - A Distributed File System   slides | aux | code | lecture
31
11/08
  Final Exam: Digital Resampling of Signals   slides | aux | code | lecture
32
11/11
  Advanced GPU Programming: The Thrust C++ Library   slides | aux | code | lecture
33
11/13
  General Computing: Task Scheduling   slides | aux | code | lecture
34
11/15
  Applications: Introduction to Computer Graphics   slides | aux | code | lecture
35
11/18
  Applications: Ray Tracing in Computer Graphics   slides | aux | code | lecture
36
11/20
  Applications: Ray Tracing on a GPU   slides | aux | code | lecture
37
11/22
  Final Exam: A Deep Dive Into Image Processing   slides | aux | code | lecture
38
12/02
  Basic GPU Programming: The NVIDIA Blackwell Architecture   slides | aux | code | lecture
39
12/04
  Applications – Introduction to Neural Networks and Deep Learning     slides | aux | code | lecture
40
12/06
  Applications – Implementation of Deep Learning Systems Using PyTorch     slides | aux | code | lecture
41
12/09
  Applications – Introduction to Quantum Computing     slides | aux | code | lecture
42
12/16
  Final Exam (8:00 AM - 10:00 AM): Competition Deadline and Student Presentations     slides | aux | code | lecture


Homework:

The homework schedule is as follows:

HW
Due Date
Item(s)
01
09/03
  Linear Algebra and DSP in C++  
02
09/09
  Fast Algorithms in C++  
03
09/16
  Numerical Libraries  
04
09/23
  My First GPU Program  
05
09/30
  Matrix Multiplication on a GPU Using CUDA  
06
10/07
  Parallel DSP Filters Using OpenMP  
07
10/14
  Parallel DSP on a GPU  
08
10/21
  Managing GPU Resources  
09
10/28
  Parallelizing Code Across Multiple GPUs  
10
11/04
  Neural Networks and Dense Graphs  
N/A
11/11
  No Late Homework Accepted After This Date  
11
12/09
  Final Exam: Data Challenge