Heading level 2 (Section)

15.1. Heading level 2 (Section)#

Overview

  • Tutorial: 10 min

    Objectives:

    1. Learn the how Numba works.

15.2. Heading level 2 (Section)#

15.2.1. Heading level 3 (Subsection)#

15.2.1.1. Heading level 4 (Sub-subsection)#

15.2.1.1.1. Heading level 5 (Paragraph)#

15.2.1.1.1.1. Heading level 6 (Subparagraph)#

Heading level 7 (Lowest level)

15.2.2. Add Imgaes#

figs/performance.png

15.2.3. Bullets#

1. Annotation and Compilation: When you use Numba’s @jit decorator on a Python function, Numba first analyzes the function’s code. This analysis determines how to compile the function to improve performance. You can also provide type hints to help Numba generate more efficient machine code.

2. Type Inference: Numba performs type inference on the function’s inputs and outputs. It determines the types of variables and ensures that operations are optimized for those types. For example, it might optimize arithmetic operations for specific numerical types.

3. Machine Code Generation: Based on the type information and analysis, Numba generates machine code tailored to the function. This code is designed to run directly on the hardware, bypassing the overhead of the Python interpreter.

15.2.4. Code Blocks#

1import numba
2from numba import jit, int32, prange, vectorize, float64, cuda

15.2.5. Notes#

Note

  1. python3/3.11.0

  2. papi/7.0.1

  3. openmpi/4.0.1

  4. cuda/12.3.2

  5. gcc/14.2.0

15.2.6. Explanations#

Explanation

  1. Numba is a JIT compiler that optimizes Python code for performance.

  2. It compiles functions at runtime, allowing for efficient execution of numerical computations.

  3. The @jit decorator is used to mark functions for optimization.

  4. Numba can handle different input types and adapt its compilation accordingly.

15.2.7. Importance#

Important

In practice weight updates do not happen after every individual sample; instead, they occur after each batch of data, depending on the batch size used.

15.2.8. Exercise#

Exercise

  1. Examine the program src/distributed_data_parallel.py. What the changes from data_parallel.ipynb?

  2. Examine the job script job_scripts/distributed_data_parallel.pbs.

  3. Run the program using the job script job_scripts/distributed_data_parallel.pbs.

Key Points

  1. Numba uses simple annonations to parallelise code.

Contents