Building Custom AI Operators with CANN TIK and TVM Training Course

Introduction to Custom Operator Development

• Rationale for building custom operators: use cases and constraints.
• Structure of the CANN runtime and points for operator integration.
• An overview of TBE, TIK, and TVM within the Huawei AI ecosystem.

Employing TIK for Low-Level Operator Programming

• Comprehending the TIK programming model and its supported APIs.
• Memory management and tiling strategies within TIK.
• Creation, compilation, and registration of custom operators with CANN.

Testing and Verifying Custom Operators

• Unit and integration testing of operators within the graph.
• Debugging performance issues at the kernel level.
• Visualising operator execution and buffer behaviour.

TVM-Driven Scheduling and Optimisation

• Overview of TVM as a compiler for tensor operations.
• Authoring schedules for custom operators in TVM.
• TVM tuning, benchmarking, and code generation for Ascend.

Integration with Frameworks and Models

• Registering custom operators for MindSpore and ONNX.
• Validating model integrity and fallback mechanisms.
• Supporting multi-operator graphs with mixed precision.

Case Studies and Specialised Optimisations

• Case study: High-efficiency convolution for small input shapes.
• Case study: Memory-aware optimisation of attention operators.
• Best practices for deploying custom operators across various devices.

Summary and Future Directions