qai-cli/examples/ai-hub/README.md

Qualcomm AI Hub Example

This example takes the ONNX model produced by the SageMaker training example and runs the Qualcomm AI Hub upload workflow:

1. Quantize
2. Compile
3. Validate
4. Profile
5. Download the compiled artifact

Prerequisites

Run the training example first and wait for it to complete:

bash examples/training/run_training.sh --config config.yaml --wait

If the dataset is already uploaded to S3, use:

bash examples/training/run_training.sh --config config.yaml --skip-upload --wait

The training artifact must contain a static-shape model.onnx. The training example exports an input named input with shape 1x3x160x160.

Your config.yaml must include AI Hub settings:

aihub:
  device: Samsung Galaxy S25 (Family)
  target_runtime: tflite
  input_specs:
    input: [[1, 3, 160, 160], float32]
  output_dir: build/qai-hub

You also need local Qualcomm AI Hub SDK authentication configured.

Prepare Inputs

AI Hub does not consume the raw JPG training images directly. It needs NumPy tensors that match the ONNX model input shape and preprocessing.

Generate calibration and validation inputs:

uv run python examples/ai-hub/prepare_inputs.py

This writes:

examples/training/data/aihub_calibration/*.npy
examples/training/data/inputs.npz

The script applies the same image preprocessing used by the training example:

• resize to 160x160
• convert to channel-first 1x3x160x160
• normalize with ImageNet mean and standard deviation

Useful options:

uv run python examples/ai-hub/prepare_inputs.py \
  --dataset-dir examples/training/data/flower_photos_sagemaker \
  --calibration-dir examples/training/data/aihub_calibration \
  --input-file examples/training/data/inputs.npz \
  --samples 16

Run AI Hub

After training completes and inputs are prepared:

bash examples/ai-hub/run_ai_hub.sh --config config.yaml

By default, the script uses the last SageMaker training job recorded in .qc-cli.json. It downloads that job's model.tar.gz, extracts model.onnx, runs the AI Hub workflow, and downloads the compiled artifact.

To use a specific training job:

bash examples/ai-hub/run_ai_hub.sh \
  --config config.yaml \
  --from-job qc-cli-YYYYMMDD-HHMMSS

To resume from a later Workbench step:

bash examples/ai-hub/run_ai_hub.sh \
  --config config.yaml \
  --from-step validate

To skip downloading the compiled artifact:

bash examples/ai-hub/run_ai_hub.sh \
  --config config.yaml \
  --skip-download

Troubleshooting

If AI Hub reports dynamic input shapes, rerun training with the current training source. AI Hub quantization requires the exported ONNX model to use static input shapes.

If run_ai_hub.sh reports missing calibration or input files, run:

uv run python examples/ai-hub/prepare_inputs.py

If validation fails with a missing input name, make sure config.yaml and the generated .npz both use input as the input name.