lpai e2e & minimum inference runtime support

[haowhsu-quic]

Test plan

Tested with QNN version 2.41.0.251128

[Initialization]

• Build hexagon & aarch64 target

# e.g.
export HEXAGON_SDK_ROOT=$WORKSPACE/aisw-tools/avante-tools/prebuilt/dsp/hexagon-sdk-6.4.0
export HEXAGON_TOOLS_ROOT=$HEXAGON_SDK_ROOT/tools/HEXAGON_Tools/21.0.0
export HEXAGON_ARCH=v81
# QNN_SDK_ROOT & ANDROID_NDK_ROOT are required as well
cd path/to/executorch
backends/qualcomm/scripts/build.sh --enable_hexagon

[aarch64-android]

• Sign artifact

# e.g.
cd path/to/executorch
backends/qualcomm/scripts/lpai_utils.sh --lpai v6

• Run e2e test

cd path/to/executorch
mkdir lpai_artifacts
python backends/qualcomm/tests/test_qnn_delegate.py TestQNNQuantizedOperator.test_qnn_backend_lpai -b build-android -s f3c0531 -m SM8850 --backend lpai -a lpai_artifacts

[hexagon direct]

• Sign & push artifacts + execution

cd path/to/executorch
backends/qualcomm/scripts/lpai_utils.sh --direct --lpai v6 --hexagon v81 --workspace /data/local/tmp/lpai_direct --artifact lpai_artifacts/ --serial f3c0531

• Verify

cd path/to/executorch
python backends/qualcomm/scripts/sqnr_verifier.py -g lpai_artifacts/golden_0* -o lpai_artifacts/output_direct/output_0* -e lpai_artifacts/encoding.txt -d uint8
# should have SQNR number for each output

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/16421

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• B200 runners are down due to network issues

❌ 10 New Failures, 1 Unrelated Failure

As of commit 61b71d5 with merge base c730feb ():

NEW FAILURES - The following jobs have failed:

• Lint / lintrunner / linux-job (gh)
  >>> Lint for examples/qualcomm/executor_runner/qnn_executor_runner.cpp:
• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  /pytorch/executorch/backends/qualcomm/runtime/backends/lpai/LpaiBackendCustomConfig.cpp:48:37: error: use of undeclared identifier 'QNN_LPAI_BACKEND_HW_VERSION_V7'
• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  /pytorch/executorch/backends/qualcomm/runtime/backends/lpai/LpaiBackendCustomConfig.cpp:48:37: error: use of undeclared identifier 'QNN_LPAI_BACKEND_HW_VERSION_V7'
• pull / test-static-llama-qnn-linux (stories_110m) / linux-job (gh)
  /pytorch/executorch/backends/qualcomm/runtime/backends/lpai/LpaiBackendCustomConfig.cpp:48:37: error: use of undeclared identifier 'QNN_LPAI_BACKEND_HW_VERSION_V7'
• pull / test-static-llama-qnn-linux (stories_260k_bc) / linux-job (gh)
  /pytorch/executorch/backends/qualcomm/runtime/backends/lpai/LpaiBackendCustomConfig.cpp:48:37: error: use of undeclared identifier 'QNN_LPAI_BACKEND_HW_VERSION_V7'
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 702523412db9ff1ae9793b4c91d1d926c59ab9d6cf84e63d139ec932cdb0ce4c /exec failed with exit code 8
• pull / unittest / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 8
• pull / unittest / windows / windows-job (gh)
  backends/xnnpack/test/recipes/test_xnnpack_recipes.py::TestXnnpackRecipes::test_all_models_with_recipes
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 732c49f6f3d18553dee79c3592a54dec43ecd20568a2bfddb2535a4002763313 /exec failed with exit code 8
• pull / unittest-editable / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 8

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

• pull / android / run-emulator (gh) (#16137)
  Timeout waiting for emulator to boot.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[haowhsu-quic]

Hi @cccclai, this PoC is for LPAI backend & minimum inference runtime on hexagon scalar core. Currently only aDSP is supported with 2 models given from RL team are verified. We'll also work on general solution in parallel (e.g. support cDSP(HTP), backend awareness quantizer, etc.).