[BUG] Some tests modify machine state

[LukaszMrugala]

Describe the bug
Some SOF tests modify machine state by setting controls to some arbitrary values and not resetting them after.

Tests I've detected on a quick look-through:

• check-keyword-detection.sh sets the volume at 40 and does not seem to reset it after.
• check-volume-levels.sh modifies a lot of amixer controls and should be looked at when handling this issue.
• volume-basic-test.sh calls the reset_sof_volume function rather than the full set_alsa_settings.
  • Note that this issue directly had its hand in the FAILs mentioned in this PR.

verify-bootsequence.sh can theoretically mess up the settings if the test crashes, but I'd like to mention it as an example for a possible new approach of a common test runner. Were we to alsactl store -f FILE the machine state before each test and then alsactl restore -f FILE after, we would be able to avoid such issues.

To Reproduce
For example, run volume-basic-test.sh and then a check-alsabat.sh test on a RT712. The latter will currently fail.

Expected behavior
Test leaves the machine in the same state as it was before the test.

Detail Info (for the aforementioned example)

1. Branch name and commit hash of the 3 repositories: sof (firmware/topology), linux (kernel driver) and sof-test (test case)
   • Kernel: 02394a2ff051
   • SOF: 9eed235129cb
   • SOF-TEST: 396efce
2. Test report ID (if you find it from test report)
   • ID: N/A
3. Test DUT Model (or a brief describtion about the device)
   • MODEL: PTLH_SDW_RT712
4. Test TPLG
   • TPLG: sof-ptl-rt712-l2-rt1320-l1
5. Test case (what test script and how you run it)
   • TESTCASE: TPLG=<topology_dir>/sof-ipc4-tplg/sof-ptl-rt712-l2-rt1320-l1.tplg MODEL=PTLH_SDW_RT712 SOF_TEST_INTERVAL=5 ~/sof-test/test-case/volume-basic-test.sh -l 50, then TPLG=<topology_dir>/sof-ipc4-tplg/sof-ptl-rt712-l2-rt1320-l1.tplg MODEL=PTLH_SDW_RT712 SOF_TEST_INTERVAL=5 ~/sof-test/test-case/check-alsabat.sh -p hw:sofsoundwire,0 -c hw:sofsoundwire,1 -C 2 -F 599

[marc-hb]

Good catch!

In my humble experience, you want a "belt and braces" approach here:

• Passing tests should try to clean up after themselves.
• At start time, every test should ALSO make an effort to cleanup and start from a good state. Because you want to try to recover from an earlier, failing test.

It's absolutely impossible to guarantee that a failing test will not mess up the system. In fact, most of the time you do NOT want cleaners to comes and mess with the "crime scene", it makes forensics and debugging MUCH harder.

So, everything and everyone should always try to clean things up when things go well. Never an exact science, always best effort.

BTW Jenkins has some logic to automatically reboot a device after each test failure and after "dangerous" tests. I forgot all the details but there are relevant links in internal issue sof-framework#623

[marc-hb]

Progress in:

• alsa: Restore test settings after each test #1279

[marc-hb]

New status needed after #1284 and #1294 were merged.