mirror of https://github.com/Qortal/Brooklyn
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
101 lines
3.9 KiB
101 lines
3.9 KiB
skqp |
|
==== |
|
|
|
`skqp <https://skia.org/docs/dev/testing/skqp/>`_ stands for SKIA Quality |
|
Program conformance tests. Basically, it has sets of rendering tests and unit |
|
tests to ensure that `SKIA <https://skia.org/>`_ is meeting its design specifications on a specific |
|
device. |
|
|
|
The rendering tests have support for GL, GLES and Vulkan backends and test some |
|
rendering scenarios. |
|
And the unit tests check the GPU behavior without rendering images. |
|
|
|
Tests |
|
----- |
|
|
|
Render tests design |
|
^^^^^^^^^^^^^^^^^^^ |
|
|
|
It is worth noting that `rendertests.txt` can bring some detail about each test |
|
expectation, so each test can have a max pixel error count, to tell skqp that it |
|
is OK to have at most that number of errors for that test. See also: |
|
https://github.com/google/skia/blob/main/tools/skqp/README_ALGORITHM.md |
|
|
|
.. _test-location: |
|
|
|
Location |
|
^^^^^^^^ |
|
|
|
Each `rendertests.txt` and `unittest.txt` file must be located inside a specific |
|
subdirectory inside skqp assets directory. |
|
|
|
+--------------+--------------------------------------------+ |
|
| Test type | Location | |
|
+==============+============================================+ |
|
| Render tests | `${SKQP_ASSETS_DIR}/skqp/rendertests.txt` | |
|
+--------------+--------------------------------------------+ |
|
| Unit tests | `${SKQP_ASSETS_DIR}/skqp/unittests.txt` | |
|
+--------------+--------------------------------------------+ |
|
|
|
The `skqp-runner.sh` script will make the necessary modifications to separate |
|
`rendertests.txt` for each backend-driver combination. As long as the test files are located in the expected place: |
|
|
|
+--------------+----------------------------------------------------------------------------------------------+ |
|
| Test type | Location | |
|
+==============+==============================================================================================+ |
|
| Render tests | `${MESA_REPOSITORY_DIR}/src/${GPU_DRIVER}/ci/${GPU_VERSION}-${SKQP_BACKEND}_rendertests.txt` | |
|
+--------------+----------------------------------------------------------------------------------------------+ |
|
| Unit tests | `${MESA_REPOSITORY_DIR}/src/${GPU_DRIVER}/ci/${GPU_VERSION}_unittests.txt` | |
|
+--------------+----------------------------------------------------------------------------------------------+ |
|
|
|
Where `SKQP_BACKEND` can be: |
|
|
|
- gl: for GL backend |
|
- gles: for GLES backend |
|
- vk: for Vulkan backend |
|
|
|
Example file |
|
"""""""""""" |
|
|
|
.. code-block:: console |
|
|
|
src/freedreno/ci/freedreno-a630-skqp-gl_rendertests.txt |
|
|
|
- GPU_DRIVER: `freedreno` |
|
- GPU_VERSION: `freedreno-a630` |
|
- SKQP_BACKEND: `gl` |
|
|
|
.. _rendertests-design: |
|
|
|
skqp reports |
|
------------ |
|
|
|
skqp generates reports after finishing its execution, they are located at the job |
|
artifacts results directory and are divided in subdirectories by rendering tests |
|
backends and unit |
|
tests. The job log has links to every generated report in order to facilitate |
|
the skqp debugging. |
|
|
|
Maintaining skqp on Mesa CI |
|
--------------------------- |
|
|
|
skqp is built alongside with another binary, namely `list_gpu_unit_tests`, it is |
|
located in the same folder where `skqp` binary is. |
|
|
|
This binary will generate the expected `unittests.txt` for the target GPU, so |
|
ideally it should be executed on every skqp update and when a new device |
|
receives skqp CI jobs. |
|
|
|
1. Generate target unit tests for the current GPU with :code:`./list_gpu_unit_tests > unittests.txt` |
|
|
|
2. Run skqp job |
|
|
|
3. If there is a failing or crashing unit test, remove it from the corresponding `unittests.txt` |
|
|
|
4. If there is a crashing render test, remove it from the corresponding `rendertests.txt` |
|
|
|
5. If there is a failing render test, visually inspect the result from the HTML report |
|
- If the render result is OK, update the max error count for that test |
|
- Otherwise, or put `-1` in the same threshold, as seen in :ref:`rendertests-design` |
|
|
|
6. Remember to put the new tests files to the locations cited in :ref:`test-location`
|
|
|