mirror of https://github.com/Qortal/Brooklyn
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
488 lines
15 KiB
488 lines
15 KiB
.. SPDX-License-Identifier: GPL-2.0 |
|
|
|
============================== |
|
drm/komeda Arm display driver |
|
============================== |
|
|
|
The drm/komeda driver supports the Arm display processor D71 and later products, |
|
this document gives a brief overview of driver design: how it works and why |
|
design it like that. |
|
|
|
Overview of D71 like display IPs |
|
================================ |
|
|
|
From D71, Arm display IP begins to adopt a flexible and modularized |
|
architecture. A display pipeline is made up of multiple individual and |
|
functional pipeline stages called components, and every component has some |
|
specific capabilities that can give the flowed pipeline pixel data a |
|
particular processing. |
|
|
|
Typical D71 components: |
|
|
|
Layer |
|
----- |
|
Layer is the first pipeline stage, which prepares the pixel data for the next |
|
stage. It fetches the pixel from memory, decodes it if it's AFBC, rotates the |
|
source image, unpacks or converts YUV pixels to the device internal RGB pixels, |
|
then adjusts the color_space of pixels if needed. |
|
|
|
Scaler |
|
------ |
|
As its name suggests, scaler takes responsibility for scaling, and D71 also |
|
supports image enhancements by scaler. |
|
The usage of scaler is very flexible and can be connected to layer output |
|
for layer scaling, or connected to compositor and scale the whole display |
|
frame and then feed the output data into wb_layer which will then write it |
|
into memory. |
|
|
|
Compositor (compiz) |
|
------------------- |
|
Compositor blends multiple layers or pixel data flows into one single display |
|
frame. its output frame can be fed into post image processor for showing it on |
|
the monitor or fed into wb_layer and written to memory at the same time. |
|
user can also insert a scaler between compositor and wb_layer to down scale |
|
the display frame first and then write to memory. |
|
|
|
Writeback Layer (wb_layer) |
|
-------------------------- |
|
Writeback layer does the opposite things of Layer, which connects to compiz |
|
and writes the composition result to memory. |
|
|
|
Post image processor (improc) |
|
----------------------------- |
|
Post image processor adjusts frame data like gamma and color space to fit the |
|
requirements of the monitor. |
|
|
|
Timing controller (timing_ctrlr) |
|
-------------------------------- |
|
Final stage of display pipeline, Timing controller is not for the pixel |
|
handling, but only for controlling the display timing. |
|
|
|
Merger |
|
------ |
|
D71 scaler mostly only has the half horizontal input/output capabilities |
|
compared with Layer, like if Layer supports 4K input size, the scaler only can |
|
support 2K input/output in the same time. To achieve the ful frame scaling, D71 |
|
introduces Layer Split, which splits the whole image to two half parts and feeds |
|
them to two Layers A and B, and does the scaling independently. After scaling |
|
the result need to be fed to merger to merge two part images together, and then |
|
output merged result to compiz. |
|
|
|
Splitter |
|
-------- |
|
Similar to Layer Split, but Splitter is used for writeback, which splits the |
|
compiz result to two parts and then feed them to two scalers. |
|
|
|
Possible D71 Pipeline usage |
|
=========================== |
|
|
|
Benefitting from the modularized architecture, D71 pipelines can be easily |
|
adjusted to fit different usages. And D71 has two pipelines, which support two |
|
types of working mode: |
|
|
|
- Dual display mode |
|
Two pipelines work independently and separately to drive two display outputs. |
|
|
|
- Single display mode |
|
Two pipelines work together to drive only one display output. |
|
|
|
On this mode, pipeline_B doesn't work indenpendently, but outputs its |
|
composition result into pipeline_A, and its pixel timing also derived from |
|
pipeline_A.timing_ctrlr. The pipeline_B works just like a "slave" of |
|
pipeline_A(master) |
|
|
|
Single pipeline data flow |
|
------------------------- |
|
|
|
.. kernel-render:: DOT |
|
:alt: Single pipeline digraph |
|
:caption: Single pipeline data flow |
|
|
|
digraph single_ppl { |
|
rankdir=LR; |
|
|
|
subgraph { |
|
"Memory"; |
|
"Monitor"; |
|
} |
|
|
|
subgraph cluster_pipeline { |
|
style=dashed |
|
node [shape=box] |
|
{ |
|
node [bgcolor=grey style=dashed] |
|
"Scaler-0"; |
|
"Scaler-1"; |
|
"Scaler-0/1" |
|
} |
|
|
|
node [bgcolor=grey style=filled] |
|
"Layer-0" -> "Scaler-0" |
|
"Layer-1" -> "Scaler-0" |
|
"Layer-2" -> "Scaler-1" |
|
"Layer-3" -> "Scaler-1" |
|
|
|
"Layer-0" -> "Compiz" |
|
"Layer-1" -> "Compiz" |
|
"Layer-2" -> "Compiz" |
|
"Layer-3" -> "Compiz" |
|
"Scaler-0" -> "Compiz" |
|
"Scaler-1" -> "Compiz" |
|
|
|
"Compiz" -> "Scaler-0/1" -> "Wb_layer" |
|
"Compiz" -> "Improc" -> "Timing Controller" |
|
} |
|
|
|
"Wb_layer" -> "Memory" |
|
"Timing Controller" -> "Monitor" |
|
} |
|
|
|
Dual pipeline with Slave enabled |
|
-------------------------------- |
|
|
|
.. kernel-render:: DOT |
|
:alt: Slave pipeline digraph |
|
:caption: Slave pipeline enabled data flow |
|
|
|
digraph slave_ppl { |
|
rankdir=LR; |
|
|
|
subgraph { |
|
"Memory"; |
|
"Monitor"; |
|
} |
|
node [shape=box] |
|
subgraph cluster_pipeline_slave { |
|
style=dashed |
|
label="Slave Pipeline_B" |
|
node [shape=box] |
|
{ |
|
node [bgcolor=grey style=dashed] |
|
"Slave.Scaler-0"; |
|
"Slave.Scaler-1"; |
|
} |
|
|
|
node [bgcolor=grey style=filled] |
|
"Slave.Layer-0" -> "Slave.Scaler-0" |
|
"Slave.Layer-1" -> "Slave.Scaler-0" |
|
"Slave.Layer-2" -> "Slave.Scaler-1" |
|
"Slave.Layer-3" -> "Slave.Scaler-1" |
|
|
|
"Slave.Layer-0" -> "Slave.Compiz" |
|
"Slave.Layer-1" -> "Slave.Compiz" |
|
"Slave.Layer-2" -> "Slave.Compiz" |
|
"Slave.Layer-3" -> "Slave.Compiz" |
|
"Slave.Scaler-0" -> "Slave.Compiz" |
|
"Slave.Scaler-1" -> "Slave.Compiz" |
|
} |
|
|
|
subgraph cluster_pipeline_master { |
|
style=dashed |
|
label="Master Pipeline_A" |
|
node [shape=box] |
|
{ |
|
node [bgcolor=grey style=dashed] |
|
"Scaler-0"; |
|
"Scaler-1"; |
|
"Scaler-0/1" |
|
} |
|
|
|
node [bgcolor=grey style=filled] |
|
"Layer-0" -> "Scaler-0" |
|
"Layer-1" -> "Scaler-0" |
|
"Layer-2" -> "Scaler-1" |
|
"Layer-3" -> "Scaler-1" |
|
|
|
"Slave.Compiz" -> "Compiz" |
|
"Layer-0" -> "Compiz" |
|
"Layer-1" -> "Compiz" |
|
"Layer-2" -> "Compiz" |
|
"Layer-3" -> "Compiz" |
|
"Scaler-0" -> "Compiz" |
|
"Scaler-1" -> "Compiz" |
|
|
|
"Compiz" -> "Scaler-0/1" -> "Wb_layer" |
|
"Compiz" -> "Improc" -> "Timing Controller" |
|
} |
|
|
|
"Wb_layer" -> "Memory" |
|
"Timing Controller" -> "Monitor" |
|
} |
|
|
|
Sub-pipelines for input and output |
|
---------------------------------- |
|
|
|
A complete display pipeline can be easily divided into three sub-pipelines |
|
according to the in/out usage. |
|
|
|
Layer(input) pipeline |
|
~~~~~~~~~~~~~~~~~~~~~ |
|
|
|
.. kernel-render:: DOT |
|
:alt: Layer data digraph |
|
:caption: Layer (input) data flow |
|
|
|
digraph layer_data_flow { |
|
rankdir=LR; |
|
node [shape=box] |
|
|
|
{ |
|
node [bgcolor=grey style=dashed] |
|
"Scaler-n"; |
|
} |
|
|
|
"Layer-n" -> "Scaler-n" -> "Compiz" |
|
} |
|
|
|
.. kernel-render:: DOT |
|
:alt: Layer Split digraph |
|
:caption: Layer Split pipeline |
|
|
|
digraph layer_data_flow { |
|
rankdir=LR; |
|
node [shape=box] |
|
|
|
"Layer-0/1" -> "Scaler-0" -> "Merger" |
|
"Layer-2/3" -> "Scaler-1" -> "Merger" |
|
"Merger" -> "Compiz" |
|
} |
|
|
|
Writeback(output) pipeline |
|
~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
.. kernel-render:: DOT |
|
:alt: writeback digraph |
|
:caption: Writeback(output) data flow |
|
|
|
digraph writeback_data_flow { |
|
rankdir=LR; |
|
node [shape=box] |
|
|
|
{ |
|
node [bgcolor=grey style=dashed] |
|
"Scaler-n"; |
|
} |
|
|
|
"Compiz" -> "Scaler-n" -> "Wb_layer" |
|
} |
|
|
|
.. kernel-render:: DOT |
|
:alt: split writeback digraph |
|
:caption: Writeback(output) Split data flow |
|
|
|
digraph writeback_data_flow { |
|
rankdir=LR; |
|
node [shape=box] |
|
|
|
"Compiz" -> "Splitter" |
|
"Splitter" -> "Scaler-0" -> "Merger" |
|
"Splitter" -> "Scaler-1" -> "Merger" |
|
"Merger" -> "Wb_layer" |
|
} |
|
|
|
Display output pipeline |
|
~~~~~~~~~~~~~~~~~~~~~~~ |
|
.. kernel-render:: DOT |
|
:alt: display digraph |
|
:caption: display output data flow |
|
|
|
digraph single_ppl { |
|
rankdir=LR; |
|
node [shape=box] |
|
|
|
"Compiz" -> "Improc" -> "Timing Controller" |
|
} |
|
|
|
In the following section we'll see these three sub-pipelines will be handled |
|
by KMS-plane/wb_conn/crtc respectively. |
|
|
|
Komeda Resource abstraction |
|
=========================== |
|
|
|
struct komeda_pipeline/component |
|
-------------------------------- |
|
|
|
To fully utilize and easily access/configure the HW, the driver side also uses |
|
a similar architecture: Pipeline/Component to describe the HW features and |
|
capabilities, and a specific component includes two parts: |
|
|
|
- Data flow controlling. |
|
- Specific component capabilities and features. |
|
|
|
So the driver defines a common header struct komeda_component to describe the |
|
data flow control and all specific components are a subclass of this base |
|
structure. |
|
|
|
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_pipeline.h |
|
:internal: |
|
|
|
Resource discovery and initialization |
|
===================================== |
|
|
|
Pipeline and component are used to describe how to handle the pixel data. We |
|
still need a @struct komeda_dev to describe the whole view of the device, and |
|
the control-abilites of device. |
|
|
|
We have &komeda_dev, &komeda_pipeline, &komeda_component. Now fill devices with |
|
pipelines. Since komeda is not for D71 only but also intended for later products, |
|
of course we’d better share as much as possible between different products. To |
|
achieve this, split the komeda device into two layers: CORE and CHIP. |
|
|
|
- CORE: for common features and capabilities handling. |
|
- CHIP: for register programing and HW specific feature (limitation) handling. |
|
|
|
CORE can access CHIP by three chip function structures: |
|
|
|
- struct komeda_dev_funcs |
|
- struct komeda_pipeline_funcs |
|
- struct komeda_component_funcs |
|
|
|
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_dev.h |
|
:internal: |
|
|
|
Format handling |
|
=============== |
|
|
|
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_format_caps.h |
|
:internal: |
|
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_framebuffer.h |
|
:internal: |
|
|
|
Attach komeda_dev to DRM-KMS |
|
============================ |
|
|
|
Komeda abstracts resources by pipeline/component, but DRM-KMS uses |
|
crtc/plane/connector. One KMS-obj cannot represent only one single component, |
|
since the requirements of a single KMS object cannot simply be achieved by a |
|
single component, usually that needs multiple components to fit the requirement. |
|
Like set mode, gamma, ctm for KMS all target on CRTC-obj, but komeda needs |
|
compiz, improc and timing_ctrlr to work together to fit these requirements. |
|
And a KMS-Plane may require multiple komeda resources: layer/scaler/compiz. |
|
|
|
So, one KMS-Obj represents a sub-pipeline of komeda resources. |
|
|
|
- Plane: `Layer(input) pipeline`_ |
|
- Wb_connector: `Writeback(output) pipeline`_ |
|
- Crtc: `Display output pipeline`_ |
|
|
|
So, for komeda, we treat KMS crtc/plane/connector as users of pipeline and |
|
component, and at any one time a pipeline/component only can be used by one |
|
user. And pipeline/component will be treated as private object of DRM-KMS; the |
|
state will be managed by drm_atomic_state as well. |
|
|
|
How to map plane to Layer(input) pipeline |
|
----------------------------------------- |
|
|
|
Komeda has multiple Layer input pipelines, see: |
|
- `Single pipeline data flow`_ |
|
- `Dual pipeline with Slave enabled`_ |
|
|
|
The easiest way is binding a plane to a fixed Layer pipeline, but consider the |
|
komeda capabilities: |
|
|
|
- Layer Split, See `Layer(input) pipeline`_ |
|
|
|
Layer_Split is quite complicated feature, which splits a big image into two |
|
parts and handles it by two layers and two scalers individually. But it |
|
imports an edge problem or effect in the middle of the image after the split. |
|
To avoid such a problem, it needs a complicated Split calculation and some |
|
special configurations to the layer and scaler. We'd better hide such HW |
|
related complexity to user mode. |
|
|
|
- Slave pipeline, See `Dual pipeline with Slave enabled`_ |
|
|
|
Since the compiz component doesn't output alpha value, the slave pipeline |
|
only can be used for bottom layers composition. The komeda driver wants to |
|
hide this limitation to the user. The way to do this is to pick a suitable |
|
Layer according to plane_state->zpos. |
|
|
|
So for komeda, the KMS-plane doesn't represent a fixed komeda layer pipeline, |
|
but multiple Layers with same capabilities. Komeda will select one or more |
|
Layers to fit the requirement of one KMS-plane. |
|
|
|
Make component/pipeline to be drm_private_obj |
|
--------------------------------------------- |
|
|
|
Add :c:type:`drm_private_obj` to :c:type:`komeda_component`, :c:type:`komeda_pipeline` |
|
|
|
.. code-block:: c |
|
|
|
struct komeda_component { |
|
struct drm_private_obj obj; |
|
... |
|
} |
|
|
|
struct komeda_pipeline { |
|
struct drm_private_obj obj; |
|
... |
|
} |
|
|
|
Tracking component_state/pipeline_state by drm_atomic_state |
|
----------------------------------------------------------- |
|
|
|
Add :c:type:`drm_private_state` and user to :c:type:`komeda_component_state`, |
|
:c:type:`komeda_pipeline_state` |
|
|
|
.. code-block:: c |
|
|
|
struct komeda_component_state { |
|
struct drm_private_state obj; |
|
void *binding_user; |
|
... |
|
} |
|
|
|
struct komeda_pipeline_state { |
|
struct drm_private_state obj; |
|
struct drm_crtc *crtc; |
|
... |
|
} |
|
|
|
komeda component validation |
|
--------------------------- |
|
|
|
Komeda has multiple types of components, but the process of validation are |
|
similar, usually including the following steps: |
|
|
|
.. code-block:: c |
|
|
|
int komeda_xxxx_validate(struct komeda_component_xxx xxx_comp, |
|
struct komeda_component_output *input_dflow, |
|
struct drm_plane/crtc/connector *user, |
|
struct drm_plane/crtc/connector_state, *user_state) |
|
{ |
|
setup 1: check if component is needed, like the scaler is optional depending |
|
on the user_state; if unneeded, just return, and the caller will |
|
put the data flow into next stage. |
|
Setup 2: check user_state with component features and capabilities to see |
|
if requirements can be met; if not, return fail. |
|
Setup 3: get component_state from drm_atomic_state, and try set to set |
|
user to component; fail if component has been assigned to another |
|
user already. |
|
Setup 3: configure the component_state, like set its input component, |
|
convert user_state to component specific state. |
|
Setup 4: adjust the input_dflow and prepare it for the next stage. |
|
} |
|
|
|
komeda_kms Abstraction |
|
---------------------- |
|
|
|
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_kms.h |
|
:internal: |
|
|
|
komde_kms Functions |
|
------------------- |
|
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_crtc.c |
|
:internal: |
|
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_plane.c |
|
:internal: |
|
|
|
Build komeda to be a Linux module driver |
|
======================================== |
|
|
|
Now we have two level devices: |
|
|
|
- komeda_dev: describes the real display hardware. |
|
- komeda_kms_dev: attachs or connects komeda_dev to DRM-KMS. |
|
|
|
All komeda operations are supplied or operated by komeda_dev or komeda_kms_dev, |
|
the module driver is only a simple wrapper to pass the Linux command |
|
(probe/remove/pm) into komeda_dev or komeda_kms_dev.
|
|
|