瑞芯微 RK3588 平台驱动开发 学习计划
简介
硕士毕业后参与工作两年多，负责了若干ip，基于arm平台linux驱动开发，围绕了soc的fpga验证，回片asic点亮到产品交付消费者已经完成了一个总的流程了，但是感觉还是得拓宽自己的视野，完善驱动开发的思维，当前在国产优秀平台瑞芯微、全志当中选择了rk3588作为认真学习的平台，巩固自己的芯片驱动开发基本功。
开发板选择
在研二实习期间基于野火的NXP imx6ull开发板完成了一些电力物联网通信节点的开发，比较喜欢野火的风格，于是找了一下相关包含rk3688的开发板，找到了鲁班猫4，有rk3588s芯片坐镇，包含usb pcie gpu，简单好用，于是搞一个来边开发边学习，边写博客。下面是对应开发板正反面的设计
| 板卡名称 | LubanCat-4 |
| 电源接口 | Type-C5V@4A 直流输入 |
| 主芯片 | RK3588S(四核 A76+ 四核 A55、 Mali-G610、 6T 算力) |
| 内存 | LPDDR4X-4/8GB |
| 存储 | eMMC-32/64/128GB |
| 以太网 | 10/100/1000M 自适应以太网口 |
| USB2.0 | Type-A 接口 x3(HOST) |
| USB3.0 | Type-A 接口 x1(HOST) |
| USB3.0 | Type-C 接口 x1(OTG)，固件烧录接口， DP 显示（支持与其他屏幕进行多屏异显） |
| Debug 串口 | 一路 Debug 串口，默认参数 1500000-8-N-1 |
| 按键 | ON/OFF(开关机键)、 MR(MaskRom)、 REC(Recovery) |
| 音频接口 | 耳机输出 + 麦克风输入 2合 1接口 |
| 40Pin 接口 | 兼容树莓派 40Pin 接口，支持 PWM,GPIO,I2C,SPI,UART 功能 |
| MINI-PCIE 接口 | 可配合全高或半高的 WIFI 网卡、 4G 模块或其他 MINI-PCIE 接口模块使用 |
| SIM 卡接口 | 需要搭配 4G 模块使用 |
| HDMI2.1 | 显示器接口, 支持与其他屏幕进行多屏异显 |
| MIPI-DSI | 2xMIPI 屏幕接口, 支持与其他屏幕进行多屏异显 |
| MIPI-CSI | 3x 摄像头接口 |
| TF 卡座 | 支持 TF 卡启动系统，最高支持 512GB |
| 红外接收 | 支持红外遥控 |
| RTC 电池接口 | 支持 RTC 功能 |
| 风扇接口 | 支持安装风扇散热 |
CPU
基于ARM公司出品的cpu集成，4 A55 + 4 A76 core，3MB 的 DSU L3缓存
GPU
基于ARM公司出品的gpu集成，Mali-G610 支持2D 3D 和 GPGPUGeneral Purpose computing on GPU (GPGPU)
使用AMBA总线之ACE master互联，缓存一致性互联强强联合
VPU
视频编解码处理器 VPU is composed by VDPU121, VDPU381, AV1, VDPU720, VEPU121(JPEG encoder only) and
VEPU580 to realize the high quality video decoding and encoding
VDPU121 can support such as H.263, MPEG1/2/4 and so on, max solution is up to 1920x1088.
VDPU381 is a multi-format video decoder which supports H265, H264, VP9 and AVS2. It also supports dual-core decoding, max solution is up 64Kx64K.
VDPU720 is JPEG decoder.
VDPU981 is AV1 decoder.
VEPU121 is JPEG encoder.
VEPU580 is H264 and H264 combo video encoder.
内存管理单元
基于ARM公司出品的MMU600集成
中断控制器
基于ARM公司出品的GIC600集成
Video Capture
The Video Capture, receives the data from Camera via DVP/MIPI, and transfers the data into system main memory by AXI bus.
⚫ Support BT601 YCbCr 422 8bit input, RAW 8/10/12bit input
⚫ Support BT656 YCbCr 422 8bit progressive/interlace input
⚫ Support BT1120 YCbCr 422 16bit progressive/interlace input, single/dual-edge sampling
⚫ Support 2/4 channels mixed BT656/BT1120 YCbCr 422 8/16bit progressive/interlace input
⚫ Support YUYV sequence configurable
⚫ Support the polarity of hsync and vsync configurable
⚫ Support receiving six interfaces of MIPI CSI/DSI, up to four IDs for each interface
⚫ Support five CSI data formats: RAW8/10/12/14, YUV422
⚫ Support three modes of HDR: virtual channel mode, identification code mode, line counter mode
⚫ Support window cropping
⚫ Support RAW data through to ISP0/1
⚫ Support four channels of 8/16/32 times down-sampling for RAW data
⚫ Support virtual stride when write to DDR
⚫ Support NV16/NV12/YUV400/YUYV output format for YUV data
⚫ Support compact/non-compact output format for RAW data
⚫ Support MMU
⚫ Support soft reset, auto-reset when DMA error
RKNN
看起来是瑞芯微自研NPU神经网络处理器
RKNN supports the following features:
⚫ Include triple NPU CORE
⚫ Support triple core co-work, dual core co-work, and work independently
⚫ AHB interface used for configuration only support single
⚫ AXI interface used to fetch data from memory
⚫ Support integer 4, integer 8, integer 16, float 16, Bfloat 16 and tf32 operation ⚫ 1024x3 integer 8 MAC operations per cycle
⚫ 512x3 integer 16 MAC operations per cycle
⚫ 512x3 float 16 MAC operations per cycle
⚫ 512x3 bfloat 16 MAC operations per cycle
⚫ 256x3 tf32 MAC operation per cycle
⚫ 2048x3 integer 4 MAC operation per cycle
⚫ 384KBx3 internal buffer
⚫ Inference Engine: TensorFlow, Caffe, Tflite, Pytorch, Onnx NN, Android NN, etc.
DMC
DDR控制器相关
The DMC includes LPDDR4/4X/5 controller (DDRCTL) and DDRPHY which are a complete memory interface solution for DDR memory subsystems.
EMMC
多媒体存储卡相关
⚫ Supports SD-HCI Host version 4 mode or less
◼ Supports same SD-HCI register set for eMMC transfers
◼ Supports Command Queuing Engine (CQE) and compliant with eMMC CQ HCI
◼ Supports the following data transfer types: CPU, SDMA, AMDM2, ADMA3
⚫ Supports eMMC protocols including eMMC 5.1
◼ Supports Auto-tuning
◼ Supports 4-bit/8-bit interface
◼ Supports legacy, High Speed SDR, High Speed DDR, HS200, and HS400 speed modes
◼ Supports boot operation and alternative boot operation
⚫ AHB Slave Interface:
◼ Supports 32-bit data width and address width
⚫ AXI Master Interface:
Supports 32-bit address width and 64-bit data width
⚫ Phase adjustment:
◼ Supports phase auto
Video Output Processor
暂时不太懂，后续学习和GPU HDMI怎么协同的
DisplayPort
PCIE
基于snps出品的PCIE控制器，gen3x4
USB
基于snps出品的USB控制器，usb3.0 & 2.0 & dp
和dp comb phy方案
SATA
MIPI
HDMI
介绍不过来了，总之rockchip 3588平台在多媒体处理方面还是非常强大的，当前的知识还不能完全理解透彻，需要学习1-3年，强化吸收，增强芯片软件驱动开发基本功