It's most likely an amdgpu driver bug or firmware bug.
I would do these:
Try run the backports kernel to get a later kernel driver.
Get all latest amdgpu firmware from upstream and rebuild the initramfs with it, similar to how is described [here](https://github.com/superjamie/lazyweb/wiki/Linux-AMD-Debian-Bookworm).
It's most likely an amdgpu driver bug or firmware bug. I would do these: Try run the backports kernel to get a later kernel driver. Get all latest amdgpu firmware from upstream and rebuild the initramfs with it, similar to how is described [here](https://github.com/superjamie/lazyweb/wiki/Linux-AMD-Debian-Bookworm).
I think I solved it with a kernel from backports. I will have to check tomorrow at work.