Thank you for all your help! I’ve finally found out what I did wrong. Gnome has it own way to enable/disable an extension which I followed here because of another bug. It is so annoying I am thinking about switching to lightdm permantly.

Thank you very much once again!

Unfortunately I can not login with GDM using X. I am having the same bug as in here :/ For wayland, I have both read/write permissions for my username.

Also I can use the PS5 controller as a mouse just fine. Another symptom I see is when I run xdotool mousemove 150 150, it will print Warning: XTEST extension unavailable on '(null)'. Some functionality may be disabled; See 'man xdotool' for more info.. The null here is concerning to me. Not sure how it says on other system.

No steam uses XTEST under the hood for controller. You can see a bug report here. You could try to disable the XTEST extensions and the steam client will crash if you connect with a controller.

For the symptom, If I do a xdpyinfo it will print a list of enabled extensions and no matter what i’ve tried it does not have the XTEST extension.