I'm a game dev and I spent a lot of time trying to merge the physics system with the player character (PC) control. You basically have the options of letting the player controls try to physically manipulate the PC by inserting forces, or skipping physics entirely for the PC and implement something separate ad hoc.
I spent a lot of time on both and read a lot about the issue, and most end up skipping physics on the PC, because it just feels worse controlling it realistically. This might be because we're used to that though. Anyway after all, you're making a game and games have to be fun to play.
I used to be a game dev, and the goal always was to create an illusion; physics and whatnot was actually used primarily in order to save the effort, and I, too, thought that quality suffered and things would come out less fun that way.
I spent a lot of time on both and read a lot about the issue, and most end up skipping physics on the PC, because it just feels worse controlling it realistically. This might be because we're used to that though. Anyway after all, you're making a game and games have to be fun to play.