They make a huge difference. Depreciation in a low land-value area gives you an extra (tax bracket * house value / 27.5) per year. For a 200K house that's easily 2900/y extra return, more if you split out internal components of the house - an extra 1.45% return, when your cap rate might be only a bit higher than 4%! Even when it's fully depreciated, you never actually take the hit from the lowered basis on sale if you keep doing 1031 exchanges.
Factors that are proportional to house value (property tax, transaction fees, maintenance costs) have a big impact on relative profitability because the marginal gains are actually small.
Most of the "advantage" of home buying comes from the ability to borrow money at (e.g.) 4% and hope the home appreciates at (e.g.) 5%. Small taxes and write offs can double or negate profitability.
(The down payment part of the investment might also be good diversification, but it is unlikely to appreciate as fast as the same cash would in stocks.)
At least as far is house value is concerned (deducting depreciation raises your wealth, but it hardly increases the value of the house).