What would be the difference if Putin invaded a NATO state and NATO was forced to defend? What would be the difference if he starts losing the war and needs to kick it up a notch? Is there any less chance he'd use nukes? Serious question. Threatening nukes is a rational move. Actually using them is not.
In general, the side that cares about the conflict more is more likely to engage in direct conflict, and escalate to nuclear use if their goals are being thwarted by conventional war.
To Russia, Ukraine is a core interest, and thus they're the most likely to escalate to tactical nuclear weapons if NATO intervenes and Russia starts losing.
With a NATO country, that calculus changes, and if for some crazy reason Russia actually attacked a NATO country, we would have a good chance of being able to fight back and expel Russia's attack without the war going nuclear, because Russia has far less interest in winning that war, as existing NATO countries aren't core interests to Russia.
A nuclear would most likely mean the end of the world.
Russia's core interest should be preventing the end of the world. If Putin is rational, he will never initiate a nuclear war. If he is irrational, he will not stop with Ukraine.
There are paths of escalation that merely risk nuclear annihilation rather than guarantee it. In particular, in a war with NATO, where Russian forces would be heavily outmatched in conventional warfare, Russian forces would consider using "deescalatory" tactical nuclear strikes on NATO troops. The side that has the most invested (e.g. for Russia where Kyiv is seen as integral to its historical Slavic / Rus heritage) is most likely to engage in escalation and brinksmanship while the other side has to consider whether to follow it up the escalatory ladder or back down.
