Presumably you still need a C++ compiler for LLVM itself though. Or... a C++ to C translator.

Edit: On second thought, that's only needed if you want to run rustc itself on the old hardware, which is probably not super useful given the main reason you would need to do this is if LLVM can't target that hardware.

For building code written in Rust for such old hardware, this would be sufficient.