"For example, it may be possible to write Peano arithmetic as a grammar (so zero would be ' ', one would be I, two would be II, etc)"
I seem to recall doing this as a homework assignment in computer science, writing some unrestricted grammars that could be used to perform certain simple operations on some simply-encoded numbers.
But I wonder if you've gotten very far into mathematics. It's actually well known that there are many things equivalent to first-order grammar and it's completely common to choose different things based on the sort of thing you're doing, just as many things are equivalent to Turing Machines and one can freely choose the most convenient one for your local problem. Peano arithmetic is chosen merely for its convenience for the simplest of proofs, and the instant it becomes inconvenient a real mathematician abandons it for something more locally useful.
I seem to recall doing this as a homework assignment in computer science, writing some unrestricted grammars that could be used to perform certain simple operations on some simply-encoded numbers.
See also http://en.wikipedia.org/wiki/Thue_%28programming_language%29
But I wonder if you've gotten very far into mathematics. It's actually well known that there are many things equivalent to first-order grammar and it's completely common to choose different things based on the sort of thing you're doing, just as many things are equivalent to Turing Machines and one can freely choose the most convenient one for your local problem. Peano arithmetic is chosen merely for its convenience for the simplest of proofs, and the instant it becomes inconvenient a real mathematician abandons it for something more locally useful.