Insects I'll consider fair enough, but fish? Are you leaning on "fish isn't meat"? Otherwise, this isn't a colloquial use I'm familiar with.

Just try referring to a fish, alive or dead or cooked, as an 'animal' to your friends. Even if it doesn't sound weird to you, your friends will feel misled in the conversation, even though it's technically/scientifically correct.