Riemann's contribution is overlooked far far too often. The early non-Euclidean geometries were spaces of constant curvature - spherical and hyperbolic - and Riemann brought the idea of a manifold, and the notion of having a geometry that changes as you move around the space. And he did it in a fantastic lecture with only one equation in 1854, a good 50 years before special relativity.
Einstein was also definitely familiar with the work of Helmholtz, who did some fascinating work on non-Euclidean geometry in the context of ophthalmology: Lenses change the amount of curvature we perceive in space (think of fish-eye lenses), and provide a great jumping off point for the notion that the universe might not be as flat as it appears.
The Dover book 'Beyond Geometry' collects a bunch of the major papers in non-Euclidean geometry leading up to relativity, and is a fantastic read.
Einstein was also definitely familiar with the work of Helmholtz, who did some fascinating work on non-Euclidean geometry in the context of ophthalmology: Lenses change the amount of curvature we perceive in space (think of fish-eye lenses), and provide a great jumping off point for the notion that the universe might not be as flat as it appears.
The Dover book 'Beyond Geometry' collects a bunch of the major papers in non-Euclidean geometry leading up to relativity, and is a fantastic read.