Lenses are actually not that irrelevant for this application. There is a huge difference between some cheap plastic and camera-grade glass. You don't want [excessive] chromatic aberrations, or [excessive] "God rays" (the Vive has that effect), or other optical artifacts.
Optics is similar to electronics: With high school physics it's all quite simple, but with high school physics a transistor can switch nearly infinitely fast ;-)
Though, I'd imagine you could use ones from cheaper 'phone-to-face' Google Cardboard-style ones.