If they're just photoconductors, you have to build up enough photocarriers to get a decent ON resistance. To reduce the amount of light required, you want a long carrier lifetime (which is easy to get in silicon, of course) but then you have to wait for them to recombine.
Photoconductors also have twice full shot noise in the photocurrent portion--both the generation and recombination are stochastic. (The literature insists on making this somehow special by calling it 'GR noise' but the physics are pretty much the same as shot noise.
Cheers
Phil Hobbs