First, go back to the data sheet and make sure you have the pins correctly connected.
The 7805 is not guaranteed to be stable unless there is a small capacitor between input and ground, and very close to the regulator. At least .22 uf ceramic or low inductance film is recommended. A .1 uf cap output to ground is also recommended, though they claim it is stable without it. The output voltage regulation is only specified for load current greater than 5 milliamps, so you might connect a 1k resistor output to ground to provide this minimum load, till you have the design verified and ready ot connect to some other load.