You say the sound comes from the power supply and the HDD is not plugged into the computer.
My diagnistic: the power-supply makes a noise when it operates at very low load (almost 0mA of current), it is probaly making the cyclic noise because of some blinking LED or another very small variation of the loaf somewhere.
This is a very common symptom of cheap power-supplies, but it doesn't necessarly mean it isn't working normally, just an annoyance.