The angle that the light is being reflected depends on the light particles' (photons') energy. Highly energetic (short wave length) light will be scattered at large angles. This is the blue light. At mid day, when the sun is directly above your head and you look at other direction (towards the horizon) the light that you will see will only be the light that is deflected by a large angle - hence the blue skies. If you look towards the sun at mid day (DON'T do it without protection) you will notice that the sky has a sort of yellowish white color, because what you are seeing then is a mixture of scattered light and light directly from the sun - which contains all wavelengths.
At dusk and dawn the sun is directly on the horizon and the light that you are seeing only has to be scattered at a small angle for you to see it. Red light is not energetic and scatters at low angles, therefore the red sky. If you were to look directly above you at this time you would notice that the sky is devoid of any red color and only remenents of blue remain. That is the light which is scattered at large angles, only this time the sun is on the horizon and it's light is scattered from particles directly above you.
It is a well known phenomena that very polluted cities (like L.A. if you live in the USA) have beutifully colored sunsets. This is because light is more scattered more intensily by the abundance of small particles in the air.
If we had a clean, atmosphere-less planet (such as mars) our skies would have been completly black and the only source of light would have been the sky and light reflected from the