The Universe started with the Big Bang about 14,000 million years ago (14,000,000,000 years). It started out very, very hot and has been expanding and cooling since then.
For the first tiny fraction of a second, everything was so hot, that we cannot describe what it was like, but we can use our knowledge of physics to give us a good idea of what was happening from as little as 10-43 seconds (that is 0.0000000000000000000000000000000000000000001 seconds!) after the beginning.
Most galaxies appear to be moving away from us. This means that the light from them is "stretched" slightly and appears a bit redder that it would otherwise do.
This is similar to what happens to sound. Imagine that a fire engine is driving past you with its siren on. When it is moving towards you, the siren it quite high pitched. When it is moving away from you, the pitch drops and it sounds lower. With light, "lower" means more red and "higher" means more blue.