The redshift of distant galaxies means that the Universe is
probably expanding. If we then go back far enough in time, everything must have been squashed together into a tiny dot. The rapid eruption from this tiny dot was the Big Bang.
Very early in its history, the whole Universe was very hot. As it expanded, this heat left behind a "glow" that fills the entire Universe. The Big Bang theory not only predicts that this glow should exist, but that it should be visible as microwaves - part of the Electromagnetic Spectrum.
This is the Cosmic Microwave Background which has been accurately measured by orbiting detectors, and is very good evidence that the Big Bang theory is correct.
As the Universe expanded and cooled down, some of the elements that we see today were created. The Big Bang theory predicts how much of each element should have been made in the early universe, and what we see in very distant galaxies and old stars is just right.
You cannot look in new stars, like the Sun, for this evidence, because they contain elements that were created in previous generations of stars. As such, the composition of new stars will be very different from the composition of stars that existed 7 billion years ago, shortly after the Big Bang.
The main alternative to the Big Bang theory of the Universe is called the Steady State theory. In this theory, the Universe does not change very much with time.
Remember that because light takes a long time to travel across the Universe, when we look at very distant galaxies, we are also looking back in time.
From this we can see that galaxies a long time ago were quite different from those today, showing that the Universe has
changed. This fits better with the Big Bang theory than the Steady State theory.