Imagine a song is made up of all these tiny tiny little samples that are all strung together to make a song. The sample rate refers to the length of these samples, the higher the sample rate then these samples are smaller and smaller so there are more of these tiny samples go into making up the song - the result of which is a higher sound quality.
Most humans can hear frequencies between 20 Hz and 20,000 Hz.
There is the Nyquist-Shannon sampling theorem, which says that analog signals can be reconstructed by using twice the maximum frequence it initially has.
That would be 40,000 Hz.
The additional 4,100Hz which are most commonly used (or 48,000 Hz for DAT and a bit higher quality) to allow for the cut-off slope of a reasonable band-pass filter.
Now and reading the other replies, you should be able to decide if using 48kHz is necessary for you.