The sample rate is crucial to the sound quality. To better understand the importance of the sampling rate we need to have an understanding of the Nyquist–Shannon sampling theorem. In short it states that perfect reconstruction of a signal is possible when the sampling frequency is greater than twice the maximum frequency of the signal being sampled. So in the case of audio, since the humans can hear up to sounds of 20kHz, a 40kHz sample rate allows for perfect reconstruction of the original signal. Nevertheless the standard is not 40kHz but 44100 Hz as explained below
44100 Hz
This is the de facto standard as set by Sony after a debate between Sony and Philips.
Although soundcards and programs now support much higher sampling rates the 44100 is still the standard when the final mixdown is ready for distribution. This is the sample frequency CDs use.
48000 Hz
Nowdays DVD and other consumer video formats use 48000 Hz as well.
There is a lot to be said about sampling frequencies, their applications and reasoning to choose one over another. Two good links for further information is this one: http://en.wikipedia.org/wiki/Sampling_rate#Audio and specifically for the 44100 Hz this one: http://en.wikipedia.org/wiki/44100_Hz