Learners of English, especially self-taught learners may have troubles in understanding speeches by native speakers. The following ways might help improve their listening ability.
First of all, do things step by step. It is not good to listen to something beyond your level. Better choose a suitable course and start with the first book. Go on to the second book only after you are sure you understand the first one.
Secondly, stick to one course of study. Don't change books often. Never let your attention be attracted by another course just because it seems to be more "fashionable" (时髦).
Thirdly, listen to the English news program over the radio from time to time. Better go through the news stories in the Chinese-language newspaper first. That will make it easy for you to understand the English news on the radio.
Fourthly, if you have time, listen to some interesting stories in Special English from the V.O.A. or other listening materials of the same level as that of your textbook.