Be careful who you listen to. Self-blame is an ugly habit that can wear you down. When an inner critic gets out of hand its time to call it out, just as you would with any voice raised against you.
If your Inner Critic is too bolshie you can do something about it. Many people worry about low self-esteem, but the more you worry about it the worse you feel. One of the features of low self-esteem is the tendency to make harsh judgements about oneself. I've found that it is more productive to focus on what you want (the solution), than it is to worry about what you have (the problem). After all, the more you worry about a problem the bigger it can appear. If harsh self criticism is a problem for you, I have a free downloadable exercise to help you develop greater self-compassion.