The Theory of Moral Sentiments is Adam Smith's account of where moral judgment comes from. Written before his work on economics, it sets aside the idea that ethics rests on cold calculation and begins instead with a fact of human nature: however selfish we may seem, we are interested in the fortunes of others and can feel sorrow at their sorrow and joy at their joy.
The engine of this fellow-feeling is the imagination. We have no direct experience of what another person feels, so we place ourselves in their situation, conceive what we ourselves would feel there, and form a weaker copy of their emotion. Smith calls this sympathy, and he extends the word beyond pity to cover fellow-feeling with any passion whatever. We approve of a person's response when, having brought their case home to ourselves, we find that our own sympathetic feeling matches theirs; this agreement is what he calls propriety.
Because approval depends on whether an onlooker can share a sentiment, Smith introduces the figure of the impartial spectator: a supposed equitable judge, fully informed and unbiased, whose view we try to take. Propriety is conduct this spectator could enter into; merit and demerit are judged by the gratitude or resentment such a spectator would feel. The spectator is not any actual crowd but an idealized standpoint that corrects for ignorance and partiality.
Smith then turns the spectator inward to explain conscience. A person raised in total solitude, he argues, could no more judge the beauty or deformity of their own mind than of their own face, having no mirror for it. Society supplies that mirror, and from the habit of being judged we build an internal judge, the man within the breast, who weighs our conduct as an impartial observer would. This inner spectator is what lets us resist self-love and prefer the greater interest of others to a small interest of our own.
From this Smith draws his standard of virtue. He separates the desire for praise from the deeper desire to be genuinely praise-worthy, and finds true satisfaction in conduct that would merit approbation whether or not it is ever seen. The book surveys the virtues of propriety, prudence, justice, and benevolence, examines how custom and fortune distort moral feeling, and reviews earlier moral philosophies, but its enduring core is the claim that conscience and moral judgment are built up socially, through sympathy and the imagined view of the impartial spectator.