Rarely used in normal conversation which is not that strange when you look at the meaning. Esoteric means intended for or understood only by a select few. It can also mean private, secret, confidential.
I have heard it used in the past to describe a person. I would guess that when someone is described as esoteric it means that they are a very private or secretive person.
1650s, from Gk. esoterikos "belonging to an inner circle," from esotero "more within," comparative adv. of eso "within," related to eis "into," en "in" (see en- (2)). In English, originally of Pythagorean doctrines. According to Lucian, the division of teachings into exoteric and esoteric originated with Aristotle.