The word comes from the Latin haeresis, a sect. It is used by Catholic Church to describe any baptised member of the church who rejects any article of faith canonised by the Catholic Church.
Medieval II: Total War Edit
In Medieval II: Total War heresy is a minor religion, cult or denomination that goes against the state religion of your faction. They are spread by heretics, rogue priests or cult leader controlled by the AI. Like player controlled religious agents they will slowly convert the people of that settlement to the heretical religion (Heresy in the religion section of the province scroll). To deal with then you must issue one of your Imams or Priests to put them to trial and execute them. The higher your piety the higher the chance the heretic will be executed. This can improve an Imam's or a Priest's Piety, but be warned, they can convert your Imam or Priest into a heretic, this will increase the piety of the Heretic. They will develop in a city that has enough Heresy to create them. They technically belong to the Rebels. If you see AI movements, they most likely belong to the Rebel faction. You will see inquisitors killing them; also if you have enough heresy in a city the Pope will lose faith in your faction.
Total War: Shogun 2 Edit
In Total War Shogun 2 heresy does not exist in the form of heretical agents but by religious rebels. Clans in the game can recruit two types of religious agents: the Monk or the Priest. As well as converting people in a settlement to their religion they may induce religious rebellions. When the rebel armies spawn they produce either Christian Rebels or Buddhist Rebels. Since they are not controlled by another faction and may not necessarily follow the same doctrine of Christianity and Buddhism as the faction who converted them they are in all purposes heretics.