(This mod will create text files on your computer and potentially crash your game, this mod is harmless and will not harm your computer) Something is in your world.. you keep see something, things spawn that you don't remember building. You can crucifix it, but the result might not be what you wanted... What are demons harmed by? Holy water, and where is holy water found? Churches but, there aren't many left, they are mostly found in plains though, and holy water is almost non existent. You will need to get iron sticks crafted from iron bars, and put the holy water and iron sticks in a cross shape to craft, "The Cross"
Spoiler
The Church contains holy water which is needed to craft the cross and four iron sticks which is crafted from 3 iron ingots. Put those into a cross shape and you have the cross, right click on Herobrine when he appears.
