In today's new green world, heating your home efficiently may be the biggest and most effective contribution one can make to the health of the planet. It is also convenient that it can save the homeowner considerable money. Most of what needs to be done is common-sense, but giving the problem structure makes it easier to accomplish.
The first, and most important, thing to consider is insulation. Keeping the heat in your house is at least as important as generating it efficiently. Many steps towards this end may have already been made, depending on age and state of repair. Thermal-pane windows, sometimes referred to as double glazing, attic insulation, and draft-sealing are the primary sources of likely improvement. It is usually best to consult professionals on these issues, and always to get at least three estimates with references.
The efficiency of the heating unit is next, and will depend partly on where the house is located. If it is far enough North that no air conditioning will be required, then only a furnace will be needed. What fuel to use in the furnace will depend on local utility rates, and fuel prices can change radically in short periods of time, as we have all come to learn recently. Gas or oil are usually the best choices, electric heat is still a poor choice, even today.
In lower latitudes a heat pump is usually the best bet. Not only is it capable of both heating and cooling the house without the expense of a separate furnace, but using one unit for all purposes will reduce the carbon footprint of the home quite a lot. Heat pumps require a back-up heat source for those times when the outside temperature falls too low for the heat pump to keep up with demand. While the backup is normally an electric-resistance heater installed in the plenum, cooler areas might want to consider a gas or oil backup furnace instead.