Unfortunately, this is yet another subject where too much of the "advice" is over-simplified, contains factual errors, or both.
But as I understand it, the benefits and drawbacks of each form of heating are as follows:
Electric resistance heaters (common characteristics)
+ low upfront cost
+ portability (can be moved around as desired)
+ ease of use (plug in and go — no installation needed)
− higher running costs and environmentally worse due to the inefficiency of electricity generation (and to a lesser extent, transmission/distribution)
? reliability (when well-made they are nearly bullet-proof, but the price pressures of the target market result in sometimes severe corner-cutting on cheapos)
It must be made clear that it's not electric heaters themselves that are to blame for their running costs. Generating electricity from available energy sources (along with storing it efficiently) has never been an easy task, and as appealing as "renewable" energy sources are, each of those has their own problems (high manufacturing cost and non-continuous availability for photovoltaic, debatedly infrasound and light interruption for wind turbines, and restricted locations for hydro-power).
I must also address the common misconception (verging on total BS) that electric heaters cannot safely be used with extension cords. Sure they can be used — provided the current rating is adequate and counterfeits are avoided (and no attempt is made to use a Class I heater — or any other Class I appliance — with 2-core extension cords still sold in North America and a few other places). That running the cord in its coiled-up state will impede cooling (particularly in the case of extension reels) is true enough, but there is no simple way around that (and it's not that big of a deal apart from reels in practice anyway). As for overloading the cords, the only way to stop people from (unwittingly) doing that is to BAN DOUBLE-ADAPTORS ALREADY!!! I would also much prefer (on 230V, anyway) that they used the IEC 60320 inlets (whichever version suits the power rating and maximum surface temperature of the heater in question), but bean-counters and lawsuit-phobes won't let that happen (apparently).
It's also true that the added resistance of an extension cord will result in power loss but this is not dangerous (unless the cord is heavily coiled up) and will usually be minimal within reasonable length limits (considering the trip hazards presented by cords across paths); it ends up in the room anyway so ultimately isn't "wasted" other than for radiant heaters.
Several styles of electric heaters exist due to their versatility: Radiant heaters "project" infrared energy that heats the "target" with minimal interaction with the surrounding air, making them great at spot heating, but they operate at very high temperatures so are not suitable for unattended operation (especially in the presence of small children) and can ignite anything flammable that comes too close. Fan heaters are smaller (and therefore can be cheaper to build) for the same power as passive convection heaters, and also often excel at even heat distribution through the room (with sensible upright axial-fan designs), but are typically noisy and sometimes short-lived (when the sleeve bearings are poorly lubricated); though I still can't help but pity them a bit in the face of popular opinion. Oil-filled column heaters are fine for bedrooms and generally more fire-safe — but are slow to warm up and if the oil sealing fails (unlikely from reputable manufacturers but quite possible on the cheapos), the result is a mess to say the least. Convectors of the plain "vented box" style (with generally zig-zagged ribbon elements) remain useful where you want near-silent operation (there's often a slight hum from the elements) with reasonably quick warm-up, and also have the potential to last incredibly long when well-built (although modern examples are typically housed in rather thin steel and the resettable thermal cut-out is still the only safety device included in many). In any case, a watt is a watt and neither type of electric heater outputs more total heat for a given power input — it's how the heat is distributed that matters.
Natural gas heating
+ usually lower running cost for the same power as resistance heaters (but higher than reverse-cycle air conditioning)
+ lower installation cost than air conditioning (but higher than resistance heaters)
− explosion risk
− unflued types release emissions into the room (requiring ventilation, which compromises system efficiency, and rendering them unsuitable for bedrooms)
− fixed installation only
− dead-end technology, nothing much can be done to improve environmental performance from what it is presently
I'd probably take it over electrical resistance heating for large spaces, but that's about it.
Reverse-cycle air conditioning
+ lowest overall running cost
+ covers both bases with one unit — heating in winter and cooling in summer
− high installation cost
− heat distribution difficulties are common
− use of refrigerant complicates disposal at end-of-life
As for the old wood fires, forget them — they produce too much smoke (as I'm all-too-familiar with, given the neighboring houses still using it), and should be avoided completely unless there's nothing better usable where you live (e.g. your local electricity supply is too wimpy for electric heating).
Modern slow-combustion wood heaters are acceptably efficient, but still more-polluting than gas and somewhat inconvenient to operate.
Overall (for Australia), I think reverse-cycle air conditioning is the best bet for large space heating (where it works), and portable resistance heaters for spot heating (radiant/conduction) or with no air conditioner (or one ineffective at heating, as the case occasionally is). If the air conditioner works but gives hot/cold spots, you can compromise by using a lower-power resistance heater to fill in the cold spots. As for cooling in summer, where the air temperature is too high for fans alone to help, it will work best to combine air conditioning of the room with local fans (preferably Vornado of course ).
Whatever method you choose, it's not OK to use it in excess.
There is no simple answer.