I'm not going to comment about flaring NG at remote wells, but in oil refineries flares are not burned just to get rid of unwanted gas. they are safety features used to help control the processes.
All vessels, tanks, and piping in a refinery is rated for a a maximum pressure and protected by relief valves. When a pressure gets too high the relief valve lifts and sends the product to flare. Understand that a typical oil refinery has a mind-boggling complexity that can't really be comprehended until you've worked inside one for a while. Hundreds of miles of piping, hundreds of thousands of valves.
All refineries would recover all that gas and use it to fire their heaters and boilers, if they could. But that is not easy to do.
In Southern California it is illegal to flare under most circumstances.
If a refinery here flares off gasses it gets hit with a really big fine. Really big.
There are some exceptions but even they usually include a FEE.
The process required to capture all excess gasses to keep them out of the flare during process upsets (like a power outage for example) is very complex and expensive which increases the cost to refine, which eventually hits the consumer at least a little.
The capability to recover all that gas does not exist in many places but eventually they'll all be required to upgrade to do it (or close down if they can't afford to do it).
Here's a pic of the 5 liquid ring compressors we installed several years ago as part of a flare gas recovery project that I was the lead mechanical engineer on (the compressors cost a half mil each, this part of the project cost around $40 mil IIRC).
