Reikan FoCal software has the ability to run AF Consistency tests, and during the development of some new features in FoCal I noticed something about contrast detect autofocus that didn’t quite go with my expectations, so I’ve run a few tests and captured some quite interesting data.
This all stems from my belief that contrast-detect autofocus (“Live” mode AF) – which uses the actual image sensor for determining the focal point – was inherently very accurate. A few months back, FoCal gained the ability to fire off a contrast detect shot at the start of the Fully Automatic test, with the assumption that this would show a good peak value for focus. But it didn’t… the quality was always a bit poor.
Forward on to now – we’ve just released the Mac and Windows version of FoCal with the new QuickCal test which makes extensive use of various focus methods in order to rapidly determine the autofocus microadjustment value. During the development, I’ve got a good understanding of the differences between the specific contrast- and phase-detect focus implementations in the cameras.
FoCal’s AF Consistency Test
First, I need to explain a bit about the tests in order to show what the results mean. Bear with me – it’s not too complicated!
The AF Consistency test works by taking a series of shots from the camera – defocusing in between the shots – and analysing the individual images for Quality of Focus (QoF). The final Consistency of Focus (CoF) result is determined by comparing (as a percentage) the average quality of all the results against the quality of the best individual shot. This means that a very consistently focused set of shots will have a CoF very close to 100%, but if there is lots of variability the percentage will drop
We take CoF values of 95% and above as indicating adequately performing autofocus, although typically a decent DSLR with well behaved lenses (usually Canon L-series or Nikon AF-S lenses) will achieve 97% regularly.
FoCal’s Peak Focus Determination
Returning back to what started all this… We wanted a way of finding out the best possible QoF value that could be achieved during the test. This is both useful for users when displayed on the chart, and also can be used to assist our Autofocus System Model in obtaining the best possible predicted result for calibration.
As I said above, the assumption was that contrast detect autofocus would provide this value for us, but it doesn’t. So we took it on ourselves to write a better contrast detect autofocus algorithm!
It’s not actually as exciting as it sounds because we have one big thing on our side: time. A camera has to focus very quickly – people get grumpy if focusing takes a second – whereas we have a bit more time so we can “check our workings” to some extent. There are improvements to be made, but the recent releases of FoCal have a fairly good algorithm to determine peak focus.
So what’s the point of all this?
I wanted to present a few graphs. I’ve taken the results of the AF Consistency test (shot on a Nikon D800, Canon 5D Mark II, 7D and 6D) over 10 test shots for each. All the tests ran the peak focus detection algorithm I described above, so we can get 2 numbers of interest from this:
1. The “Focus Error” is the average QoF value for the test divided by the QoF of the peak focus (determined with the peak-focus detection algorithm above). If this value is above about 0.1 it shows a significant error in focus compared to ideal. This is shown by the blue bar in the charts below.
2. The “CoF Drop” is the difference from perfect CoF (100%) and the test result, divided by 100. So a good result of 95% would be shown with a red bar height of 0.05 below.
So, to recap in simpler terms:
- The higher the blue bar, the more out of focus the shots were (on average)
- The higher the red bar, the more variability there is in the focus system.
It’s worth pointing out here that the cameras all had their AF Microadjustment/Fine Tune calibrated before running the tests. If there was a fixed focus error (like you would have from a badly adjusted camera), the blue bar would be taller without any effect on the red bar in the charts below.
The Nikon shots were taken with the Nikon 50mm f/1.4 AF-S lens, and the Canon shots with the Canon EF 100mm f/2.8L IS Macro lens.
Here are the charts:
What does it all mean?
It’s fairly plain to see that the Nikon D800 and the Canon 6D perform well. The blue bar is low which shows the shots would all be nicely in focus, and the red bar is low showing that each shot would be very similar to the others.
The 5D Mark II and the 7D charts look quite scary initially, but if we analyse phase detect autofocus first we can see that the 5D Mark II isn’t much different to the Nikon D800, although when you don’t defocus between shots the 5Dmk2 seems to perform worse which is a bit surprising and counterintuitive. Looking at the 7D we see the same thing but to a larger extent – non defocused phase detect AF is starting to get quite bad indeed.
But the big surprise for me was the difference between phase detect and contrast detect. The contrast detect results on the 5Dmk2 and 7D show a big focus error with a reasonable amount of variability between the shots too. This just translates to out of focus shots – plain and simple.
Conclusion (for now)
I did wonder if there were some problems with the analysis, but some carefully conducted manual tests showed the same results – Live View focus is just not very good!
…At least for the cameras that I used. This was a fairly limited test but I’ll add the 5D Mark III, 1D Mark IV and D7000 to it shortly, as well as making the generation of results a bit more automated.
So watch this space for more info!