AF Consistency Comparison: Phase Detect vs Contrast Detect

Reikan FoCal software has the ability to run AF Consistency tests, and during the development of some new features in FoCal I noticed something about contrast detect autofocus that didn’t quite go with my expectations, so I’ve run a few tests and captured some quite interesting data.

This all stems from my belief that contrast-detect autofocus (“Live” mode AF) – which uses the actual image sensor for determining the focal point – was inherently very accurate.  A few months back, FoCal gained the ability to fire off a contrast detect shot at the start of the Fully Automatic test, with the assumption that this would show a good peak value for focus.  But it didn’t… the quality was always a bit poor.

Forward on to now – we’ve just released the Mac and Windows version of FoCal with the new QuickCal test which makes extensive use of various focus methods in order to rapidly determine the autofocus microadjustment value.  During the development, I’ve got a good understanding of the differences between the specific contrast- and phase-detect focus implementations in the cameras.

FoCal’s AF Consistency Test

First, I need to explain a bit about the tests in order to show what the results mean.  Bear with me – it’s not too complicated!

The AF Consistency test works by taking a series of shots from the camera – defocusing in between the shots – and analysing the individual images for  Quality of Focus (QoF).  The final Consistency of Focus (CoF) result is determined by comparing (as a percentage) the average quality of all the results against the quality of the best individual shot.  This means that a very consistently focused set of shots will have a CoF very close to 100%, but if there is lots of variability the percentage will drop

We take CoF values of 95% and above as indicating adequately performing autofocus, although typically a decent DSLR with well behaved lenses (usually Canon L-series or Nikon AF-S lenses) will achieve 97% regularly.

FoCal’s Peak Focus Determination

Returning back to what started all this… We wanted a way of finding out the best possible QoF value that could be achieved during the test.  This is both useful for users when displayed on the chart, and also can be used to assist our Autofocus System Model in obtaining the best possible predicted result for calibration.

As I said above, the assumption was that contrast detect autofocus would provide this value for us, but it doesn’t.  So we took it on ourselves to write a better contrast detect autofocus algorithm!

It’s not actually as exciting as it sounds because we have one big thing on our side: time.  A camera has to focus very quickly – people get grumpy if focusing takes a second – whereas we have a bit more time so we can “check our workings” to some extent.  There are improvements to be made, but the recent releases of FoCal have a fairly good algorithm to determine peak focus.

So what’s the point of all this?

I wanted to present a few graphs.  I’ve taken the results of the AF Consistency test (shot on a Nikon D800, Canon 5D Mark II, 7D and 6D) over 10 test shots for each.  All the tests ran the peak focus detection algorithm I described above, so we can get 2 numbers of interest from this:

1. The “Focus Error” is the average QoF value for the test divided by the QoF of the peak focus (determined with the peak-focus detection algorithm above).  If this value is above about 0.1 it shows a significant error in focus compared to ideal.  This is shown by the blue bar in the charts below.

2. The “CoF Drop” is the difference from perfect CoF (100%) and the test result, divided by 100.  So a good result of 95% would be shown with a red bar height of 0.05 below.

So, to recap in simpler terms:

  • The higher the blue bar, the more out of focus the shots were (on average)
  • The higher the red bar, the more variability there is in the focus system.
For each camera, I ran the test 4 times – twice with phase-detect autofocus, and twice with contrast-detect.  For each type of focus, I ran with both defocusing between shots (turning the lens towards infinity so the autofocus system has to start from scratch), and no defocusing at all (so just taking one shot after the other with no change in focus between shots).

It’s worth pointing out here that the cameras all had their AF Microadjustment/Fine Tune calibrated before running the tests.  If there was a fixed focus error (like you would have from a badly adjusted camera), the blue bar would be taller  without any effect on the red bar in the charts below.

The Results

The Nikon shots were taken with the Nikon 50mm f/1.4 AF-S lens, and the Canon shots with the Canon EF 100mm f/2.8L IS Macro lens.

Here are the charts:

Nikon D800 AF Consistency
image-900

Canon 5D Mark II AF Consistency
image-901

Canon 7D AF Consistency
image-902

Canon 6D AF Consistency
image-903

What does it all mean?

It’s fairly plain to see that the Nikon D800 and the Canon 6D perform well.  The blue bar is low which shows the shots would all be nicely in focus, and the red bar is low showing that each shot would be very similar to the others.

The 5D Mark II and the 7D charts look quite scary initially, but if we analyse phase detect autofocus first we can see that the 5D Mark II isn’t much different to the Nikon D800, although when you don’t defocus between shots the 5Dmk2 seems to perform worse which is a bit surprising and counterintuitive.  Looking at the 7D we see the same thing but to a larger extent – non defocused phase detect AF is starting to get quite bad indeed.

But the big surprise for me was the difference between phase detect and contrast detect.  The contrast detect results on the 5Dmk2 and 7D show a big focus error with a reasonable amount of variability between the shots too.  This just translates to out of focus shots – plain and simple.

Conclusion (for now)

I did wonder if there were some problems with the analysis, but some carefully conducted manual tests showed the same results – Live View focus is just not very good!

…At least for the cameras that I used.  This was a fairly limited test but I’ll add the 5D Mark III, 1D Mark IV and D7000 to it shortly, as well as making the generation of results a bit more automated.

So watch this space for more info!

About

Lead developer of Reikan FoCal automatic lens calibrations software.

Posted in Interesting
10 comments on “AF Consistency Comparison: Phase Detect vs Contrast Detect
  1. Michelle says:

    Wish you could work with Magic Lantern guys and implement your slower but more accurate live view focusing routine through ML!

    Thanks for the article, Rich.

  2. Jim Peterson says:

    Great research and testing! I would love to see similar testing on cameras such as the NEX7, which relies exclusively on CDAF, and also the new A99 that uses a combination in a very new implementation.

  3. Very interesting results. Looks like the 6d is doing pretty well for focus accuracy and is a step up from the 5dii.

    Looking forward to the 5d3 and 1dx results!

  4. Ronnie Chan says:

    Interesting,looks like Canon 6D is pretty a good choice as my 2nd body

    • Rich says:

      Hi Ronnie

      The 6D is a superb camera. We have one in the lab and I often use it out and about. It’s a great size, has brilliant IQ, superb focusing and I’m particularly fond of it with the Canon 40STM lens as it makes a really compact but excellently performing setup. In fact, most of the time I pick the 6D up over the 5Dmk3 for any personal shooting!

      Rich

  5. Marco says:

    I upgraded to Focal pro (1.8.1 W) and tested my 7D with 17-55 2.8 IS for center focus point consistency.
    Using PDFAR: 97,6 % (consistency of focus)
    Using PDNone: 97.3 % (consistency of focus)

    So (with running the test only once i found no difference beteen defocussing or not in between shots.

    However, the relation between the COF in the report and the red and blue bar is not really clear (after reading the description multiple times).

  6. jacqui mills says:

    That is an Excellent chart Rich ! I have been researching 7D Vs 6D Vs 5DM3 for a few weeks trying to decide which one to get. The decisions came down to 6D Vs 5DM3 ( For IQ) and I was hoping that the 6D didn’t disappoint me with the AF system ( on moving objects, animals and sports). It seems that from the plots above, the AF system of the 6D is not that disappointing ?

    • Rich says:

      Hi Jacqui. The 6D does indeed have really reliable AF, especially on the centre point (which is also very sensitive – down to -3EV). The other points on the 6D are linear (either horizontal or vertical rather than cross type) which may cause problems with certain subjects, but in my experience of using the 6D for a few months it seems to be surprisingly robust on those outer points. All in all, I really like the 6D for both IQ and AF, as well as the extra features like WiFi (which I use quite a lot) and the GPS (I use it less so, but it’s useful sometimes). I’m glad you found the post useful.

  7. Graham says:

    Thank you. Real data instead of guessing and repeating what someone else said that someone else said.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>