Full frame of the contrast/saturation boosted image.
How good do your DSLR darks need to be?
This is to see how "careful" we need to be when we take DSLR dark frames, including
Test Setup
The camera used for this test is a modified Nikon D800. It has fairly low dark current (0.295 e-/sec at about 25.4 deg C ambient, and 0.032 e-/sec at about 6 deg C ambient), but is not the lowest one at this moment (2015). (D810A is probably the lowest one, about 2x to 3x lower than D800.) Therefore, D800 may be considered as a representative model as for 2015.
10 dithered 3-minute on-sky exposures were taken under ambient temperature of 7.1 to 7.2 deg C. The camera body was sufficiently warmed up and the ambient temperature was pretty stable. The optics is Takahashi E180ED (D=180mm, F=2.8). The location is Maunakea Visitor Information Center where the sky brightness is about 21.5 to 22 mag/sq-arcsec, very dark. The dark sky is offset by the fast optics, so the photon rate on the sensor should not be terribly lower than more typical cases (somewhat slower optics under a somewhat brighter sky). Another key point to know is that the dark noise is much lower than the sky photon noise under my imaging conditions (see this article.)
Several scenarios of dark frames are included:
Image calibration and stacking was done in PixInsight. Four different processing scenarios were used (from the two combinations):
All images are equally linearly stretched and saturation boosted. The contrast stretch and saturation boost are somewhat stronger than what we normally do, in order to better show the differences among all the cases. However, they are not ridiculously strong. Below shows the full image of the stretched result, and as can be seen, the stretches are not crazy. A region of 300x300 pixels near the center is extracted, and 2x enlarged with nearest neighbor method (no interpolation).
|
In all the above scenarios, it is expected that the combination of dark optimization + clipped stacking + 20 dark frames taken at the right temperature should give the best results. In the following, all results are compared against this best-case scenario. Move your mouse over the images to see the best case and compare (unless otherwise mentioned). In the figure caption, the difference between the best case is in red.
1. When temperature is right.
1a. sigma clip or not
|
To my surprise, the above mentioned "best-case" (mouse on) is not the best. The stacked image without sigma clipping (mouse off) is apparently better when everything else is right. The unclipped stack has a few hot pixels leftover but overall the image background is less noisy (confirmed with measurements of pixel standard deviation in the background). This is caused by the clipping parameters I used in PixInsight (sigma low = 3.5, sigma high = 2.0, the latter being kind of too aggressive), which removed too many useful data. Relaxing the clipping parameters does improve the result. The conclusion here is that when everything else is right, a moderate clipping is enough to clean up the image. Don't be too aggressive.
1b. dark optimization or not
|
With the same amount of sigma clipping and when everything else is right, dark optimization doesn't bring much difference. However, we can confirm that it did try to do something, as the image background has slightly different color and brightness.
1c. no dark optimization, no sigma clipping
|
Again not much difference, except for a few hot pixels.
2. Right temperature, but only a few dark frames
We already know dark optimization doesn't do much when dark frames are taken under the right temperature. So I won't compare results with/without dark optimization.
2a. sigma clipped
|
The difference is really small. This says that when dark noise is not the major noise term (sky photon noise is), we do not need many dark frames. As long as the dark frames are taken at the right temperature and when the images are dithered, just a few dark frames will be sufficient.
2b. no sigma clipping
|
More hot pixels than the case of many dark frames (case 1c). Otherwise it looks fine.
3. No dark frames at all
3a. sigma clipped
|
Appears very similar to the best-case. There is only a hint of higher background noise in the image of no dark subtraction.
3b. no sigma clipping
|
Not surprisingly, it is a mess. Images like this are not useable.
4. Dark temperature too high
4a. no dark optimization, no sigma clipping
|
There is difference, but not overwhelming. If we look carefully, we see pixels darker than what they should be. This is the over-subtraction caused by the fact that the dark frames are too hot.
4b. no dark optimization, sigma clipped
|
Looks very good. Sigma clipping takes care of the dark pixels we see in case 4a.
4c. with dark optimization, no sigma clipping
|
Now here comes a surprise. PixInsight's dark optimization doesn't seem to work well. However, if we look carefully, we see that it takes care of the dark pixels in case 4a, but leaves hot pixels. This means that PixInsight's algorithm pays more attention to the majority of the background pixels, rather than the small amount of hot pixels.
4d. with dark optimization, sigma clipped
|
Looks quite good. Sigma clipping seems to be the key, not dark optimization.
5. Dark temperature too low
5a. no dark optimization, no sigma clipping
|
The opposite of case 4a. Instead of leaving dark pixels, here there are hot pixels caused by under-subtraction.
5b. no dark optimization, sigma clipped
|
Once again, sigma clipping takes care of the hot pixels.
5c. with dark optimization, no sigma clipping
|
Opposite to 4c, here there is hint of over-subtraction at the location of hot pixels in 5a. However, overall it looks OK.
5d. with dark optimization, sigma clipped
|
Looks very OK.
6. Wrong dark or no dark?
6a. no dark vs hot dark
 mouse off: no dark subtraction, sigma clipped mouse on: hot dark, dark optimized, sigma clipped |
The difference is very small. It is hard to say which one is better.
6b. no dark vs cold dark
mouse off: no dark subtraction, sigma clipped mouse on: cold dark, dark optimized, sigma clipped |
Again the difference is very small.
Summary
It's a bit frustrated to see these results. What it implies is that all my past effort on carefully taking dark frames is unnecessary. In my imaging condition, the key seems to be employing sigma clipping in the stacking. Everything else doesn't matter that much. It doesn't matter whether darks are taken at the right temperature or not. It doesn't matter how many dark frames are used. It doesn't even matter whether there are dark frames or not! Dark optimization can help, but the effect is way smaller than the effect of sigma clipping. The only thing that matters is the use of sigma clipping. Only when everything is theoretically right (correct temperature for darks, enough dark frames, dark optimization), aggressive sigma clipping can slightly damage the image.
After showing that cooling is not necessary in my imaging condition (see this article), now I can go as far as saying that taking dark frames is not necessary with modern DSLRs. This seems absurd, but the data support this.
Now, there are a few caveats. Here I only look at the effect of dark noise in the background and the effect of hot/cold pixels, in the center of the image. We know that DSLR images behave differently along the edges. There could be ampglow. The temperature there could be higher too, causing higher dark current. These are not tested here, as these effects on D800 are not that strong. However, different cameras can behave differently. To cleanly remove these edge effects, it is still highly recommended to carefully subtract dark.
Another caveat is that my site is relatively cool and the camera is relatively quiet, so the dark noise is weaker than sky photon noise. For those who image under much warmer weather, or those whose cameras have higher dark current, the importance of dark noise will be much higher. Under such circumstances, careful dark subtraction (i.e., enough frames, similar temperature) may be still necessary for removing the fixed dark pattern in the images. What my results here show is that dark subtraction may be more tolerable than what we used to believe. I don't want to jump to a conclusion saying that dark subtraction is totally not needed. You should at least make a test on your own images with/without dark subtraction, to draw your own conclusion.
Finally, for sigma clipping to work, the images have to be dithered. In many cases, even if we do not dither the guided exposures, they are naturally dithered by the flexure of the instrument. However, the flexure induced dither often happens along a fixed direction, which may introduce "walking noise" to the stacked image. Sigma clipping may or may not take care the walking noise. It is still highly recommended to dither the image through guiding to somewhat randomize the dither footprint.
