Some positive results. v2 is uniformly higher than v1, but at nearly 1.8, not 1.5. Some of this could be because of a different calibration curve (which is worth checking on; I'll do that later).
While the pixel-to-pixel comparison yields values ~1.75, the aperture photometry is perhaps even more severe:
Those are a difference of a factor of 2.5, which is rather enormous.
So the code to add point sources to the maps and perform basic aperture photometry has started working to the point where I can share some results.
For maps with no underlying sky structure, only point sources, recovery is very nearly perfect, with PearsonR coefficients essentially equal to 1 until the peak amplitude of the point sources drops below 2.0 (and even then they only drop to ~0.997)
Below is a sample of the difference residual of output-input flux vs input flux for 40", 80" and 120" apertures
flux vs input flux will show just how correlated these are:
Note that for the blank maps, these are taken from simulations using a power law distribution of point sources from 0.1 to 2.0 Jy in magnitude (if we allow the range to extend above 2.0 Jy, the recoveries become even more closely correlated.
As for simulations with background sky structure we see similar patterns for the most part, however a few interesting pieces appear:
For the 4 different source ranges I ran, the 80" aperture had the highest pearson r coefficient across all simulations. Generally sitting around 0.95 with the 40" and 120" apertures floating from 0.75-0.9. This may be a fluke, however, I need to run more simulations to see if this is reproducible with other seeds.
Also, in the difference residuals we can clearly see an underrecovery of flux as input flux increases, which we should probably work to quantify, as seen by this example:
A demonstration that the spatial transfer function can be improved to ~80% at ~300 arcseconds by reducing the number of PCA components subtracted. However, image fidelity and noise backgrounds suffer, so these sorts of maps are probably less ideal for point source extraction. Below:
0 PCA 3 PCA 10 PCA 15 PCA
Last report was a bit of a fiasco. There were problems all over the place I didn't understand. I still don't but I've fixed them. My best guess at this point is that a pass-by-reference led to an unacceptable modification of an image. That doesn't even make sense - there was no place it could have happened - but, there you have it. So, going through the process step by step. This is the effect of smoothing an image:
Note from the 3rd figure that 100% recovery isn't reached until ~700 arcseconds. Next question: What is happening at large spatial scales in the flat-spectrum simulations?
No obvious problems there.
Hmm, no apparent problem here either, though one might ask why the two curves approach each other in sky05 (alpha=-0.5). So it appears that the reason for the bump up at low frequencies (long wavelengths) must be because of edge effects. After much hassle, I've addressed that by cropping images. Finally, the averaged results:
So we've got an Official Spatial Transfer Function. However, of course, we must note that there is a dependence on the atmosphere amplitude to source amplitude ratio: it appears that large-scale structure is *easier* to recover when the atmosphere is at higher amplitude. This makes sense: it is easier to distinguish faint astrophysical signal from bright atmosphere in this case. The reason I didn't run simulations to test this more is that the S/N ratio on small scales becomes poor for the low astrophysical amplitudes.
Below are plots of the spatial transfer function for a variety of simulation parameters. Each line represents the median (i.e., mean of the center 2 of 4) spatial transfer function of 4 realizations of the listed parameters.
A few unfortunate details are apparent:
Note for comparison that the recovery fraction for point sources is much more nearly 1, at least in the calibrator type maps. We need to wait for Jared's simulations with point sources in a full size map to confirm this, but it seems pretty likely that the STF is different for point sources and extended structures.
Just in case we were wondering, the V1 bolocat is completely inconsistent with a lognormal distribution, but is perfectly consistent (or... at least reasonably consistent...) with a power law distribution.
These plots show power-law fits (red) and lognormal fits (blue) to the data (black). It's pretty obvious that the lognormal is a bad fit, but in case you're unconvinced, the ks test for the "source flux" has a probability 1.6e-7, and it is the highest likelihood by 17 orders of magnitude out of the 4 flux types. By contrast, the simulations are on average (though not uniformly) more consistent with lognormal than powerlaw distributions:
Even in those examples where the KS test is slightly more favorable for the powerlaw distribution, the lognormal is a pretty good fit, and the failure points for the two distributions are in about the same place. The smoothness of the lognormal distribution is required to reproduce the observed distribution.
Note that the first 4 plots are for the whole BGPS survey. What about an individual field? For obvious reasons, I choose l30 again.
This gets to be a little more interesting - apparently the "source flux" has a tendency to pick up the power-law distributed background structure, since it is consistent with a lognormal (but note that it is also consistent with a powerlaw! The ks test doesn't really say definitively which is better). Although the fits look bad at high flux, note that this is a log-log plot and therefore the difference in probability is rather small. What does this all indicate? It's not entirely clear whether individual fields are genuinely more lognormally-distributed or whether the number statistics are just worse. However, even the source flux is consistent with a power-law, while many realizations of the simulations are not. Therefore, we should perform the next logical test - add point sources drawn from a power-law distribution (and a log-normal distribution?) and see what bolocat retrieves. We can at least say now that the point source contribution cannot be ignored, since there is no power-law distribution that can reproduce the observed bolocat flux distribution.
This figure shows the flux recovery of bolocat. The Y axis shows the Input flux (green) and the Filtered flux (blue/cyan/purple) with a few different large-scale filters. The red line is the 1-1 line. Obviously, bolocat doesn't recover everything that was there - we removed a lot of flux, so recovery fractions are in the few % range. However, bolocat does a much better job than the simple filter at recovering positive fluxes. Therefore, the filter function still isn't good enough.
that's all.
I still haven't dealt with http://bolocam.blogspot.com/2011/07/additional-problems.html, or essentiall any of http://bolocam.blogspot.com/2011/07/minor-ongoing-problems.html. However, they're probably related to this problem: % CLEAN_ITER_STRUCT: There were 1 bolometers with high weights: 13.5047 indices: 142 or flagged indices 142 which indicates that in a simulation in which there can be no outliers (in terms of weight/scale), there is one being rejected as an outlier. That indicates that weights are being computed incorrectly, despite the fact that scales look right (so far): Relsens calibration: Scaled to bolometer # 0% RELSENS_CAL_PCA: There were 0 NAN scales and 0 very low scales% RELSENS_CAL_PCA: Scale Median/Mad: 1.0005510+/- 0.010990833 led to 0 scales set to zero for a total of 0 bad bolos% RELSENS_CAL_PCA: Scales avg+/-std = 1.0007304355062783+/- 0.0103252880998329Relsens calibration: Scaled to bolometer # 0% RELSENS_CAL_PCA: There were 0 NAN scales and 0 very low scales% RELSENS_CAL_PCA: Scale Median/Mad: 0.99955294+/- 0.0093122063 led to 0 scales set to zero for a total of 0 bad bolos% RELSENS_CAL_PCA: Scales avg+/-std = 0.9990063123041220+/- 0.0096663438876613