Recovery as a function of bolometer noise part 2

In the Experiment 7 simulations I was running, I observed greater noise than expected, causing me to question the results of the previous post. I therefore ran Experiment 8, which is the same as Experiment 6 from that post with a larger (320x320) map with a larger step size. The noise recovery remains linear, but the scaling is quite different - a factor of ~4 instead of ~12. The step size is the most likely culprit, since an 8x larger step size should result in sqrt(8)~2.8 worse noise per pixel.

There are some curious / worrisome artifacts that turn up and are evident in the recovery fraction plot. For the low-noise cases, the middle bolometers get totally flagged out because they are over-weighted (by orders of magnitude).

So I'm forced to explore via pyflagger. I will almost certainly need to re-run all experiments after making a change to how weights are computed. Well, it turns out the problem is that those 28 bolos are scaled to zero, even though there is nothing obvious (or even suggestive) in their timestream plots. This is only true when varyrelscale is off. Apparently varying the relative scales leads to a different problem. AHA! The noise is so low that the relative scales are SO well correlated that the signal is enough to cause problems! A plausible solution is therefore no change to the pipeline, but to add minimal (nominal) noise to the relative scales to increase the MAD so that the others don't get flagged out. So I added a 1% variation, which prevented flagging at the scale stage, but there are still some disturbing artifacts in the map:

Unfortunately, this problem requires further examination in detail. Exp 9/10 should probably be gaussians and airys on larger step-size maps, but the solution will require something else, possibly even a change in the pipeline. On the plus side, I think I can re-run experiment 7 with a factor of 4 instead of 12 scaling for the noise and expect it to work.

Comments