I cut down the number of maps made by the pipeline from 80 to 30 by merging regions with severe overlap. I don't think this will result in any net change in efficiency of the mapping process because a few of the maps will go over to swap, but it will reduce the amount of overlap / noisy edges and make debugging simpler. It should also reduce hard drive usage once I decide that I'm satisfied with the results and remove the old (v0.6) redundant data. As part of this process, the reference fields for coalignment have been re-set to the v0.6.2 coadds + some (crappy) mosaics from v0.6.2 so that we no longer need to rely on 3x1's for the alignment. See /data/bgps/releases/v0.7/readme.txt for details (reproduced below) Created 01/14/09 Version 0.7 does not exist yet. This is the 'release notes' indicating what will change -Overlapping redundant fields will be eliminated Mapping of field name to area covered, followed by number of observations included: gemob1: l189-l192 11 - 2 = 9 w5: l135-l138 46 = 46 l133: l133-l134 34 - 1 = 33 l111: l110-l111 88 = 88 l086: l085-l086 17 = 17 l082: l080-l084 33 = 33 l077: l075-l080 43 - 1 = 42 l072: l070-l074 10 - 2 = 8 l065: l059-l069 19 = 19 l055: l053-l058 10 - 1 = 9 l050: l048-l052 19 = 19 l045: l043-l047 20 - 4 = 16 l040: l038-l042 33 - 4 = 27 l035: l034-l037 53 = 53 l032: l031-l033 61 = 61 l030: l030 61 - 3 = 58 l029: l028-l029 53 - 4 = 49 l024: l021-l027 29 = 29 l018: l015-l021 29 - 3 = 26 l012: l011-l014 7 = 7 l009: l008-l010 11 = 11 l006: l005-l007 15 - 1 = 14 l003: l002-l004 25 - 1 = 24 l001: l001 20 = 20 l000: l000 23 - 1 = 23 l359: l359 14 = 14 l357: l356-l358 9 - 1 = 8 l354: l353-l355 3 = 3 l351: l350-l352 9 = 9
Articles in the bgps category
Gem OB1
With Miranda's help, I discovered some serious errors in Gem OB1 and fixed them.
- L189 was using L134 data for no apparent reason
- many September 2007 observations have the WRONG rotation angle in place in array_params! Unfortunately it's not clear yet which ones suffer from this problem: I need a list of when rotangle was/was not used, and it looks like it'll be a pain to fix the problem.
- GemOB1link wasn't mapped, it will be now.
Back to work
I started working again yesterday, mostly looking at the 31PCA reductions I did. I have no strong opinions yet on their quality. It's time to work towards a final data release, though, and to that end, I'm going to start removing duplicate fields. e.g. l023 and l024 have identical data, so only one of them should ever be used.
L111 observation notes
In L111, the failures noted in the previous post are specific to September 2005. By comparing the pointing model images to the "raw cso pointing" and "no pointing model" (the latter still have FAZO/FZAO applied, all that has been done to them is aberration/nutation and precession correction) it appears that the Sep 05 model is simply wrong. I haven't had time to check on W3/4/5 and L033, but presumably they have a similar issue. W3/4/5 includes late July 07 and September 06. L033 includes July 05, June 06, and July 07. I will need to generate pointing plots for September 05 and 06, and probably regenerate June 06. I have the rest and I think Meredith's models are fine for those dates - they certainly don't create recognizable errors in the maps. It is possible that my pointing_model.pro has some pointing models (the problematic ones) entered incorrectly. It would be nice to get those double-checked.
Pointing model failure
I attempted to map L111, L033, and W3/4/5 with the pointing model corrections applied. That was a failure. Two images are attached to illustrate the problem - relative pointing is clearly not correct (middle of L33, bottom left of L111). So.... possibilities:
- Pointing models are incorrect
- Pointing model signs have been misinterpreted (don't think this one is possible based on other graphs)
- Uhh... ideas?
Next step, I'll use the individual maps with no ptg mdl and find out what the offsets should be....
Deconvolution vs. Not
I've done some by-eye comparisons of deconvolutions vs no deconvolution. The no-deconvolution clearly does better on the bright sources: probably the deconvolved beam size is a little bit different from (larger? smaller?) the actual source size. Deconvolution does better in putting noise from noisy regions into the noisemap and keeping it out of the astromap. Both might be useful - the deconvolved maps may end up being prettier, but the no-deconvolve maps will probably have more reliable fluxes. This all probably relies on testing / simulation.
4 errors
l047, l083, l359 (l000 is in /usb, l359 is in /scratch), l136p15 Only 83 is a code error, the others are file location errors. l083 is an error in 'pixshift' with subscripting and will require real debugging.
def_user_common modified
The NP doesn't work without USER_COMMON defined because at a single point it makes use of the 'which' function. def_user_common is a disaster, though, because it calls get_screen_size(), which forces an X connection to be launched, which means that if you start a remote session and close X IDL crashes. This has been a constant nagging problem and has cost me ~a few days of work. So I commented out the offending (and offensive) line. The worst of it is, I'm not even convinced there's anything that uses any of that information. The common block is only needed to get paths, so which could have been written better. The pipeline ground to a halt for an unknown reason at an unknown point so I added a lot more timing outputs to try to figure out what's going on. Back to the grind...
flagger
Discovered that the flagger causes some pretty serious problems if you try to run it on a non-coadd for obvious reasons: there's practically no coverage! Individual maps should NOT go through the flagger, so I have added a piece of code that turns off the flagger if only one observation is being used. That code is in map_iter where the flagger is called.
v0.6.1 done