Created 'beam_locations_0707.txt' from uranus_070702_o42 with a few contributions from g34.3_070630_o34. The rest were created by averaging over all of the beam location files
0707 distortion maps
They're consistent but not very close to each other.
beam locations & peak fluxes
The results of my distortion mapping work below. Note that, especially for 06, there are a LOT of cases where no beam locations correction (noBL) had a higher peak than the distortion corrected map. I have no explanation for this yet. More work to come...
PEAK COMPARISON BL: 5.95824 noBL: 3.09836 uranus_050619_o23 PEAK COMPARISON BL: 3.93725 noBL: 2.85479 uranus_050619_o24 PEAK COMPARISON BL: 15.5613 noBL: 523.035 neptune_050626_o19 PEAK COMPARISON BL: 16.4601 noBL: 16.0038 neptune_050626_o20 PEAK COMPARISON BL: 426.671 noBL: 375.174 mars_050627_o31 PEAK COMPARISON BL: 415.968 noBL: 413.327 mars_050627_o32 PEAK COMPARISON BL: 15.2464 noBL: 33.8593 uranus_050628_o33 PEAK COMPARISON BL: 34.6631 noBL: 35.5086 uranus_050628_o34 PEAK COMPARISON BL: 164.832 noBL: 403.189 uranus_050904_o31 PEAK COMPARISON BL: 216.820 noBL: 425.613 uranus_050904_o32 PEAK COMPARISON BL: 134.972 noBL: 156.196 uranus_050911_ob8 PEAK COMPARISON BL: 11.0957 noBL: 11.1993 neptune_060602_o30 PEAK COMPARISON BL: 12.0000 noBL: 11.0947 neptune_060602_o31 PEAK COMPARISON BL: 2478.20 noBL: 2365.17 mars_060605_ob1 PEAK COMPARISON BL: 2144.68 noBL: 2147.18 mars_060605_ob2 PEAK COMPARISON BL: 17.7354 noBL: 25.7041 uranus_060621_o29 PEAK COMPARISON BL: 18.7889 noBL: 25.6599 uranus_060621_o30 PEAK COMPARISON BL: 28.1957 noBL: 31.1013 uranus_060625_o46 PEAK COMPARISON BL: 23.0236 noBL: 27.8556 uranus_060905_ob6 PEAK COMPARISON BL: 18.8731 noBL: 28.4964 uranus_060906_o12 PEAK COMPARISON BL: 23.3481 noBL: 29.8294 uranus_060908_o13 PEAK COMPARISON BL: 20.6238 noBL: 26.9424 uranus_060909_o12 PEAK COMPARISON BL: 21.1049 noBL: 28.8533 uranus_060910_o12 PEAK COMPARISON BL: 24.0231 noBL: 32.0877 uranus_060914_o10 PEAK COMPARISON BL: 23.2590 noBL: 33.3496 uranus_060914_o11 PEAK COMPARISON BL: 24.0538 noBL: 30.0552 uranus_060919_ob9 PEAK COMPARISON BL: 355.467 noBL: 669.416 g34.3_070630_o34 PEAK COMPARISON BL: 246.803 noBL: 296.252 g34.3_070630_o35 PEAK COMPARISON BL: 724.874 noBL: 807.152 uranus_070702_o42 PEAK COMPARISON BL: 7.60370 noBL: 9.02960 uranus_070912_o27 PEAK COMPARISON BL: 98.8058 noBL: 88.9957 mars_070913_o22 PEAK COMPARISON BL: 82.2708 noBL: 89.3476 mars_070913_o23
sign errors
pinned down the problem. Was a minor sign error in the offsets. Why is it that simple sign errors are ALWAYS the hardest things to track down? Now, open question: should x,y scaling be free parameters or not? What I mean is, when I measure the positions of bolometers on the array using the planet map, should I allow the X and Y stretch (the bolometer spacing) to change? Should it be a uniform stretch in X and Y or should it be allowed to 'distort' too? My opinion is, none of the above: I'm measuring their actual positions (in terms of a fixed spacing) and therefore stretching or distorting to match the nominal positions is not necessary.
BL/noBL Mars
Mars using uranus_070702_o42 distortion map:
BL070706_o38_raw_ds1.nc_indiv0 134.8757324 -0.2325330 0.0721872 3282.5439453 2.0985703 2.1959550 3147.6682129 2.3311033 2.1237679BL070706_o39_raw_ds1.nc_indiv0 188.0588379 -0.0947621 -0.0643225 3782.3984375 2.1416662 1.9909470 3594.3395996 2.2364283 2.0552695BL070706_o40_raw_ds1.nc_indiv0 451.6955566 -0.0427303 -0.3303463 3461.9191895 2.1925442 2.0726480 3010.2236328 2.2352746 2.4029944BL070713_o36_raw_ds1.nc_indiv0 294.5898438 -0.2068889 0.0493107 5975.9482422 1.9935303 2.1527038 5681.3583984 2.2004192 2.1033931BL070713_o37_raw_ds1.nc_indiv0 813.5678711 -0.4597032 0.0361557 5515.4501953 2.0334179 2.1772604 4701.8823242 2.4931211 2.1411047BL070713_o38_raw_ds1.nc_indiv0 286.2934570 -0.1904230 0.0520797 5967.7011719 1.9853363 2.1624830 5681.4077148 2.1757593 2.1104033BL070713_o39_raw_ds1.nc_indiv0 773.2104492 -0.0816238 -0.3387871 5409.8344727 2.0317609 2.1736624 4636.6240234 2.1133847 2.5124495BL070714_o76_raw_ds1.nc_indiv0 244.2031250 -0.2079357 0.0572422 5312.0419922 1.9982606 2.1597764 5067.8388672 2.2061963 2.1025343BL070714_o77_raw_ds1.nc_indiv0 688.7365723 -0.1176772 -0.3138330 4706.1503906 2.0379941 2.1989338 4017.4138184 2.1556714 2.5127668BL070715_o45_raw_ds1.nc_indiv0 108.5583496 -0.0652246 -0.0818899 2397.9819336 2.1608708 1.9907858 2289.4235840 2.2260954 2.0726757BL070715_o46_raw_ds1.nc_indiv0 268.4321289 -0.2927990 -0.0692098 2236.4533691 2.1387808 2.0159581 1968.0212402 2.4315798 2.0851679BL070717_o47_raw_ds1.nc_indiv0 372.6528320 -0.2188239 0.0690587 7070.0649414 1.9820960 2.1670835 6697.4121094 2.2009199 2.0980248BL070717_o48_raw_ds1.nc_indiv0 498.4628906 -0.2993519 0.0410304 6636.1396484 2.0418782 2.1088474 6137.6767578 2.3412302 2.0678170BL070718_o41_raw_ds1.nc_indiv0 287.1591797 -0.2016468 0.0641356 6684.1699219 1.9891322 2.1470959 6397.0107422 2.1907790 2.0829604BL070718_o42_raw_ds1.nc_indiv0 841.6083984 -0.0809386 -0.3511798 5934.2832031 2.0332539 2.1427581 5092.6748047 2.1141925 2.4939380BL070725_o34_raw_ds1.nc_indiv0 245.6381836 -0.0810674 -0.0582435 5918.6728516 1.9807132 2.1547492 5673.0346680 2.0617807 2.2129927BL070725_o35_raw_ds1.nc_indiv0 697.7910156 0.1512802 -0.6174591 4873.1469727 2.3222449 2.0499101 4175.3559570 2.1709647 2.6673691BL070726_o33_raw_ds1.nc_indiv0 216.2124023 -0.0697458 -0.0627950 5056.3364258 2.1531394 1.9873902 4840.1240234 2.2228851 2.0501852BL070726_o34_raw_ds1.nc_indiv0 433.0415039 -0.0154374 -0.3017542 4586.9204102 2.1483676 2.0385454 4153.8789062 2.1638050 2.3402996BL070730_o23_raw_ds1.nc_indiv0 105.4489746 0.1025155 -0.2241490 2679.8452148 2.1567283 1.9786894 2574.3962402 2.0542128 2.2028384BL070730_o24_raw_ds1.nc_indiv0 440.7482910 -0.3968222 -0.0447321 2595.1044922 2.0618360 2.1782801 2154.3562012 2.4586582 2.2230122
distortion mapping failures
So, despite the fact that I clearly have a distortion mapper working, applying the distortion maps that come out of it do NOT improve the peak or FWHM of the data.
filling factors
to do: histogram per square degree, plot cumulative > some bin number (100 mJy, 300 mJy, 1 Jy) vs l, then do it again for b=+/-.1, +/-.3.
distortion mapping
Between sign errors, failures to fit, and.... I really don't know WHAT was going on, I finally figured out how to get the cursed distortion mapping to work. I still haven't even started testing, unfortunately. I don't know why I can fit the way the OP did, fitting R and Theta, and I apparently can't fit x,y. It's frustrating.
Flagger images
James requested sample images from the flagger for the methods paper. Below are images + description: .. image:: http://3.bp.blogspot.com/_lsgW26mWZnU/SXHsZcqegtI/AAAAAAAAEs4/3PjUtU2IXtQ/s400/sample_waterfall_070911_o15_highFnoisepng.png This is a "noise-dominated" scan in the sense that the high/low pixels are set by noise, not signal. Despite the clear high frequency noise, this image actually maps out pretty well - I think the high-noise bolometers get downweighted and the high/low pixels probably get clipped by my hot pixel rejection procedure. .. image:: http://3.bp.blogspot.com/_lsgW26mWZnU/SXHsY7Rgq3I/AAAAAAAAEsY/0NT8IB26JUo/s400/flagger_marked_source_050708_o15.png An image of the galactic center with a source southeast of center identified. .. image:: http://1.bp.blogspot.com/_lsgW26mWZnU/SXHsYyHCEGI/AAAAAAAAEsg/3iwoklY0--0/s400/sample_waterfall_050708_o15_glitchandsources.png A scan from the GC image above. I forgot to mark the source, I should go back and do that. The glitch is obvious. .. image:: http://1.bp.blogspot.com/_lsgW26mWZnU/SXHsZHVZ4OI/AAAAAAAAEso/pOXwGFn7rGg/s400/sample_waterfall_050708_o15_glitchflagged.png I drew a box to flag out the region affected by the glitch. .. image:: http://4.bp.blogspot.com/_lsgW26mWZnU/SXHsZG_MmWI/AAAAAAAAEsw/RA6tqAWMnLM/s400/sample_waterfall_050708_o15_glitchgone.png This is what happens when I redraw after flagging out the glitch - the colorbar is rescaled and no more glitch. .. image:: http://4.bp.blogspot.com/_lsgW26mWZnU/SXIDILh3hfI/AAAAAAAAEtQ/cMuOhlFO_fA/s400/sample_waterfall_050708_o15_glitchandsources_marked.png Timestream with glitches and sources marked (one pixel in the map is hit by 3 different points in this scan). .. image:: http://4.bp.blogspot.com/_lsgW26mWZnU/SXIDH3uIjHI/AAAAAAAAEtI/vijTeEWf_nA/s400/sample_waterfall_050708_o15_glitchandsources_gray.png Grayscale version of above (ok, I lied about grayscale being impossible) with a different pixel marked. .. image:: http://1.bp.blogspot.com/_lsgW26mWZnU/SXIDHbYLbiI/AAAAAAAAEtA/nSh2rM6GhMc/s400/flagger_marked_source_footprint_050708_o15.png A zoom-in around the 'kidney bean' source with the Bolocam footprint overlaid and a pixel marked. Note that this pixel corresponds to the 3 points in the color waterfall above.
0709 rotators fixed, latest run
0709 11-13 needed to be 83.88 degrees, they're all fixed now. I flagged the l060 070911 maps; I'm not convinced that was weather, there were some fluctuations up to 2400 Jy! Maybe a cloud would do that though.
Problems in the latest run:
l359 - missing files? Reading files from /scratch/sliced/INFILES/l359\_infile.txt FIELD v0.7\_l359 BEGUN at Fri Jan 16 20:11:34 2009 MRDFITS: File access error % HEULER: ERROR - First parameter must be a FITS header or astrometry structure l012 - missing files? % READFITS: ERROR - Unable to locate file /scratch/adam\_work/l012/060614\_o10\_raw\_ds5.nc\_indiv13pca\_map01.fi ts
