Adam Ginsburg's blog

Articles by Adam (keflavich@gmail.com)

• Python 64-bit on Mac OS X 10.6 Snow Leopard

  After yesterday's disastrous attempt to install various python packages, I started from scratch. First, I got rid of all of my python frameworks (backed up but removed from the path). Then, I compiled python 2.7 from scratch:

  I got some help from http://blog.mahmoudimus.com/2009/12/python-2-6-4-and-twisted-9-on-os-x-10-6-snow-leopard/ ./configure --enable-framework --enable-universalsdk=/Developer/SDKs/MacOSX10.6.sdk MACOSX_DEPLOYMENT_TARGET=10.6 --with-universal-archs=intel -with-readline-dir=/usr/localmake -j 17make -j 17 test make results:

  Python build finished, but the necessary bits to build these modules were
  not found:
      _bsddb
      dl
      gdbm
      imageop
      linuxaudiodev
      ossaudiodev
      spwd
      sunaudiodev
  To find the necessary bits, look in setup.py in detect_modules() for the
  module's name.
  

  I'm not concerned about these - I don't use any of them and I assume I need to install some other packages to get them to work. During make test, I had two failures that resulted in "python crash" pop-up boxes:

  test_subprocess.
  this bit of output is from a test of stdout in a different process ....
  this bit of output is from a test of stdout in a different process ...
  test_sunaudiodev
  

  Then, I got some malloc errors:

  test_ioTesting large file ops skipped on darwin.It requires 2147483648
  bytes and a long time.Use 'regrtest.py -u largefile test_io' to run
  it.Testing large file ops skipped on darwin.It requires 2147483648 bytes
  and a long time.Use 'regrtest.py -u largefile test_io' to run
  it.python.exe(22914,0x7fff70d3ebe0) malloc: ***
  mmap(size=9223372036854775808) failed (error code=12)*** error: can't
  allocate region*** set a breakpoint in malloc_error_break to
  debugpython.exe(22914,0x7fff70d3ebe0) malloc: ***
  mmap(size=9223372036854775808) failed (error code=12)*** error: can't
  allocate region*** set a breakpoint in malloc_error_break to
  debugpython.exe(22914,0x7fff70d3ebe0) malloc: ***
  mmap(size=9223372036854775808) failed (error code=12)*** error: can't
  allocate region*** set a breakpoint in malloc_error_break to
  debugtest_ioctl
  

---

• Python magic / advanced numpy indexing

  Yeah, indexing python arrays should really be easy. Stefan van der Walt's page In [85]: bi = (f.bolo_indices[np.newaxis,:] + zeros([7751,1])).astype('int')In [86]: whc = (whscan[:,np.newaxis] + zeros([1,107])).astype('int')In [87]: array2d[whc,bi] = temp2d

---

• Python: one-line arrays

  Ahhh, refreshing: `` whscan = asarray([arange(scanlen)+i for i in scans_info[:,0]]).ravel()`` Not like IDL, which takes at least 4 lines b/c of the variable declaration. There's probably a better way to do that too.

---

• Python: setting matplotlib defaults

  Setting matplotlib defaults is a lot more difficult than it should be. matplotlib.defaultParams['image.origin']='lower'matplotlib.defaultParams['image.interpolation']='nearest'

---

• python stuff

  to get imshow defaults to be nearest neighbor, need to edit ~/.matplotlib/matplotlibrc. Still don't know how to change default command-line output format.

---

• Quicksilver and Awesome

  Set up f17-19 today to play amusing sounds. Also, recalled that I use ScreenSaverEngine to lock my screen with a shortcut key. Quicksilver triggers are the way to do it. Mac is stupid w/o quicksilver.

---

• Quicksilver sounds

  To switch sound source from the command line: switchaudio Use this to make scripts such as: #!/bin/bash/Users/adam/humor/SwitchAudioSource -s "Built-in Line Output"afplay /Users/adam/humor/losinghorn.wav/Users/adam/humor/SwitchAudioSource -s "Built-in Output" Then make triggers in Quicksilver by:

  1. Go to trigger pane, make new hotkey trigger
  2. press "." to allow you to type a command
  3. make sure the action is "Run"
  4. hook up a hotkey
  5. if it doesn't work, just try again. Persist through crashes, they happen often.

  Follow-up: You can also control the volume! http://discussions.apple.com/thread.jspa?threadID=585781 `` osascript -e 'set volume output volume 100'``

---

• RATRAN on Mac OS X

  Mac OS X doesn't like the defaults built in to RATRAN. It died unhappily with errors like: ld_classic: can't locate file for: -lcrt0.o and ld: warning: in /usr/local/lib//libcfitsio.a, file is not of required architecture In order to get it to run, I had to do the following:

  1. Install CFITSIO with CFLAGS="-arch x86_64 -arch i386 -g -O2" to /usr/local/lib
  2. Edit the sky/Makefile OPT variable (line 23) to read: OPT = -I. -O2 -fno-automatic -arch x86_64

  Also, you need to set up system variables:

  export RATRAN=/path/to/Ratran export RATRANRUN=/path/to/Ratran/run
  

---

• Rebuilding python from scratch again

  I got scipy working a week or two ago, but doing so killed matplotib's tkagg. So, I switched to the MacOSX backend, which worked ok until I realized that the interactive (connect) features of macosx failed miserably. This led me to try to get matplotlib working.... which broke with those awful "symbol not found" errors in ft2font.so and _path.so, which I've determined all have to do with linking to the wrong library files. The most worrisome part of this process was discovering that a full Time Machine recovery of /usr and /Library/Frameworks and /Library/Python did *not* restore python - it stayed dead with IDENTICAL errors. So there are probably additional layers of hidden links. The process below is based on hyperjeff's blog post but differs substantially based on Sam Skillman's recommendations and the very big issue I ran into that my /usr/local files appeared to be corrupted. After this install, my path no longer includes /usr/local/bin and /sw has been moved to /_sw... hopefully one of these days I'll be ballsy enough to delete it.

  1. Install python 2.6.4
     1. Needed a clean terminal with no flags set at all. Don't know why - all I had set were a bunch of -arch x86_64 flags. export LD_LIBRARY_PATH="/usr/local/lib:/usr/X11/lib"./configure --enable-framework=/Library/Frameworks MACOSX_DEPLOYMENT_TARGET=10.6 make -j 17sudo make install
     2. Reset PYTHONPATH to blank
     3. `` alias clearflags='export CFLAGS=""; export CCFLAGS=""; export CXXFLAGS=""; export LDFLAGS=""; export FFLAGS="";'`` to make sure
  2. Install FFTW cd ~/tmpcurl -O http://www.fftw.org/fftw-3.2.2.tar.gztar xf fftw-3.2.2.tar.gzcd fftw-3.2.2clearflags./configure CC="gcc -arch x86_64" CXX="g++ -arch x86_64" CPP="gcc -E" CXXCPP="g++ -E"make -j 17sudo make install
  3. Install UMFPACK cd ~/tmpcurl -O http://www.cise.ufl.edu/research/sparse/umfpack/current/UMFPACK.tar.gzcurl -O http://www.cise.ufl.edu/research/sparse/UFconfig/current/UFconfig.tar.gzcurl -O http://www.cise.ufl.edu/research/sparse/amd/current/AMD.tar.gztar xf AMD.tar.gztar xf UFconfig.tar.gztar xf UMFPACK.tar.gzsed -ibck 's/F77 = f77/F77 = gfortran/' UFconfig/UFconfig.mk sed -ibck '299,303s/# //' UFconfig/UFconfig.mkcp UFconfig/UFconfig.h AMD/Include/cp UFconfig/UFconfig.h UMFPACK/Include/cd UMFPACKmake -j 17make hbmake clean
  4. Install numpy
     1. Set environment variables export MACOSX_DEPLOYMENT_TARGET=10.6export CFLAGS="-arch x86_64"export FFLAGS="-m64"export LDFLAGS="-Wall -undefined dynamic_lookup -bundle -arch x86_64"export PYTHONPATH="/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/"echo "[amd]library_dirs = /Users/adam/tmp/AMD/Libinclude_dirs = /Users/adam/tmp/AMD/Includeamd_libs = amd[umfpack]library_dirs = /Users/adam/tmp/UMFPACK/Libinclude_dirs = /Users/adam/tmp/UMFPACK/Includeumfpack_libs = umfpack" > site.cfg
     2. Setup & Install python setup.py build --fcompiler=gnu95sudo python setup.py install
     3. Test: python -c "import numpy"
  5. Install scipy. The important thing is to use g++-4.2 because g++-4.5 doesn't accept the -arch flag. Also, get rid of /sw if it's on your computer at all. sudo mv /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/numpy /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/numpyXcd ~/repos/scipy-0.7.1python setup.py cleanrm -rf buildclearflagsFFLAGS="-m64" CFLAGS="-arch x86_64 -I/usr/local/include/freetype2 -I/usr/X11/include -L/usr/X11/lib"  LDFLAGS="-Wall -undefined dynamic_lookup -bundle  -lpng -arch x86_64" CXX="/usr/bin/g++-4.2" CC="/usr/bin/gcc-4.2" python setup.py buildpython setup.py install Test the install: python -c "import scipy, scipy.fftpack, scipy.interpolate"
  6. 1. Install matplotlib. MAKE SURE /usr/bin/texbin is in front of /usr/local/bin and /sw/bin so that dvipng comes from MacTEX. I also ended up having to remove /usr/local/bin from my path completely sudo mv /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/numpy /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/numpyXcd ~/repos/matplotlib-svnpython setup.py cleanrm -rf buildclearflags
     2. Do hyperjeff's recommended edits except don't use /usr/local because it's f'd: make.osx: MACOSX_DEPLOYMENT_TARGET=10.6PREFIX=/usrPYTHON=/Library/Frameworks/Python.framework/Versions/Current/bin/python## You shouldn't need to configure past this point (and yet…)PKG_CONFIG_PATH="${PREFIX}/lib/pkgconfig"CFLAGS="-arch i386 -arch x86_64 -I${PREFIX}/include -I${PREFIX}/include/freetype2 -isysroot /Developer/SDKs/MacOSX10.6.sdk"LDFLAGS="-arch i386 -arch x86_64 -L${PREFIX}/lib -syslibroot,/Developer/SDKs/MacOSX10.6.sdk"FFLAGS="-arch i386 -arch x86_64" setup.cfg: wxagg = False
     3. Do the install (different from hyperjeff b/c I don't want root) `` sudo make -f make.osx fetch deps make -f mpl_build mpl_installpython setup.py install``
  7. Install setuptools
  8. easy_install ipython
  9. install everything else pythonically

---

• Reinstalling a system

  Python died tonight when I foolishly tried to update numpy or matplotlib. One was not compatible with the other in upgraded form, and the update for matplotlib wouldn't install because of GTK issues that were totally opaque. So, I reinstalled EVERYTHING - fink AND macports - from scratch. If I don't get things running by morning I'm going to have to do a restore, which is ugly as death. Things I need to do post-install: `` sudo port install python_select sudo python_select python25`` test ipython test matplotlib test numpy

---

« Page 15 / 19 »