Adam Ginsburg's blog - Articles by Adam (keflavich@gmail.com)

Articles by Adam (keflavich@gmail.com)

Python 64 bit!

2012/09/20

I got python 64 bit to compile, but it required a number of tricky steps. First, this guy has the instructions I followed: captnswing However, it didn't work entirely as advertised. I ran the configure as advertised: ./configure --enable-framework=/Library/Frameworks \--enable-universalsdk=/ \MACOSX_DEPLOYMENT_TARGET=10.5 \--with-universal-archs=all \--with-readline-dir=/usr/local then the make install, but /usr/local/bin/python pointed to the wrong place, so I replaced the symbolic link in my python path with the correct one: sudo rm /Library/Frameworks/Python.framework/Versions/2.6/bin/python sudo ln -s /Library/Frameworks/Python.framework/Versions/2.6/bin/python-64 /Library/Frameworks/Python.framework/Versions/2.6/bin/python Now python is 64 bit: eta ~$ python -c "import sys; print sys.maxint"9223372036854775807 I haven't checked whether it works yet though... Update: Had to reinstall with gnu readline installed. Also have to install PyQt4 and might have to recompile numpy... numpy won't compile with python 2.6.2: C compiler: gcc -arch i386 -arch ppc -arch ppc64 -arch x86_64 -isysroot / -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypescompile options: '-Inumpy/core/include -Ibuild/src.macosx-10.5-universal-2.6/numpy/core/include/numpy -Inumpy/core/src -Inumpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c'gcc: build/src.macosx-10.5-universal-2.6/numpy/core/src/_sortmodule.cIn file included from numpy/core/include/numpy/ndarrayobject.h:26, from numpy/core/include/numpy/noprefix.h:7, from numpy/core/src/_sortmodule.c.src:29:numpy/core/include/numpy/npy_endian.h:33:10: error: #error Unknown CPU: can not set endiannesslipo: can't figure out the architecture type of: /var/folders/ni/ni+DtdqFGMeSMH13AvkNkU+++TI/-Tmp-//cceaWIvZ.outIn file included from numpy/core/include/numpy/ndarrayobject.h:26, from numpy/core/include/numpy/noprefix.h:7, from numpy/core/src/_sortmodule.c.src:29:numpy/core/include/numpy/npy_endian.h:33:10: error: #error Unknown CPU: can not set endiannesslipo: can't figure out the architecture type of: /var/folders/ni/ni+DtdqFGMeSMH13AvkNkU+++TI/-Tmp-//cceaWIvZ.outerror: Command "gcc -arch i386 -arch ppc -arch ppc64 -arch x86_64 -isysroot / -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Inumpy/core/include -Ibuild/src.macosx-10.5-universal-2.6/numpy/core/include/numpy -Inumpy/core/src -Inumpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c build/src.macosx-10.5-universal-2.6/numpy/core/src/_sortmodule.c -o build/temp.macosx-10.5-universal-2.6/build/src.macosx-10.5-universal-2.6/numpy/core/src/_sortmodule.o" failed with exit status 1 That sucks.

Python 64-bit on Mac OS X 10.6 Snow Leopard

After yesterday's disastrous attempt to install various python packages, I started from scratch. First, I got rid of all of my python frameworks (backed up but removed from the path). Then, I compiled python 2.7 from scratch:

I got some help from http://blog.mahmoudimus.com/2009/12/python-2-6-4-and-twisted-9-on-os-x-10-6-snow-leopard/ ./configure --enable-framework --enable-universalsdk=/Developer/SDKs/MacOSX10.6.sdk MACOSX_DEPLOYMENT_TARGET=10.6 --with-universal-archs=intel -with-readline-dir=/usr/localmake -j 17make -j 17 test make results:

Python build finished, but the necessary bits to build these modules were
not found:
    _bsddb
    dl
    gdbm
    imageop
    linuxaudiodev
    ossaudiodev
    spwd
    sunaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the
module's name.

I'm not concerned about these - I don't use any of them and I assume I need to install some other packages to get them to work. During make test, I had two failures that resulted in "python crash" pop-up boxes:

test_subprocess.
this bit of output is from a test of stdout in a different process ....
this bit of output is from a test of stdout in a different process ...
test_sunaudiodev

Then, I got some malloc errors:

test_ioTesting large file ops skipped on darwin.It requires 2147483648
bytes and a long time.Use 'regrtest.py -u largefile test_io' to run
it.Testing large file ops skipped on darwin.It requires 2147483648 bytes
and a long time.Use 'regrtest.py -u largefile test_io' to run
it.python.exe(22914,0x7fff70d3ebe0) malloc: ***
mmap(size=9223372036854775808) failed (error code=12)*** error: can't
allocate region*** set a breakpoint in malloc_error_break to
debugpython.exe(22914,0x7fff70d3ebe0) malloc: ***
mmap(size=9223372036854775808) failed (error code=12)*** error: can't
allocate region*** set a breakpoint in malloc_error_break to
debugpython.exe(22914,0x7fff70d3ebe0) malloc: ***
mmap(size=9223372036854775808) failed (error code=12)*** error: can't
allocate region*** set a breakpoint in malloc_error_break to
debugtest_ioctl

Python magic / advanced numpy indexing

2012/09/20

Yeah, indexing python arrays should really be easy. Stefan van der Walt's page In [85]: bi = (f.bolo_indices[np.newaxis,:] + zeros([7751,1])).astype('int')In [86]: whc = (whscan[:,np.newaxis] + zeros([1,107])).astype('int')In [87]: array2d[whc,bi] = temp2d

Python: one-line arrays

2012/09/20

Ahhh, refreshing: `` whscan = asarray([arange(scanlen)+i for i in scans_info[:,0]]).ravel()`` Not like IDL, which takes at least 4 lines b/c of the variable declaration. There's probably a better way to do that too.

Python: setting matplotlib defaults

2012/09/20

Setting matplotlib defaults is a lot more difficult than it should be. matplotlib.defaultParams['image.origin']='lower'matplotlib.defaultParams['image.interpolation']='nearest'

python stuff

2012/09/20

to get imshow defaults to be nearest neighbor, need to edit ~/.matplotlib/matplotlibrc. Still don't know how to change default command-line output format.

Quicksilver and Awesome

2012/09/20

Set up f17-19 today to play amusing sounds. Also, recalled that I use ScreenSaverEngine to lock my screen with a shortcut key. Quicksilver triggers are the way to do it. Mac is stupid w/o quicksilver.

Quicksilver sounds

2012/09/20
To switch sound source from the command line: switchaudio Use this to make scripts such as: #!/bin/bash/Users/adam/humor/SwitchAudioSource -s "Built-in Line Output"afplay /Users/adam/humor/losinghorn.wav/Users/adam/humor/SwitchAudioSource -s "Built-in Output" Then make triggers in Quicksilver by:
1. Go to trigger pane, make new hotkey trigger
2. press "." to allow you to type a command
3. make sure the action is "Run"
4. hook up a hotkey
5. if it doesn't work, just try again. Persist through crashes, they happen often.
Follow-up: You can also control the volume! http://discussions.apple.com/thread.jspa?threadID=585781 `` osascript -e 'set volume output volume 100'``

RATRAN on Mac OS X

2012/09/20
Mac OS X doesn't like the defaults built in to RATRAN. It died unhappily with errors like: ld_classic: can't locate file for: -lcrt0.o and ld: warning: in /usr/local/lib//libcfitsio.a, file is not of required architecture In order to get it to run, I had to do the following:
1. Install CFITSIO with CFLAGS="-arch x86_64 -arch i386 -g -O2" to /usr/local/lib
2. Edit the sky/Makefile OPT variable (line 23) to read: OPT = -I. -O2 -fno-automatic -arch x86_64
Also, you need to set up system variables:
```
export RATRAN=/path/to/Ratran export RATRANRUN=/path/to/Ratran/run
```

Rebuilding python from scratch again

2012/09/20
I got scipy working a week or two ago, but doing so killed matplotib's tkagg. So, I switched to the MacOSX backend, which worked ok until I realized that the interactive (connect) features of macosx failed miserably. This led me to try to get matplotlib working.... which broke with those awful "symbol not found" errors in ft2font.so and _path.so, which I've determined all have to do with linking to the wrong library files. The most worrisome part of this process was discovering that a full Time Machine recovery of /usr and /Library/Frameworks and /Library/Python did *not* restore python - it stayed dead with IDENTICAL errors. So there are probably additional layers of hidden links. The process below is based on hyperjeff's blog post but differs substantially based on Sam Skillman's recommendations and the very big issue I ran into that my /usr/local files appeared to be corrupted. After this install, my path no longer includes /usr/local/bin and /sw has been moved to /_sw... hopefully one of these days I'll be ballsy enough to delete it.
1. Install python 2.6.4
  1. Needed a clean terminal with no flags set at all. Don't know why - all I had set were a bunch of -arch x86_64 flags. export LD_LIBRARY_PATH="/usr/local/lib:/usr/X11/lib"./configure --enable-framework=/Library/Frameworks MACOSX_DEPLOYMENT_TARGET=10.6 make -j 17sudo make install
  2. Reset PYTHONPATH to blank
  3. `` alias clearflags='export CFLAGS=""; export CCFLAGS=""; export CXXFLAGS=""; export LDFLAGS=""; export FFLAGS="";'`` to make sure
2. Install FFTW cd ~/tmpcurl -O http://www.fftw.org/fftw-3.2.2.tar.gztar xf fftw-3.2.2.tar.gzcd fftw-3.2.2clearflags./configure CC="gcc -arch x86_64" CXX="g++ -arch x86_64" CPP="gcc -E" CXXCPP="g++ -E"make -j 17sudo make install
3. Install UMFPACK cd ~/tmpcurl -O http://www.cise.ufl.edu/research/sparse/umfpack/current/UMFPACK.tar.gzcurl -O http://www.cise.ufl.edu/research/sparse/UFconfig/current/UFconfig.tar.gzcurl -O http://www.cise.ufl.edu/research/sparse/amd/current/AMD.tar.gztar xf AMD.tar.gztar xf UFconfig.tar.gztar xf UMFPACK.tar.gzsed -ibck 's/F77 = f77/F77 = gfortran/' UFconfig/UFconfig.mk sed -ibck '299,303s/# //' UFconfig/UFconfig.mkcp UFconfig/UFconfig.h AMD/Include/cp UFconfig/UFconfig.h UMFPACK/Include/cd UMFPACKmake -j 17make hbmake clean
4. Install numpy
  1. Set environment variables export MACOSX_DEPLOYMENT_TARGET=10.6export CFLAGS="-arch x86_64"export FFLAGS="-m64"export LDFLAGS="-Wall -undefined dynamic_lookup -bundle -arch x86_64"export PYTHONPATH="/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/"echo "[amd]library_dirs = /Users/adam/tmp/AMD/Libinclude_dirs = /Users/adam/tmp/AMD/Includeamd_libs = amd[umfpack]library_dirs = /Users/adam/tmp/UMFPACK/Libinclude_dirs = /Users/adam/tmp/UMFPACK/Includeumfpack_libs = umfpack" > site.cfg
  2. Setup & Install python setup.py build --fcompiler=gnu95sudo python setup.py install
  3. Test: python -c "import numpy"
5. Install scipy. The important thing is to use g++-4.2 because g++-4.5 doesn't accept the -arch flag. Also, get rid of /sw if it's on your computer at all. sudo mv /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/numpy /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/numpyXcd ~/repos/scipy-0.7.1python setup.py cleanrm -rf buildclearflagsFFLAGS="-m64" CFLAGS="-arch x86_64 -I/usr/local/include/freetype2 -I/usr/X11/include -L/usr/X11/lib" LDFLAGS="-Wall -undefined dynamic_lookup -bundle -lpng -arch x86_64" CXX="/usr/bin/g++-4.2" CC="/usr/bin/gcc-4.2" python setup.py buildpython setup.py install Test the install: python -c "import scipy, scipy.fftpack, scipy.interpolate"
6. 1. Install matplotlib. MAKE SURE /usr/bin/texbin is in front of /usr/local/bin and /sw/bin so that dvipng comes from MacTEX. I also ended up having to remove /usr/local/bin from my path completely sudo mv /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/numpy /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/numpyXcd ~/repos/matplotlib-svnpython setup.py cleanrm -rf buildclearflags
  2. Do hyperjeff's recommended edits except don't use /usr/local because it's f'd: make.osx: MACOSX_DEPLOYMENT_TARGET=10.6PREFIX=/usrPYTHON=/Library/Frameworks/Python.framework/Versions/Current/bin/python## You shouldn't need to configure past this point (and yet…)PKG_CONFIG_PATH="${PREFIX}/lib/pkgconfig"CFLAGS="-arch i386 -arch x86_64 -I${PREFIX}/include -I${PREFIX}/include/freetype2 -isysroot /Developer/SDKs/MacOSX10.6.sdk"LDFLAGS="-arch i386 -arch x86_64 -L${PREFIX}/lib -syslibroot,/Developer/SDKs/MacOSX10.6.sdk"FFLAGS="-arch i386 -arch x86_64" setup.cfg: wxagg = False
  3. Do the install (different from hyperjeff b/c I don't want root) `` sudo make -f make.osx fetch deps make -f mpl_build mpl_installpython setup.py install``
7. Install setuptools
8. easy_install ipython
9. install everything else pythonically

« Page 15 / 19 »

Recent Posts

Articles by Adam (keflavich@gmail.com)