How to setup a Mac with Python dev tools
来源：互联网 发布：hello聊天软件下载 编辑：程序博客网 时间：2024/06/02 00:45
How to setup a Mac with Python dev tools
Notes for how to setup a 64-bit Mac with important Python development tools
Deprecated – click here for updated page for Yosemite version
Created by Trond Kristiansen on 27.7.2009, and last updated 05.0.2.2012
Updated and modified after installation of Snow Leopard on 05/11/2009
Re-written for publication on www.trondkristiansen.com: 06.02.2010, updated after comment 10.4.2010
Updated completely and edited to work for Python v2.6.5 and Numpy v1.4 02.06.2010.
This page was updated again on 05.02.2012 after updating my installation of Intel Fortran to the XE composer (v12). A major step of the installation was to update the HDF5 and NetCDF4 , which caused several problems to arise.
This page contains my own notes for how to install a variety of packages that I use on a regular basis on Intel Mac computers. This particular setup described below is the latest version of these notes and was written when I wanted to re-install a number of packages as 64-bit on my new Mac with Snow Leopard. My particular interest is to have a solid python installation in 64-bit with a functional numpy, matplotlib, netcdf4, f2py, Intel fortran compiler, and hdf5. It took me a long time to get this to work correctly and I always seem to forget how I did this the previous time I had to install all these packages so I decided to write up everything I did. I hope other people will find these notes useful. Comments, hints, suggestion, and improvements on the 20 steps of installation procedures are very welcome.
A few hints/comments before start:
- Some of the suggestions found on this page was originally found here
- One critical thing to remember when going through these steps and you install using “python setup.py build …” is to always go into the build directory and delete everything that is there. If you don’t do that, you may involuntarily use old compiled files that was compiled with an old configuration setup.
- Enable 64 bit kernel on your Snow Leopard following this tip can boost performance: sudo systemsetup -setkernelbootarchitecture x86_64
.
- In some cases I transfer files from one computer to another using email and in some cases this has led to a “DOSified” version of the file. A very useful tool for making sure that the line end of critical files such as .bash_profile are correct is to use the dos2unix online tool found here. Or alternatively, use the command line option: tr ‘\r’ ‘\n’ < file-with-dos-newline.txt > file-with-mac-newline.txt
STEP 1: Important flags you have to set before you start compiling
export CXXFLAGS="-arch x86_64 -Os -pipe"
export FFLAGS="-m64"
export FCLAGS="-m64"
export MACOSX_DEPLOYMENT_TARGET=10.6
export LDFLAGS=""
STEP 2: Install/update your local installation of subversion
Download subversion for Mac (Universal build for 32 and 64 bit Mac) http://www.open.collab.net/downloads/apple/index.html. The download requires you to register at Collabnet before you can download. This package will install to
so after installation add the following to your .bash_profile file
STEP 3: Install LibPNG
make check
sudo make install
STEP 3 alternative: Install LibPNG
When I wanted to use version 1.4.2 I had to manually download the file from here, before I installed it with
make check
sudo make install
STEP 4: Install ZLIB
Zlib is used for compression/decompression in reading/writing netcdf4 files. The package can be downloaded here and installs by default to /usr/local.
make
make check
sudo make install
After installation you want to add this to your .profile (or .bash_profile) file:
then
ZLIBDIR="/usr/local"; export ZLIBDIR
else
ZLIBDIR="/usr/local:${ZLIBDIR}"; export ZLIBDIR
fi
export LIB=/usr/local/lib:$LIB
export INCLUDE=/usr/local/include:$INCLUDE
STEP 5: Install Python 64-bit
bzip2 -d Python-2.6.5.tar.bz2
tar xf Python-2.6.5.tar
cd Python-2.6.5
unset DYLD_LIBRARY_PATH
unset LD_LIBRARY_PATH
./configure --enable-framework=/Library/Frameworks \
MACOSX_DEPLOYMENT_TARGET=10.6 \
--with-universal-archs=intel \
--with-readline-dir=/usr/local/lib \
--with-zlib=/usr/local/include \
--enable-toolbox-glue \
--enable-universalsdk=/ MACOSX_DEPLOYMENT_TARGET=10.6
make
make test
sudo make install
Next, I made a copy/backup of the original Python installation (Python 2.6.1) by running:
Then I relinked the new 64-bit Python 2.6.5:
Test to check that you get correct 64-bit versionof the Python binary
And you should see something like this:
And you should see this when you type in “python” in the terminal window:
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
STEP 6: Update your easy_install installation
Here we install easy_install for python 2.6.5 on a computer that previously had python version 2.5. The current easy_install is linked to python 2.5 and will install packages to site-packages/2.5 which is not what we want. Therefore we install easy_install-2.6. After the installation we will have both easy_install-2.5 and easy_install-2.6 available, so use the one that corresponds to the python version you use.
Also, see this link and this link for more details on these topics.
The installation of easy_install can be done in two ways 1) get the source codehere so you can build against the 64-bit python:
sudo python setup.py install
Or 2) get the egg from this link and install using
STEP 7: Install readline with support for X86_64
STEP 8: install numpy for 64-bit
Since python was installed as a universal architecture (ppc,i386,x86_64), but I only wanted numpy as 64-bit, I needed to edit the python makefile. I removed all the references in python Makefile to other archs except x86_64. So I edited the file (see two options depending on Leopard or Snow Leopard installation)
/System/Library/Frameworks/Python.framework/Versions/Current/lib/python2.6/config/Makefile (Snow Leopard)
and kept only the occurrence of “-arch x86_64″ and removed the other ones. Then I saved the file and build numpy (get numpy here) using the commands:
sudo python setup.py install
A note of warning: There is a problem using ifort and icc from the command-line under Snow Leopard. If you get an error message of the formld: in /opt/intel/fce/10.1.006/lib/libimf.a(libm_reduce_pio2.o),ObjectFileAddressSpace::mappedAddress(0xFFFFFFFFFFFFFFFC) not in any section at the end of the compilation process you’ve hit a known bug in the Intel suite. This problem is solved by adding the flag -shared-intel into the disutils file “/Library/Python/2.6/site-packages/numpy/distutils/fcompiler/intel.py” (relative path differs with installation of python 2.5 and 2.6).
Since I wanted to compile Fortran programs in 64-bit mode using the Intel Fortran Compiler (ifort), I edited the file “/Library/Python/2.6/site-packages/numpy/distutils/fcompiler/intel.py”. In the file, I found the section that contained specifications for my compiler which was “class IntelEM64TFCompiler(IntelFCompiler):”, and then I added “-shared-intel” in the “linker_so” options so that it looked like this:
compiler_type = 'intelem'
compiler_aliases = ()
description = 'Intel Fortran Compiler for EM64T-based apps'
version_match = intel_version_match('EM64T-based|Intel\\(R\\) 64')
possible_executables = ['ifort', 'efort', 'efc']
executables = {
'version_cmd' : None,
'compiler_f77' : [None, "-FI", "-w90", "-w95"],
'compiler_fix' : [None, "-FI"],
'compiler_f90' : [None],
'linker_so' : ['',"-O0","-shared-intel","-bundle",'-dynamiclib', '-Wl,-undefined,dynamic_lookup', '-Wl,-framework,Python'],
'archiver' : ["ar", "-cr"],
'ranlib' : ["ranlib"]
}
....
Update 21.04.2010: This problem disappears if you upgrade to the latest Intel Fortran compiler version 11.1 according to L. Pagani.
Update 05.02.2012: This problem is gone with the Intel XE composer.
After a successful installation of Numpy I added these lines to my .bash_profile script:
export INCLUDE=/Library/Frameworks/Python.framework/Versions/2.6/include:$INCLUDE
export LIB=/Library/Frameworks/Python.framework/Versions/2.6/lib:$LIB
STEP 9: Howto get F2PY to work for 64-bit python
To get f2py to compile together with 64-bit fortran Intel compiler, you have to edit the following file in disutils (notice two options, one for Leopard and one for Snow Leopard):
Edit the line that contains “linker_so” for both intel (32-bit) and intelem (64-bit Fortran compilers) so that it looks like this:
Then you can for example compile Fortran modules using Intel Fortran compiler with the following option in f2py:
The compiled module can be imported to python and called using import barotropic
Step 10: Install UDUNITS
Udunits is used by netcdf4 an hdf5 which will be installed later.
cd into the src directory:
make
make test
sudo make install
after installing UDUNITS add this to your .profile:
export UDUNITS_INC=/Users/trond/GMT/udunits-1.12.7/include
export UDUNITS_LIB=/Users/trond/GMT/udunits-1.12.7/lib
export UDUNITS_PATH=/Users/trond/GMT/udunits-1.12.7
Step 11: Install IPython and the dependencies IPython relies on
Download source code from: http://ipython.scipy.org/dist/ or use curl below (see further down). IPython requires a number of other programs to function optimal:
1. pygments (syntax highlighter) which you can install using: sudo easy_install-2.6 Pygments
2. Next, to create documentation you should install sphinx: sudo easy_install-2.6 sphinx
3. Parallel computing with foolscap: sudo easy_install-2.6Foolscap
4. Nose required for running test suite: sudo easy_install-2.6nose
5. Pexpect required for running standalone doctest: sudo easy_install-2.6 pexpect
Next complete the installation by installing Ipython from scratch
tar xf ipython-0.10.tar.gz
cd ipython-0.10
python setup.py build
sudo python setup.py install
An alternative to compiling the sources for IPython is to use easy_install:
sudo easy_install-2.6 ipython
STEP 12: Install Freetype2
Get the package here: http://mirror.its.uidaho.edu/pub/savannah/freetype/
make
sudo make install
STEP 13: Install the PIL (Imaging library)
To be able to for example to use the blue marble background image in matplotlib you have to install PIL (Imaging library). Download the package from: http://www.pythonware.com/products/pil/ and install by issuing the following commands:
sudo python setup.py install
STEP 14: Install PKG-CONFIG
Main website :http://pkg-config.freedesktop.org/wiki/
Download the package from here: http://pkgconfig.freedesktop.org/releases/
make
sudo make install
Step 15: Install Matplotlib
Matplotlib requires a number of libraries to be installed prior to installation. Howto install these libraries have been described in the previous steps, however, the paths to these ibraries (libpng, pkg-config, and freetype2) have to be set below in the make.osx file that you find in the root of the matplotlib source directory.
You can download Matplotlib from here: http://matplotlib.sourceforge.net/users/installing.html or grab the latest source from thesubversion trunk:
cd matplotlib
The matplotlib package was installed using the suggestion in README.osx
STEP 16: Install Basemap for Matplotlib
Basemap is another very useful module for python created by Jeffrey Whitaker for making maps in matplotlib. Prior to the installation of Basemap you need to compile and install the GEOS package (I could only get version 2.3.3 to work and had no success with version 3.1). Download the GEOS package from here : http://trac.osgeo.org/geos/
Compile and install:
make
sudo make install
Add this line to .bash_profile
Download the Basemap source code from here.
When your download has completed, cd to root level of basemap source directory and install Basemap using :
sudo python setup.py install
Finally, you can test the success of the installation by trying to import the Basemap module in python:
STEP 17: Install HDF5
Download HDF5 here: http://www.hdfgroup.org/HDF5/release/obtain5.html
I compiled HDF5 (version HDF5 version 1.8.8) in static mode for use with Fortran (--enable-fortran) and netcdf4. If you want to compile HDF5 as a shared library you can not create Fortran libraries and have to use the option --disable-fortran.
make
make check
sudo make install
Next add these lines to your .profile so that when we install netcdf4 HDF5 will be visible to the configurator:
If [ -z "${LD_LIBRARY_PATH}" ]
then
LD_LIBRARY_PATH="/usr/local/lib"; export LD_LIBRARY_PATH
else
LD_LIBRARY_PATH="/usr/local/lib:${LD_LIBRARY_PATH}"; export LD_LIBRARY_PATH
fi
if [ -z "${LIBDIR}" ]
then
LIBDIR="/usr/local/lib"; export LIBDIR
else
LIBDIR="/usr/local/lib:${LIBDIR}"; export LIBDIR
fi
export INCLUDE=/usr/local/lib:$Include
export PATH=/usr/local/bin:$PATH
export HDF5_LIB_DIR=/usr/local/lib
export HDF5DIR=/usr/local
export HDF5_DIR=/usr/local
export CPPFLAGS="-I$HDF5_DIR/include -I/usr/local/include"
export LDFLAGS="-L$HDF5_DIR/lib -L/usr/local/lib"
STEP 18: Install MacTeX
Download and install (DMG) the latest version of LaTeX from here. A working installation of LaTeX is required to create documemnts and man pages for NetCDF. When installation is complete, add this line to your .bash_profile script to make all the binaries available in PATH.
STEP 19: Install NETCDF
NetCDF source files can be downloaded from here. As of the writing of this, latest stable version is v4.1.3. This version was particularily difficult to get to compile in 64 bit mode (at least for me) and it took me several hours to make sure that both HDF 1.8.8 was compiled as 64 bit and that it could be linked to the comilation of the netCDF4. I had to set several environmental variables to get this to work, including setting the GNU GCC compileer to version 4.6 (as described here):
export CXX=/usr/local/bin/g++
export FC=ifort
export F90=ifort
export F77=ifort
export FCFLAGS=-m64
export FFLAGS=-m64
Next, I still had problems with teh compilation until I found a tip at the Unidata website that suggested a solution to my problem. The trick to get NetCDF4+HDF5+Intel XE composer to work together was a simple statement:
. The following commands enabled me to compile and load NetCDF4 using Intel Fortran X:
make
make check
make install
After installation of netCDF4 add this to your .profile:
export NETCDF_PREFIX=/usr/local
export NETCDF_LIB=/usr/local/lib
export NETCDF4_ROOT=/usr/local
export NETCDF4_DIR=/usr/local
export NETCDF_INC=/usr/local/include
export PATH=/usr/local/bin:$PATH
STEP 20: Install NetCDF for Python
Jeffrey Whitaker has written an extremely useful module for interfacing python with netCDF4 called netcdf4-python. The module can be downloaded from here http://code.google.com/p/netcdf4-python/. After updating NetCDF to version 4.1.3 , HDF5 1.8.8, and netCDF4-Python to 0.9.9, I no longer had to manually edit the setup.py file to point to HDF5 and NC dirs.
I was able to install the package (version netCDF4-0.9.9) using these commands:
python setup.py install
cd test; python run_all.py
STEP 21: Install H5PY (HDF5 Python support)
Se here for a link to h5py
Install cython
Then grab the latest source code of h5py :
Next, cd into h5py working directory (h5py-read-only) and run the following commands:
python setup.py cython
python setup.py configure --api=18 --hdf5=/usr/local/SharedButNoFortranHDF5
python setup.py build
sudo python setup.py install
If you update to a newer revision or have modified h5py, be sure to run the
following to prevent the use of stale files:
python setup.py cython --clean
STEP 22: Install/update MySQL
Download MySQL from MySQL downloads website. I also
installed a separate MySQL Startup Item package to enable the automatic startup of MySQL on system startup. The Startup Item starts the server by invoking mysql.server
and you can see this option as part of
. See this link for more details. I added the installation to my .bash_profile script:
Next I started up MySQL running:
I then needed to set the root passwords for my MySQL installation (for details see here):
mysql>; SET PASSWORD FOR 'root'@'localhost' = PASSWORD('newpwd');
mysql>; SET PASSWORD FOR 'root'@'127.0.0.1' = PASSWORD('newpwd');
Next I edited the rights for reading and writing to the data directory (/usr/local/mysql/data) by issuing these commands:
sudo chgrp -R mysql /usr/local/mysql/data
Finally, one of the reasons I use all of these packages is because of my interest in ecosystem modeling. I have a few projects related to ecosystem modeling where I combine Python and Fortran tools. These projects are available on my SVN website: http://www.assembla.com/profile/trond. My major project is my soda2roms toolbox that enables automatic generation of forcing files for the Regional Ocean Modeling System (ROMS). This toolbox is written using a combination of Python, and Fortran programming languages and compiles using f2py. This project could benefit tremendously if more people would be involved and help development. So please email me if you are interested in developing and improving the toolbox. One of the major goals is to expand it so that it can be used with a number of forcing files such as HYCOM or ROMS input data. Currently the toolbox supports SODA data as input (more info).
6 thoughts on “How to setup a Mac with Python dev tools”
- How to setup a Mac with Python dev tools
- How to Setup a Linux Firewall with PPPoE/NAT/iptables
- How to setup a GIT server with gitosis and gitweb
- How To Setup a CA
- How to Setup A Database
- How to setup a test/dev Datazen Server instance in Microsoft Azure
- How to setup FlumeNG with example configuration
- How to setup multiple sites hosted on your Mac with OSX 10.8 + (MAMP Part 5)
- how-to setup apache on mac
- Google Merchant How To Setup A Live XML Data Feed With MySql and PHP
- How To Setup a Rails 4 App With Apache and Passenger on CentOS 6
- How to Create a High Availability Setup with Pacemaker, Corosync and Floating IPs on CentOS 7
- How to setup a Ubuntu Server?
- How to Do Everything with Google Tools
- How to setup Apache + Python + Django + mod_wsgi
- How to become a qualified dev&ops
- how to setup a new user to hdfs
- how to setup DatabaseMail(with MS Exchange Server)
- (二十八)monggodb和maven零散笔记
- 转发，windows下安装php、apache、mysql，并使得外网可访问内网的机器，搭建svn服务器提供版本控制
- 小波变换教程(四)
- PDB文件：每个开发人员都必须知道的
- Navicat Premium破解
- How to setup a Mac with Python dev tools
- 小波变换教程(五)
- LeetCode 13 Roman to Integer
- spring基础
- 怎么使得cxDBTreeList的节点内容可以多行显示
- 第七周项目一 vc++6.0初体验
- 蛇形矩阵是由1开始的自然数依次排列成的一个矩阵上三角形
- GridView和ListView简单使用
- 字符串匹配代码
I have the same problem as you when compiling a fortran program with ifort (in O2 mode, not in debug mode). I have tried to include -shared-intel in the command line for the ld step or in the compiler step but none worked. Can you tell me more precisely how to pass this option ?
Hi Laurent. In my case, since I wanted to compile Fortran programs in 64-bit mode using the Intel Fortran Compiler (ifort), I edited the file “/Library/Python/2.6/site-packages/numpy/distutils/fcompiler/intel.py”. In the file, I found the section that contained specifications for my compiler which was “class IntelEM64TFCompiler(IntelFCompiler):”, and then I added “-shared-intel” in the “linker_so” options so that it looked like this:
I have added some more details to the top of this page to make it more clear where to add the “-shared-intel” option. Thanks for your comment, and I hope this helps you. Cheers, Trond
Did not seem to work. I don’t want to spoil this page with my many questions. Would you mind to contact me at my email so that I can ask you a few more questions ? I won’t spam you, of course.
Hello,
I’m actually trying to wrap a few fortran subroutines to speed up my python code (dynamic relaxation for my thesis). I’m on OSX, using python 2.6 and numpy 1.4. I’ve installed gfortran and the latest ifort.
I’ve written a simple myhello.f90 subroutine and :
I can compile it from XCode with ifort
I can compile it from the Terminal both with ifort and gfortran using “ifort / gfortran -c myhello.f90″ and get a myhello.o file
I can create the signature file myhello.pyf using “f2py -m myhello -h myhello.f2py myhello.f90″
… But I can’t get the “f2py -c -m myhello myhello.f90″ working correctly (even if It creates a myhello.so file)
ld: warning: in /var/folders/pz/pzBDuWF6HOqd3Ai8dGBUAk+++TI/-Tmp-/tmpE49iUm/var/folders/pz/pzBDuWF6HOqd3Ai8dGBUAk+++TI/-Tmp-/tmpE49iUm/src.macosx-10.3-fat-2.6/myhellomodule.o, missing required architecture x86_64 in file
.. Neither I can’t get the “f2py -c –fcompiler=intelem -m myhello myhello.f90″ working correctly (even if It creates a myhello.so file)
ipo: warning #11010: file format not recognized for /Library/Frameworks/Python.framework/Python
ipo: warning #11010: file format not recognized for /var/folders/pz/pzBDuWF6HOqd3Ai8dGBUAk+++TI/-Tmp-/tmp9RLX9e/var/folders/pz/pzBDuWF6HOqd3Ai8dGBUAk+++TI/-Tmp-/tmp9RLX9e/src.macosx-10.3-fat-2.6/myhellomodule.o
ipo: warning #11010: file format not recognized for /var/folders/pz/pzBDuWF6HOqd3Ai8dGBUAk+++TI/-Tmp-/tmp9RLX9e/var/folders/pz/pzBDuWF6HOqd3Ai8dGBUAk+++TI/-Tmp-/tmp9RLX9e/src.macosx-10.3-fat-2.6/fortranobject.o
ld: library not found for -lgcc_s
So I can’t import my module into python (import myhello -> “not found”)
I’m stuck here since 3 days, looking every possible solution on the Net : but I’m not really familiar with compiled languages.
Could you take a bit of your time to give me few advices ?
Thanks,
Lionel
Hi Lionel.
It seems to me that you have a problem with libraries and that it crashes at the linker level (ld). I think the reason for your problems is that you try to link 32 bit libraries from GCC with 64 bit Fortran and Python code. I suggest three steps to check your code and tools setup:
First, make sure that your python 2.6 installation together with Numpy is compiled with 64 bit support. For example, that you compiled Python with universal support 32 and 64 bit Intel support. I myself compile it using “–with-universal-archs=intel”.
Second, make sure that you actually use the 64 bit version of the compiler (e.g. /opt/intel/fce and not /opt/intel/fc).
Third, it could be that you GCC installation only supports 32 bit and you are trying to compile and link 64-bit programs using a 32-bit linker. In that case you can install 64-bit GCC as I suggest on my website:
http://www.trondkristiansen.com/?page_id=1006
Also, please email me your programs and I can see if I can compile them on my computer. If so, then I guess one of the above suggestions is the reason for your troubles. Given the error message in your code: “missing required architecture x86_64 in file” I think your problem may be solved installing a 64-bit version of gcc.
Good luck and let me know if it worked.
Cheers, Trond
Thank you for this VERY useful page.
However there is a simple way to turn around the Intel libimf problem :
just do a static link with /opt/intel/fce/10.1.006/lib/libimf.dylib ! It works
ifort -o myRuntime myCode.o …. /opt/intel/fce/10.1.006/lib/libimf.dylib
and that’s it !