IDL Mini-tutorial (C. Miller 08/10/05)
In this mini-tutorial session, we briefly describe how an astronomer can use IDL in a variety research environments, from image analysis to theoretical modelling, to phenomenological studies.Documentation for IDL (6.1) is provided online by NASA Goddard.
Install:
Install IDL from the internet (~150Mb download) or from the CDs
provided. The final installation is ~250MB.
INSTALL directions are provided on the CDROM in the WINDOWS, LINUX or
MACOSX directory.
LINUX users can (safely?) ignore the symbolic link errors and web
server locations. Follow the install directions. It is easier from ROOT.
WINDOWS users: make sure the JAVA_BRIDGE is selected.
Import the LICENSE KEY and expiration date using the RSI License
Manager :
LINUX: rsi/idl_6.1/bin/rsilicense You might have to set the
IDL_DIR environment
(csh type== setenv IDL_DIR /idl_install_location or bash
type IDL_DIR=/idl_install_location; export IDL_DIR)
OSX: rsi/idl_6.1/RSILicense.app
WINDOWS:
Select Evaluation License
Select (from Pulldown) IDL 6.1
Enter Date and KEY provided on the CDROM (or via email)
There are some missing path dependencies in the setup_idl files in nvoss2005/idl/VOlib_0.1. After the line:
setenv IDL_PATH
${IDL_PATH}:+$NVOSS_IDL_HOME/VOlib_0.1/pro (if .cshrc)
or
set IDL_PATH=%IDL_PATH%;%NVOSS_IDL_HOME%\VOlib_0.1\pro
(.cmd)
ADD:
setenv IDL_PATH
${IDL_PATH}:+$NVOSS_IDL_HOME/VOlib_0.1/dependencies (if .cshrc)
or
set IDL_PATH=%IDL_PATH%;%NVOSS_IDL_HOME%\VOlib_0.1\dependencies
(if .cmd)
The .sh (bash-shell) setup_idl.sh should look something like:
CLASSPATH=${CLASSPATH}:$NVOSS_IDL_HOME/VOlib_0.1/classes
export CLASSPATH
IDL_PATH=${IDL_PATH}:$NVOSS_IDL_HOME/VOlib_0.1/pro
IDL_PATH=${IDL_PATH}:$NVOSS_IDL_HOME/VOlib_0.1/dependencies
Contents:
The Command-line
Some Datatypes
Some
Built-in Functions
Some Mathematical
Relations
Arrays/Matrices
Some
Commonly Used Commands
Data I/O
Astronomy IDL
Libraries/Resources
Data I/O (again)
Objects and
Structures
What
Does that Routine Do?
Plot/Display
Data
Images
Programing Methods
Batch
Procedures
Functions
FOR Loops
WHILE Loops
IF THEN ELSE
External Compiled Libraries
IDL and the VO
The Command-line
To install IDL, download from www.rsinc.com and follow these
instructions to install the NVOSS 2005 license.
To start IDL, type "idl" and any UNIX/Linux prompt, or start the IDL GUI (e.g., for Windows users). If you are using the GUI, the "command-line" is at the bottom of the window.
Datatypes
IDL supports more than a dozen of datatypes, including integers,
long integers,
floats, doubles, strings, etc.
Try the following examples. Note that IDL ignores capitalization.
IDL> A = 3*5
IDL> print, a
15
IDL> a = 2020
IDL> a = double(a); or a = float(a), or a = long(a), etc.
IDL> print, a
2020.0000
IDL> filename = '/home/user/nvoss05/root' + string(fix(a)) + '.fit'
IDL> print, filename
/home/user/idl/root 2020.fit
IDL> filename = '/home/user/nvoss05/cluster' + strtrim(string(fix(a)),2) + '.fit'
IDL> print, filename
/home/user/idl/cluster2020.fit
Some Built-in Mathematical Functions
These include: -,+,/,*,sin(), cos(), asin(), cosh(), sqrt(), a^2, alog10(), alog(), exp(), abs(). Note the datatype of the returned value.Some IDL relations
<, <=, >, >=, =, != translate to lt, le, gt, ge, eq, ne (FORTANish)Arrays and Matrices
IDL is row-major (like FORTRAN)
IDL> a = fltarr(2); or intarr(2), or dblarr(2), or strarr(2,2), etc
IDL> a = [5,3]
IDL> print, a[0]
5
IDL> a = fltarr(2)
IDL> a[1] = 2
IDL> print, a
0.00000 2.00000
IDL> a = [[2,2],[2,2]]
IDL> b = [[0.5,0.5],[0.5,0.5]]
IDL> print, a#b
2.00000 2.00000
2.00000 2.00000
IDL> print, a##b
2.00000 2.00000
2.00000 2.00000
IDL> d = [[1,5],[2,2]]
IDL> print, d##a; COLUMN-MAJOR
12 12
8 8
IDL> print, d#a; ROW-MAJOR
6 14
6 14
An example of IDL provide matrix manipulation is the ability to solve for Eignenvalues (and vectors) in a simple command line call:
IDL> evals = Eigenql(covar, Eigenvectors=evecs)
Some Commonly Used Commands
There are a number of commands that regular IDL users re-use often.
IDL> ra = findgen(360) & dec = findgen(360)/2.0 - 90.0 & z = findgen(360)/1800.0 & eps = 10
IDL> print, size(ra); # elements, size of each dimension, datatype, # elements
1 360 4 360
IDL> print, n_elements(ra)
360
IDL> result = where( (abs(ra - 180) le eps) and (abs(dec - 0) le eps) and z ge 0.05 and z le 0.095)
IDL> print, ra[result[0:1]], dec[result[0:1]], z[result[0:1]], format = '(6(1x,f12.8))'
170.00000000 171.00000000 -5.00000000 -4.50000000 0.09444445 0.09500000
IDL> result = randomu(seed, 100)
IDL> print, result[0:5]
0.139039 0.481758 0.594192 0.431110 0.854777 0.585114
IDL> result = randomn(seed, 100); or randomu(seed, 100, /normal)
IDL> print, avg(result)
Data Input/Output
IDL contains many methods to ready many types of data formats.Common techniques include readf, readcol, rdfloat, and fits binary table readers (discussed below).
Similarly, IDL outputs data in a variety of formats using a variety of methods.
Common techniques include printf and fits table writers (discussed below).
IDL>$vi ascii_datafile.dat ; or $emacs ascii_datafile.dat
1 2 3 4 5 6 7 8 9 10
10 9 8 7 6 5 4 3 2 1
5 6 7 8 9 10 4 3 2 1
4 3 2 1 5 6 7 8 9 10
1 1 1 1 1 1 1 1 1 10
10 2 2 2 2 2 2 2 2 2
2 1 2 1 2 1 2 1 2 1
10 10 10 10 10 10 10 10 10 10
10 5 2 10 4 2 10 8 4 1
1 1 1 1 1 1 1 1 1 1
:wq ; or ctrl-q
IDL> openr, 1, 'ascii_datafile.dat'
IDL> delvar, a
IDL> readf,1, a
IDL> print, a
IDL> 1.00000
IDL> close, 1
IDL> openr, 1, 'ascii_datafile.dat'
IDL> a = intarr(10, 10)
IDL> readf,1, a
IDL> print, a
1 2 3 4 5 6 7 8 9 10
10 9 8 7 6 5 4 3 2 1
5 6 7 8 9 10 4 3 2 1
4 3 2 1 5 6 7 8 9 10
1 1 1 1 1 1 1 1 1 10
10 2 2 2 2 2 2 2 2 2
2 1 2 1 2 1 2 1 2 1
10 10 10 10 10 10 10 10 10 10
10 5 2 10 4 2 10 8 4 1
1 1 1 1 1 1 1 1 1 1
IDL> close,1 ; or close, /all
IDL> openw,1, 'test_write.txt'
IDL> printf,1, 'Test for IDL file writing with PRINTF'
IDL> $more test_write.txt
Test for IDL file writing with PRINTF
The IDL GUI (IDLDE)
The Astronomy Libraries and Resources
The NASA Goddard Space Flight Center (GSFC) IDL astronomy libary (astrolib) is widely used.For detailed descriptions of the available routines, please see:
http://idlastro.gsfc.nasa.gov/contents.html
For these to work, make sure your IDL_PATH contains the correct location of your ASTROLIB.
setenv IDL_PATH ${IDL_PATH}:+/Users/chrism/idl/astron_lib/pro
Below are some highlights of routines in ASTROLIB (you'll need this file in your working directory:( sdss_c4_dr2_published.fit)
IDL> rdfloat, 'ascii_datafile.dat', a1,a2,a3,a4, /double
IDL> tabledata = mrdfits('sdss_c4_dr2_published.fit',1,hdr)
% Compiled module: MRDFITS.
% Compiled module: FXPOSIT.
% Compiled module: MRD_HREAD.
% Compiled module: FXPAR.
% Compiled module: VALID_NUM.
% Compiled module: MRD_SKIP.
% Compiled module: MRD_STRUCT.
% Compiled module: MRD_STRUCTTEMP.
MRDFITS: Binary table. 126 columns by 748 rows.
% Compiled module: IS_IEEE_BIG.
IDL> image = readfits('../../../../proto/irafws-clients/data/cluster.fits',hdr)
IDL> hprint, hdr
IDL> extast, hdr, astr
IDL> help, /str,astr
** Structure <38d6c10>, 8 tags, length=112, data length=108, refs=1:
CD DOUBLE Array[2, 2]
CDELT DOUBLE Array[2]
CRPIX FLOAT Array[2]
CRVAL DOUBLE Array[2]
CTYPE STRING Array[2]
LONGPOLE FLOAT 180.000
PROJP1 FLOAT -1.00000
PROJP2 FLOAT -2.00000
IDL> print, lumdist()
% Compiled module: LUMDIST.
Syntax: result = lumdist(z, H0 = ,k=, Lambda0 = ])
Returns luminosity distance in Mpc
IDL> print, zang()
% Compiled module: ZANG.
Sytnax - angsiz = zang( dl, z, [H0 =, Omega_m =, Lambda0 = , q0 = , k =, /SILENT])
IDL> tabledata = mrdfits('sdss_c4_dr2_published.fit',1,hdr)
IDL> result = robust_linefit(alog10(tabledata.lum_r), alog10(tabledata.biwt1500), $
IDL> yfit, sig, coef_sig)
IDL> plot, alog10(tabledata.lum_r), yfit, psym=-3
IDL> oplot, alog10(tabledata.lum_r), alog10(tabledata.biwt1500), psym=3
IDL> h = histogram(tabledata.z, binsize=0.004, omin=omin, max=0.4, min=0.)
IDL> plothist,tabledata.z,bin=0.004
- Astronomical
Utilities
A_b() Compute B band interstellar extinction as a function of Galactic coords.
ADSTRING() Format RA and DEC as a character string
DAYCNV Convert from Julian Date to calendar date.
PREMAT() Returns precession matrix from equinox 1 to equinox 2
SUNPOS Compute the RA and Dec of the Sun at a given date
ZANG() Compute angular size as a function of redshift in a Friedman cosmology
- DAOPHOT-Type
Photometry Procedures
APER Circular APERture photometry
GETPSF Obtain a PSF (Gaussian + residuals) from isolated stars.
MMM (Mean, Median, Mode) sophisticated sky background computation
SKY Compute image sky level using MMM
- FITS
Astrometry and Calibration
ADXY Use FITS header to convert celestial (RA,Dec) to pixel coordinates
EXTAST EXTract ASTrometry parameters from a FITS header into an IDL structure
HASTROM Rotate, Congrid, and/or shift an image until astrometry matches that in a reference FITS header. Used to align images.
HREBIN REBIN an image and update the astrometry in a FITS header
WCSSPH2XY Convert between longitude,latitude to X,Y angular coordinates for 25 different map projection types
- FITS I/O
MRDFITS() Read FITS file mapping table columns into IDL structure
MWRFITS Write a FITS file from a IDL array or structure
READFITS() Read a disk FITS file into an IDL data and header array.
- Image
Manipulation
CONVOLVE() Convolve an image with a PSF using the product of Fourier Transforms
FILTER_IMAGE() Like MEDIAN or SMOOTH but handles edges & allows iteration
- Math
and Statistics
AVG() Return the average value of an array or 1 dimension of an array.
KSTWO Compute the two-sided Kolmogorov-Smirnov statistic
PCA Perform a principal component analysis (Karhunen-Loeve expansion)
- Plotting
Procedures
PLOTHIST Plot the histogram of an array
PLOTSYM Define useful plotting symbols not in the standard PSYM definition
- Robust
Statistics Procedures
ROBUST_LINEFIT Robust fit of Y vs X (or bisector of Y vs X and X vs Y)
- IDL
Structure procedures
CREATE_STRUCT Create an IDL structure from a list of tag types and dimensions
BIWEIGHT_MEAN Iterative biweighted determination of mean and std. dev.
- Web
Socket Procedures
QUERYUSNO() Query the USNO-A2.0 Catalog at the ESO/ST-ECF Archive by position
WEBGET() Use the IDL SOCKET procedure to get data from http servers
- TV
Display Procedures
BLINK Blink two or more windows in an image display
CURVAL Interactive display of image intensities and astronomical coordinates
- Miscellaneous
(Non-Astronomy) Procedures
HPRINT Pretty terminal display of a FITS header (or other string array)
READCOL Read a file of free-format ASCII data into IDL vectors
RDFLOAT Quickly read an ASCII file with columns of data into IDL vectors
Structures: an aside
In the above example where we read in tabledata, we have created an IDL structure. Structures are important data "types" in IDL in that they compactly store highly complex data. For example, read in the table again and follow these steps:
IDL> tabledata = mrdfits('sdss_c4_dr2_published.fit',1,hdr)
IDL> hprint, hdr
XTENSION= 'BINTABLE' /Binary table written by MWRFITS v1.4a
BITPIX = 8 /Required value
NAXIS = 2 /Required value
NAXIS1 = 1030 /Number of bytes per row
NAXIS2 = 748 /Number of rows
PCOUNT = 0 /Normally 0 (no varying arrays)
GCOUNT = 1 /Required value
TFIELDS = 126 /Number of columns in table
COMMENT
COMMENT *** End of mandatory fields ***
.
.
.
COMMENT *** Column names ***
COMMENT
TTYPE1 = 'RA_BCG ' /
TTYPE2 = 'DEC_BCG ' /
TTYPE3 = 'RA_BCGPHOT ' /
TTYPE4 = 'DEC_BCGPHOT ' /
TTYPE5 = 'Z_BCG ' /
TTYPE6 = 'ZERR_BCG ' /
TTYPE7 = 'RA_MEAN ' /
TTYPE8 = 'DEC_MEAN ' /
.
.
.
COMMENT *** Column formats ***
COMMENT
TFORM1 = 'D ' /
TFORM2 = 'D ' /
TFORM3 = 'D ' /
TFORM4 = 'D ' /
TFORM5 = 'D ' /
TFORM6 = 'D ' /
TFORM7 = 'D ' /
TFORM8 = 'D ' /
TFORM9 = 'D ' /
TFORM10 = 'D ' /
TFORM11 = 'D ' /
TFORM12 = 'D ' /
TFORM13 = 'I ' /
IDL> help, /str, tabledata
** Structure <8287a54>, 126 tags, length=996, data length=992, refs=1:
RA_BCG DOUBLE 258.12006
DEC_BCG DOUBLE 64.060760
RA_BCGPHOT DOUBLE 258.12006
DEC_BCGPHOT DOUBLE 64.060760
Z_BCG DOUBLE 0.073402300
ZERR_BCG DOUBLE 0.00020854300
RA_MEAN DOUBLE 258.12720
DEC_MEAN DOUBLE 64.016602
RA_GEOM DOUBLE 258.13622
DEC_GEOM DOUBLE 64.018345
Z DOUBLE 0.080783151
Z2 DOUBLE 0.080979899
NC4 INT 203
RADC4 DOUBLE 0.47838822
BIWT500 DOUBLE 1342.1189
.
.
.
IDL> print, size(tabledata)
1 748 8 748
IDL> print, tabledata[0].(0), tabledata[0].(1)
258.12006 64.060760
IDL> print, tabledata[0:10].(0)
258.12006 213.74054 133.95474 229.21742 257.73026 240.34569 255.63577 228.82294 202.79596
358.55704 234.12422
IDL> print, tabledata[0].ra_bcg, tabledata[0].dec_bcg
258.12006 64.060760
Understanding all these routines: an aside
When IDL are properly written, users have access to the documentation provided by the individual who wrote the function/procedure.
IDL> doc_library, 'readcol'
----- Documentation for /home/chrism/idl/astron_lib/pro/readcol.pro -----
NAME:
READCOL
PURPOSE:
Read a free-format ASCII file with columns of data into IDL vectors
EXPLANATION:
Lines of data not meeting the specified format (e.g. comments) are
ignored. Columns may be separated by commas or spaces.
Use READFMT to read a fixed-format ASCII file. Use RDFLOAT for
much faster I/O (but less flexibility). Use FORPRINT to write
columns of data (inverse of READCOL).
CALLING SEQUENCE:
READCOL, name, v1, [ v2, v3, v4, v5, ... v25 , COMMENT=
DELIMITER= ,FORMAT = , /DEBUG , /SILENT , SKIPLINE = , NUMLINE = ]
.
.
.
IDL> doc_library, 'zang'
----- Documentation for /home/chrism/idl/astron_lib/pro/zang.pro -----
NAME:
ZANG
PURPOSE:
Determine the angular size of an object as a function of redshift
EXPLANATION:
Requires an input size in kpc and returns an angular size in arc seconds
Default cosmology has a Hubble constant of 70 km/s/Mpc, Omega (matter)
=0.3 and a normalized cosmological constant Lambda = 0.7; however these
values can be changed with apropriate keywords.
CALLING SEQUENCE:
angsiz = zang( dl, [ z, H0 =, Omega_m =, Lambda0 = , q0 = , k =,
/SILENT] )
.
.
.
Plotting/Display
IDL provides numerous techniques to display data and images. The ones shown here are only the tip of the iceberg.First, let's make figures and display data.
Data display
Here, we include simple and publication quality figures, histograms, and contour plots.For a nice IDL library to interactively make plots, try PUBPLOT.
IDL> a = findgen(100) & b = findgen(100)
IDL> plot, a,b
IDL> a = sin(a*!PI/180 + !PI/2) & b = cos(b*!PI/180 +!PI/2)
IDL> c = dblarr(2,100)
IDL> c[0,*] = a & c[1,*] = b
IDL> contour, c
IDL> surface, c, zr=[-0.5,0.5]
IDL> w = where(tabledata.sub le 1)
IDL> plot, tabledata[w].lum_r,tabledata[w].biwt1500, psym=3, /xlog, /ylog, xrange=[5d10,4d12], yrange=[100,2200], xstyle=1,ystyle=1,$
ytitle = 'Velocity Disp. within 1.5Mpc (km/s)', xtitle='r-band luminosity (solar)'
Image display
IDL comes standard with some simple image display routines.For fancier and more useful libraries, try ATV, LOOK, etc.
IDL> image = readfits('../../../../proto/irafws-clients/data/cluster.fits', hdr)
IDL> tv, image
IDL> loadct
% Compiled module: LOADCT.
% Compiled module: PATH_SEP.
0- B-W LINEAR 14- STEPS 28- Hardcandy
1- BLUE/WHITE 15- STERN SPECIAL 29- Nature
2- GRN-RED-BLU-WHT 16- Haze 30- Ocean
3- RED TEMPERATURE 17- Blue - Pastel - R 31- Peppermint
4- BLUE/GREEN/RED/YE 18- Pastels 32- Plasma
5- STD GAMMA-II 19- Hue Sat Lightness 33- Blue-Red
6- PRISM 20- Hue Sat Lightness 34- Rainbow
7- RED-PURPLE 21- Hue Sat Value 1 35- Blue Waves
8- GREEN/WHITE LINEA 22- Hue Sat Value 2 36- Volcano
9- GRN/WHT EXPONENTI 23- Purple-Red + Stri 37- Waves
10- GREEN-PINK 24- Beach 38- Rainbow18
11- BLUE-RED 25- Mac Style 39- Rainbow + white
12- 16 LEVEL 26- Eos A 40- Rainbow + black
13- RAINBOW 27- Eos B
Enter table number:
13
IDL> tv,image
IDL> atv,image
IDL> imdisp, image
IDL> image24 = TVRD(True=1)
IDL> image2d = Color_Quan(image24,1,r,g,b,Cube=6)
IDL> write_jpeg, 'cluster.jpg', image2d
idl> read_jpeg, 'cluster.jpg', image
idl> tvscl, image
Programming Methods
While the command-line provides simple and immediate access to IDL,
programs (procedures and functions) written by the user are a more
powerful technique.SUBROUTINES/PROCEDURES:
Consider the file separ.pro
PRO SEPAR, separ, ra1,ra2,dec1,dec2
;Returns the separation of 2 points on the sky
;(in RA,DEC) in arcminutes. Uses spherical trig.
phi1 = ra1/57.29578
theta1 = (90.0 - dec1)/57.29578
phi2 = ra2/57.29578
theta2 = (90.0 - dec2)/57.29578
costh3=cos(theta1)*cos(theta2)+sin(theta1)*sin(theta2)*cos(phi2-phi1)
separ = (acos(costh3)*57.296)*60.0
RETURN
END
IDL> .com separ
IDL> separ, sep,180,179,45,45
% Compiled module: SEPAR.
IDL> print, sep
42.4159
IDL> separ, sep,180,179,0,0
IDL> print, sep
59.9973
FUNCTIONS:
Consider the file directory.pro
FUNCTION DIRECTORY, t
name = STRUPCASE(t) ;Convert to uppercase
CASE name OF
'JANE': d='555-1212'
'BILL': d='555-2121'
ELSE: d='Not Found'
ENDCASE
RETURN,D
END
IDL> number = directory('jane')
IDL> print, number
555-1212
FOR Loops:
WHILE Loops:
IDL> testme = findgen(15) & zran = dblarr(100,n_elements(testme)) & probd=zran
IDL> FOR I = 0, 99 DO BEGIN zran[I,*] = randomu(iseed, n_elements(testme))*15.0
IDL> FOR I = 0, 99 DO BEGIN kstwo, testme, zran[I,*],d,prob & probd[I] = prob
IDL> print, avg(probd)
PRO MAKE_BCG_DATA
c4data = mrdfits('sdss_c4_dr2_published.fit',1,hdr)
openw,1,'bcg_data.dat'
FOR I = 0, n_elements(c4data.ra_mean)- 1 DO BEGIN
printf,1, c4data[I].ra_bcg, c4data[I].dec_bcg, c4data[I].z_bcg, $
format = '(3(1x, f16.10))'
ENDFOR
close, 1
END
IDL> make_bcg_data
IDL> $ls
IDL> openr,1,'bcg_data.dat'
IDL> WHILE NOT(EOF(1)) DO BEGIN READF,1,a,b,c & print,a,b,c
IF THEN ELSE
IDL> openr,1,'bcg_data.dat'
IDL> WHILE NOT(EOF(1)) DO BEGIN READF,1,a,b,c & IF (c gt 0.15) THEN PRINT, a,b,c
260.042 26.6256 0.159325
196.274 62.2164 0.169516
15.2565 -9.95473 0.150398
348.409 14.8376 0.163154
PRO SEPAR, separ, ra1,ra2,dec1,dec2
;Returns the separation of 2 points on the sky
;(in RA,DEC) in arcminutes. Uses spherical trig.
IF (n_elements(ra1) eq 0 or n_elements(ra2) or eq 0 n_elements(dec1) eq 0 or n_elements(dec2) eq 0) THEN BEGIN
PRINT, 'Error: invalid coordinates'
RETURN
ENDIF ELSE BEGIN
phi1 = ra1/57.29578
theta1 = (90.0 - dec1)/57.29578
phi2 = ra2/57.29578
theta2 = (90.0 - dec2)/57.29578
costh3=cos(theta1)*cos(theta2)+sin(theta1)*sin(theta2)*cos(phi2-phi1)
separ = (acos(costh3)*57.296)*60.0
ENDELSE
RETURN
END
IDL> .com separ
IDL> separ, sep
Error: invalid coordinates
The IDL VO Libraries
The NVOSS 2005 software distribution contains a directory
nvoss2005/idl/ which in turn contains a sub-directory containing
a registry calling procedure (G. Greene) and another sub-directory
containing a number of VO libraries (VOLib_0.1 --C.
Miller). The VOLib contains procedures to read VOTables, SIAP calls,
Cone Searches, Open Sky Query calls, Registry calls, and WESIX calls.
These will be discussed in more detail in the Lessons Learned talk
(Saturday) after we're all a little more comfortable with some of these Standards.