PythonScriptingforArcGIS

PaulZandbergen Department p ofGeography g p y UniversityofNewMexico

OutlineofTopics
Introduction d
Examples,PythonandArcGIS,Pythonversions

Fundamentalsofgeoprocessing inArcGIS Pythonlanguagefundamentals

WheretorunPythoncode Datatypes:numbers,strings,lists Functionsandmodules Controlling gworkflow

ArcPy:Geoprocessing usingPython
Usingtools,functions,classes Describing b d data,l listingdata, d working k with hl lists

Creatingcustomtools
Script p tools, ,toolp parameters

Resources

WorkshopMaterialsPosted

posteduntilOctober24

h // http://www.paulzandbergen.com/workshops l db / kh

ForthcomingBook
PythonScriptingforArcGIS Esri Press Sometimein2012 UpdatedforArcGIS 10.1

Sampleexercisesposted(for10.0)

Introduction

PriorKnowledgeandExperience
UsingArcGIS 9.3or10.0?
Workshop pisfor10.0

PriorPythonexperience?
Imnotassuming gany y

Other O h programming i experience? i ?

Imnotassumingany

Example1
Scripttocopyallshapefiles inafolderintoageodatabase
import arcpy from arcpy import env env.overwriteOutput = True env.workspace k = "c:/workshop/ex01" " / k h / 01" fclist = arcpy.ListFeatureClasses() for fc in fclist: py ( ) fcdesc = arcpy.Describe(fc) arcpy.CopyFeatures_management(fc, "c:/workshop/ex01/study.mdb/" + fcdesc.basename)

Example2
Scripttooltogenerateaknearestneighbortable RunsanexistingArcGIS toolmultipletimes,writestheresult
import arcpy from arcpy import env env.overwriteoutput it t t = True T infc = arcpy.GetParameterAsText(0) output = arcpy.GetParameterAsText(1) k = arcpy.GetParameter(2) arcpy GetParameter(2) n = 1 f = open(output, "w") while n <= < k: result = arcpy.CalculateDistanceBand_stats(infc, n) f.write(str(n) + " " + str(result[1])+ "\n") n = n + 1 f.close()

Example3
ScripttooltorunHuffmodel SophisticatedanalysisnotavailableinArcGIS

Example3

WhatisPythonScripting?
Addfunctionality f i li toArcGIS A GIS
IntegratedintoArcGIS interface B ild uponexisting Builds i i functionality f i li Automatesrepetitivetasks Expandsanalysisoptions

Share Sh newf functionality ti lit

Scripttoolsworkjustlikeregulartools Can C be b integrated i t t d i into t models, d l tools t l Easytosharewithothers(free)

WhyPython?
Free,opensource Objectoriented BasicscriptingANDcomplexobjectorientedprogramming Batteriesincluded Embracedbygeospatialcommunity,includingESRI Manylibraries

PythonCommunity

http://www.python.org

PythonandArcGIS
Python P h i isthe h preferred f dscripting i i l languagef forArcGIS A GIS 1. YoucanrunPythonfromwithinArcGIS
PythonWindowworkslikeaninteractiveinterpreter ImportArcPy togetfulllibraryoftools ExtendfunctionalityofArcGIS

2. AlltoolsinArcToolbox canbeaccessedfromPython 3 Python 3. P th scripts i t canbe b made d i into t t tools l 4 Supportforotherscriptinglanguageswillgoaway 4.

VBScriptandJScript beingreplacedbyPython

PythonVersionsandArcGIS
Versions:
CurrentversionofPythonis3.2.2 PythonthatworkswithArcGIS 10.0 10 0is2 2.6.x 6x PythonthatworkswithArcGIS 10.1is2.7.x MovetoPython y 3.xlikely yonly ywithArcGIS 11

ArcGIS only yworkswithaspecific p versionofPython: y

UsetheonethatcomesinstalledwithArcGIS DontinstallyourownversionofPython

InstallingPython
RemoveanyexistinginstallationsofPython InstallArcGIS 10.0
Python2.6.5willbeinstalledbydefault

InstallaPythoneditor ConfiguretheeditortoworkwithArcGIS Note:YoucanrundifferentversionsofPythononone machine however,acleaninstallofPython2.6.5 withArcGIS 10.0isrecommended

Demo:CheckArcGIS andPythoninstallation

FundamentalsofGeoprocessing inArcGIS

Geoprocessing Tools

ToolOrganization

ToolDialogs

ToolParameters
Parameters
Required q Optional

Errors Warning

EnvironmentSettings

Geoprocessing Options

Demo:Geoprocessing Fundamentals

RunningPythonCode

TwowaystorunPythonCode
1. UsinganInteractiveInterpreter
Codeisexecuteddirectly ylineby yline

2 By 2. B running i ascript i t
Codesavedina.py file RunfromwithinaPythoneditorordirectlyfrom p gsystem y operating

WheretotypeandrunPythoncode?
1. PythonwindowinArcGIS
BuiltintoanyArcGIS Desktopapplication Goodfortestingcode,veryshortscripts

2. Pythoneditor
IDLEinstalledbydefault Manyothers,PythonWin isagoodonetostart Goodformorecomplexcode code,savingscripts

PythonWindowinArcGIS

PythonWindowinArcGIS
Workswithcurrentmapdocument Interactiveinterpreter:
Executescodedirectlylinebyline

Goodfortestingshortcode Codecanbesaved No N errorchecking h ki /d debugging b i

PythonEditor IDLE

PythonEditor PythonWin

PythonEditor
Standalone outsideofArcGIS Interactiveinterpreter:
Executescodedirectlylinebyline

Savecodeasscriptfiles(.py) ( ) Goodfororganizingmorecomplexcode

Demo:RunningsimplePythoncode

PythonDocumentation

PythonDocumentation

http://www.python.org

PythonDocumentation
Versionspecific! p

http://docs.python.org

PythonBeginnersGuide

http://wiki.python.org/moin/BeginnersGuide

PythonBooks
Versionspecific!

N None of fth thesebooks b k i including l di anything thi onArcGIS A GIS orgeoprocessing! i !

PythonLanguageFundamentals

PythonDataTypes
Number(integerandfloat) String List Tuple Dictionary

Strings,listsandtuples aresequences Strings,numbersandtuples areimmutable Listanddictionariesaremutable

Numbers
Integers
Wholenumber,i.e.nodecimals e.g.34

Floats
Decimalpoint e.g. g 34.8307

NumericalOperators

Operator * / % +

Description Multiplication Division Modulus Addition Subtraction

Integer Example Result 9*2 18 9/2 4 9%2 1 9+2 11 9 2 7

Floatingpoint Example Result 9*2.0 18.0 9/2.0 4.5 9%2.0 1.0 9+2.0 11.0 9 2.0 7.0

Demo:NumericalOperators

Strings
Asetofcharacterssurroundedbyquotesiscalleda stringliteral Tocreateastringvariable,assignastringliteraltoit
>>> mytext = "Crime hotspot maps are cool." >>> print mytext Crime hotspot maps are cool.

QuotesinPython
InPythonsingleanddoublequotesarethesame "NIJ" isthesameas'NIJ' >>> print "I I said: 'Let's Let s go!'" go! QuotesinPythonarestraightup "text" or 'text', nottext ortext Beawareofcopy/pasteandautoformatting

Variables
Pythonscriptsusevariables tostoreinformation Toworkwithvariablesuseanassignment statement >>> x = 17 >>> x * 2 34

Variables
Python P h usesdynamic d i assignment i >>> x = 17 >>> type(x) <type 'int'> int > >>> x = "GIS" >>> type(x) <type 'str'> Noneedtodeclarevariables Valuedefinesthetype

VariableNames
Rules
Letters,digitsandunderscores Cannotstartwithadigit Dontusekeywords(print,import,etc.)

Recommendations
Bedescriptive(count insteadofc) Keepitshort(count insteadofcount_of_records count of records) Followconvention:alllowercase,useunderscores

StatementandExpressions
APythonexpression isavalue >>> 2 * 17 34 APythonstatement isaninstructiontodosomething >>> x = 2 * 17

WorkingwithStrings
Concatenatestrings >>> >>> >>> >>> GIS x = "G" y = "I" I z = "S" print x + y + z

ConvertingtoString
>>> temp = 100 >>> print "The temperature is " + temp + " degrees" TypeError: cannot concatenate 'str' str and 'int' int objects

>>print "The temperature is " + str(temp) + " degrees"

Convertingthevalueofavariablefromonetypetoanotheris knownascasting

Lists
APythonlistisanorderedsetofitems Thelistofitemsissurroundedbysquarebrackets[],andthe it items areseparated t db bycommas( (,) ) Itemscanconsistofnumbers,stringsandotherdatatypes mylist = [1, 2, 4, 8, 16, 32] mywords ["jpg", j "bmp", "tif"] i Listsareverywidelyusedingeoprocessing:
e.g.listoffeatureclasses,listofrecords,listoffields,etc.

PythonFunctions
Afunction carriesoutacertainaction Pythonhasmanybuiltinfunctions <function>(<arguments>) >> pow(2,3) 8 Usingafunctionisreferredtoascalling afunction Additionalfunctionscanbeaccessedusingmodules

PythonMethods
Amethod isafunctionthatiscloselycoupledtosomeobject <object>.<method>(<arguments>) >>> topic = "Crime Mapping" >>> topic.count("i") 2 ManyofPythonsdatatypeshavemethods

StringIndexing
Python P th strings t i have h anindex i d positioning iti i system t >>> mystring = "Crime Crime Mapping" Mapping >>> mystring[0] 'C' >>> mystring[-1] 'g' Stringscanbeslicedintosmallerstringsusingslicing >>> mystring[0:5] 'Crime

WorkingwithList
Python h l listsh haveanindex d positioningsystem >>> crimes = [ ["arson", arson , "burglary", burglary , "robbery"] robbery ] >>> cities[1] 'burglary' Therearemanylistmethods >>> crimes.append("homicide") >>> crimes.remove( crimes.remove("arson") arson ) >>> crimes ['burglary', 'robbery', 'homicide']

WorkingwithPathnames
Pathnames P h arecritical i i lwhen h writing i i scripts: i
Exampleworkspace:c:\data\results Example E l shapefile: h fil c:\data\results\streams.shp \d \ l \ h

InPythonabackslash(\)isanescapecharacter PathnamesinPythonshouldthereforelooklikeone ofthefollowing

"c:/data" "c:\\data" \\ r"c:\data" (raw string)

PythonModules
M Modules d l arelike lik extensions i that h canbe b imported i dinto i Python P h toextenditscapabilities >>> import time Atypicalmodulecontainsanumberofspecializedfunctions whichcanbecalledoncethemodulehasbeenimported <module>.<function> >>> time.localtime()

ConditionalStatements
Branching canbeusedtocontrolworkflow import random x = random.randint(0,6) print x if x == 6: print = "You win!

Syntax:keywordif,followedbyacondition,followedby(:)

IndentationinPython
Indentedcodeisreferredtoasablock Usetabsorspaces beconsistent Recommended:4spaces Tip:becarefulwithcopy/pastefromother applications

MoreConditionalStatements
Useof felif and delse isoptional l
import random x = random.randint(0,6) print x p if x == 6: print "You win!" elif x == 5: print "Try again!" else: print "You lose!"

LoopStructures:While
L Loop structuresallow ll youtorepeatacertain i partof fyour code Awhile looprepeatsuntilaparticularconditionisreached i = 0 while i <= 10: print i p i += 1 Thewhile statementusesasentryvariableintheexit condition

LoopStructures:For
Afor loop l repeatsabl block kof fcode d f foreach helement l of fa sequence
mylist = ["A", "B", "C", "D"] for letter in mylist: print letter

Intheexample,letteristhenameofavariableandforeach iterationoftheloopthisvaraible isassignedadifferentvalue

ArcPy:Geoprocessing usingPython

WhatisArcPy?
A ArcPy P wasintroduced i d dwith i hArcGIS A GIS 10.0 10 0 ArcPy isacollectionofmodules,classesand f functions i which hi hgive i accesstoall llthe h geoprocessing i toolsinArcGIS fromwithinPython Most M geoprocessing i scripts i will illstartwith: ih import arcpy Note:ArcPy replacestheolderarcgisscripting module

SettingCurrentWorkspace
AfterimportingArcPy,mostscriptsstartwithsettinga workspacetoretrieveandstorefiles import arcpy arcpy.env.workspace k = "c:/workshop" " / k h " Inthe h code d above b env is i aclass l and dworkspace is i a propertyofthisclass arcpy.<class>.<property>

UsingTools
ArcPy givesyouaccesstoalltoolsinArcToolbox Alltoolsareprovidedasfunctions arcpy.<toolname_toolboxalias>(<parameters>) Example:
import arcpy arcpy env workspace = "c:/data" arcpy.env.workspace arcpy.Clip_analysis("streams.shp", "study.shp", "result.shp")

ToolParameters
Agoodunderstandingoftoolparametersisessential Parametershaveproperties:
Name Type(featureclass,integer,etc.) Direction(inputoroutput) Requiredoroptional

Example: p Clip ptool

ToolSyntax
Tooldialog:

Pythonsyntax:

Clip_analysis(in_features, clip_features, out_feature_class, {cluster tolerance}) {cluster_tolerance}) Clip_analysis("streams.shp","study.shp", " "result.shp") lt h ")

Example:

OptionalParameters
Requiredtoolparametersarelistedfirst Optionaltoolparameterscanbeleftout
Butwhatifsomeneedtobeset?
Buffer_analysis (in_features, out_feature_class buffer_distance_or_field, {line_side}, {line_end_type}, {dissolve_option}, { p }, { {dissolve_field}) }) arcpy.Buffer_analysis("roads", "buffer", "100 METERS", "", LIST , "Code") Code ) "", "LIST" arcpy.Buffer_analysis("roads", "buffer", "100 METERS", di dissolve_option=LIST, l ti LIST dissolve_field=Code) di l fi ld C d )

HardcodedParameters
Considertheexample
import arcpy arcpy.env.workspace = "c:/data" arcpy.Clip_analysis("streams.shp", "study.shp", "result.shp")

Howcanwemakethiscodemoreusable?

UsingVariablesforParameters
import arcpy arcpy.env.workspace = "c:/data" infc = "streams.shp" clipfc = "study.shp" outfc = "result.shp" arcpy.Clip_analysis(infc, clipfc, outfc)

VariablesProvidedbyaUser
import arcpy infc = arcpy.GetParameterAsText(0) clipfc = arcpy.GetParameterAsText(1) outfc = arcpy.GetParameterAsText(2) arcpy.Clip_analysis(infc, clipfc, outfc)

ResultObjects
ArcPy returnstheoutputofatoolasaResultobject
import arcpy arcpy.env.workspace = "c:/data" myresult = arcpy.Clip_analysis("streams.shp,"study.shp","result.shp") print myresult

Thiswillprintthepathtotheoutputdataset
c:/data/result.shp

MultipleOperationsusingResultObjects
Resultobjectscanbeusedastheinputintoanother function
import arcpy arcpy.env.workspace = "c:/data/study.gdb" buffer = arcpy.Buffer_analysis("str","str_buf","100 METERS") py _management(buffer) g count = arcpy.GetCount print count

Thisallowscomplexgeoprocessing operations

ArcPy Classes
Sometoolparametersarecomplicated/detailed
e.g.coordinatesystem

ArcPy classesareusedtoworkwiththeseparameters
Classesareusedtocreateobjects j Classeshavepropertiesandmethods

Generalsyntax
arcpy.<classname>(<parameters>)

ArcPy Classes:Example
Thefollowingisanexampleofthecontentsofa.prj file

Toavoidhavingtoworkwiththisactualstring,wecanusea SpatialReference class

ArcPy Classes:Example
Th Thefollowing f ll i example l createsaspatial i lreference f object bj basedonanexisting.prj file propertiesofthisobjectcan thenbeused
import arcpy prjfile = "c:/data/streams c:/data/streams.prj prj" spatialref = arcpy.SpatialReference(prjfile) myref = spatialRef.name print myRef

Thiswillprint
NAD_1983_StatePlane_Florida_East_FIPS_0901_Feet

ArcPy Classes:Example
Thefollowingexamplecreatesaspatialreference objectandusethistodefinethecoordinatesystem ofanewfeatureclass
import arcpy out_path = "c:/data" out_name = "lines.shp" prjfile = "c:/data/streams.prj" spatialref = arcpy.SpatialReference(prjfile) arcpy.CreateFeatureclass management(out path, out_name, arcpy.CreateFeatureclass_management(out_path, out name, "POLYLINE", "", "", "", spatialref)

ArcPy Functions
Allgeoprocessing toolsareArcPy functions AdditionalArcPy functions:
listingdata Retrieving gandsetting gproperties p p Manymore

Generalsyntax
arcpy <functionname>(<arguments>) arcpy.<functionname>(<arguments>)

ArcPy Functions
Cursors C Describingdata Environmentandsettings Fields General Generaldatafunctions Gettingandsettingparameters Licensingandinstallation Listingdata Messaginganderrorhandling Progressdialog Toolsandtoolboxes

DescribingandListingData

DescribingData
The Th Describe D ib f function ti is i used dto t determine d t i properties ti of f d dataset t t Generalsyntax import arcpy <variable> = arcpy.Describe(<input dataset>) Example: import arcpy desc = arcpy.Describe("c:/data/streams.shp") print desc.shapeType desc shapeType

DescribingData:Example
import i t arcpy arcpy.env.workspace = "c:/data" infc = "streams.shp" clipfc = "study.shp" outfc = "streams_clip.shp" desc = arcpy.Describe(clipfc) arcpy Describe(clipfc) type = desc.shapeType if type == "Polygon": arcpy.Clip_analysis(infc, clipfc, outfc) else: print "The The clip features are not polygons polygons."

ListingData
Li Listing ti d data t i isverycommon SeveraldifferentlistfunctionsinArcPy
ListFields ListIndexes ListDataset ListFeatureClasses ListFiles ListRasters ListTables ListWorkspaces ListVersions

Similarlogic: g
Createalist Iterateoverthelistusingafor loop

ListingFeatureClasses
Th TheListFeatureClasses Li tF t Cl f ti returns function t alist li tof f featureclassesinthecurrentworkspace Generalsyntax: y
ListFeatureClasses ({wild_card}, {feature_type}, {feature dataset}) {feature_dataset})

Example:
import arcpy from arcpy import env env.workspace = "c:/data" fclist = arcpy.ListFeatureClasses()

ListingFeatureClasses
No N filtering: filt i
fclist = arcpy.ListFeatureClasses() arcpy ListFeatureClasses()

Filteringbasedonwildcard
fclist = arcpy.ListFeatureClasses("w*")

Filteringbasedonfeaturetype
fclist = arcpy.ListFeatureClasses("", "point")

ListingFields
The h ListFields function f lists l the h fields f ld inafeature f class l or tableinaspecifieddataset. General G lsyntax: t
ListFields (dataset, (dataset {wild {wild_card}, card} {field {field_type}) type})

Example p
import arcpy arcpy.env.workspace = "c:/data" fieldlist = arcpy.ListFields("roads.shp")

UsingListsinfor loops
Thefollowingscriptcreatesalistoffieldsoftype Stringanddeterminesforeachtextfieldwhatthe lengthofthefieldis
import arcpy arcpy.env.workspace = "c:/data" fieldlist = arcpy.ListFields("roads.shp", "", "String") for field in fieldlist: print field.name + " " + str(field.length)

UsingListsinfor loops
The h f following ll i script i createsalist li of fTIFFfiles fil and d iteratesthrougheachfileinthelisttobuild pyramids id
import arcpy from arcpy import env env.workspace = "c:/data" tifflist = arcpy.ListRasters("","TIF") for tiff in tifflist: arcpy.BuildParamids_management(tiff)

CreatingCustomTools

WaystoExecuteaScript
1 As 1. A astand dalone l script i
Thescriptisexecutedfromtheoperatingsystemorfrom withinaPythoneditorsuchasPythonWin WhenusingArcPy,ArcGIS needstobeinstalledand licensed NoArcGIS Desktopapplicationneedstobeopen

2. AsascripttoolwithinArcGIS
Atooldialogiscreatedtoexecutethescript ScripttoollookslikeanyothertoolinArcToolbox ToolexecutioniscontrolledfromArcGIS Desktop p

PythonScriptsasTools

WhyCreateScriptTools?
Tooldialogmakesiteasiertouse Tooldialogvalidatesuserinputs Becomespartofallgeoprocessing Environmentsettingsarepassedon WritesmessagestotheResultswindow Easiertoshare DoesnotrequireusertoknowPython

StepstoCreateScriptTools
1. 2. 3. 4. CreateaPythonscript(.py) CreateacustomToolbox(.tbx) AddatooltotheToolboxusingAddScript Modifythescriptwithinputsandoutputs

ExampleScript:HardcodedVariables
import impo t arcpy a cp from arcpy import env env.overwriteoutput = True infc = "c:/data/points.shp" / / output = "c:/data/result.txt" k = 10 n = 1 f = open(output, "w") while n <= k: result = arcpy.CalculateDistanceBand_stats(infc, n) f.write(str(n) + " " + str(result[1])+ "\n") n = n + 1 f.close()

ToolParametersandDialog

ExampleScript:UserProvidedParameters
import impo t arcpy a cp from arcpy import env env.overwriteoutput = True infc = arcpy.GetParameterAsText(0) output = arcpy.GetParameterAsText(1) k = arcpy.GetParameter(2) n = 1 f = open(output, "w") while n <= k: result = arcpy.CalculateDistanceBand_stats(infc, n) f.write(str(n) + " " + str(result[1])+ "\n") n = n + 1 f.close()

MoreArcPy Functionality

MoreArcPy Functionality
Cursorstoworkwithrowsandgeometry
Retrieve,edit,create

arcpy.sa moduletoworkwithrasters a arcpy.mapping cpy. app g moduleformapautomation Creatingcustomfunctionsandclasses

ResourcesforPythonScriptinginArcGIS

ArcGIS DesktopHelp

VirtualCampusCourses

http://training.esri.com

ArcScripts

http://arcscripts.esri.com

ArcGIS ResourceCenter

http://resources.arcgis.com

ArcGIS ResourceCenter

http://resources.arcgis.com/content/geoprocessing/10.0/about

BeyondArcGIS

UsingPySAL forSpatialAnalysis

http://geodacenter.asu.edu/pysal

PySAL
Pythonlibraryofspatialanalysismethods ESDA,spatialstatistics,geostatistics Growingandexpandable

UsingRforSpatialAnalysis
Opensourcelanguagefordataanalysis Librarieshavebeendevelopedforspatialmethods Largeandactiveusercommunity Growingandexpandable

ArcGIS andR

ScriptTool

PythonscriptthatcallsR

EvaluatingRStatements

ConcludingRemarks
Pythonisarelativelyeasytolearnlanguage ArcGIS isbecomingmorePythonesque Creatingtimesavingsscriptsforrepetitivetasksdoes not t t take k a l lot tof fcode d Easytosharescripttools

PaulZandbergen
DepartmentofGeography zandberg@unm.edu db @ d www.paulzandbergen.com p g

WorkshopMaterialsPosted

posteduntilOctober24

h // http://www.paulzandbergen.com/workshops l db / kh