Reconstruction Pipeline Howto

version0.1

Concept
Thepipelineconsistsoffourparts.
1. Structurefrommotion(sfm):estimatescamerapositionsandintrinsicparametersfrom
images
2. Multiviewstereo(mvs):estimatesdepth(distancefromcamera)foreachpixelineach
image
3. Surfacereconstruction:mergesalldepthmapsintoagloballyconsistentmesh
4. Export:exportsmeshverticesintoPMVSfileformatrequiredforimagebasedrendering

Licenses
Forstructurefrommotion,weuseathirdpartysoftwarewhichisfreefornonprofituse.Thisis
notaproblemfortheprereleasecandidate.Later,thiswillbereplacedbyourowncode.
ThesourcecodeforMVSisourownimplementation.ItispubliclyavailableundertheGPL,but
sinceitisourowncodeweshouldbeabletoshipitunderadifferentlicense,too?TheMVE
pipelinecontainsonlybinaries.Iamnotsureaboutthelicensesthere.
Thesameholdsforthegraphicaluserinterface(umve)whichisshippedwithMVE.
Furthermore,umvereliesonQtandwepackedtherelevantdllsintotheziparchivefor
convenience.Iamnotsurewhetherthisisokaylicensewise.

Install
1. Createadirectoryforallcomponents,e.g.crplay.
2. Downloadandextractthenewestziparchivefromhttp://ccwu.me/vsfm/into
crplay\VisualSFM.ThisshouldyieldasetofdllsandtheVisualSFM.exe.
3. ExtracttheMVEpipelineziparchiveintocrplay(thisyieldstwosubdirectoriesmveand
gui).
4. Optional.Extracttheimagesfromtheexampledatasettower.zip.

Run
1. Createafoldertostoreallthedata,e.g.crplay\data\tower.
2. Copyallinputimagesintoasubfolder,e.g.crplay\data\tower\images.
3. Startacommandpromptandnavigateintothecrplayfolder.
4. RuntheSfMstep:VisualSFM\VisualSFM.exesfm+sortdata\tower\images
data\tower\bundle.nvm.Thesecondargumentisthefoldercontainingtheinputimages.
Thethirdargumentistheoutputfilewhichstorestheestimatedcameraparameters.
 5. ImportimagesandcameraparametersintoanMVEscene(thishastoberunfromthe
directorycontainingbundle.nvm):cddata\tower\&..\..\mve\makescene.exebundle.nvm.
&cd..\..
6. Optional.Openthesceneinumve(theGUItool)andcheckthatcamerapositionsmake
sensebyclickingonthesceneinspecttabandrotatingtheviewuntilyouseethepoint
cloudofthescene:cdgui&umve.exe..\data\tower&cd..
7. RuntheMVSstep:mve\dmrecon.exedata\tower.Thisreconstructsadepthmapper
imageandmighttakesometime.Youcanspecifyas1ors2commandline
argumenttoreconstructhalf/quartersizedimages.
8. Optional.Openthesceneinumveandcheckthereconstructionbybrowsingthedepth
embeddingsintheviewinspecttab:cdgui&umve.exe..\data\tower&cd..Youcan
alsovisualizeadepthmapin3DbynavigatingtotheOperationstabwithintheScene
InspecttabandselectingtherespectivedepthembeddingundertheDMTriangulate
section.
9. Combinetheindividualdepthmapsintoalargepointcloud:mve\scene2pset.exensc
data\towerdata\tower\point_set.ply.IfyourantheMVSreconstructionondownscaled
images,youalsohavetospecifyddepthLXiundistLXasarguments(whereXisthe
scalenumberusedinstep7,e.g.mve\scene2pset.exeddepthL2iundistL2nsc
data\towerdata\tower\point_set.ply).
10. Insertallpointsintoanoctree:mve\fssr_octree.exedata\tower\point_set.ply
data\tower\oct.ree.
11. Extractthemeshfromthesigneddistancefieldstoredintheoctree:
mve\fssr_surface.exedata\tower\oct.reedata\tower\mesh.ply.Youmightwantto
experimentwiththetandccommandlineswitchestoremovemoreunreliable
geometry(e.g.mve\fssr_surface.exet5.0c500data\tower\oct.ree
data\tower\mesh.ply).
12. Optional:Openthesceneinumveandcheckthemeshreconstruction.Switchtothe
sceneinspecttabandclicktheopenmeshbutton:cdgui&umve.exe..\data\tower&
cd..
13. ExportthemeshintothePMVSpatchfileformatforimagebasedrendering(thishasto
berunfromthedirectorycontainingthebinary):cdgui&scene2patch.exe
originalresmax=2592creategldepthmaps..\data\tower..\data\tower\mesh.ply
..\data\tower\patches.pmvs&cd..

Thisisanexamplescreenshotofthecoloredmeshobtainedforthetowerdataset: