Journal of the Optical Society of America, 15(7)1777-1786, 1988. Range Estimation by Optica

RangeEstimationbyOpticalDifferentiation

HanyFarid

PerceptualScienceGroup

DepartmentofBrainandCognitiveSciencesMassachusettsInstituteofTechnology

Cambridge,MA02139

farid@psyche.mit.edu

EeroP.Simoncelli

CenterforNeuralScience,andCourantInst.ofMathematicalSciences

NewYorkUniversityNewYork,NY10003-6603

eero.simoncelli@nyu.edu

Abstract

Wedescribeanovelformulationoftherangerecoveryproblembasedoncomputationofthedifferen-tialvariationinimageintensitieswithrespecttochangesincameraposition.Thismethodusesasinglestationarycameraandapairofcalibratedopticalmaskstodirectlymeasurethisdifferentialquantity.Wealsodescribeavariantbasedonchangesinaperturesize.Thesubsequentcomputationoftherangeimageinvolvessimplearithmeticoperations,andissuitableforreal-timeimplementation.Wepresentthetheoryofthistechniqueandshowresultsfromaprototypecamerawhichwehaveconstructed.

1

1Introduction

Visualimagesareformedviatheprojectionoflightfromthethree-dimensionalworldontoatwo-dimensionalsensor.Inanidealizedpinholecamera,allpointslyingonaraypassingthroughthepinholewillbeprojectedontothesameimageposition.Thus,informationaboutthedistancetoobjectsinthescene(i.e.,range)islost.Rangeinformationcanberecoveredbymeasuringthechangeinappearanceoftheworldresultingfromachangeinviewingposition(i.e.,parallax).Traditionally,thisisaccomplishedviasimultaneousmeasurementswithtwocamerasatdifferentpositions(binocularstereo),orviasequentialmeasurementscollectedfromamovingcamera(structurefrommotion).

Therecoveryofrangeintheseapproachesfrequentlyreliesontheassumptionofbrightnesscon-stancy[1]:thebrightnessoftheimageofapointintheworldisconstantwhenseenfromdifferentviewpoints.Considertheformulationofthisassumptioninonedimension(theextensiontotwodimensionsisstraightforward).Letsystem.Thevariableaxis).Thevariable

;

describetheintensityfunction,asmeasuredthroughapinholecamera

correspondstothepinholeposition(alonganaxisperpendiculartotheoptical

parameterizesthepositiononthesensor.ThisconfigurationisillustratedinFigure1.

Accordingtotheassumption,theintensityfunction

;

isoftheform:

;

0

0

2

dZI(x) = f(x;0)x=0V=0f(x;v)x=0x=vd/ZVV=vFigure1:Geometryforabinocularstereosystemwithpinholecameras.Thevariableparameterizesthepositionofthecamerapinholes.Accordingtothebrightnessconstancyconstraint,theintensityofapointintheworld,asrecordedbythetwopinholecameras,shouldbethesame.

(3)

Combiningthesetwodifferentialmeasurementsgives:

2OpticalDifferentiation

Wenowshowadirectmethodforthemeasurementoftherequiredderivatives(Equation(4))fromasinglestationarycameraandapairofopticalattenuationmasks.Conceptually,thisismadepossiblebyreplacingthepinholecameramodelwithamorerealistic“thinlens”model,inwhichthecameracollectslightfromarangeofviewpoints.Thisviewpointinformationislostoncethelightisimagedonthesensorplane.Nonetheless,atthefrontofthelens,theinformationisavailable.Itispreciselythisinformationthatweexploit.

Consideraworldconsistingofasinglepointlightsourceandalens-basedimagingsystemwithavariable-opacityopticalmask,

,placeddirectlyinfrontofthelens(leftside,Figure2).Thelight

strikingthelensisattenuatedbythevalueoftheopticalmaskfunctionatthatparticularspatiallocation(weassumethatthevaluesofsuchamaskfunctionarerealnumbersintherange[0,1],andthattheopticaltransferfunctionoftheimagingsystemisconstant).Withsuchaconfiguration,theimageofthepointsourcewillbeascaledanddilatedversionoftheopticalmaskfunction:

1

(5)

asillustratedinFigure2.Thescalefactor,,isamonotonicfunctionofthedistancetothepointsource,,

andiseasilyderivedfromtheimaginggeometryandthelensequation:

1

(6)

where

isthedistancebetweenlensandsensor,and

isthefocallengthofthelens.

2.1OpticalViewpointDifferentiation

InthesystemshownontheleftsideofFigure2,theeffectiveviewpointmaybealteredbytranslatingthemask,whileleavingthelensandsensorstationary.Forexample,consideramaskwithasinglepinhole;differentviewsoftheworldareobtainedbyslidingthepinholeacrossthefrontofthelens.Thegeneralizedimageintensityfunctionforamaskcenteredatposition

;

1

iswrittenas:

(7)

assumingthatthenon-zeroportionofthemaskdoesnotextendpasttheedgeofthelens.

4

Point Light SourceZOptical MaskLensM(u)M’(u)SensorI(x) = 1/α M(x/α)-I (x) = 1/α M’(x/α) vd/dx2I (x) = 1/α M’(x/α)xαFigure2:Illustrationofdirectdifferentialrangedeterminationforasinglepointsource.Imagesofapointlight

anditsderivative,sourceareformedusingtwodifferentopticalmasks,correspondingtothefunction

.Ineachcase,theimageformedisascaledanddilatedcopyofthemaskfunction(byafactor).

producesanimagethatisComputingthespatial(image)derivativeoftheimageformedundermask

,exceptforascalefactor.Thus,mayidenticaltotheimageformedunderthederivativemask,

beestimatedastheratioofthetwoimages.RangeisthencomputedfromusingtherelationshipgiveninEquation(6).

Thedifferentialchangeintheimagewithrespecttoachangeinthemaskpositionisobtainedbytakingthederivativeofthisequationwithrespecttothemaskposition,,andevaluatingat

0:

;

0

1

(9)

5

Combiningthesetwoequationsgivesarelationshipbetweenthetwoderivatives:

1

2

(12)

Byintegratingoverasmallpatchintheimage,theleast-squaressolutionavoidssingularitiesatlocationswherethespatialderivative,

,iszero.However,sincethedenominatorstillcontainsan

term

(integratedoverasmallimagepatch),asingularitystillexistswhen

iszeroovertheentireimage

patch.Thissingularitymaybeavoidedbyconsideringamaximumaposteriori(MAP)estimatorwithaGaussianprioron

(asin[3]).Forapriorvarianceof

2

,theresultingestimateis:

Thisalgorithmextendstoatwo-dimensionalimageplane:weneedonlyconsidertwo-dimensionalmasks

,andthehorizontalpartialderivative

.Foramorerobust

implementation,theverticalpartialderivativemask,Theleast-squareserrorfunctionbecomes:

,mayalsobeincluded.

2

2

(14)

Asabove,theMAPestimatorgives:

factorisincludedtoensurethatthechangesintheaperturesizedonotchangethemeanintensityof

theimage.

Thedifferentialchangeintheimagewithrespecttoaperturesizemaybecomputedbytakingthederivativeofthisequationwithrespecttoaperturesize,

,evaluatedat

1:

2

2

andthespatialderivativeofthisimageis:

1

(19)

whichisequaltoEquation(17)withanadditionalfactorofmeasurementscouldbeusedtocompute

.Thus,theratioofthesetwoimage

(andthusrange).

AparticularlyinterestingchoiceforamaskfunctionisaGaussian:

;

1

2

2

forwhich

1

(20)

whichmayberelatedtothespatialsecondderivative:

1

1

2

2

(23)

Thereareseveralnotabledifferencesbetweenthisformulationbasedonaperturesizederivativesandthepreviousformulationbasedonviewpointderivatives.First,asecond-orderspatialderivativeisrequired.Second,theratiooftheaperturesizederivativeandspatialderivativeisproportionaltothesquareoftheparameter.Assuch,onlytheabsolutevalueof

canbedetermined.Asecondlookattheopticalmasks8

revealswhythismustbeso.Whereastheviewpointderivativemaskisanti-symmetricwithrespecttothecenterofthelens,theaperturesizederivativemaskissymmetric.Asaresult,pointspositionedonoppositesidesofthefocalplanewilldifferbyasigninthecaseoftheanti-symmetricmask,butcouldappearidenticalinthecaseofthesymmetricmask.Thisambiguitymaybeeliminatedbyfocusingthecameraatinfinity,thusensuringthat

0.

3RangeMapEstimationbyOpticalDifferentiation

Equations(10)and(21)embodythefundamentalrelationshipsusedfortheopticaldifferentialcomputationofrangeforasinglepointlightsource.Aworldconsistingofacollectionofmanysuchpointsourcesimagedthroughanopticalmaskwillproduceanimageconsistingofasuperpositionofscaledanddilatedversionsofthemasks.Inparticular,inthecaseoftheviewpointderivative,anexpressionfortheimagecanbewrittenbyintegratingovertheimagesofthevisiblepoints,,intheworld:

;

1

(24)

wheretheintegralisperformedoverthevariable

,thepositioninthesensorofapointprojectedthrough

thecenterofthelens.Theintensityoftheworldpoint

isdenotedas

,and

ismonotonically

relatedtothedistanceto(asinEquation(6)).Notethateachpointsourceisassumedtoproduceauniformlightintensityacrosstheopticalmask(i.e.,brightnessconstancy).Again,considerthederivativesofwithrespecttoviewingposition,,andimageposition,,evaluatedat

;

0:

;

0

1

(26)

Anexactsolutionfor

isnontrivial,sinceitisembeddedintheintegrandanddependsontheintegration

variable.Nevertheless,thecomputationofEquation(10)givesanestimate:

ˆ

1

2

2

systems,astereopairofimagesisgeneratedbytwofixedmirrors,ata45anglewiththecamera’soptical

axisandarotatingmirrormadeparalleltoeachofthefixedmirrors.Inthesecondsystem,arotatingglassplateplacedinfrontofthemainlens,shiftstheopticalaxis,simulatingtwocameraswithparallelaxis.Thelastsystemplacestwoangledmirrorsinfrontofacameraproducinganimagewheretheleftandrighthalfoftheimagecorrespondtotheviewfromapairofvergedvirtualcameras.Ineachcase,rangeiscalculatedusingstandardstereomatchingalgorithms.Thebenefitoftheseapproachesisthattheyeliminatetheneedforextrinsiccameracalibration(i.e.,determinationoftherelativepositionsoftwoormorecameras),buttheydorequireslightlymorecomplicatedintrinsiccalibrationofthecameraoptics.

5Experiments

Wehaveverifiedtheprinciplesofopticaldifferentiationforrangeestimationbothinsimulationandexperimentation[16].Thissectiondiscussestheconstructionofaprototypecamera,andshowsexamplerangemapscomputedusingthiscamera.

5.1PrototypeCamera

Wehaveconstructedaprototypecameraforvalidatingthedifferentialapproachtorangeestimation.AsillustratedinFigure3,thecameraconsistsofanopticalattenuationmasksandwichedbetweenapairofplanar-convexlenses,andplacedinfrontofaCCDcamera.Thisarrangementplacesthemaskatthecenteroftheopticalsystem,wheretheviewpointinformationisisolated.ThecameraisaSonymodelXC-77R,thelensesare25mmindiameter,50mmfocallength,andwereplaced31mmfromthesensor.Thus,thecamerawasfocusedatadistanceof130mm.Wehaveemployedaliquidcrystalspatiallightmodulator(LCSLM),purchashedfromCRLSmecticTechnology(Middlesex,UK),foruseasanopticalattenuationmask,alsoshowninFigure3.Thisdeviceisafullyprogrammable,fast-switching,twistednematicliquidcrystaldisplaymeasuring38mm(W)

42mm(H)

4.3mm(D),withadisplayareaof28.48mm(W)

20.16mm(H).Thespatialresolutionis0480pixels,with4possibletransmittancevalues.Thedisplay

iscontrolledthroughastandardVGAvideointerface,suppliedbythemanufacturer.TheLCSLMrefreshesitsdisplayat30Hz;whensynchronizedwiththecamera,thepairofimagesmaybeacquiredbytemporalinterleavingatarateof15Hz.Alternatively,apairofimagescouldbeacquiredsimultaneously(i.e.,30Hz),byemployinganadditionalcamera,somebeam-splittingoptics,andtwofixedopticalmasks,asin[8].

11

Figure3:PrototypeCamera.Illustratedontheleftisafastswitchingliquidcrystalspatiallightmodulator(LCSLM)employedasanopticalattenuationmask.Illustratedontherightisourrangecameraconsistingofanoff-the-shelfCCDcameraandtheLCSLMsandwichedbetweenapairofplanar-convexlenses.Thetargetconsistsofapieceofpaperwitharandomtexturepattern.

Thesubsequentprocessing(i.e.,convolutionsandarithmeticcombinations)canbeperformedinreal-timeonageneral-purposeDSPchiporperhapsevenonafastRISCmicroprocesser.

5.2OpticalMasks

Themostessentialcomponentofourrangecameraistheopticalattenuationmask.Thefunctionalformofanysuchmaskmustonlycontainvaluesintherange01,whereavalueof0correspondstofullattenuation

andavalueof1correspondstofulltransmittance.Theviewpointandaperturesizederivativemasksbothcontainnegativevalues,andthusmaynotbeuseddirectly.Furthermore,addingapositiveconstanttothederivativemaskdestroysthederivativerelationship.Nonethelessapairofnon-negativemaskscanbeconstructedbytakingtheappropriatelinearcombinationoftheoriginalmasks.Inparticular,considerthefollowingconstructionofapairofnon-negativemasks:

1

1

1

and

2

2

2

(28)

wherethescalingparametersThedesiredmasks,

12

and

12

arechosensuchthat

1

and

2

lieintherange01.

and

canthenbereconstructedthroughasimplelinearcombinationsofthe:

2

non-negativemasks,

1

and

1

2

2

1

12

21

(29)

and

Iftheimagingsystemislinear,thedesiredimagesformedunderthemasks

12

canbe

Figure4:Gaussianaperturemasks.Left:Atwo-dimensionalGaussianmask,derivative,.Right:Twonon-negativeaperturemasks,1and2fromthepairofleft-mostmasksusingEquation(28).

anditspartial.Thesearecomputed

determinedfromtheimagesformedunderthemasks

1

and

2

:

(30)

,respectively.Clearly,

21

12

12

21

where

1

and

2

aretheimagesformedunderthemasks

1

and

2

thisconstructionextendsto2-Dopticalmasksaswell.IllustratedinFigure4isa2-DGaussianmask,

1

7

16

351

wherethe

representsthequantizedpixel,andthepositionoftheweightsrepresentspatialpositionona

rectangularsamplinglattice.Sincethisalgorithmmakesonlyasinglepassthroughtheimage,theneighborsreceivingaportionoftheerrormustconsistonlyofthosepixelsnotalreadyvisited(i.e.,thealgorithmshouldbecausal).Inordertoavoidsomeofthevisualartifactsduetothedeterministicnatureofthisalgorithm,stochasticvariationsmaybeintroduced.Alongtheselines,wehavetakenthestandarderror

13

diffusionalgorithmandrandomizedtheerror(byafactordistributeduniformlybetween90%and110%)beforedistributingittoitsneighbors,andalternatedthescanningdirection(oddlinesarescannedfromlefttoright,andevenlinesarescannedfromrighttoleft).5.2.2

Calibration

Inoursystem,thereareatleasttwopotentialnon-linearitiesthatneedtobeeliminated.Thefirstisthenon-linearityinthelighttransmittanceoftheopticalattenuationmasks.Anon-linearityatthisstagewillaffectthederivativerelationshipoftheopticalmasks.AsillustratedinFigure5,theLCSLMusedtogeneratetheopticalmasksishighlynon-linear.Showninthefirstpanelofthisfigureisthelighttransmittance(measuredwithaphotometer)throughauniformmasksettoeachofthefourLCDlevels.Ifthisdevicewerelinear,thenthesemeasurementswouldliealongaunit-slopeline.Clearlytheydonot.

Thisnon-linearitymaybecorrectedintheditheringprocess.Morespecifically,intheditheringalgorithmtheerrorbetweenthequantizedpixelvalueandthedesiredvalueisdistributedtoitsneighbors.Inordertocorrectforthenon-linearityintheLCSLM,thediffusederrormaybeassignedthedifferencebetweenthedesiredandmeasuredlighttransmittancesofthequantizedpixel.NotethatweareassumingthattheLCSLMpixelsareindependent(i.e.,therearenointeractionsbetweenneighboringtransmittancevalues).IllustratedinFigure5isthemeasuredlighttransmittancethrough32constantmasks,ditheredusingtothistechnique.Thedataareseentobemuchmorelinearthantheoriginalmeasurements.

Thesecondpotentialnon-linearitytoconsiderisintheimagingsensor.Withtheopticalmasklinearized,itispossibletotestthelinearityoftheimagingsensorbymeasuringthepixelintensityoftheimageofapointlightsourceimagedthroughaseriesof(dithered)uniformgraymasks.AsshowninFigure5,theimagingsensorisfairlylinear.Inparticular,thedatainthisfigurecloselyresemblesthemeasuredlighttransmittanceoftheLCSLMafterlinearization(Figure5).Thus,itisassumedthattheimagingsensorislinear,andasecondlinearizationcorrectionisnotnecessary.

Thefinalcalibrationthatneedstobeconsideredisthatofthecamera’sintrinsicpointspreadfunction(PSF).Morespecifically,indescribingtheformationofanimagethroughanopticalattenuationmaskwehavebeenassumingthatthecamera’sPSFisconstantacrossthelensdiameter(i.e.,theimageofapointlightsourceisassumedtobeahard-edgedrectangularfunction).Thisisgenerallynotthecaseinarealcamera:thePSFtypicallytakesonaGaussian-likeshape.ThePSFandtheopticalmaskwillbecombinedinamultiplicativefashion.Whereasbefore,weemployedamatchedpairofmasks,

14

and

,with

Normalized Light TransmittanceNormalized Light Transmittance11Normalized CCD Intensity10.50.50.5000.51000.51000.51Figure5:CalibrationofLCDmaskandCCDsensor.Showninthefirstpanelisthenormalizedlighttransmittance(incd/m2,asmeasuredwithaphotometer)throughconstantmaskssettoeachofthefourLCDvalues.Showninthesecondpanelisthenormalizedlighttransmittancemeasuredthrougheachof32uniformditheredandgamma-correctedmasks,averagedoverfivetrials.Ifourditheringandgamma-correctionwereperfect,themeasurements(circles)wouldliealongaunit-slopeline(dashedline).ShowninthethirdpanelisthenormalizedCCDpixelintensityofapointlightsourceasimagedthroughaseriesof32uniform,ditheredopticalmasks(withgammacorrection),averagedoverfivetrials,andspatiallyintegratedovera55pixelneighborhood.Ifboththeopticalmaskandimagingsensorwerelinear,thenthesemeasurements(circles)wouldliealongaunit-slopeline(dashedline).

(31)

where

isthecameraPSF.Thatis,thederivativerelationshipshouldbeimposedontheproductofthe

opticalmaskandthePSF.Wehavenotincludedthiscalibrationinourexperiments.

5.3Results

Wehaveverifiedtheprinciplesofrangeestimationbyopticaldifferentiationwithaprototypecamerawhichwehaveconstructed(seeFigure3).Accordingtoourinitialobservationweexpectthattheimageofapointlightsourcetobeascaledanddilatedcopyofthemaskfunction.IllustratedinFigure6isanexampleofthisbehavior:shownare1-Dslicesofimagestakenthroughapairofnon-negativeGaussian-basedopticalmasksprintedontoasheetoftransparentplastic(laterexperimentsutilizedtheLCSLMdescribedabove).Theappropriatelinearcombinationoftheseslices(Equation(30)),andtheresultingslicesof

and

.

Intheremainingexperimentsthetargetconsistedofasheetofpaperwitharandomtexturepatternandback-illuminatedwithanincandescentlamptohelpcounterthelowlighttransmittanceoftheLCSLMopticalmask.Spatialderivativeswerecomputedusingapairof5-tapfilterkernelsdescribedin[18].For

15

Figure6:Illustratedinthefirsttwopanelsare1-Dslicesoftheimageofa“pointlightsource”takenthroughapairofnon-negativeGaussian-basedmasks,1and2.Showninthethirdpanelare1-Dslicesofthelinearcombinationofthemeasurements(seeEquation(30)).Showninthefourthpanelare1-Dslicesoftheresultingimages(solid)and(dashed).Theseimagesshouldberelatedtoeachotherbyascalefactorof(seeEquation(10)).

example,the-derivativeiscomputedviaseparableconvolutionwiththeone-dimensionalderivativekernel

inthe

direction,andwithaone-dimensionalblurringkernelinthe

direction.Theviewpointderivative

wasfilteredwiththeblurringkernelinbothspatialdirections.Rangewasestimatedusingtheleast-squaresformulation(Equations(15)or(23)),withaspatialintegrationneighborhoodof31

31pixels.

IllustratedinFigures7and8areapairofrecoveredrangemapsforfrontal-parallelsurfacesplacedatdistancesof11and17cmfromthecamera.Thesefiguresillustratetherangemapscomputedusingopticalviewpointandaperturesizedifferentiation,respectively.Thecameraisfocusedatadistanceof13cm.Inthecaseoftheviewpointdifferentiation,therecoveredrangemapshadameanof10.9and17.0cm,withastandarddeviationof0.27and0.75cm,andaminimum/maximumestimateof10.1/11.8cmand15.1/19.4cm,respectively.Inthecaseoftheaperturesizedifferentiation,therecoveredrangemapshadameanof11.0and17.0cm,withastandarddeviationof0.06and0.16cmandaminimum/maximumestimateof10.8/11.2cmand16.5/17.5cm,respectively.Itwassomewhatsurprisingtodiscoverthattheaperturesizedifferentiationgavesignificantlybetterresultsthantheviewpointdifferentiation(intermsofstandarddeviation).Wesuspectthatonepossiblereasonforthisisthattheaperturesizemaskshaveahighertotallighttransmittance:fortheGaussian-basedopticalmasks,themeanlighttransmittanceis0.37,ascomparedtoameanof0.20fortheviewpointmasks.Increasedlighttransmittanceproducesahighersignal-to-noiseratiointhemeasurements.Althoughtheaperturesizedifferentiationhassmallererrorsinthisexample,itsuffersfromasignambiguity(i.e.,surfacesoneithersideofthefocalplanecanbeequallydefocused).IllustratedinFigure9isarecoveredrangemapforaplanersurfaceorientedapproximately30degreesrelativetothesensorplane,withthecenteroftheplane14cmfromthecamera,andapairofoccludingsurfacesplacedat11and17cm.Therecoveredrangemapsinthisfigureweredeterminedusing

16

theviewpointdifferentiationformulation.Qualitatively,theserangemapslookquitereasonable.

5.4Sensitivity

Aswithmostothertechniques,theinherentsensitivityofourmethodofrangeestimationisdependentonthebasicrulesoftriangulation.Inparticular,fromclassicalbinocularstereoweknowthatrangeisinverselyproportionaltodisparity:

(32)

Inoursystem,

playstheroleofdisparity,andeffectivebaselineisproportionaltolensdiameterand

dependentonthechoiceofopticalmasks.

Inaddition,errorsinestimating

willbeproportionalto

2.

Morespecifically,weconsidertheeffects

ofadditivenoiseinthedifferentialmeasurements:

ˆ

∆

2

∆∆

2∆

1

2∆

1

2

∆

(34)

Thatis,measurementerrorsleadtorangeerrorsthatscaleasthesquareofthedistancefromthefocalplane.

6Discussion

Wehavepresentedthetheory,analysis,andimplementationofanoveltechniqueforestimatingrangefromasinglestationarycamera.Thecomputationofrangeisdeterminedfromapairofimagestakenthroughoneoftwoopticalattenuationmasks.Thesubsequentprocessingoftheseimagesissimple,involvingonlyafew1Dconvolutionsandarithmeticoperations.

17

2010020100Figure7:Illustratedaretherecoveredrangemapsusingopticalviewpointdifferentiationforapairoffrontal-parallelsurfacesatadistanceof11and17cmfromthecamera.Thecomputedrangemapshaveameanof10.9and17.0cmwithastandarddeviationof0.27and0.75cm,respectively.

2010020100Figure8:Illustratedaretherecoveredrangemapscomputedusingopticalaperturesizedifferentiationforapairoffrontal-parallelsurfacesatadistanceof11and17cmfromthecamera.Thecomputedrangemapshaveameanof11.0and17.0cmwithastandarddeviationof0.06and0.16cm,respectively.

2010020100Figure9:Illustratedontheleftistherecoveredrangemapcomputedusingopticalviewpointdifferentiationforaslantedsurfaceorientedapproximately30degreesrelativetothesensorplanewiththecenteroftheplaneatadepthof14cm.Illustratedontherightistherecoveredrangemapforapairofoccludingsurfacesatadepthof11and17cm.

18

Thesimplicityofthistechniquehassomeclearadvantages.Inparticular,theuseofasinglestationarycamerareducesthecost,sizeandcalibrationoftheoverallsystem,andthesimpleandfastcomputationsrequiredtoestimaterangemakesthistechniqueamenabletoareal-timeimplementation.Incomparisontoclassicalstereoapproaches,ourapproachcompletelyavoidsthedifficultandcomputationallydemanding“correspondence”problem.Inaddition,withonlyasinglestationarycamera,weavoidtheneedforextrinsiccameracalibration.Therearesomedisadvantagesaswell.Mostnotably,theconstructionofanon-standardimagingsystem,andthelimitedrangeaccuracyduetothesmalleffectivebaseline.

Acounterintuitiveaspectofourtechniqueisthatitreliesonthedefocusoftheimage.Inparticular,aperfectlyfocusedimagecorrespondsto

0,leadingtoasingularityinEquation(10).Wehavepartially

overcomethisproblembyimposingapriordensityonthatbiasessolutionstowardthefocalplane.But

ingeneral,accuracywillbebestforsurfacesoutsideofthefocalplane.

Theresultspresentedherecanbeimprovedinanumberofways.Abettermaskdesign,whichincludestheeffectsofthePSFofthecameraopticsandoptimizeslighttransmittancewhilesatisfyingthedesiredderivativerelationshipcouldhavealargeeffectonthequalityoftheestimator.Intheproposedcamera,apairofimagesareacquiredinatemporallyinterleavedfashion,sothatmotioninthescenewillbemisinterpretedasfalserangeinformation.Amoresophisticatedalgorithmshouldbedevelopedthatcompensatesforanyinter-framemotion.Alternatively,thetechniquecouldbemodifiedtomeasurethetwoimagessimultaneously(usingabeamsplitter,asin[8]).Finally,aswithallintensity-basedrangeimagingapproaches,theresultsmaybeimprovedbyilluminatingthescenewithstructuredlight.

Acknowledgments

ThisresearchwasperformedwhileHFwasintheGRASPLaboratoryattheUniversityofPennsylvania,wherehewassupportedbyARODAAH04-96-1-0007,DARPAN00014-92-J-17,andNSFSBR-20230.HFiscurrentlyatMITwhereheissupportedbyNIHGrantEY11005-04andMURIGrantN00014-95-1-0699.ThisresearchwasperformedwhileEPSwasintheGRASPLaboratoryattheUniversityofPennsylvania,wherehewaspartiallysupportedbyARO/MURIDAAH04-96-1-0007.EPSiscurrentlyatNYU,whereheispartiallysupportedbyNSFCAREERgrant9624855.Portionsofthisworkhaveappearedin[19,20,21,16].

19

References

[1]B.K.P.Horn.RobotVision.MITPress,Cambridge,MA,1986.

[2]B.D.LucasandT.Kanade.Aniterativeimageregistrationtechniquewithanapplicationtostereo

vision.InProceedingsofthe7thInternationalJointConferenceonArtificialIntelligence,pages674–679,Vancouver,1981.

[3]E.P.Simoncelli,E.H.Adelson,andD.J.Heeger.Probabilitydistributionsofopticalflow.InProc

ConfonComputerVisionandPatternRecognition,pages310–315,Mauii,Hawaii,June1991.IEEEComputerSociety.

[4]A.P.Pentland.Anewsensefordepthoffield.IEEETransactionsonPatternAnalysisandMachine

Intelligence,9(4):523–531,1987.

[5]M.Subbarao.Paralleldepthrecoverybychangingcameraparameters.InProceedingsoftheIn-ternationalConferenceonComputerVision,pages149–155,Tampa,FL,1988.IEEE,NewYork,NY.

[6]Y.XiongandS.Shafer.Depthfromfocusinganddefocusing.InProceedingsoftheDARPAImage

UnderstandingWorkshop,pages967–976,NewYork,NY,1993.IEEE,Piscataway,NJ.

[7]A.M.SubbaraoandG.Surya.Depthfromdefocus:Aspatialdomainapproach.InternationalJournal

ofComputerVision,13(3):271–294,1994.

[8]S.K.Nayar,M.Watanabe,andM.Noguchi.Real-timefocusrangesensor.IEEETransactionson

PatternAnalysisandMachineIntelligence,18(12):1186–1198,1995.

[9]M.WatanabeandS.Nayar.Minimaloperatorsetforpassivedepthfromdefocus.InProceedings

oftheConferenceonComputerVisionandPatternRecognition,pages431–438,SanFrancisco,CA,1996.IEEE,LosAlasmitos,CA.

[10]E.R.DowskiandW.T.Cathey.Single-lenssingle-imageincoherentpassive-rangingsystems.Applied

Optics,33(29):6762–6773,1994.

[11]D.G.JonesandD.G.Lamb.Analyzingthevisualecho:Passive3-dimagingwithamultipleaperture

camera.TechnicalReportCIM-93-3,DepartmentofElectricalEngineering,McGillUniversity,1993.

20

[12]E.H.AdelsonandJ.Y.A.Wang.Singlelensstereowithaplenopticcamera.IEEETransactionson

PatternAnalysisandMachineIntelligence,14(2):99–106,1992.

[13]W.TeohandX.D.Zhang.Aninexpensivestereoscopicvisionsystemforrobots.InInternational

ConferenceonRobotics,pages186–1,1984.

[14]Y.NishimotoandY.Shirai.Afeature-basedstereomodelusingsmalldisparities.InProceedingsof

theConferenceonComputerVisionandPatternRecognition,pages192–196,Tokyo,1987.[15]A.GoshtasbyandW.A.Gruver.Designofasingle-lensstereocamerasystem.PatternRecognition,

26(6):923–937,1993.

[16]H.Farid.RangeEstimationbyOpticalDifferentiation.PhDthesis,DepartmentofComputerand

InformationScience,UniversityofPennsylvania,Philadelphia,PA.,1997.

[17]R.W.FloydandL.Steinberg.Anadaptivealgorithmforspatialgreyscale.ProceedingsoftheSociety

forInformationDisplay,17(2):75–77,1976.

[18]H.FaridandE.P.Simoncelli.Optimallyrotation-equivariantdirectionalderivativekernels.InCom-puterAnalysisofImagesandPatterns,Kiel,Germany,1997.

[19]E.P.SimoncelliandH.Farid.Directdifferentialrangeestimationfromaperturederivatives.In

ProceedingsoftheEuropeanConferenceonComputerVision,pages82–93(volumeII),Cambridge,England,1996.

[20]E.P.SimoncelliandH.Farid.Single-lensrangeimagingusingopticalderivativemasks,1995.U.S.

PatentPending(filed14Nov1995).InternationalPatentPending(filed13Nov1996).

[21]H.FaridandE.P.Simoncelli.Adifferentialopticalrangecamera.InProceedingsoftheAnnual

MeetingoftheOpticalSocietyofAmerica,Rochester,NY,1996.

21