This text is replaced by the Flash movie.

 

 
 

 

 

JPT Nº12

 Sponsors & Supporters

 

 

 

 

How does photogrammetry work?

Photogrammetry, or more strictly speaking stereophotogrammetry, is the derivation of 3D information on points, lines and areas on objects or terrain from photographic image sequences. It is a non-contact technique which creates virtual reality 3D scenes with real-life textured models. Created data can be used for measurements and interpretation of objects by providing precise 3D point coordinates and other geometric and semantic object information such as size and volume information. It can be used in digital, graphical and orthophoto forms as maps, charts and overlays, but also 3D-printed to create precise copies of the original objects.

As of spring 2014, there are more than 40 different computer programs available that are capable of creating photogrammetricmodels (e.g.,see https://en.wikipedia.org/wiki/Compari son of photogrammetry software). Cost-free programs include VisualSFM (Wu, 2007, 2011; Wu et al., 2011) and Autodesk's 123D Catch (however, the latter can be used free of charge, but the fine print says that Autodesk actually owns any scans you make and can use it for a wide variety of purposes, including marketing). We present our examples with the commercial software Agisoft Photoscan Professional (www.agisoft.ru), which we believe to currently be one of the most user-friendly programs for photogrammetry. However, it must be noted that Photoscan Pro offers comparatively little options for user control. In the context of the methods presented here, this limitation  does  not  matter,  as  we  expect readers to be able to produce photographs that do not result in the necessity to fine-tune and tweak the program input. If, however, older photographs  without EXIF data  or other problematic data sets are to be used, other programs may be better suited to the task.

Advances     in    technology   and  the software development in photogrammetry are still very rapid  and  have  resulted  in  vastly  improved accuracy and user-friendliness in recent years. It is likely that some aspects of our guide will become obsolete through future technological and  software  improvements.  For example, Falkingham (pers. communication April 2014) found  that between  his submission  of Falkingham(2012) and  its  publication, “photogrammetry methods  saw a  speed increase of ~10-100x models that had to be done over a couple of days can now be done in an hour or so”. However, we believe that the majority of our suggestions for photographic procedures will always help in creating a clean dataset of paleontological objects.

Photogrammetry  strongly depends  on the source data material and is a good example for the  GIGO” principle  (Garbage  In,  Garbage Out). On one hand, if the photos are not of good  quality,  they  limit the  quality  of  the resulting 3D model, no matter how good the software  is.  On  the  other  hand,  with good source photos, one can always reprocess the images with an updated version or a different computer program at a later time. Essentially, photogrammetry is about taking good photos with some differences to normal photography work. However, this does not mean that useful 3D models cannot be obtained from low-quality photographs,  or  images  taken  without  the intent  to  use  them  for photogrammetry.  In fact, as long as several images from different viewpoint  exist,  models  can  be created.  An exciting example of photogrammetric modeling in       paleontology    from             old           images is the reconstruction  of  the  famous  Paluxy  River sauropod  and  theropod  dinosaur trackways from 12 photographs taken  in  1940 (Falkingham et al., 2014).

The actual process of creating a 3D model from photographs starts with the program using, e.g., the SIFT algorithm (Lowe, 1999) to find specific points on each image. In the next step, points from all images are compared to each other, with the aim to find corresponding image parts. This so-called correspondence problem (https://en.wikipedia.org/wiki/Correspondence_problem) is often solved using the RANSAC algorithm (Fischler and  Bolles, 1980), which simultaneously also calculates the fundamental matrix (https://en.wikipedia.org/wiki/Fundamental_matrix_%28computer_vision%29) that defines mathematically how the image parts are related. SIFT and RANSAC are also used for panorama stitching (Brown and Lowe, 2007), for example in the free software hugin (http://hugin.sourceforge.net). The output of this step is called a sparse point cloud, consisting of the 3D coordinates of those points that were used to align the images. Because alignment does not require the use of all available points in high resolution images, typically only a small percentage of points are used to keep the calculation times tolerable.

In the next step, all points are used to create a high density point cloud, termed dense cloud, that is a very close representation of the real physical objects in the photographs. This cloud can then be turned into a polygon mesh or polymesh which consists of triangles that connect the points. Color information can be included by directly taking the color of the various points from the photographs or by separately calculating a texture for the mesh. Figure 1 shows a diagram of the work process with a recommended workflow.

 

PHOTOGRAMMETRY IN PALEONTOLOGY

 

In  paleontology,  the  most  common  use  of photogrammetry  is the  three-dimensional digitization of  specimens.  Previously, the complexity of the required calculations allowed only the measuring of individual points (e.g. Gunga et al., 2008), not entire point clouds. The massive increase in computing power and advances     in  programming during  the  last decade,  however,  have  made  such  limited forms  of  data collection  outdated.  Research areas  of  interest  are  mainly  biomechanics, including  locomotion,  ranges  of motion  and body mass, or the reconstruction soft tissue volumes, as well as morphometric studies and ichnology. Removing the necessity to handle heavy,  fragile  and  rare  fossil  specimens  for research,  digital models allow     computer simulations which widely  expand  possible research topics. Immobile or very heavy specimens often require significant effort and expenses  to  study, whereas  the  ease  with which   a    high-detail  3D model  can be transferred, viewed and measured makes them accessible to practically everybody. Also, scans do  not  decay,  whereas  real  fossils  can  be damaged by  exposure  to  the  elements  or during research. A major advantage of using digital files is the ability to simply save any configuration of many objects at any time,  and to compare various configurations directly to each other, all without any risk to the actual physical specimens.

The documentation of excavation progress and the exact geospatial relationship of fossils and/or sites can be accurately recorded. Maps of fossils in the quarry can then be produced at high accuracy with ease, and if several series of photographs are taken at different times, the maps can be produced even as 3D maps that show the vertical arrangement of the fossils including exact distances.

Aside from musculoskeletal modeling (e.g. Hutchinson et al., 2005), paleoichnology, the study of fossil tracks, is the research area that can probably profit the most from the availability of a rapid, versatile, field-suited and highly accurate 3D digitizing method, i.e. photogrammetry. The application of photogrammetry to fossil tracks had indeed a pioneering role in paleontology (e.g. Matthews

& Breithaupt, 2001; Matthews, 2008). Many tracksites are physically inaccessible during certain times of the day or year (e.g., covered by water in the intertidal zone, in rivers and lakes). Tracks which are otherwise affected or even destroyed by weathering and erosion because they cannot be excavated due to size, location or other reasons can be documented and the data preserved. In fact, Adams et al. (2010) suggested that digitypes, digital holotypes, should be introduced for that very reason. Digital track models are far easier to compare with existing data and represent an objective dataset, virtually not influenced by interpretation. For example, the often-used line drawings of track fossils can differ significantly for the same track if done by different researchers (Bock, 1952: plate 44).

Figure 1: Workflow diagram. Red steps are minimum work path for production of dense point cloud and polymesh final output. Orange steps are highly recommended for any kind of specimen. Blue steps are additionals for multi-chunk use or one-chunk method with several photo sets. Green steps are additional in-program work to salvage data sets that do not work out well. Bold blue arrows mark regular minimum-effort workflow.

 

Workflow

 

The workflow of photogrammetry in paleontology consists of two or three main parts:

1.Photography

2. Image editing (may not be necessary)

3.     Model calculation

Model calculation itself consists in principle of the following steps:

1.     In-program  data  setup  (includes  scaling

preparations)

2.     Alignment (includes scaling) including spare point cloud generation

3.     Dense  point  cloud  generation  (may  be skipped if a polygon mesh is to be calculated. However, meshes from dense clouds are usually superior)

4.     Polygon mesh generation (if desired)

5.     Texture generation (if desired)

6.     Data export

We provide detailed instructions for the photography  part  below.  The  image  editing required is normally limited to masking and requires no further explanations. We also detail the  model  calculation  methods  we use  as suggestions for a general workflow. However, programs change so rapidly that it is highly recommended to use our methods as a basis for own experimentation.

We attempt to give all information relevant to one step of the work process in the respective section, which    means    that    information pertaining to one topic may be spread out across various parts of the text or repeated. However, we feel that photogrammetry novices are better served by this organization, as it facilitates the use of this work as a take-along handbook for work during travels.

 

 Equipment list

 

Essentially, all you need to start photogrammetry is a digital camera and a suitable computer program. Additionally, a scale is important because the scientific value of 3D-data is much higher if the dimensions of the object are known.

 

Bare minimum

      Digital camera (“point & shoot” consumer camera or cell phone)

      Object  with  known  dimensions  that  can serve as a scale

 

Travel-suited

      DSLR with several lenses, ideally a very fast zoom  lens  from  wide-angle  to  moderate telephoto (e.g., 28-70mm), and a very fast prime lense (e.g., 50mm 1:1.8). If using a DSLR with cropped (APS-C) sensor, use a lens between 28mm and 100mm. Select the lens  based   on high sharpness and low distortion at about f/8 - f/13. Because of the often difficult positioning  of   the camera necessitated by  immobile  specimens  in exhibits or the field, it can be preferable to buy a camera with a lower-resolution sensor but with a rotatable live-view touch screen that allows shot composition and point-of- focus selection by touch (e.g., Canon 650D, Nikon D5100). For very small specimens a macro lens or close-up filter is required.

Polarizing filter, which allows reducing reflections on shiny surfaces - mandatory for photography in exhibitions with glass covers (Figure 2).

 

Figure 2: Mounted partial skeleton of the thyreophoran dinosaur Edmontonia rugosidens AMNH 5665 in the AMNH exhibit. The specimen is surrounded by walls and a glass cage. A. Photograph taken without polarizing filter, camera hand-held close to the glass but not flush with it. B. As in A., but with a polarizing filter and a tripod. Note how the use of a tripod allowed taking a better exposed and sharper image, and how the polarizing filter weakened reflections to the point where they can only be noticed on uniform backgrounds.

 

     Tripod (Figure 2), size depending on space and weight restrictions and specimen size. For  large  or exhibition  specimens  and  for work on laboratory tables a large tripod is useful. Choose a tripod that allows setting the camera  up  nearly  on  ground  level  (e.g. Manfrotto 055XPROB with 804RC2 head) or bring a second mini-tripod (e.g. Cullmann Magnesit Copter Tripod with cb 2.7 Ballhead) so that large immobile specimens can be photographed at shallow angles.

      White and black cardboard or cloth (locally procured, t-shirts also work) or a green- screen, and pages of printed text, e.g. a newspaper.

      Folding yardstick with meter scale and 10- or 20-centimeter scale bar.

      Separate hand-hel flas (no strictly necessary).

      LED-set for lighting specimens from shallow angle (not strictly necessary); do not forget to     bring     multiple-outlet      power     strip     or several adaptors if traveling abroad.

      mid-sized to large mirror (acquired locally, for photographing immobile specimens that the camera cannot be placed behind).

On the road, during museum visits or excavatio work th mai factor are durability, easy setup, space and weight restrictions. Special care during the photography process, in order to achieve high quality images,  is  recommended  for  non-repeatable situations and objects (e.g., during excavations). It is always possible to amend the suggested equipment with gear mentioned in the lab-setup section (i.e., a remote shutter release can be very useful, but is not essential). Museum curators are often willing to organize materials in advance at the visitor's cost, so that they are available for the full time of a collections visit.

 

Optimal lab setup

      DSLR with several suitable lenses (see above). For the controlled conditions of a laboratory, the maximum resolution sensor camera should be acquired for optimal model resolution. An adjustable camera stand may be necessary for macro lenses that do not have a focus ring.

      Remote shutter release to avoid vibrations.

      Polarizing filter. Allows reducing reflections on specimens that have shiny surfaces.

      Tripod (large and solid) with ballhead foreasy adjustment.

      Reflectors.

      Flash array (optional).

      Light/LED setup with several rows of focused LEDs for even lighting of specimens (can be combined with or used instead of reflectors).

      Neutral white/black background (can be made from white/black cardboard, professional drapes preferred) or (ideally) green-screen an sheets   with   printed patterns with human-recognizable features (e.g., black-on-white text pages or colorful advertising supplements).

      Turntable (potentially motorized) with featureless cover and degree markings on the vertical outer surface. Note that continuous turning is not desirable, so that a motorized turntable needs to be computer- controlled and stoppable at predefined intervals.

      Glass/perspex  cubes  of  many  sizes  for specimen support.

      White packing foam (tiny bubbles, so no points can be found by the program) and putty for specimen support.

      Scales with centimeter and millimeter markings in various sizes, non-reflective.

 

A  permanent  lab  setup  aimed  at  rapid  and optimal quality digitizing offers the advantage that the photography kit can be optimized for best photograph quality and suitability for photogrammetric reconstructions, without any concessions regarding ambient lights or background.

A key decision to make is whether to use a flash  setup  or  strong  lights.  Stronger  lights allow lower ISO values and shorter exposure times. However, if a specimen is shiny (e.g., has  been  covered  with lacquer  or  has  a crystalline surface) the photographs will show variations in color or even light spikes from tiny reflections. In this case, directional light or the use of flashes can ameliorate or worsen the problem, thus it is necessary to experiment. 

The ideal background for easy model creation is a green-screen, placed sufficiently far back that reflected light from it does not fall on the specimen. Evenly lit it can be selected with a magi wan too i th photogrammetry software or  a graphics editing program and masked/deleted. White or black cloth or even cardboard can also be used, but it is harder to light it so evenly that magic wand tools are effective. However, if the background is featureless, the photogrammetry software will not be able to pick up points anyways. The color of the background, black or white, must be chosen with the lighting of the images in mind. A white background for a very dark (coal) fossil will either lead to underexposure of the fossil, or white glare around the fossil's edges. In such cases, a black background is preferable.

As discussed below, for specimens on which few features are detected by the photogrammetry program, a highly structured background can be used to improve alignment at the cost of time spent for masking or cropping later in the modeling process.

A motorized turntable that rotates continuously and slowly (then, very  short exposure times are required), or rotates a fixed small angle and then remains stationary for a short time, obviates the need of someone stepping and reaching into the picture between shots.

If very rapid turnaround times are desired it is possible to use several cameras simultaneously. However, achieving even lighting is then difficult. Currently, efforts are underway to produce an automated 3D digitizing line that uses many cameras and mobile  lights  for  photogrammetry  and  other techniques (Santos, 2013). However, this and similar set-ups are (yet) not sufficiently mobile, not variable in the optics used and not affordable for individual scientists' needs.

 

Legal issues

 

There are to our knowledge practically no established practices and rules governing the use of data derived from photogrammetric modeling. It is therefore always advisable to contact the responsible curators in advance and agree to a bilateral agreement on file ownership, copyright and file use permissions. This approach is especially  important for photography conditions that are not covered by free-use rules, e.g. in collection rooms of museums.

Additionally, a very important issue is the automatic copyright transfer included in the Terms of Use of some (often cloud-based) free photogrammetry software programs. Use of such programs should be avoided at all cost, especially when digitizing specimens owned by third parties. Such a transfer of ownership is the reason why we exclude, for example, the no-fee software 123D Catch.

 

PHOTOGRAPHY FOR PHOTOGRAMMETRY      

 

The photography phase has an enormous impact on the final output file. It is often easier to reshoot a new series of source photographs instead of trying to save and improve a set of photos which does not work right away. An important trick to master is the ability to visualize, while taking the photographs, the computer program aligning the photographs, so that one can plan ahead and find angles that potentially were missed. We suggest developing and following a certain routine in taking the photos. If possible, all images should be transferred to the computer right away and a low-resolution alignment calculated while the next specimen(s) is/are being photographed. This allows re-shooting certain images, or additional photographs to fill gaps, while the specimen is still accessible and while markers that were possibly placed on it have not yet been removed. In difficult scenarios, it may be necessary to reshoot multiple times for a suitable model. The quality of pictures is the key factor for the quality of the final model.

 

 

Image formats

 

Most digital cameras can save photographs in both an uncompressed, lossless format (usually a RAW/TIFF version) and a compressed format (usually JPEG). Agisoft Photoscan and some other photogrammetry programs can handle lossless formats. RAW files contain the maximum amount of information and original data generated by the sensor and have been considered the only scientifically justifiable file format (Verhoeven, 2010). However, we have found that usually, the artifacts caused by mild and high-quality compression algorithms, such as those used for the maximum-size JPEGs in high-quality DSLR cameras, do not significantly influence the quality of the alignment or final model. Similarly, the many image manipulation options available when using RAW format are not needed for normal photogrammetric modeling. On the other hand, RAW shooting takes more hard disk space, more time, as the much larger files require longer copying times, and the photographs may need to be converted into TIFF images before processing. We therefore recommend shooting maximum- quality JPEGs in normal circumstances.

 

General rules: ideal photograph quality

 

Photogrammetry software essentially needs clean, sharp, evenly lit images; with every point of the surface of the specimen visible from at least three angles within the entire photo set. The task of aligning the camera positions requires parallax (different positions and angles) between the images, which for the creation of panoramic images would be fatal. Leave out or mask off undesired parts of the image, either with a green screen during photography or later in an image editor or the photogrammetry software. Specific features which are the highlights of artistic photography (e.g., dramatic lighting, wide-angle distortion, selective depth of field) are completely counter-productive for photogrammetry.

Because the quality of the pictures is the key factor for the quality of the final model, following these instructions is suggested:

      Choose the highest-quality settings your camera allows. In rare cases, usually on cheap point-&-shoot cameras, a resolution lower than the maximum the camera offers should be chosen, if it offers higher image quality. In all other cases the maximum resolution  with  minimal  or  without  image compression should be used.

   Fill the frame with the specimen, or move even closer so that each photograph shows only part of the specimen. The closer the camera is to the specimen, the higher will the final resolution of the model be. However, if too little of the specimen is visible in each photograph, a large number of photographs is required, so that the calculation time and file size become impracticable and alignment errors between photographs may degrade the overall quality of the model.

   When photographing immobile specimens, ignore obstacles hiding part of the specimen from view. As opposed to regular photography, it does not matter if support struts or other objects are in front of the specimen. It is more important to obtain a good coverage of the entire surface from many angles than avoiding undesired objects. Figure 3 shows an example of a photograph that contains useful photogrammetry data, but would normally be considered to be of bad quality.

   Every detail that is not in the pictures will not be in the final mesh. Move underneath and above the specimen to take photos. Make sure you have the coverage you need to get all the details you want, because reshooting in the exact same conditions is difficult.

   Take in-focus photographs. Focus on part of the specimen that is in the middle of the distance range that all the specimens parts have to the camera. Otherwise, you risk the close or far end being out of focus. Use a mid- to high-range f-stop for a large depth of field and maximum sharpness. This requires exposure times typically too long for hand-held shooting, necessitating the use of a tripod.

  Use the middle of the zoom range of the lens, up to the higher end (which is a tradeoff with exposure time), to minimize distortion.

● Use base or low ISO for minimal fuzziness/noise. ISO value versus exposure time and f-stop is a three-way trade-off, in which the exact best choice depends on the exact circumstances and specimens.

● Avoid vibrations during exposure (passing tram, walking around near the tripod during long exposures, pressing the trigger roughly, etc.) – delay the trigger with the automatic release, use touch-screen release or remote shutter release.

● Aim for even lighting of the specimen to avoid shadows, ideally identical in all photographs. This is especially important for recesses and parts underneath the specimen.

● Consciously pick a place to start the image set, so that you can form a mental picture of the camera positions and judge if further images are required. Once you have taken the first few images, zoom in on the camera LCD and check the focus and level of noise.

● Avoid taking photos against the light because detrimental light reflections on the specimen are possible. Do not photograph into strong lights that can cause lens flare. Sometimes, when photographing specimens in exhibitions, it is possible to block individual lights or reflections by simply holding a finger or the entire hand in front of the lens (Figure 3 C). Alternatively, get another person to help you by holding up a piece of cardboard.

 

General rules: photoset suitability

 

● Move the camera in relation to the specimen (or vice versa) to create parallax; do not take panorama photos (many photographs from one camera position). The latter can be acceptable but even small motions of the camera can drastically improve the quality of photograph alignment.

● Each point on the specimen must be well visible and in focus on at least two images. Any point not visible directly will not be included in the final model (line-of-sight rule). Take care to point the camera into recesses.

● Take photographs with 40-60% overlap as rule of thumb. Avoid near-identical photographs, including photographs that differ only in long-axis rotation (i.e. portrait versus panorama format).

● Overview photographs can be supplemented by close-ups, but much overlap is required.

● Take more photographs than necessary, because unsuitable photographs can later beexcluded from model creation and replaced by others, and gaps can be closed.
 

Figure 3: Mounted skeleton of the ceratopsian dinosaur Protoceratops andrewsi AMNH 6417 in the AMNH exhibit. A. Photograph taken through the glass cage that contains the specimen, and bears inscriptions on one side. For a normal photograph, the presence of the letters would be considered inacceptable, but as can be seen in B., where the features found by Agisoft Photoscan Pro (search target in this case was 10.000 points) are displayed as dots, the image is well suited for photogrammetry. Grey dots indicate specific features found for the alignment process, blue dots those used for the alignment. The specimen is in focus and thus delivers good points, whereas the type on the glass is out of focus and delivers no points. C. Photograph showing masking of strong reflections on the glass by blocking them with the photographer’s hand.

 

Lighting

● ‘Even lighting’ is the key criterion: less light (especially in combination with a tripod) but no spotlights can be better than more but uneven light; fluorescent ceiling lights and white collection cabinets surrounding the specimen (acting as reflectors) work surprisingly well!

● Diffuse lights if necessary (e.g., diffuse the sun with translucent plastic sheets).

● Cardboard (easily bought everywhere) or white t-shirts for background and as reflectors to lighten up shadows on the opposite side and underneath. Aluminum foil can also work, but beware of possible light reflexes. Use darker materials for dark specimens to avoid too much contrast in the images.

● Potentially use LED lights (easily transportable, too) from low angle to light up shadows.

● If light conditions are poor and a tripod is not available or feasible (e.g., wall-mounted specimens can require work on a ladder), the exposure time may be too long for sharp photographs. Consider using higher ISO values. If all else fails, photography with the camera’s internal flash can produce in-focus pictures. However, flash-based photography will always produce strong shadows and overexposed highlights. It is essential to avoid shadows and white-outs on a sufficient number of pictures of each part of the specimen to allow photogrammetric calculation. Alternation of the shooting angles is very important. Shadowed areas must later be removed from the calculation in the photogrammetry software, which may require masking.

Pitfalls


● Poor Focus – This is the most basic factor to adjust correctly. Focused parts should ideally include all important parts of the image.

● Wrong f-stop causing poor Depth of Field – Shooting at low f-stop as often happens with automatic camera settings can cause a major part of your object to be out of focus. For the best focus, a stop down between f/8 and f/16 is recommended; above these f- stop values, diffraction sets in and reduces the overall sharpness, rather than increasing it. Check on internet resources (i.e., www.DPReview.com) to find the best f-stop for certain lenses.

● Too few images usually result in bad alignment, and thus gaps and errors in the model.

● Too little overlap (as above).

● Too many images increase calculation time for no gain. Worse, too many images can lead to artifacts by grouped alignment creating two points where there should be only one. However, it is advisable to take many photos and use only a selection for photogrammetry, especially if the specimen is not at one’s disposal.

● Too much overlap, duplicate photographs (as above).

● Highly compressed images contain artifacts that create false features which will be tracked by the software. RAW images or high-quality JPEG are recommended.

● Unintended motion of the specimen can lead to alignment problems if background features are included in the alignment.

● Movement/Motion blur – Constantly spinning a specimen on a turntable while shooting in burst mode will likely fail because the shutter speed of most cameras is too slow. Slow shutter introduces blur in the direction of the moving object, lowering your effective resolution. Heavy specimens should be positioned absolutely stable – this is important if placed e.g. in a sandbed.

● Lens distortion – The level of distortion in most normal lenses is no problem for modern photogrammetric software. However, it is recommended to avoid fish eye lenses with high barrel distortion.

● Changes in lighting (doors opening, the sun moving, clouds, partial shade under trees, etc.) can lead to alignment problems and/or gaps in the model. If a texture is calculated it can contain artifacts. Moving shadows can be tracked as false movements and also affect textures. Static light gives more consistent results; if feasible, it is recommended to wait for an overcast sky for outdoor photography.

● Strong shadows – Dark shadows often do not work well for point detection because they suffer from excessive noise. Furthermore, the resulting textures are likely far from realistic colors.

● Taking pictures into the direction of a light source can cause reflections on smooth surfaces or lens flare, so that no or erroneous points will be found. At worst, “phantom objects” can be caused by lens flare.

● Under-or over exposure – A significantly under- or overexposed photograph loses usable detail. Inconsistent exposure causes light and dark patches on the model.

● Unequal exposure within image set – If the exposure differs significantly between photographs it may be impossible for the program to find corresponding features. Even if images align well, the following steps may partly fail, so that the final surface has artifacts and unrealistic texture colors.

● Transparent and shiny surfaces – Gathering data from specimens behind display cases or shiny surfaces (i.g., caused by lacquer, liquids, or enamel) is tricky. Current software cannot distinguish between white pixels on the object and the reflection of light, as it assumes all surfaces are non- reflective and opaque. Reflections of surroundings in showcases do not move consistently with the surface of the object and the angle of the photo; they may cause “ghost images” or holes in the model. Furthermore, glass surfaces may distort the objects.

● Repetitive features – Tracking certain repetitive pattern, such as architectural elements in the background, ripple marks on track surfaces or honeycombs, can cause the software to wrongly detect different instances as corresponding features. Such a “jump” can result in strange conglomerate models with parts of the model presented repeatedly at different positions.

● Featureless textures – Plain surface textures, such as at blank walls or even dusty surfaces, make feature identification difficult; the software may fail in rebuilding depth of the model, or even at building a model at all.

● Very thin specimens – If the resolution of your photos of thin specimens (such as vertebrate ribs or mollusk shells) is not high enough, the software-generated point cloud does not contain enough points to accurately reproduce the shape of your object. The points may not be placed in the exact same place along the length of the thin object.


PHOTOGRAPHING SPECIMENS

Because photogrammetry requires parallax either the camera or the specimen must move between shots, or both. Depending on the size of the specimen and its mobility, as well as the space available, one of two main methods is usually adopted: the turntable method, or the walk-around method. For the former the camera is usually stationary, ideally on a tripod. For the latter, the specimen usually does not move and the camera is moved around it, either hand-held or on a tripod. Both methods have their specific advantages and demands. Only occasionally is it advantageous to move both the specimen and the camera, for example when the turntable method is used with a scale bar that cannot be placed on the turntable with the specimen.

Aside from choosing between the methods the image acquisition process is influenced also by considerations regarding the potential risk of failure of the in-program model creation. Photography aimed at optimizing the work process for the one-chunk model creation method described below results in images that, if the one-chunk method fails, do not make full use of the superior image alignment capabilities of the multi-chunk method. We therefore recommend reading the sections Alignment methods for more than one set of images and Background – blank or structured? beforehand.
 

The turntable method: basic principles

The turntable method has the camera on a tripod, and a series of photographs is taken of the specimen on a turntable that is rotated across a small angle between shots. Figure 4 shows the typical process for photographing a specimen on a turntable. The photograph series thus forms a perfect circle of camera positions around the specimen, with the camera always aimed at the central vertical axis of the turntable. In order to cover the surface of a specimen with a complex geometry it is usually necessary to vary the height of the camera position in relation to the specimen by repeating the process shown in Figure 4 with the camera at a different height, so that several concentric circles of photographs are taken. Undercuts may require additional photographs, with the camera pointed off the main vertical axis. These images are usually created not by moving the camera, but by manually shifting the specimen on the turntable and optimizing the position for each photograph. The underside of the specimen has to be photographed by flipping it over, and sometimes more than two positions are required to capture the full surface geometry. The biggest advantages of the turntable method are the ability to control the lighting, photograph speedily, with fixed camera settings, and the control one has over the background. The non-turning background must be as devoid of features as possible or masked, so that no points are detected on it. Masking is additional work and should thus be avoided, but it can allow combining photograph series from different specimen positions into one photogrammetric reconstructions without any need to adjust partial models to each other (see one-chunk method below), delivering the highest quality models for the least amount of fiddling and editing.

The biggest drawback of the turntable method is, aside from the requirement of a featureless background, that it requires the specimens to be mobile, and sturdy enough to be handled. Additionally, even the sturdiest turntables have weight limits, so that very large specimens like sauropod dinosaur sacra or elephant skulls are too heavy.
 

 

Figure 4: Video showing the basic photography process for specimens that can be placed on a turntable.

 

Turntable method for small mobile specimens - using a turntable

The basic process is shown in Figure 4. Use a turntable for even lighting and speedy photography. Heavy-duty turntables for TV screens cost less than US$30. You may also use a Lazy Susan, even an office chair that spins might be sufficient for some objects. Cover the turntable with a featureless cover (white/black cardboard) and mark its vertical outer side at 5° intervals as a visual guide for the rotation between photographs. If possible, lift the specimen away from the turntable by placing it on glass/perspex supports so no points are detected on the turntable, as its cover will be out of focus. Use white packing foam bits to stabilize objects that do not rest well in desired position.
Take photos at a shallow angle (0-15° from horizontal) at ~5° to 10° intervals by turning the turntable. Then, lift the camera higher on the tripod (30-60°); repeat. Then, lift the camera high above the specimen and take a few photographs (3 or 4) from ~70-80° up. The more complex the specimen's geometry, the more photographs are necessary.

Turn the specimen over (180° if possible, but smaller angles are sufficient if enough of the lateral surfaces is captured in each set to combine the two, i.e. if there is good overlap). Repeat photo procedure as above.

For tiny specimens the use of a high-quality macro lens or a microscope with digital camera is recommended. Because such set-ups often do not allow a large depth of field, it may be necessary to shoot focus stacks of photographs, i.e. take the same photograph repeatedly but with a slightly different focus distance, and use dedicated computer software to compute these stacks into individual images (note: the final images should preferably have EXIF data, which you can add manually).

 

Turntable method for small mobile specimens - without turntable

Use a sandbox or a very sturdy piece of cardboard as a makeshift turntable (Figure 5). If this is not possible, use a piece of white cardboard to blank the table surface (if you have any), set the specimen on the table with a scale adjacent, take several photographs from different positions, then begin the process of rotating the specimen to complete the picture set as above. When processing, set in-program markers for scale on the photos but mask the scale bar along with rest of the background (if masking the latter is necessary).
 

Figure 5: Digitizing a small specimen (caudal vertebra of Citipati in the AMNH collection) without a turntable. A. Setup with camera on tripod, light source sub-parallel to camera view and a cardboard box with sand as a makeshift turntable. Printed paper serves as feature-rich background in case the specimen offers too few features. A caliper serves as scale. In this case, the specimen offered ample points; see Figures 10 and 11. No markers were put on the specimen due to its fragility; easy model creation could only be realized using the one-chunk method. B and C. Images from the two series, showing the two positions the specimen was placed in. Note that a matchbook was used to prop up the specimen in B. All parts of the surface were still adequately documented in several images, so that the blocking of parts of the specimen from view in some images had no negative consequences for model creation.

 

Turntable method: scale bars

The normal procedure for placing a scale bar is to put it next to the specimen, on the turntable if one is used, and leave it there for the duration of the photographing session. If it is necessary to move the specimen between photographs that belong to one set, the scale bar should be removed at this time, or it must later be masked along with the rest of the background in all images taken after this time. A scale bar does not have to be modeled in 3D, but must at least be present in two photographs. If it can be exactly marked in more photographs, the overall accuracy can be improved. Several scale bars are better than one, especially for very large specimens of which each photograph will show only a part, as they reduce the measurement error. They should be placed on opposite sides of the specimen, and can then help reduce distortion. The larger the scale bar, the better, because the error caused by photograph resolution and/or marker placement in the software becomes relatively smaller. Ideally, scale bars get rotated with the object.

Even for the one-chunk method, any scale needs not be in same relative position to the object after flipping over. Theoretically, it is sufficient that a scale is present and fully visible in two photographs of one set only, as long as it does not move relative to the specimen between them, provided these two photographs align with the rest of the set. However, more photographs should be taken to be on the safe side. Scales should also always be included in the setup for the second (and any further) set of photographs, in case the multi-chunk method must be used if the one- chunk method fails (see below).

 

Turntable method: using physical markers on specimens

If at least three markers are placed on the specimen (see Figure 4) so that each is visible in two photos, the markers can be used to align models calculated from each of the two (or more) separate sets of photographs required to cover the whole surface (required for the multi- chunk method). It is generally a good idea to place markers even if the use of the one-chunk method is intended, as they greatly ease the task of alignment of parts if the one-chunk method fails.

Because all markers placed on the specimen will be visible in the final model, markers should be as small as possible. Tiny pieces of white self- adhesive labels marked with the letter X (use the crossing point of the two lines for the in- program marker) and a number (for easier identification on the photos) work well, if care is taken that the markers do not fall off when the specimen is flipped upside down. Make sure to use only materials from which no chemicals can leach into the specimen, and which do not leave any residue when removed.

Markers must be well visible in all positions that the specimen will be placed in during photography. It is advisable to turn the specimen over after placing them, to check if they really are visible in the alternate positions.

It is possible to produce a marker-free model with added effort, by taking additional photographs with the markers removed, and masking the markers in all photographs. Care must be taken to take a sufficient number of images, so that each spot that was covered by a marker is well represented in sufficient detail in at least two additional photographs. The marker- free images then provide the surface information for those parts of the surface that is under the marker. However, because this approach involves talking near-duplicate photographs, the calculation time for the model will rise significantly.
 

The walk-around method: basic principles

The walk-around method inverts the roles of camera and specimen, in that the latter is stationary, and the former moves. Thus, the obvious advantage of the walk-around method is that it allows capturing completely immobile specimens, as well as those that cannot be placed on a turntable or manually rotated. The disadvantages are lack of or limited ability to control lighting, often a lack of line-of-sight onto parts of the specimen (especially in exhibitions), the need to adjust camera settings for each photograph, and an increased need to construct a mental image of the sum of camera positions to judge if the specimen surface has been sufficiently captured.

Walk-around method: very large or barely mobile specimens

If a specimen is too large, heavy or fragile to be placed on a turntable, but can be otherwise moved (e.g. on a trolley), it is usually advisable to position it so that there is room to move the tripod all around it (see Figure 6 for a series of photographs taken with this method). Also, it is usually worthwhile to spend some time searching for a position with good light.

Photographs should be taken so that the relative positions of camera and specimen are roughly the same as in the protocol for using a turntable above. The sole difference is that the role of the stationary part and the mobile part are exchanged. As a consequence, the lighting, exposure time and depth of field must be checked for each camera position separately. Care must be taken to avoid reflections on the specimen caused by light sources such as windows behind it, especially if it is covered by lacquer. There is no requirement for the camera positions to be as regularly spaced as the use of a turntable will usually make them. Figure 7 shows the sparse point cloud, camera positions and dense point cloud generated from the photo set shown in Figure 6. Note how several different elevations of the camera were used to ensure sufficient capture of the surface.

If additional lights are placed they need to stay in place during the entire photography, as it is nearly impossible to reconstruct their exact positions and directions during later parts of the photography process. Alternatively, a set-up producing even light can be moved with the camera, so that the entire view in each photograph is evenly lit. Photography should be conducted rapidly to avoid changing light conditions. It can be preferable to ignore people walking through, as they can be masked. In such an event it is best to take more photographs.

Specimens too large to be moved at all must be treated like in situ specimens outdoors, except for the restrictions imposed by sunlight.

 

Walk-around method: in situ and exhibition specimens

Specimens that are immobile can only be photographed in situ by walking around them as far as local conditions allow. This places restrictions on the lighting conditions, and may mean that parts of a specimen cannot be digitized because the camera cannot be brought into positions required to photograph them. Figure 8 shows an example of a mounted dinosaur skeleton in an exhibit hall populated by obstacles in the form of other exhibit specimens. Especially problematic are specimens outdoors, where one is dependent on weather conditions, and often has no access to electrical power available. Try to avoid strong sunlight as it causes high contrasts with dark shadows (also true in exhibition spaces with natural light, such as the AMNH dinosaur halls); use reflectors/flash to brighten them up. A light but uniform cloud cover is preferable. Avoid shooting during times of day where the sun shines at a shallow angle onto the surface of interest (i.e., for sub- horizontal surfaces prefer shooting during mid-day, for strongly inclined surfaces choose times accordingly).

When a specimen is subject to changing light conditions, work rapidly to minimize the differences and ignore people walking through view (they can be masked out; take more photographs to ensure that all parts of the specimen are sufficiently captured).

 

Figure 6: Image set of specimen Khaan mckennai IGM 100/1127 (currently stored at AMNH) on a trolley, set up in the middle of the room so that there was space to move a tripod around the specimen.
 

Figure 7: Model of specimen Khaan mckennai IGM 100/1127 (currently stored at AMNH) created from image set shown in Figure 6. A. Sparse point cloud and camera positions. B. Dense point cloud in same oblique view as A. C. Dense cloud, medium density. D. Closer view of the skull. Note the regular arrangement of cameras in A caused by the identical tripod height for several sets of photographs, and note the height lines in the dense point cloud.

 

Mounted skeletons, long trackways and similar very large and complex objects can be calculated in chunks and merged later (see below). If this approach is chosen, the photography should be adjusted by shooting series of pictures of parts, with markers for the later alignment placed beforehand. Each part should contain its own scale bar, as large as possible. It is for example possible to measure railings or other architecturally defined distances. You may also use a complete measuring tape or, otherwise, place a yardstick (ideally at least 2 m) next to each section of the specimen in a stable position while you photograph it. To facilitate correct photogrammetric calculation of repetitive pattern such as ripple marks on trackways, place uniquely colored/shaped objects (e.g., clothes, tools, etc.) around the specimen. Typically, specimens with complex shapes in exhibition settings suffer from uneven lighting caused by top-down light or spotlights, causing strong shadows on the undersides of individual elements. Ground-mounted spotlights sometimes ameliorate this effect, but are normally insufficient to allow straightforward digitizing. If possible, ask for access outside normal visiting hours, and for the exhibition lights to be turned off. Many museums have “cleaning light” that more evenly lights the specimen. Use reflectors and additional light sources, and use a flash if nothing else helps. A second, hand-held and automatically triggered flash can also be helpful. If nothing else works, block spotlights or reflections with your hand (Figure 3C).

The actual process of photography consists of moving around the specimen and taking photographs. As simple as this sounds, there are a number of pitfalls. First of all, it is important to retain enough overlap between images, especially if the view of the specimen is edge-on. Here, it is even more important than in the turntable method to shoot more images than one believes to be necessary. Care must be taken to not shoot panorama series (in which the camera is pointed in different directions but remains stationary), as these image sets have little to no parallax between individual images. For huge specimens, especially those too large to fit entirely in each photograph, e.g. because it is impossible to gain enough distance or because a higher resolution of the model is required, it can help to place visual markers of used positions on the ground, e.g. small pieces of paper, so that one can keep an overview of what has already been photographed.
 

Figure 8: Four views (A. anterior, B. lateral, C. top, D. oblique) of sparse point cloud and camera positions of an image set of the torso of the Museum für Naturkunde Berlin Diplodocus mount in the Dinosaur Hall. Note how the neighboring Giraffatitan and Dicraeosaurus skeletons have also been captured, and how they block line-of-sight to the right side of the Diplodocus mount (gaps in camera positions at the top of C). Also, the other specimens and the pedestal they rest on require variations in the distance between camera and specimen. Note how the tails and necks of the mounts are badly represented in the sparse point cloud, because they are captured only on the fringes of some images. In this example, erroneous feature matching leads to sub-optimal alignment, causing increasingly visible distortion with increasing distance from what roughly is the focal point of the image set (the torso of Diplodocus), especially notable on the right lateral sides of Dicraeosaurus and Giraffatitan in the top view C.

Exposure must be checked and adapted for each photograph, as differences can easily cause color differences which make feature matching difficult for the program, degrading or failing alignment. It may even be necessary to adjust the white balance between shots. RAW format photographs, in which settings can be developed later, can be useful under these circumstances. Especially outdoors, but also in indoor places with low-mounted light sources, it is important to avoid casting shadows on the specimen.

Also, it is often difficult to capture the top surfaces of huge specimens sufficiently from ground level. In this case, arrange for a ladder and a second person to steady it. Long telescopic masts, the use of kites or UAVs (Unmanned Aerial Vehicles; e.g. Watts et al. 2012; see http://www.mikrokopter.de) are useful while photographing large areas.


Walk-around method: specimens behind glass

Museum specimens on exhibit behind glass present an especially difficult case for photogrammetry, because typically the glass is not perfectly clean and reflects lights and well- lit structures (Figure 2A). These reflections interfere with model creation. Often, a polarizing filter can remove most of the reflections (Figure 2B). Placing the camera very close to (ideally almost touching) the glass and careful observation of the focus point usually makes the dust and grime layer on the glass invisible on the photos (Figure 2B). At worst, a separate flash held at an acute angle (combined with a camera placement very close to the glass) can be used to reduce the relative strength of reflections, by lighting the specimen so much that the reflections are outshined. One also may use optically dense materials like a hand or a piece of dark cardboard to block reflections (Figure 3C).

 

Walk-around method: scale bars

The normal procedure for placing a scale bar is to put it next to the specimen and leave it there as long as the photographing session takes place. If it is necessary to move the specimen between photographs that belong to one set, the scale bar should be removed at this time, or it must later be masked along with the rest of the background in all images taken after this time.

Walk-around method: using mirrors

If it is physically impossible or impracticable to place the camera behind the specimen, e.g. when a specimen is mounted close to a wall, it is sometimes possible to place a mirror behind it. Photographs can then be taken of the specimen in the mirror, and later treated like any other photograph in the model creation phase. The same approach can be used for photographs from below. Make sure to take a sufficient number of photographs with sufficient overlap to those taken without the use of a mirror so that alignment is possible. Because it is not always possible to fill the frame with only the mirror additional masking of photographs may become necessary.

It is not necessary to mirror photographs taken using a mirror in a photograph editing program before they can be used in photogrammetry. A mirror simply re-directs light, so that the calculated camera position is the hypothetical location from where the camera would see the object as it is shown in the photograph.

Background – blank or structured?

The background behind and especially under the specimen has a profound influence on the model building phase. If the program detects features on it, these points can be used in the alignment of the photographs, where they often play a helpful role because of the large parallax-induced motions they experience between photographs. The features will result in points being created in the dense point cloud, and accordingly polygons in the polygon mesh. As such points and polygons are typically not desired and must be cropped, a feature-rich background and support for the specimen typically means an increase in work time. Alternatively, the background must be masked by hand in all photographs, taking often even more time. For specimens on a turntable features on the background make masking mandatory as they otherwise impede the alignment process.

A blank and out-of-focus background means no masking or cropping is necessary and makes the one-chunk method easier to use, but may decrease the chances of a successful alignment within a set of images. Therefore, the effort in time and money necessary to mask photographs must be weighed against the effort of re-shooting a specimen for which modeling fails. If a specimen is accessible only during a special occasion, or if the cost of re- shooting is high due to travel costs, we recommend using a structured background that can help with alignment. On the other hand, easily accessible specimens should be photographed using a blank and out-of-focus background. The example shown in Figure 5 shows the use of a structured background, because the cost of another intercontinental trip far outweighs the effort of masking about 2000 photographs taken during the visit. In fact, the vertebra shown in Figure 5 aligned well using the one-chunk method, but other, smaller vertebrae from the same series required the use of background features for alignment.
A background that allows easy distinction of features by a human, such as a page of printed text (easily available anywhere), also allows the manual placement of markers to aid the alignment process, and is therefore better than a feature-rich but difficult to assess wild pattern.

Photography tips & tricks

Do not use a camcorder

Although photogrammetric models can be calculated from video frames, it is not recommended because motion blur does not permit crisp images and reduces the quality of the resulting model.

 

Easier photography with a tilt screen

A DSLR with a rotatable (touch-) screen and live view that shows the view of the lens before the shutter is pressed (e.g. Canon EOS 650D; Nikon D5100) makes it simpler to choose the best focal point, and allows speedier and easier photography in cases where the camera has to be held very high or low, or over a barrier.
 

Keeping photograph series apart
Sorting the thousands of photos from a single session is most easily achieved by using the large thumbnail folder view. In order to identify each series of photographs speedily, it is advisable to write down the first and last image numbers, or to photograph before and after the series a piece of paper with the specimen number and potentially other useful information (i.e., “back set”, “part 1”, etc.) as shown in Figure 4. This paper can be held in front of the camera when it is mounted on the tripod, so no time is lost altering the camera position. The text should be written with a dark marker thick enough to be readable in the file explorer thumbnail view.

 

When series are separated into sub-folders, these photographs should be moved with them.

IN-PROGRAM WORK

Selecting images

Review and remove bad and redundant images because too many photos may overwhelm the software. Select only high-confidence information for processing. If the background interferes with alignment, mask it out. Also, even when using the multiple-chunk method, it is often faster and easier to mask the background than to remove erroneous points from the finished model.

If the alignment is not correct for a certain image, re-set and re-run or delete it. Human brains are still smarter at filtering than the software. Check the camera position display (e.g., Figures 7, 8 - all camera positions are fine, and Image 9 - three positions are obviously faulty) for images that were supposedly taken 'inside' the specimen, from below the floor, or from unrealistic distances. Photograph sets taken with the turntable method should align in perfectly regular circles, and images taken at identical tripod heights must be on one level.

Also, if the set contains many photographs, consider using only a subset for the model. Using more images not only increases the total calculation time, but can also lead to artifacts on the surface. If images are redundant, tiny and unavoidable inaccuracies in alignment will lead to the program finding several points with minimal differences in position where there only should be one point. As a consequence, the surface can become wrinkled, may contain pyramid artifacts caused by individual points floating just above or below the rest of the surface, or may even be created as two sub- parallel instances.

If a project is very large it can be a viable alternative to split it into chunks (see below). Each chunk should contain images that overlap well. For example, if a mounted skeleton of an elephant is to be modeled, one chunk could hold the skull, neck and front limb images, another the hind limbs, and a third the main body. Once the images within each chunk are aligned, the chunks can then be aligned via markers (see below).

 

Scaling the model

Scaling needs to be done manually, because the methods suitable for paleontological digitizing usually do not allow the use of targets with scales that the software can recognize automatically. Therefore, at least one object of exactly known length must be visible in at least two of the photographs of one set, and may not move relative to the specimen between the two images. It is important that the two ends of the known distance can be found with ease on the photographs. Therefore, measuring a distance on the specimen is not recommended, unless it is between two tiny discrete marks. For example, the 'maximum length' of a bone cannot be used because it will be nearly impossible to identify the two measuring points down to one pixel in the photographs. We recommend using printed scale bars or (folding) yard sticks marked in centimeters (practically all photogrammetric software uses the metric system). It is usually best to place in-program markers on the images, not the 3D model or dense point cloud.
1. Create one marker at one end of the scale object and place the same on a second image.
2. Create a second marker at the other end of the scale object and place the same on another image. Note that it is not required that both the first and second marker are created on the same image, nor is it necessary to use the same second image for placing the second instance!
3. Use the two markers to create a scale bar.
4. Set the length for the scale bar.
5. Repeat for as many scale objects as are available.
Note that in Agisoft Photoscan Pro the scaling step can be prepared before the Alignment, but can also be performed after the images have been aligned, via the Update option.

Masking

Undesirable parts of photographs containing moving objects or repetitive background features cannot always be avoided. Such areas need to be made unavailable for feature detection; otherwise they can ruin the alignment and model creation processes. Masking can be performed in several ways: in a graphics program, where the area can be blanked (any uniform color will do) manually by painting in or lassoing, or automatically using a magic wand tool. We have found, however, that a very uniform background suitable for use of a magic wand tool usually is so constant that it does not allow the photogrammetry software to pick up features anyway, and thus needs not to be masked. The second option for masking is the in-program masking of the photogrammetry program.
 

Figure 9: Top view of sparse point cloud and camera positions calculated from an image set of Dimetrodon mount at the Royal Belgian Institute of Sciences, Brussels. Camera positions that are nonsensical (inside the mount’s skull) or a massive mismatch to the images (almost in touch with fore- and hind limbs) are highlighted in red.

 

In both cases, the exact separation of desired and undesired data is important. There is a natural tendency to draw the mask so that all parts of the specimen are retained in the un- masked area. The direct consequence is that the contact area between the specimen and the background is in smaller or larger parts also contained in the allowed area, and will be used by the photogrammetry software. However, there is no 100% clear distinction of the 'this pixel is specimen, the next one is background' kind. Therefore, drawing the masking line this way will have pixels included in the alignment and model creation that contain information that is not strictly speaking part of the specimen. Usually, if the rim of included background and contact area pixels is narrow, the effect will be minimal, and amounts only to a small number of erroneous points floating close to the main model. In the end, it comes to a tradeoff between time spent masking and time spent removing artifacts from the dense point cloud. However, it is easy to avoid the creation of these points by simply masking the border area with the background, i.e. by leaving only those parts of the image un- masked that show only the specimen (Figure 10). Similarly, dark shadows on the underside of specimens can also be masked, resulting in more realistic textures, provided there is sufficient overlap within the remaining photograph parts.

If features on the background are needed to achieve good alignment of the images, it is still recommended to mask a thin strip of pixels around the specimen, especially in places where the specimen contacts the ground. This strip will later result in a gap in the dense point cloud that makes cropping the undesired background points easy and fast.

Alignment

Although running the alignment can be as easy as simply choosing the corresponding step in the program menu, there are settings that can have a strong influence on the quality of the resulting calculation.
 

Figure 10: Image of a caudal vertebra of the oviraptorine dinosaur Citipati osmolskae IGM 100/978 (currently stored at the AMNH). A. Masked image. Note that the markers used for scaling the model can be in the masked area. B. Points marking detected features in the image. Note how no features are detected in the masked areas

 

Figure 11: Model creation process with the one-chunk method on the example of a caudal vertebra of the oviraptorine dinosaur Citipati osmolskae IGM 100/978 (currently stored at the AMNH). A. Aligned images and sparse point cloud. Each blue rectangle shows one reconstructed camera position with the matching image filename. Note that the two circles of photographs are not sub-parallel to each other, as the geometry of the specimen did not allow rotating it by ~180°. However, all parts of the surface are sufficiently captured to deliver a complete model. Two markers were used to set scale (see Figure 10). B. Closer view of the sparse point cloud with 127,994 points. Because of extensive masking in all images there are no points representing the background. C-D. Dense point cloud with 5,733,491 points with (C) and without (D) color. E-G. Polygon mesh with 7,108,439 vertices and 14,216,842 polygon faces with color (E), shaded (F) and wireframe display (G). H-K. Polygon mesh reduces to (H) 1,500,018 vertices and 3,000,000 polygon faces, 500,012 vertices and 1,000,000 polygons (I), 50,006 vertices and 100,000 polygons (J) and 5,000 vertices and 10,000 polygons (K). L. Photograph of the specimen.

 


The regular approach should always be the use of the highest quality alignment the program offers. However, if this approach fails, it is recommended to use a low-quality setting initially, eliminate the images that are obviously mis-aligned, and re-run the alignment at high quality. Similarly, the number of points the program is supposed to use for the alignment can be increased at the cost of much longer calculations times. Increasing the number per image three- to five-fold can, however, rescue data sets that otherwise do not align at all. Lastly, choosing generic pair-selection (if available in your software) can also help to achieve better alignment.

Additionally, it is always worth checking the points display for all images (Figures 3B and 10B) to see if erroneous points are being used in the alignment. Such points can be on parts of the image that should not be used, e.g. background that was insufficiently masked, or can be on parts of the image that should be used but produce false-positives in the search for identical features. Typical for the latter case are points on building parts or other repetitive structures in the background, or points on repetitive features on the specimen itself. The former can be avoided by masking, the latter can be ameliorated by the method described in the next paragraph.

The last resort, when a photograph set that finally fails alignment cannot be easily re- taken, consists of adding in-program markers manually to some or all images that align well with each other and to all images that do not align well, and re-run alignment. Depending on the reasons for the non-alignment this method can sometimes lead to perfect alignments for a tolerable amount of work, or can require several dozens of markers to be placed on hundreds of images, a task that is usually impractical. In this case it may be a viable alternative to split the project into chunks, and use markers to align the chunks.

 

Alignment methods for more than one set of images

Method 1: one-chunk (preferred). Summary of photography method suitable for one- chunk method: One set of photographs on turntable, flip specimen over, take second set. Repeat until the entire surface is well represented in the photograph sets. Include scale bar in each set. Try to make background unsuitable for detection of points: use featureless materials, distance specimen from turntable (place on perspex cube; cover turntable with white cardboard; covering with white or black cloth possible but may have enough structure to give points). Set depth of field so that full specimen but not background/turntable is in focus. Blank or out- of-focus background can make masking unnecessary.

See Figure 11.

1. Add all sets of photos to one chunk, including the photograph that separates the two sets. Make the latter unavailable for alignment; it only serves as an easy clue in the photo list to distinguish the two sets.

2. Add markers to the scale bar in one or both sets, create in-program markers and the scale bars and set the appropriate lengths.

3. Mask the entire background in all photographs, including the scale bars (those parts of the background on which no points can be detected can be omitted from the masking). The more accurately the masking outline follows the specimen outline, the better the final model will be and the fewer erroneous points will be created along the edges of the model (this step may be unnecessary if no points are found on the background. Run a test alignment with a low number of photos and a high point detection number to check).

4. Align the photos, optimize for all scale bars as described above.

5. Generate a dense point cloud. This point cloud should then be inspected for erroneous points along the areas where the model is based on one set of photos; typically, this area features some dark points that pertain to areas outside the specimen, and stem from erroneous points found in the shadows the specimen cast on the table. If the masks in the photos reach very close to the specimen, there will be few such points. Small numbers of erroneous points can however be ignored as their influence in polygon mesh creation is negligible.

6. Generate the polygon mesh.

If the one-chunk method fails, use the multi- chunk method on the same photographs. Only if this method also fails will it be necessary to take more photographs.

Method 2: multiple-chunk. Summary of photography method suitable for multiple- chunk method: one set of photos (with or without turntable), flip specimen over, take next set (repeat if necessary). Include scale bar in each set. For walk-around method of large specimens: Take one set of one end of the specimen, take a separator photograph, take set of next section, ensuring that the two sections overlap. Repeat if necessary. See Figure 12.

The multi-chunk method allows using the background for the alignment step within each chunk (those parts of the background that do not move relative to the specimen during the photography of one set of photos), thus it is a good method for specimens where few points for alignment will be found on the specimen itself. A background well suitable for point detection should be chosen (newspaper, Persian rug, etc.). Also, using the multi-chunk method means less care is needed to set up the turntable than for the one-chunk method, and the preparation in the photogrammetry program is easier, as background blanking or masking is not needed. However, much fiddling may later be necessary to adjust the alignment between the two chunks, or the fit between the two model halves. The method also makes handling very large projects easier, by reducing the overall calculation time and improving the chance at good alignment. The method works with more than two series of photos and shells as well, but more parts usually mean more trouble aligning them.

 

Figure 12: Model creation process with the multi-chunk method on the example of a fibula of a sauropod dinosaur from the Utah Field House of Natural History State Park Museum in Vernal. A. View of the program interface of Agisoft Photoscan Pro with (left) the chunks 1 and 2 containing the two image sets. Note that markers have been added to both sets. Markers called ‘X 1’, ‘X2’ and ‘X 4’ in both chunks mark the positions of the physical markers on the specimen, as seen on the display of the image (right). Additional markers (‘point 1’ through ‘point 4’) in chunk 2 are intended for the creation of scale bars. B. From the additional markers (circled in red) four scale bars have been created (circled in orange), and their lengths set. Via the “wizard wand” button on the top left an existing alignment can be refined to include the scaling information. Alternatively, the scale bars can be added before alignment. In this case, the sparse point cloud for chunk 2 is shown on the right. C. The two chunks have been aligned based on the markers with identical names, and merged. The new chunk ‘merged chunks’ contains all images and all markers (green circle). Note that the merged sparse point cloud shows slight alignment problems on the lower left edge of the bone shaft near the two markers ‘X 4’. However, the near-perfect alignment of the markers, so close that in the model view they differ by less than three pixels each, shows that each individual set of images was well- aligned (otherwise the 3D placement of the makers would not fit the other set). D. Dense point cloud computed from the merged sparse point clouds. Note that there are a number of artefacts around the edges of the bone, but that the overall shape of the bone and most details are well represented. Compare C to Figure 13.
 

Figure 13: Sparse point cloud from the same image sets as in Figure 12, but computed with the one-chunk method. Comparison to Figure 12 C shows that the one-chunk method here results in a large number of alignment problems. The most obvious resulting erroneous points are circled in red.

 

As in the one-chunk method, it is possible to mask the background for a cleaner model provided the features detected on the specimen itself suffice for alignment within each image set. In the example shown in Figure 12 this is the case; but barely: several images are not entirely correctly aligned (visible by the handful of ‘floating’ points in the lower left quadrant next to the bone shaft; caused by slight misalignment of several images). If the same data set is run via the one-chunk method, the alignment is less good (Figure 13).

In order to ameliorate the fiddling necessary to align the chunks, it is advisable to mark the specimen with points that can be used to place markers with high accuracy (Figure 12A). Three markers are the minimum necessary, more are advisable. However, the markers will be visible in the texture of the finished model. If two separate dense point clouds are generated and later combined in a different CAD program, there typically will be artifacts from a sub-perfect fit of the two halves and a lot of manual correction is needed.

1. Add each photograph series to a separate chunk in one overall file.

2. Add markers to the scale bar in one or both sets, create the scale bars in the program and set the appropriate length. Note that the markers used to create the scale bars should have different names in each chunk, as they will otherwise be included for a marker- based alignment.

3. Mask in all chunks only those parts of the background that move relative to the specimen, or mask the full background if you expect the specimen to deliver sufficient features for alignment.

4. Align the images in all sets (use batch process).

5. Place markers on the photographs on the physical markers you put on the specimen. Rename the markers so that the markers corresponding to the same physical marker have the same name in all chunks. Leave one physical marker unused; it can later be used to check the chunk alignment.

6. Align the chunks, marker-based.

Alternatively, align them point-based, if all background in all photos has been masked.

7. Merge the chunks.

8. Add a marker to the previously unused physical marker in two photographs, and check its position in all other photographs. If there are systematic divergences, one chunk is not well aligned. Note that such misalignment may be obscured on the model!

9. Remove background points in the sparse point cloud of the merged chunks.

10. Calculate the dense point cloud.

11. Remove any remaining background points from the dense point cloud.

12. Calculate the polygon mesh.
 

Very large specimens (trackways, excavations, mounted skeletons). The calculation time for alignment, dense cloud generation and mesh creation increases exponentially with the number of photographs used. If the vast majority of photographs overlap with a significant portion of other photographs nothing can be done to accelerate the process. If, on the other hand, each photograph overlaps only with a handful of others, as is the case if a long sequence of photographs documents a large area such as a trackway, the set can be split into chunks that each calculate quickly. If the alignment is performed in chunks, these can later be merged based on markers, as described above for the multi-chunk method, which takes little time. To ease this task it is useful to place numbered markers before the photography. Tracks, for example, may be photographed with scale bars next to them and a piece of paper carrying a number. Later, distinct points on the scale bars can easily be used to create in-program markers for chunk alignment. Additionally, for large numbers of photographs it is usually worthwhile to turn on generic pair selection which filters out those photos that likely overlap for alignment and discounts all other photo pairs, thus reducing the calculation time significantly.

For dense point cloud and polygon mesh generation it is similarly advisable to select only a section of the entire model, calculate the dense cloud and mesh, save to a new file name (e.g., the name of the entire file with ‘_part01’ appended), select the next part, calculate the dense cloud and mesh and save with a new name (‘_part02’), and so on (Figure 14). Alternatively, in the program the box used for selecting the volume for dense cloud computation can be moved ahead by dragging a corner point from the beginning of the model past the points marking the end of the first, already calculated dense cloud part, so that the borders between dense clouds are an exact touch without any gap or overlap. This process results in manageable file sizes, but each partial dense cloud is aligned with all others, and most research tasks can be performed by alternately using the individual files.

In order to scale large models at high accuracy it is best to include markers at the far extremes of the specimen and measure their physical distance in the field with a measuring tape. For aligning several versions of models based on photograph series from different times, e.g. to document the advance of an excavation and the relative position of bones removed at different times of the digging season, or even across several seasons, it is advisable to place several immobile and clearly visible markers (e.g. chisels or poles cemented in drilled holes).

If large complex specimens were photographed in chunks, the multi-chunk method described above can also be used for alignment. For segmented specimens (e.g. skeletons) it is advisable to calculate the dense point clouds for the separate chunks after alignment, as editing is much faster when done on smaller overall point clouds. Instead of merging all chunks (step 7 above), the chunks can be kept separate and only the finished polygon mesh models of the individual segments should be combined into one file, usually in the separate CAD software used for further processing.
 

Using remote computing facilities effectively

If remote computing is available, it can speed up the calculation of photogrammetric models considerably. However, not all processes during the creation of a 3D-model are equally suitable for remote computing. It is recommended to finish all CPU-heavy steps (alignment, dense point cloud generation, polygon mesh generation) on remote, while all interaction heavy steps involving a lot of data loading (masking, marker placement, point cloud and mesh editing), and thus on a remote computer a lot of data transfer, should be carried out on a local computer.

Additionally, for all programs offering batch processing, it is advisable to set up batch operations that run autonomously, e.g. overnight. Although sometimes steps have to be repeated later, e.g. because the program does not automatically set crop parameters appropriately for dense point cloud calculations, the avoidance of idle time is usually a significant gain in overall work speed. If available, a save-after-each-step option should be used. It allows interrupting calculation of large batches with minimal loss of completed work.
 

Figure 14: Trackway of a theropod dinosaur in the Wesling Quarry in Münchehagen, Germany. A. Oblique view of sparse point cloud and camera positions calculated from 86 images. B. Roughly top view of sparse point cloud. C. and D. show piecemeal creation of meshes of only the key areas. In C the area around a single footprint has been selected for mesh generation (denoted by red points), in D the selection area has been shifted to the next footprint. E. Rhinoceros 5.0 (McNeel Associates; www.rhino3d.com) view of all separate meshes. Note how the separate meshes are all to the same scale and in correct relative position to each other, although a complete dense point cloud or mesh of the entire trackway was never calculated. The total file size (as Stanford PLY) of all tracks is under 100 MB, whereas a mesh covering all of them together would be over 1 GB in size before cropping. We estimate the time saved at about 70% compared to calculating one single huge mesh and cropping it.
 


Tips & Tricks

Specimens that are flat are best not placed on the two flat surfaces, because each set of pictures then captures the flat surface perfectly, but the area with high curvature connecting the flat surfaces is usually captured (and thus reconstructed) less well, with little overlap between the two photograph sets. It is therefore advisable to place the specimens on edge on a support (e.g. on modeling clay; remember to use a plastic film so that no chemicals from the clay contaminate the specimens), so that the flat surfaces are vertical. When the flat surfaces face the camera, large angles can be covered between photographs; whereas the edge-on positions require small intervals (<5°). This setup creates large overlap between the two picture sets showing the flat areas, ensuring excellent alignment and model creation.
Specific, irregular shaped objects, such as dinosaur ulnae, tend not to rest in two positions that are roughly 180° from each other. In such cases, it is advisable to take three or more sets of photographs, using the minimum number of positions the bone can easily be placed in. Both the one-chunk and the multiple-chunk method can be used, but it is always worth the extra trouble to take photographs suitable for the one-chunk method, as the time invested in aligning many chunks in the many-shell method is usually significantly higher.

Back                                                                                                                                                                                                                 Next

 

 
   

Copyright © 2006

Optimized  IE 1024x768

About us    |  Contact us    |