Contents


Using CloudCompare + ICE for many images

If you have not installed Microsoft's Image Compositing Editor (ICE) download the latest version from:

Microsoft Research


before attempting to process a large set of images. 

Install ICE.  See the instructions in the Installation Notes.


In AirPhotoSE, after having loaded multiple images and then the Run option, 

Click

to start the external programme CloudCompare (CC).  An options window appears:



The first time you use CloudCompare all options may be checked, However CloudCompare has limited memory. If you are going to process a lot of images, do not use the "Generate 2D " option  which displays them in CC.  Install Microsoft's Image Compositing Editor which can handle very large images and blend the seams from orthophoto overlays as well.  If AirPhotoSE detects it's presence, it will be called automatically when CloudCompare terminates and CC's results passed on to it for further processing.

Turn off all but options of interest by unchecking the appropriate boxes.  When only one option at time is selected, this permits treating the largest number of images, for example:

For DTMs      For Orthophotos:


When you click OK, CC computation starts.  If there are a large number of images, nothing appears to happen for some time.  Wait until the progress bar has reached the end or the DTM is displayed if you have selected that option.   You can view and orient it by dragging with the left mouse button, move it with the right one and zoom in and out with the mouse wheel. If you have selected "Keep Images in Memory" you will see:



when processing is complete.  However, if you have more than about 30 to 40 images, there may not be enough free memory available for other operations.  In that case, do not use this option. A small window with a progress bar will show that CC is working.

If you have selected Orthophotos, exit from CloudCompare.  Again, there will be a delay, and then you will be asked:



Click on Yes and ICE will start in a separate window.  If you can't see this window, hold down Alt-Tab on the keyboard to switch to it.

ICE blends overlapping parts of the images and corrects minor positional differences so that seams between the images are no longer visible.  You may use the internal cropping tool from ICE to cut off undesired portions of the result if you so wish.

  

ICE attempts to detect the geometric relationships between the images and use an appropriate model for stitching them into a single image.  For orthophotos, the best model is Planar  Motion.  If it's automatic test chooses Rotating Motion, re-select the Planar Motion 3 option.  If ICE persists in trying to use rotating motion despite this change, there may be one or more images in the set processed by CloudCompare which are causing the problem.  Usually the cause is an extremely oblique image or one which shows the horizon.  Deleting or changing it's extension name so that it is not loaded may help without having to recompute the whole orthophoto operation from the beginning. The part-images of the orthophoto are copied to the directory where the source images for the orthophoto operation were selected. Part images from older programme runs are not erased, but they will not be in the list passed to ICE for a new run.

If you haven't installed ICE, the programme will default to the image container option:



Warnings:

The "image scale factor" default should always be left at 1.  It scales an output image made in CC by the factor shown to make smaller, not larger images.  The "import image option" should always be left checked, and the source of the images as shown should not be changed.  Do not use "Alternative keypoints" or "Import Keypoints as Cloud".unless you fully understand the comment below by Daniel Girardeau-Montaut and wish to experiment using other features in CC.

When closing ICE, you will be asked if you want to save the ICE project:

  

This permits you to reload ICE in it's stand-alone mode along with the images without having  to repeat the computation in AirPhotoSE in order to make changes in output types, cropping etc.

If you have selected , and if you have selected high resolution in Orthophoto Options, the finest detail in the DTM can be seen by increasing the number of vertices from the default value visible when CC starts for the first time.  You can save multiple orientation DTM images using CloudCompare's "Render to File" and set the size of the output image there. However, output may fail if the value chosen is too large for CC's available memory. 

You may ignore the message:

  

unless you want to experiment with the many options available in CloudCompare or have chosen to use point cloud output created with the PtCloud button.

Note:

The main orthophoto code gets keypoints from each source image by sampling from a window which may range in size from 400 to 2400 pixels.  The larger the window, the more keypoints will be found.  However, processing time for finding associated keypoints (approximate near-neighbours) rises drastically when this number is large.  It is usually best to have 4000 or fewer keypoints in each image.  The threshold option in the Orthophoto Options dialogue permits selection of important keypoints without reducing the size of the sample window.  If there are a very large number of images, reducing the sample window size plus the threshold to as small value will speed up calculation, but there may not be enough keypoints found in some images and the image will be excluded from the CloudCompare result.  It is recommended to start with three to five overlapping images and choose an effective sample window size plus a threshold before attempting to process the entire set of images. 

Using the Point Cloud for keypoints:

Daniel Girardeau-Montaut, the author of CloudCompare writes:

"[There is] a new option in the 'Bundler file' import dialogue to use an alternative cloud (or mesh) as keypoints. I let the user specify any file he desires, then CC tries to open it and expects to find either one cloud or one mesh (and only one).

The points are then simply back projected in the images and used as the 'native' ones in the ortho-rectification algorithm. When using it with the PMVS ply file I generally use for my tests,  I realised that while looking very dense and clean, it had in fact several groups of outliers. So if we use the file as is, the resulting ortho-rectification process is about as good as with the 'native' keypoints.However, if we clean it first (either manually or with a connected components extraction - keeping only the biggest one), we can achieve slightly better results.

And of course it's the same when trying to use the PMVS ply file as a DTM. The result is nicer if we first load the ply file, clean it, and eventually apply Poisson Surface reconstruction. If we save the resulting mesh in a new ply file, and use this file as the alternative 'keypoints' file, CC will automatically use it as a DTM. In order to avoid any use of Delaunay 2D (which is definitely too weak with such point clouds), I only sample a large number of points on this DTM mesh and then colour the points with the image information. If the cloud density is high, it already permits visualising the result without too many holes (at a reasonable scale). And the scale can't be very big, because while the ply file is much nicer than the default keypoints, it's not so clean that we can get precise details like houses shapes, trees, etc."

CloudCompare Error Messages:

If you receive this error message from CloudCompare:



click Cancel and then close CloudCompare by clicking on the exit button in the upper right corner or via Quit on the File menu. First, try cropping the input images in the main AirPhotoSE window so that the upper and side portions of extreme obliques are cut off.  

Then click the Options button and look at the values you have chosen for the sample window size and the High Resolution check box.  If the High Resolution checkbox is not on, try turning it on.  If it is already on, try increasing the value of the sample window size.  Then click the Run button to re-run with the same images and other parameters.  

If you run out of memory when loading many large images with High Resolution enabled in Options, you can either reduce the number of images selected or resize them with an image processing utility.  AirPhotoSE can open and process more images than CloudCompare can display. If you turn off all but "generate 2D orthophotos" in the initial CC window, this will usually reduce memory requirements to a minimum and enable treatment to continue.