New Technology for a New Century
International Conference 
FIG Working Week 2001, Seoul, Korea 6–11 May 2001

Session 4 - Digital Photogrammetry and Remote Sensing

Key words: DCRP, Segmentation, Ellipse Fitting, Displacement.

In this study, this researcher designed the routine of image analysis with object concept, so as to increase the efficiency of displacement interpretation of structure by digital image analysis. For the hierarchical connection performance of analysis routine, this researcher formed the constant attribute of objectification of photogrammetry process by constituting the classes such as target location, transformation of coordinates of scan image, bundle adjustment, and direct linear transformation (DLT). And, this researcher performed the efficient location of sub-pixel coordinates by manufacturing the routine of coordinates location of image including target recognition, grouping, image segmentation and location etc. and the module of additional functions such as the establishment of pixel size and the indication function of recognized target.

For the development of component of digital photogrammetry that the location of sub-pixel of high accuracy is possible, this researcher extracted the image segmentation (T-3) method which uses the average and standard error of greylevel values in search area, and ellipse fitting (ELI) method by edge detection and thinning by executing error analysis by the methods of coordinates location of sub-pixel, and could know that the accuracy of location is improved in accordance with the increase of threshold value. And, by developing the component to be based on the establishment of hierarchical diagram of classes through designing each subject of process of image analysis with the class of objectification concept, more efficient displacement interpretation of structure was possible.

1. OOP METHOD

Existing application program was the program of procedure type. Then, user could not help performing only in accordance with fixed routine regardless of the processing course or flow of program. But, the development of program to be able to reinforce the interface with user came to be possible, as the establishment of system of windows environment of GUI came to be possible. Object oriented program solves the problem through the transmission of message mutually among objects by prescribing the issues to require the processing of data of characteristic of engineering as objects and generating event. Object is performed by message transmission through method as the conglomerate to include data and operation, and the class for expressing object has the general behavioral style and the state variable within. Like this, Object oriented programming technique is the programming of compromise type which uses the core element to be object and has the merit to be able to take the reuse, expansion and conservation of object efficiently from the viewpoint of characteristic of software.In this study, this researcher used the object oriented programming technique which introduces the object oriented design concept in the software engineering. Fig. 1 is what showed the hierarchical diagram of class of windows application program to move on the basis of message. Then, user gets to complete program by defining the reaction on event. (Bae,1999)

Fig. 1 Hierarchical Diagram of Class of Digital Photogrammetry

1.1 Manufacture of Class

Class is for defining the data structure and behavioral style of similar objects, and object gets to show event in the class which defined the data form, structure and reaction indication of itself. So, this researcher designed hierarchical diagram, so that the hierarchical connection performance through the co-ownership of class may be possible.

1.1.1 Sub-pixel

For the target recognition and the coordinates location of target, this researcher manufactured target class by constituting the object which defined relevant method and property as one class. This researcher enabled the property to be suitable for the demand of user to be equipped, by forming the automatic recognition routing of target through making image into digital data and by objectifying the method on image segmentation and sub-pixel location for the improvement of location accuracy of image coordinates. And, in the performance of target class which uses the data of scan image, this researcher defined the object of format of scan image for the maintenance of realiability of measured result. As target occupies a part of digital image to be expressed with the greylevel value of 0-255 and forms the group of similar greylevel value, this researcher performed target grouping to use the information of greaylevel of the data format of lattice type in search area so as to prescribe the form of target. This is one of the course of image analysis which is necessary for the routine of target recognition of automation concept. This researcher decided the pixel beyond the initial greylevel value in search area by using the information of greylevel of image and designed the routine of automatic/semi-automatic target recognition, so that the case that the interval among pixels is within 5 pixel may be recognized as one independent target, and planned the minimum size of target for recognizing with image as 5 pixel. And, so as to extract more improved method of coordinates location of sub-pixel of accuracy, this researcher established and used the algorithm of 6 kinds of methods of image segmentation and of 5 kinds of location methods like table 1.(Chapman,1992, Jansa, 1995, Shortis,1995, etc)

Fig. 2 offers pixel coordinates, image coordinates, and correction coordinates of lens distortion by the location methods of sub-pixel of target as what showed the windows of coordinates location which is performed in system area. And, it is involving the additional functions such as the illustration of central position of target which is based on interpretation method, the offer of pixel information of target area, the establishment of activity of target, and the change of No. of adjustment point etc. Image coordinates extracts message in turn with pixel unit and interpretation unit in accordance with the event to use mouse, and user may make target active or non-active passively with the automatic deletion of improper target by reject limit.

Fig. 2 Windows of Coordinates Location

1.1.2 Lease Square Method

LSM class decides the unknown value by performing the data processing to use surplus observation value. As for LSM, expression is possible with the procession of AX=L as the control method to be used most for the processing of observation value. So as to obtain unknown matrix, object oriented method regards individual procession to pass through operation course as object and shares these under definition to be class. That is, it gets to handle the operation of procession by passing through the action course of object to form method and property. LSM class realizes unknown procession and co-variance procession by using Cholesky method. Like this, the data to be presented through property and the behavior styles to show reaction indication get to form the performance of hierarchical operation course or the reconstitution of data by being transmitted to lower class.

1.1.3 Scan Image

So as to interpret the film image acquired by using general camera, the control point of transformation of index role for transforming pixel coordinates(X-Y) into photo coordinates(X'-Y') is necessary. Generally, in case of non-metric camera, the system of photo coordinates is established by measuring the corner of film. But, with this, it is difficult to obtain the location reliability of high accuracy including the location error of observer. In this study, this researcher established the coordinates system of image through the control of LSM of redundancy as the method for the improvement of accuracy of interpretation of scan image.

This researcher decided primary polynomial by observing film edge that location is easier than corner with surplus as the method for the decide of image format of scan image like Fig. 3 and established the system of image coordinates by verifying 4 line forms which were extracted.

Fig. 3 Image Format of Scan Image

Formula (1) is the basic formula for the establishment of the system of image coordinates, and we may calculate the solution of LSM of surplus observation value by arranging with the procession formulas like formula(2).

This researcher decided the rotary angle of central coordinates (Xc,Yc) of image, X' axis and Y' axis by using ai(1~4) and bi(1~4) of X procession and transformed pixel coordinates into image coordinates by the phase transformation of unequal angle of 2 dimension. This researcher measured the sub-pixel coordinates of point that the slant is greatest by analyzing greylevel information toward horizontal or vertical direction from the central point in the search area of edge. This researcher got to derive simple equation by using surplus data beyond minimum 3 points which were observed like this.

1.1.4  3 dimensional Location

This researcher designed the mutual connection nature of analysis module and classes of digital photogrammetry and introduced the semi-automation concept to the process from the input section of image to the location interpretation. As for DLT, analysis course is simple and initial parameter is not required as the solution of linear relation. But, as the bundle adjustment technique for the image interpretation of high accuracy is non-linear solution to require initial parameter, this researcher manufactured DLT class for leading the automation processing of this. In DLT class, receiving the object control point in space and the image point to correspond with it from the section of input data and extracting the coefficient(L1,-L11) of DLT through the co-ownership of procession class get to decide the initial value of outside look element of w, j, k, X0, Y0 and Z0. And, this researcher sought the efficiency of technique of 3 dimensional location interpretation through on-line data processing by manufacturing the bundle adjustment class to share DLT class. As DLT routine requires the control point of 6 or more at lease and the image point to correspond with it for extracting 11 transformation parameters and it is linear solution, interpretation result changes sensitively in accordance with control point, image point and the arrangement. Therefore, this researcher designed bundle adjustment class to be able to decide 3 dimensional position by dynamic adjustment solution like Robust theory which is not static solution in consideration of the weight of location accuracy of control point and image coordinates. This is more roundabout non-linear solution to calculate the coordinates value and interpretation accuracy of unknown point by the repeated performance of space resection and space intersection.

1.2 Digital Photogrammetry System

As windows environment enables the integral environment establishment which is based on the object oriented technique centering around event by reinforcing the interface with user, we should complete the systematic design about right recognition of analysis routine, approachableness, and processing etc. and the hierarchical diagram about the mutual compatibility and constant attribute etc. among classes, for the efficiency increase of digital photogrammetry. (Han,1995)

So, in this study, this researcher established the digital photogrammetry system of windows system, by designing the analysis routine that the independent connection performance is possible about input of 8 bit image, independent taking of picture information, abstraction of target image, decision of target area, reject limit, image segmentation, location, correction of distortion, decision of pixel size, pixel coordinates transformation, coordinates transformation of scan image, DLT, and bundle adjustment, passing through the preprocessing course of raw image. Established system gets to perform the analysis module to want through mouse or keyboard that the interface with user is possible and enables visual analysis in addition to quantitative analysis to be based on simple data processing. And, we may acquire the location accuracy of sub-pixel beyond the location accuracy of characteristic of hardware about the image of size of standardized pixel space acquired from horizontal/vertical frame graver from the viewpoint of software, by defining the class about target recognition and location on the basis of pixel information of 256 stages. Besides, so as to extract more effective method of target location, we may choose the analysis routine to be suitable for the hierarchical of system most, by arranging 6 kinds of methods of image segmentation and 5 kinds of methods of location.

We may extract DLT coefficient and outside look element by utilizing the result of control point and the image coordinates with the data of Cholesky procession formula and may offer the convenience of system management for image analysis by designing the routine to process data to be able to measure more exact 3 dimensional position through using them as the initial value of bundle control.

Like this, this researcher could uplift the convenience, clearness and intuitive nature of system by establishing the integral environment of digital photogrammetry through forming the hierarchical relation and framework about the module of image analysis and the classes. And, about the independent data of various process courses that automation level is different, consistent analysis came to be possible through the systematic establishment to reach from lower class to upper class. As each analysis module exists in the form of objectification, the additional establishment of module and class that user may help analysis is easy, and the geometrical analysis to correspond mutually among images came to be possible without all the knowledge about image structure. Fig. 4 is what showed system window as the basic panel of digital photogrammetry system established for the location of image coordinates of target.

Fig. 4 System Windows

2. MONITORING OF STRUCTURE, AND ANALYSIS OF ACCURACY

2.1 Image Acquisition and Control Point Survey

So as to verify the possibility of 3 dimensional geometrical image analysis about structure, this researcher manufactured the imitation bridge (120 (l)×28 (w)×42 (h)) in the form of suspension bridge. So as to compare the extraction of displacement quantity which is based on load change and the standard error which is based on image capture media by using the application program of established windows environment, this researcher planned the photography for the image acquisition of same time zone by manufacturing simultaneous shutter. And, so as to acquire the image of time zone which is same as the steel video camera by using digital camcoder that momentary image capture beyond 10 frame per second is possible, this researcher manufactured and installed timer. Fig. 5 is what showed the scene of image acquisition of imitation bridge including photography apparatus and all the systems used for image acquisition.

This researcher acquired image by using Kodak DCS200 and DC50, Samsung Camcoder SV-D100(I),(II), Nikon F-801 camera and changed 35 film into the scan image of 15 and 30 per pixel by using the PhotoScan of Intergraph. Table 2 is what showed the photography condition by the image capture media. Target attached to interpretation point leads the strong contrast between the target and the information of greylevel of background as the retro-reflective target of 8 size. This researcher executed the location of control point by arranging the reflective target like this as control point and using total station SET2B around object. After choosing 2 reference point toward the direction to be parallel with object, this researcher established 3 dimensional right angle coordinates system in space which establishes baseline direction as X axis, and vertical direction as Z axis, and right-angled direction to it as Y axis. And, this Fig. 5 Scene of Image Acquisition of Imitation Bridge researcher tried to measure displacement by loading in the range of middle of land to being many displacements to the model of tube axis.

2.2 Accuracy Analysis

So as to improve the location accuracy of image coordinates, the processing level of analysis system, the proper image processing, and the location of sub-pixel of high accuracy are required. Therefore, in this study, this researcher executed the error analysis which is based on the method of image segmentation and the method of location which have influence on the location accuracy of target by using established program.

2.2.1 Standard Error to be based on the Method of Image Segmentation

One of important courses of image processing which have influence on the location accuracy of target is the establishment of threshold value by image segmentation. As, the size and form of target change by threshold value, the method of image segmentation for dichotomizing the background image that greylevel value is close to 0 and the target image that it is close to 255 is the course of image processing to have to consider necessarily for the improvement of location accuracy of target.

As the result that this researcher executed error analysis by applying the method of image segmentation and the location method like table 1 to the image of DCS200 camera and the image of same time zone of scan 15, this researcher could extract the average standard error and 3 dimensional position error of XYZ axes like table 3. From this, this researcher could know that T-3 method of image segmentation which uses the average and standard error greylevel information in search area shows relatively better accuracy to show the relation between threshold value and accuracy which are based on the method of image segmentation on the basis of result of error analysis of table 3 is as is in table 4. This showed the tendency that the number of target to include decreases and the accuracy is improved as threshold value is big and that the number of target to include increases and the accuracy decreases relatively as threshold value is small as what showed the relation among method of image segmentation, threshold value, target size, and accuracy. From this, this researcher could know that T-3 method of image segmentation which uses the average and standard error of greylevel value in search area shows high efficiency about the pixel information of target image.

2.2.2 Standard Error to be based on the Location Method

This researcher calculated the coordinates result of target and the error analysis by applying the technique of centroid location which enables the sub-pixel location of image coordinates and the ellipse fitting(EL) method to be based on edge detection through subdividing the pixel to be the physical location unit of system. Fig. 6 is what illustrated 3 dimensional position error by the method of image segmentation which is based on the location method. This researcher could see that fig. 6 shows minute size but better accuracy in comparison with the location method that EL(I) method is different as the result that this researcher executed the error analysis to be based on location method by using the image of same time period which was acquired with 2 SV-D100

Fig. 6 3-dimensional Position Error by the Method of Image Segmentation which is based on the Location Method camcoders. And, out of centroid location methods, CT2 method which used greylevel value as weight showed more improved accuracy. The result interpreted by applying location EL(II) method showed the accuracy distribution which is low relatively in comparison with the accuracy interpreted by applying other location algorithm, and it is considered that this became issue in the algorithm arrangement of mathematical model formula.

2.2.3  2 dimensional Target Location Accuracy

This researcher analyzed the location accuracy of sub-pixel of target by the image capture media by applying T-3 method of image segmentation and EL1 method of location to the image of same time zone. Table 5 is the result which extracted 2 dimensional location accuracy of target by analyzing the solid model to be based on the image capture media afterwards. In case of interpreting the image of same time zone of DCS200 and scan 15?, this researcher could obtain 2 dimensional target location accuracy of 3? or so and the pixel accuracy of 1/354. And, this researcher could obtain the precision of 1:800,000 or so at 2.6m or so of photography distance. Fig. 7 is what illustrated the relation between 2 dimensional target location accuracy and 3 dimensional position error which were interpreted by the method of image segmentation and the location method by the image capture media. Then, this researcher could know that 3 dimensional position error which is based on interpretation method becomes larger as irregular aspect as the location error of target becomes larger. Therefore, we can see that we should improve the location accuracy of sub-pixel of target and that the establishment of location algorithm of sub-pixel of high accuracy and the development of processing software are essential with the development of hardware that the image acquisition of high resolution is possible.

Fig. 7 2-dimensional Target Location Accuracy and 3 dimensional Position Error

2.3 Location of Displacement

So as to measure the displacement of observation model which is based on load change, this researcher interpreted 22 image points attached to beam by loading total 9.6kg by 1.6kg by stages and applying T-3 method of image segmentation and EL1 method of location at initial state. In case of ICM-A, this researcher acquired the image of same time zone which is based on the load change of 7 stages by using DC50 and SV-D100, and it showed the standard error of Y axis (hanging direction) of 1.8918 or so. This is considered as the result induced from the systematic instability and low resolution of DC50 camera which showed serious lens distortion. The case of ICM-C showed the standard error of Y axis of 0.9641? as the result analyzed by capturing the image of same time zone which uses timer from the continuous frame acquired by using 2 SV-D100. As this may reduce the required time to be based on the acquisition and processing of on-line data in comparison with steel video camera, it is expected that establishing the hardware such as the image capture board of high resolution which can capture the increase of frame number per second and the momentary frame may increase the application for the location of displacement and momentary conduct of structure.

Fig. 8 3 dimensional Position Error by Load which is based on the Image Capture Media axis of 0.2 or so.

The case of ICM-D analyzed by using the image of DCS200 and the image of 35 film showed relatively most improved standard error of vertical direction of 0.1816 , and the case of ICM-B and ICM-E interpreted by scanning film image showed the standard error of the direction of Y.

Fig. 8 is the result which illustrated 3 dimensional position error by load which is based on the image capture media. In case of analyzing the image of same time zone of scan of DCS200 and 15 film, the hanging quantity of 5.648? was generated at the time of loading 1.6kg, and the hanging quantity of 8.433? was generated at the time of loading 9.6kg. Thus, it could be seen that 70% or so of whole hanging quantity is generated at the time of initial loading. At this time, the standard error of vertical direction is 0.1819. Fig. 9 is the result which illustrated the displacement quantity of vertical direction which is based on load in initial state as relative coordinates result.

Like this, as I could create the base of module of image analysis that the addition and renewal of diverse camera in digital image, it is considered that many applications in diverse fields will be possible.analysis modules are easy in this study, it is considered that management will be possible by grafting various modules for performing the displacement interpretation of structure more efficiently through expanding the class of objectification concept of digital photogrammetry. And, as the interface and processing of on-line data are possible from CCD camera or digital.

Fig. 9 Vertical Displacement of Imitation Bridge which is based on Load

It is expected that the possibility to put digital photogrammetry to practical use will be very great as the location technique of version for the interpretation of displacement of large-size structure in civil engineering and diverse industry fields, if much study on the development of digital camera and analysis system is performed

3. CONCLUSION

1. For the analysis of digital image, this researcher developed object oriented digital photogrammetry component by designing and establishing the hierarchical diagram about the class of user and the processing routine of data with objectification concept.
   
2. For the hierarchical connection performance of analysis routine, this researcher created the objectification structure of photogrammetry process by constituting the classes such as target location, coordinates transformation of scan image, bundle adjustment, and DLT.
   
3. This researcher could uplift the reliability of coordinates location of target and could establish reject limit for promoting the efficiency of target, as this researcher established the system of image coordinates by deciding the outer block part of scan image through the method of edge detection.
   
4. This researcher could see that the method of image segmentation (T-4) which uses the average and average error of greylevel information in search area and the ellipse fitting method(EL1) to be based on edge detection are ideal for the location of sub-pixel of target.
   
5. This researcher could perform the displacement location of imitation bridge more efficiently by developing the component to be able to expand the class of diverse subjects through designing each subject of digital photogrammetry process with the class of objectification concept.
   
6. It is expected that the application will be expanded to various industrial fields as well as construction field, if we seek the on-line of system that the location of real structure of real-time is possible with the continuous study for the improvement of accuracy of digital photogrammetry, in the future.

BIBLIOGRAPHY

1. Amirn Saeed Homainejad(1996). The Problems of Real Time Photogrammetry. The Univ. of New south Wales.
2. Bae, Sang-Ho, Kang, Jook-Mook, Song, Seung-Ho (1999). The Construction of Digital Photogrammetry System Using Windows Environment. The Journal of Korean Society Civil Engineering, Vol.19, ?-2; 325~334.
3. Clive S. Fraser(1991). A resume of some industrial applications of photogrammetry. ISPRS Journal of Photogrammetry and Remote Sensing.
4. C.W. Urquhart, J.P. Siebert(1993). Towards Real-Time Dynamic Close Range Photogrammetry. SPIE Videometric, Vol. 2067: 240~251.
5. D. Cosandier and M.A. Chapman(1992). High precision target location for industrial metrology. SPIE,Vol.1820: 111-121, Videometrics.
6. Firooz A.Sadjadi(1997). Automatic target Recognition . SPIE.
7. Han, Seung-Hee(1995). 3D Modeling of Automobile part Based on CCD camera. SPIE, Videometric?, Vol. 2598: 149~160.
8. J.C. Trinder,J. Jansa and Y. Huang(1995). An assessment of the precision and accuracy of methods of digital target location. ISPRS:12-20.
9. M.R. Shortis, T.A. Clarke, and S. Robson(1995). Practical Testing of the Precision and Accuracy of Target Image Centering Algorithms. SPIE Videometrics, Vol. 2598: 65-76.

CONTACT

Sangho Bae
Full Time Lecturer, Civil Eng.
Daelim College
526-7 Bisan Dong
Dongan Gu
Anyang
KOREA
Tel. + 82 31 467 4912
Fax + 82 31 467 4910
E-mail: [email protected] 

13 April 2001

This page is maintained by the FIG Office. Last revised on 15-03-16.