High precision trajectory determination of an UAS by integrating camera and laser scanner data with generalised object models
Final Report Abstract
Especially in geodesy, there are many MSSs, which require accurate and reliable georeferencing regardless of the environment and the application. This is indispensable for derived subsequent products, such as highly accurate three-dimensional point clouds for 3D city models or Building Information Modelling (BIM) applications. Increasingly, unmanned aerial vehicles (UAV) are employed as a platform and lead to new applications, some of which involve low flight altitudes and specific requirements such as low weight and low cost of the sensors. For image orientation, additional information is needed to determine not only relative measurements but also position and attitude in a global coordinate system. In view of these requirements and especially for flights between obstacles in urban areas, the classically used information from navigation satellite (GNSS) and intertial measurement systems (IMU) or even specially marked control points (GCP) is often not available or inaccurate. The idea addressed in this project is to improve the orientation of UAVs with camera and laser scanner data as well as existing generalized building model. Such models are increasingly available and provide a way to compensate for inaccurate or unavailable GNSS camera positions and drift effects of orientation. Although the model differs from the observed scene due to its generalization, relationships of sensor measurements to the building model will be found and used for orientation. In the image-based part of this project, three approaches for assigning link points to model planes in object space and a hybrid bundle adjustment for image orientation using a generalized building model that is global or window-based are presented. The assignments lead to notional observations for the distance from link points to model planes and are refined during iterations of the bundle adjustment. Experiments with an image sequence acquired between buildings show an improvement in image orientation from the meter range purely with GNSS measurements to the decimeter range when using the generalized building model with the simplest assignment method based on point-to-plane distances. No improvement in point-to-plane mappings is observed by searching planes in the point cloud. In summary, the developed method successfully improves UAV image orientation using a generalized building model. If there is no excessive computing power available to take into account the vast amounts of observation data, recursive methods are usually recommended. In this case, an iterative estimation of the requested quantities is performed, whereby the comprehensive total data set is divided into several individual epochs. If the most recent observations are successively available for each epoch, a filtering algorithm can be applied. Thus, an efficient estimation is carried out and, with respect to a comprehensive overall adjustment, generally larger observation sets can be considered. However, such filtering algorithms exist so far almost exclusively for explicit relations between the available observations and the requested estimation quantities. If this mathematical relationship is implicit, which is certainly the case for several practical issues, only a few methods exist or, in the case of recursive parameter estimation, none at all. This circumstance is accompanied by the fact that the combination of implicit relationships with constraints regarding the parameters to be estimated has not yet been investigated at all. In this project, a versatile filter algorithm is presented, which is valid for explicit and for implicit mathematical relations as well. For the first time, methods for the consideration of constraints are given, especially for implicit relations. The developed methodology will be comprehensively validated and evaluated by simulations and realworld application examples of practical relevance. The usage of real data is directly related to kinematic MSSs and the related tasks of calibration and georeferencing. The latter especially with regard to complex inner-city environments. In such challenging environments, the requirements for georeferencing under integrity aspects are of special importance. Therefore, the simultaneous use of independent and complementary information sources is applied in this thesis. This enables a reliable georeferencing solution to be achieved and a prompt notification to be issued in case of integrity violations.
Publications
- (2016): Integration of a generalised building model into the pose estimation of UAS images. In: Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XLI-B1, 1057-1064
Unger, J.; Rottensteiner, F.; Heipke, C.
(See online at https://doi.org/10.5194/isprs-archives-XLI-B1-1057-2016) - (2017): Assigning Tie Points to a Generalised Building Model for UAS Image Orientation. In: Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XLII-2/W6, 385-392
Unger, J.; Rottensteiner, F.; Heipke, C.
(See online at https://doi.org/10.5194/isprs-archives-XLII-2-W6-385-2017) - (2018). Iterated Extended Kalman Filter with Implicit Measurement Equation and Nonlinear Constraints for Information-Based Georeferencing. in 2018 21st International Conference on Information Fusion, FUSION 2018 (S. 1209-1216). [8455258] Institute of Electrical and Electronics Engineers Inc.
Vogel, S., Alkhatib, H., & Neumann, I.
(See online at https://doi.org/10.23919/ICIF.2018.8455258) - (2019). Geo-Referenzierung von Unmanned Aerial Systems über Laserscannermessungen und 3D-Gebäudemodelle. in Terrestrisches Laserscanning 2019 (Band 96, S. 63-74). (DVW-Schriftenreihe). Wissner Verlag
Bureick, J., Vogel, S., Neumann, I., Diener, D., & Alkhatib, H.
- (2019). Georeferencing of Laser Scanner-Based Kinematic Multi-Sensor Systems in the Context of Iterated Extended Kalman Filters Using Geometrical Constraints. Sensors (Basel, Switzerland), 19(10)
Vogel, S., Alkhatib, H., Bureick, J., Moftizadeh, R., & Neumann, I.
(See online at https://doi.org/10.3390/s19102280) - (2019): Georeferencing of an Unmanned Aerial System by Means of an Iterated Extended Kalman Filter Using a 3D City Model, In: PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation Science, Volume 87, Issue 5-6, 229-247
Bureick, J.; Vogel, S.; Neumann, I.; Unger, J.; Alkhatib, H.
(See online at https://doi.org/10.1007/s41064-019-00084-x) - (2020). Information-based georeferencing by dual state iterated extended kalman filter with implicit measurement equations and nonlinear geometrical constraints. in Proceedings of 2020 23rd International Conference on Information Fusion, FUSION 2020 [9190414] Institute of Electrical and Electronics Engineers Inc.
Moftizadeh, R., Bureick, J., Vogel, S., Neumann, I., & Alkhatib, H.
(See online at https://doi.org/10.23919/FUSION45008.2020.9190414) - (2021). Data fusion for georeferencing a laser scanner based multi-sensor system in a city environment. in 2021 IEEE 24th International Conference on Information Fusion (FUSION) Institute of Electrical and Electronics Engineers Inc.
Ernst, D., Jüngerink, J., Kindervater, L., Moftizadeh, R., Alkhatib, H., & Vogel, S.
(See online at https://doi.org/10.23919/FUSION49465.2021.9627026) - (2021). Information-Based Georeferencing of an Unmanned Aerial Vehicle by Dual State Kalman Filter with Implicit Measurement Equations. Remote sensing, 13(16)
Moftizadeh, R., Vogel, S., Neumann, I., Bureick, J., & Alkhatib, H.
(See online at https://doi.org/10.3390/rs13163205) - (2021): Mounting calibration of a multi-view camera system on a UAV platform. In: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences V-1-2021, pp. 97–104
Mohammadi M., Khami A., Rottensteiner F., Neumann I., Heipke, C.
(See online at https://doi.org/10.5194/isprs-annals-V-1-2021-97-2021)
