AUTOMATION OF TRAFFIC FLOW MEASUREMENT USING VIDEO IMAGES This thesis was submitted to the Department of Computer Science and Engineering of the University of Moratuwa In partial fulfillment of the requirements of the Degree of M. Eng. By G. C. de Silva Department of Computer Science and Engineering University of Moratuwa Sri Lanka 2002 77182 Abstract This research focuses on the design and implementation of a system for automating the task of measuring vehicular traffic flow on a motorway, using video image sequences. Such a system is useful when designing traffic management systems such as signal lights at road junctions. It can also function as part of an intelligent transportation system, providing useful quantitative information, such as number and size of vehicles, speed and direction of movement etc, for managing vehicular traffic effectively. The system processes a sequence of images captured by a video camera mounted above a motorway. It performs motion segmentation on the images using background subtraction, and derives traffic parameters using a combination of image processing techniques such as statistical methods and solving systems of equations. While making use of existing algorithms for image processing and image sequence analysis, new algorithms have been developed and employed for background extraction and occlusion modeling. The system has been designed to handle inconsistencies caused by occlusion of vehicles, by predicting the motion of vehicles using the derived parameters of motion. Knowledge about the traffic scene has been used by the system to improve the accuracy of results. The system relies on a centralized 'knowledge base', a source of information related to the traffic scene. This knowledge base is loosely coupled to the system in such a way that the system can be customized to perform well in different traffic scenes by changing only the content of the knowledge base. The system has been implemented to work on an IBM PC based system with an Intel Celeron 450 MHz processor and 64MB Memory. The tools for implementation have been selected after a study of their performance. This automated traffic flow measurement system has been tested using three image sequences from different traffic scenes. A number of tests have been conducted by varying the frame rate, image resolution and the method used to create the background image. The results have been compared with each other, as well as the ground truth, while recording the processing time involved with each test case. The automated system measures traffic parameters with an average accuracy of 90%. The new technique that has been proposed in this work for background construction produces the most accurate result. Processing speed can be improved by reducing the frame rate and the resolution of images, with a slight reduction in accuracy of the results. D e c l a r a t i o n I . G . C . d e S i l v a . h e r e b y c e r t i f y t h a t t h e w o r k i n c l u d e d i n t h i s t h e s i s h a s n o t b e e n s u b m i t t e d i n p a r t o r \ V h o l e f o r a n y o t h e r a c a d e m i c q u a l i f i c a t i o n a t a n y i n s t i t u t i o n . C i . C . d e S i l v a R e s e a r c h S t u d e n t D r . N . K . W i c k r a m a r a c h c h i S u p e r v i s o r C o n t e n t s • A b s t r a c t • A c k n o w l e d g m e n t s • L i s t o f f i g u r e s • L i s t o f t a b l e s C h a p t e r 1 : I n t r o d u c t i o n 1 1 . 1 B a c k g r o u n d . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 . 2 A i n 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1 . 3 O b j e c t i v e s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1 . 4 O r g a n i z a t i o n o f t h e t h e s i s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 C h a p t e r 2 : R e v i e w o f t h e s t a t e o f t h e a r t 2 . 1 I n t r o d u c t i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 . 2 C o m p u t e r v i s i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 . 3 I m a g e s e q u e n c e a n a l y s i s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 . 4 T r a f f i c i m a g e a n a l y s i s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 . 5 A n o v e r v i e w o f t r a f f i c d e t e c t i o n s y s t e m s i n u s e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 . 6 L i m i t a t i o n s i n e x i s t i n g s y s t e m s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 . 7 S u 1 n n 1 a r y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . , . . . . . . . . . . . . . . . . . C h a p t e r 3 : P r o b l e m S t a t e m e n t a n d A p p r o a c h 3 - . , _ ) - . , . ) 6 7 1 0 1 1 1 1 1 2 3 . 1 I n t r o d u c t i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 3 . 2 P r o b l e m d e f i n i t i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 3 . 3 A p p r o a c h . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 3 . 4 A p p l i c a b i l i t y o f t h e s y s t e m . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 4 3 . 5 S u m m a r y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 5 C h a p t e r 4 : S y s t e m d e s i g n 1 6 4 . 1 I n t r o d u c t i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 6 4 . 2 F u n c t i o n a l d e s i g n a n d a l g o r i t h m s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 9 4 . 3 D a t a d e s i g n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 8 4 . 4 K n o w l e d g e b a s e d e s i g n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 4 . 5 U s e r i n t e r f a c e d e s i g n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 8 4 . 6 S u m m a r y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 9 C h a p t e r 5 : I m p l e m e n t a t i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 0 5 . 1 I n t r o d u c t i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 0 5 . 2 S e l e c t i o n o f t o o l s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 0 5 . 3 P l a t f o r m . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1 5 . 4 F u n c t i o n a l m o d u l e s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2 5 . 5 D a t a b a s e o f t h e s y s t e m . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3 5 . 6 K n o w l e d g e b a s e o f t h e s y s t e m . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 4 5 . 7 T h e g r a p h i c a l u s e r i n t e r f a c e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 6 5 . 8 S u m m a r y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3 C h a p t e r 6 : E x p e r i m e n t a l r e s u l t s a n d e v a l u a t i o n 5 4 6 . 1 I n t r o d u c t i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 4 6 . 2 I m a g e s e q u e n c e s a n d g r o u n d t r u t h . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 4 6 . 3 A n o v e r v i e w o f r e s u l t s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 7 6 . 4 T e s t i n g s t r a t e g y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1 6 . 5 T e s t c a s e s a n d r e s u l t s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 6 . 6 C o m p a r i s o n o f r e s u l t s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 6 6 . 7 S u m m a r y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 9 C h a p t e r 7 : C o n c l u s i o n a n d r e c o m m e n d a t i o n s . . . . . . . . . . . . . . . . . . . . 1 0 0 7 . 1 C o n c l u s i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0 0 7 . 2 S u m m a r y o f c o n t r i b u t i o n s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0 1 7 . 3 R e c o m m e n d a t i o n s a n d f u t u r e w o r k . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0 1 • R e f e r e n c e s 1 0 3 • A p p e n d i x A : R e l a t e d t h e o r y 1 0 9 • A p p e n d i x B : S o u r c e c o d e 1 1 9 • A p p e n d i x C : C o n t e n t s o f t h e C D R O M 1 3 3 A c k n o w l e d g m e n t s H a v i n g c o m p l e t e d t h e M . E n g . D e g r e e p r o g r a m a t t h e d e p a r t m e n t o f C o m p u t e r S c i e n c e a n d E n g i n e e r i n g , u n i v e r s i t y o f M o r a t m v a . I w o u l d l i k e t o t a k e t h i s o p p o r t u n i t y t o t h a n k t h o s e w h o h e l p e d m e a l o t i n c o m p l e t i n g i t a t t h i s l e v e l . F i r s t o f a l l . I w o u l d l i k e t o t h a n k m y s u p e r v i s o r s . D r . N a l i n W i c k r a m a r a c h c h i a n d D r . N i h a l K o d i k a r a . f o r t h e i r s u p e r v i s i o n a n d g u i d a n c e d e s p i t e h a v i n g b u s y s c h e d u l e s . D r . W i c k r a m a r a c h c h i , a s H e a d o f t h e D e p a r t m e n t o f C o m p u t e r S c i e n c e a n d E n g i n e e r i n g , m a d e s u r e t h a t I h a d s u f f i c i e n t t i m e a n d s t u d y l e a v e f o r r e s e a r c h w o r k a l t h o u g h t h e d e p a r t m e n t w a s c o n s t r a i n e d b y t h e l a c k o f s t a f f D r . G i h a n D i a s . R e s e a r c h c o o r d i n a t o r i n t h e D e p a r t m e n t o f C o m p u t e r S c i e n c e a n d E n g i n e e r i n g , m o t i v a t e d m e t o s t a r t t h i s p r o g r a m a n d g a v e m e v a l u a b l e g u i d e l i n e s t o k e e p p r o g r e s s i n g . A l s o . I w o u l d l i k e t o t h a n k D r . L a l i t h G a m a g e , e x t e r n a l e x a m i n e r , f o r h i s c o m m e n t s a n d s u g g e s t i o n s d u r i n g t h e p r o g r e s s r e v 1 e w s . D r . D i e t e r K o l l e r . F a c u l t y o f I n f o r m a t i c s , U n i v e r s i t y o f K a r l s r u h e , G e r m a n y a n d M r . A n d r e a P r a t i . U n i v e r s i t y o f M o d e n a . I t a l y , h e l p e d m e b y p r o v i d i n g i m a g e s e q u e n c e s a n d a n u m b e r o f r e s e a r c h p a p e r s r e q u i r e d f o r t h e p r o j e c t . M r . T . H . G . S i l v a o f B o m i r i y a C e n t r a l C o l l e g e v v a s v e r y h e l p f u l i n d e r i v i n g m a t h e m a t i c a l f o r m u l a e r e l a t e d t o t h e p r o j e c t . M y s i n c e r e t h a n k s g o t o P r o f . ( M r s . ) N . R a t h n a y a k e , D i r e c t o r o f p o s t g r a d u a t e S t u d i e s a n d a l l t h e s t a f l i n t h e P o s t g r a d u a t e S t u d i e s D i v i s i o n f o r t h e i r s u p p o r t i n a d m i n i s t r a t i v e m a t t e r s . I w o u l d l i k e t o e x p r e s s m y g r a t i t u d e t o m y f e l l o w r e s e a r c h s t u d e n t s i n t h e C E E E r e s e a r c h l a b f o r a l l t h e a s s i s t a n c e a n d e n c o u r a g e m e n t g i v e n t o m e . F i n a l l y , 1 w o u l d l i k e t o t h a n k M r . N i l h a n N i l e s . l e c t u r e r o n t h e d e p a r t m e n t o f T e x t i l e E n g i n e e r i n g . f o r h i s v a l u a b l e c o m m e n t s t h a t m a d e m y t h e s i s m u c h b e t t e r t h a n i t p r e v i o u s l y \ V a s . C h a p t e r 3 F i g u r e 3 . 1 C h a p t e r 4 F i g u r e 4 . 1 f i g u r e 4 . 2 F i g u r e 4 . 3 F i g u r e 4 . 4 F i g u r e 4 . 5 f i g u r e 4 . 6 F i g u r e 4 . 7 F i g u r e 4 . 8 F i g u r e 4 . 9 C h a p t e r 5 F i g u r e 5 . 1 F i g u r e 5 . 2 f i g u r e 5 . 3 F i g u r e 5 . 4 F i g u r e 5 . 5 F i g u r e 5 . 6 F i g u r e 5 . 7 F i g u r e 5 . 8 F i g u r e 5 . 9 F i g u r e 5 . 1 0 F i g u r e 5 . 1 1 F i g u r e 5 . 1 2 F i g u r e 5 . 1 3 F i g u r e 5 . 1 4 F i g u r e 5 . 1 5 F i g u r e 5 . 1 6 C h a p t e r 6 F i g u r e 6 . 1 F i g u r e 6 . 2 F i g u r e 6 . 3 F i g u r e 6 . 4 L i s t o f F i g u r e s S y s t e m o v e r v i e w T o p l e v e l f u n c t i o n a l m o d e l E n t i t y - r e l a t i o n s h i p m o d e l o f t h e s y s t e m d a t a L e v e l 1 d a t a f l o w d i a g r a m R e l a t i o n a l d a t a b a s e s c h e m a S p e c i f i c a t i o n o f a n u n p r o c e s s e d r e g i o n I m a g e w i t h i d e n t i f i e d t r a c k s A s c e n a r i o w h e r e p h o t o g r a m m t e r i c c o r r e c t i o n i s n o t r e q u i r e d A s c e n a r i o w h e r e p h o t o g r a m m t e r i c c o r r e c t i o n i s r e q u i r e d P a r a m e t e r s r e q u i r e d f o r c a l c u l a t i n g t h e c o r r e c t i o n f a c t o r . I m a g e w i t h s c e n e i n f o r m a t i o n T h e g r a p h i c a l u s e r i n t e r f a c e h i e r a r c h y M a i n w i n d o w a t s y s t e m s t a r t u p U s i n g F i l e m e n u t o l o a d a s e q u e n c e S p e c i f y i n g t h e d i r e c t o r y t o l o a d i m a g e s f i · o m M a i n w i n d o w a f t e r l o a d i n g t h e i m a g e s e q u e n c e · M e a s u r i n g t r a f f i c f l o v v S p e c i f y i n g t h e n u m b e r o f f r a m e s u s i n g t h e d i a l o g b o x M a i n w i n d o w a f t e r m e a s u r i n g t r a f f i c f l o w V i e w i n g r e s u l t s b y t r a c k W i n d o w s h o w i n g r e s u l t s g r o u p e d b y t r a c k S a v i n g r e s u l t s i n a t i l e V i e w i n g r e s u l t s b y v e h i c l e t y p e W i n d o w s h o w i n g r e s u l t s g r o u p e d b y v e h i c l e t y p e R e t u r n i n g t o t h e m a i n w i n d o w E x i t i n g s y s t e m S e q u e n c e 1 : i m a g e w i t h t r a c k s a n d u n p r o c e s s e d a r e a s S e q u e n c e 2 : I m a g e w i t h t r a c k s a n d u n p r o c e s s e d a r e a s B a c k g r o u n d i m a g e S a m p l e f r a m e f r o m i m a g e s e q u e n c e 1 3 1 6 1 7 1 9 ' " ' ! J~ 3 4 3 5 3 6 3 6 3 7 4 4 4 7 4 8 4 8 4 8 4 9 4 9 4 9 5 0 5 0 5 1 5 1 5 1 5 2 5 2 - , )~ 5 5 5 6 5 7 5 8 f i g u r e 6 . 5 F i g u r e 6 . 6 F i g u r e 6 . 7 F i g u r e 6 . 8 f i g u r e 6 . 9 F i g u r e 6 . 1 0 F i g u r e 6 . 1 1 F i g u r e 6 . 1 2 F i g u r e 6 . 1 3 F i g u r e 6 . 1 4 F i g u r e 6 . 1 5 F i g u r e 6 . 1 6 F i g u r e 6 . 1 7 F i g u r e 6 . 1 8 F i g u r e 6 . 1 9 F i g u r e 6 . 2 0 F i g u r e 6 . 2 1 F i g u r e 6 . 2 2 F i g u r e 6 . 2 3 F i g u r e 6 . 2 4 F i g u r e 6 . 2 5 F i g u r e 6 . 2 6 F i g u r e 6 . 2 7 S e t o f b l o b s d e t e c t e d b y p r o c e s s i n g a f r a m e S e q u e n c e 1 . o r i g i n a l s i z e : B a c k g r o u n d i m a g e c o n s t r u c t e d b y u s i n g t h e a v e r a g e p i x e l i n t e n s i t y S e q u e n c e 1 . o r i g i n a l s i z e : B a c k g r o u n d i m a g e c o n s t r u c t e d b y u s i n g t h e m e d i a n o f p i x e l i n t e n s i t i e s S e q u e n c e 1 , o r i g i n a l s i z e : B a c k g r o u n d i m a g e c o n s t r u c t e d b y u s i n g t h e m o d e o f p i x e l i n t e n s i t i e s S e q u e n c e 1 . s c a l e d d o w n t o 5 6 % : B a c k g r o u n d i m a g e c o n s t r u c t e d b y u s i n g t h e a v e r a g e p i x e l i n t e n s i t y S e q u e n c e 1 . s c a l e d d o w n t o 5 6 % : B a c k g r o u n d i m a g e c o n s t r u c t e d b y u s i n g t h e m e d i a n o f p i x e l i n t e n s i t i e s S e q u e n c e 1 , s c a l e d d o w n t o 5 6 % : B a c k g r o u n d i m a g e c o n s t r u c t e d b y u s i n g t h e m o d e o f p i x e l i n t e n s i t i e s S e q u e n c e 1 . s c a l e d d o w n t o 2 5 % : B a c k g r o u n d i m a g e c o n s t r u c t e d b y u s i n g t h e a v e r a g e p i x e l i n t e n s i t y S e q u e n c e 1 . s c a l e d d o v v n t o 2 5 % : B a c k g r o u n d i m a g e c o n s t r u c t e d b y u s i n g t h e m e d i a n o f p i x e l i n t e n s i t i e s S e q u e n c e 1 . s c a l e d d o w n t o 2 5 % : B a c k g r o u n d i m a g e c o n s t r u c t e d b y u s i n g t h e m o d e o f p i x e l i n t e n s i t i e s S e q u e n c e 2 , o r i g i n a l s i z e : B a c k g r o u n d i m a g e c o n s t r u c t e d b y u s i n g t h e a v e r a g e p i x e l i n t e n s i t y S e q u e n c e 2 . o r i g i n a l s i z e : B a c k g r o u n d i m a g e c o n s t r u c t e d b y u s i n g t h e m e d i a n o f p i x e l i n t e n s i t i e s S e q u e n c e 2 . o r i g i n a l s i z e : B a c k g r o u n d i m a g e c o n s t r u c t e d b y u s i n g t h e m o d e o f p i x e l i n t e n s i t i e s S e q u e n c e 2 , s c a l e d d o w n t o 5 6 % : B a c k g r o u n d i m a g e c o n s t r u c t e d b y u s i n g t h e a v e r a g e p i x e l i n t e n s i t y S e q u e n c e 2 . s c a l e d d o w n t o 5 6 % : B a c k g r o u n d i m a g e c o n s t r u c t e d b y u s i n g t h e m e d i a n o f p i x e l i n t e n s i t i e s S e q u e n c e 2 . s c a l e d d o v v n t o 5 6 % : B a c k g r o u n d i m a g e c o n s t r u c t e d b y u s i n g t h e m o d e o f p i x e l i n t e n s i t i e s S e q u e n c e 2 . s c a l e d d o v m t o 2 Y Y o : B a c k g r o u n d i m a g e c o n s t r u c t e d b y u s i n g t h e a v e r a g e p i x e l i n t e n s i t y S e q u e n c e 2 . s c a l e d d o w n t o 2 5 % : B a c k g r o u n d i m a g e c o n s t r u c t e d b y u s i n g t h e m e d i a n o f p i x e l i n t e n s i t i e s S e q u e n c e 2 , s c a l e d d o w n t o 2 5 % : B a c k g r o u n d i m a g e c o n s t r u c t e d b y u s i n g t h e m o d e o f p i x e l i n t e n s i t i e s C o m p a r i s o n o f p r o c e s s i n g t i m e s f o r b a c k g r o u n d e x t r a c t i o n f o r d i f f e r e n t i m a g e s e q u e n c e s C o m p a r i s o n o f e r r o r i n b a c k g r o u n d e x t r a c t i o n f o r d i f f e r e n t i m a g e s e q u e n c e s C o m p a r i s o n o f m o t i o n s e g m e n t a t i o n e r r o r s w i t h d i f f e r e n t b a c k g r o u n d e x t r a c t i o n t e c h n i q u e s C o m p a r i s o n o f m o t i o n s e g m e n t a t i o n e r r o r s w i t h d i f T c r e n t f r a m e r e s o l u t i o n s 5 8 6 2 6 3 6 4 6 6 6 7 6 8 7 0 7 1 7 2 7 4 7 5 7 6 7 8 7 9 8 0 8 2 8 3 8 4 8 7 8 8 8 9 9 0 F i g u r e 6 . 2 8 F i g u r e 6 . 2 9 F i g u r e 6 . 3 0 F i g u r e 6 . 3 1 F i g u r e 6 . 3 2 F i g u r e 6 . 3 3 F i g u r e 6 . 3 4 F i g u r e 6 . 3 5 A p p e n d i x A F i g u r e A - 1 F i g u r e A - 2 F i g u r e A - 3 F i g u r e A - 4 F i g u r e A - 5 F i g u r e A - 6 F i g u r e A - 7 F i g u r e A - 8 C o m p a r i s o n o f v e h i c l e t r a c k i n g e r r o r s v v i t h d i f f e r e n t b a c k g r o u n d e x t r a c t i o n t e c h n i q u e s C o m p a r i s o n o f v e h i c l e t r a c k i n g e r r o r s w i t h d i f f e r e n t f r a m e r e s o l u t i o n s C o m p a r i s o n o f v e h i c l e t y p e c l a s s i f i c a t i o n e r r o r s w i t h d i f f e r e n t b a c k g r o u n d e x t r a c t i o n t e c h n i q u e s C o m p a r i s o n o f v e h i c l e t y p e c l a s s i f i c a t i o n e r r o r s w i t h d i f f e r e n t f r a m e r e s o l u t i o n s E f f e c t o f f r a m e r c s o l u t i o n o n p r o c e s s i n g t i m e C o m p a r i s o n o f v e h i c l e t r a c k i n g e r r o r s \ \ i t h d i f f e r e n t f r a m e r a t e s C o m p a r i s o n o f v e h i c l e t y p e cla~sification e r r o r s w i t h d i f f e r e n t f r a m e r a t e s C o m p a r i s o n o f e f l e c t i v e p r o c e s s i n g t i m e s f o r d i f f e r e n t 1 m a g e s e q u e n c e s 9 1 9 2 9 3 9 4 9 5 9 6 9 7 9 8 C a l c u l a t i n g t h e s c a l e f a c t o r w h e n t h e C C O s u r f a c e i s I 0 9 p a r a l l e l t o t h e g r o u n d p l a n e C a l c u l a t i n g t h e s c a l e f a c t o r w h e n t h e C C O s u r f a c e i s a t a n I I 0 a n g l e t o t h e g r o u n d p l a n e T h e e f f e c t o f p h o t o g r a m m c t r i c c o r r e c t i o n I I 0 P r o j e c t i n g t h e i m a g e f o r p h o t o g r a m m c t r i c c o r r e c t i o n I l l C a l c u l a t i o n f o r t h e t o p h a l f o f t h e i m a g e 1 1 3 C a l c u l a t i o n s a l o n g a l i n e i n t h e t o p h a l f o f t h e i m a g e 1 1 5 C a l c u l a t i o n f o r t h e b o t t o m h a l f o f t h e i m a g e 1 1 6 C a l c u l a t i o n s a l o n g a l i n e i n t h e b o t t o m h a l f o f t h e i m a g e 1 1 8 C h a p t e r 4 T a b l e 4 . 1 T a b l e 4 . 2 T a b l e 4 . 3 T a b l e 4 . 4 T a b l e 4 . 5 T a b l e 4 . 6 T a b l e 4 . 7 T a b l e 4 . 8 T a b l e 4 . 9 T a b l e 4 . 1 0 C h a p t e r 5 T a b l e 5 . 1 T a b l e 5 . 2 C h a p t e r 6 T a b l e 6 . 1 T a b l e 6 . 2 T a b l e 6 . 3 T a b l e 6 . 4 T a b l e 6 . 5 T a b l e 6 . 6 T a b l e 6 . 7 T a b l e 6 . 8 T a b l e 6 . 9 T a b l e 6 . 1 0 T a b l e 6 . 1 1 T a b l e 6 . 1 2 T a b l e 6 . 1 3 T a b l e 6 . 1 4 · r a b l e 6 . 1 5 L i s t o f T a b l e s E n t i t i e s i n t h e d a t a m o d e l M a p p i n g b e t w e e n t h e d a t a i n t h e k n o w l e d g e b a s e a n d t h e s y s t e m d a t a m o d e l T w o m a t c h i n g b l o b s w i t h t h e i r a t t r i b u t e s A t t r i b u t e s o f t h e S c e n e e n t i t y A t t r i b u t e s o f t h e T r a c k e n t i t y A t t r i b u t e s o f t h e U n p r o c e s s e d ! J e K i o n e n t i t y A t t r i b u t e s o f t h e V e h i c l e T } p e e n t i t y A t t r i b u t e s o f t h e V e h i c l e e n t i t y A t t r i b u t e s o f t h e F r a m e F e a t u r e e n t i t y . R e s u l t s o b t a i n e d b y f i t t i n g a 2 n d o r d e r p o l y n o m i a l t o e a c h t r a c k F u n c t i o n a l m o d u l e s o f t h e s v s t e m F i l e s c r e a t e d b y t h e s y s t e m 1 8 1 8 2 4 2 8 2 9 2 9 3 0 3 0 3 1 3 5 4 2 4 3 S e q u e n c e 1 : k n o w l e d g e b a s e 5 4 S e q u e n c e 1 : G r o u n d t r u t h 5 5 S e q u e n c e 2 : k n o w l e d g e b a s e 5 6 S e q u e n c e 2 : G r o u n d t r u t h 5 7 T h e s e t o f f r a m e f e a t u r e s f o r t h e s a m p l e f r a m e 5 9 M a t c h e d v e h i c l e f e a t u r e s 5 9 R e s u l t s g r o u p e d b y t r a c k 5 9 R e s u l t s g r o u p e d b y v e h i c l e t y p e 6 0 R e s u l t s f o r s e q u e n c e 1 . o r i g i n a l s i z e : b a c k g r o u n d u s i n g t h e 6 2 a v e r a g e v a l u e o f p i x e l i n t e n s i t i e s R e s u l t s f o r s e q u e n c e 1 . o r i g i n a l s i z e : b a c k g r o u n d u s i n g t h e 6 3 m e d i a n o f p i x e l i n t e n s i t i e s R e s u l t s f o r s e q u e n c e 1 . o r i g i n a l s i z e : b a c k g r o u n d u s i n g t h e 6 4 m o d e o f p i x e l i n t e n s i t i e s R e s u l t s f o r s e q u e n c e 1 , o r i g i n a l s i z e : w i t h h a l f t h e o r i g i n a l 6 5 f r a m e r a t e R e s u l t s f o r s e q u e n c e 1 , o r i g i n a l s i z e : w i t h o n e t h i r d t h e 6 5 o r i g i n a l f r a m e r a t e R e s u l t s f o r s e q u e n c e 1 . s c a l e d d o w n t o 5 6 % : b a c k g r o u n d 6 6 u s i n g t h e a v e r a g e v a l u e o f p i x e l i n t e n s i t i e s R e s u l t s f o r s e q u e n c e 1 . s c a l e d d O \ v n t o 5 6 % : b a c k g r o u n d 6 7 u s i n g t h e m e d i a n o f p i x e l i n t e n s i t i e s T a b l e 6 . 1 6 R e s u l t s f o r s e q u e n c e 1 , s c a l e d d o v , n t o 5 6 % : b a c k g r o u n d 6 8 u s i n g t h e m o d e o f p i x e l i n t e n s i t i e s T a b l e 6 . 1 7 R e s u l t s f o r s e q u e n c e 1 , s c a l e d d o w n t o 5 6 % : w i t h h a l f t h e 6 9 o r i g i n a l f r a m e r a t e T a b l e 6 . 1 8 R e s u l t s f o r s e q u e n c e 1 , s c a l e d d o v v n t o 5 6 % : w i t h o n e t h i r d 6 9 t h e o r i g i n a l f r a m e r a t e T a b l e 6 . 1 9 R e s u l t s f o r s e q u e n c e 1 , s c a l e d d o w n t o 2 5 % : b a c k g r o u n d 7 0 u s i n g t h e a v e r a g e v a l u e o f p i x e l i n t e n s i t i e s T a b l e 6 . 2 0 R e s u l t s f o r s e q u e n c e 1 , s c a l e d d o w n t o 2 5 % : b a c k g r o u n d 7 1 u s i n g t h e m e d i a n o f p i x e l i n t e n s i t i e s T a b l e 6 . 2 1 R e s u l t s f o r s e q u e n c e 1 , s c a l e d d o w n t o 2 5 % : b a c k g r o u n d 7 2 u s i n g t h e m o d e o f p i x e l i n t e n s i t i e s T a b l e 6 . 2 2 R e s u l t s f o r s e q u e n c e 1 , s c a l e d d o w n t o 2 5 % : \ V i t h h a l f t h e 7 3 o r i g i n a l f r a m e r a t e T a b l e 6 . 2 3 R e s u l t s ! o r s e q u e n c e 1 , s c a l e d d o w n t o 2 5 % : w i t h o n e t h i r d 7 3 t h e o r i g i n a l f r a m e r a t e T a b l e 6 . 2 4 R e s u l t s f o r s e q u e n c e 2 , o r i g i n a l s i z e : b a c k g r o u n d u s i n g t h e 7 4 a v e r a g e v a l u e o f p i x e l i n t e n s i t i e s T a b l e 6 . 2 5 R e s u l t s t o r s e q u e n c e 2 , o r i g i n a l s i z e : b a c k g r o u n d u s i n g t h e 7 5 m e d i a n o f p i x e l i n t e n s i t i e s T a b l e 6 . 2 6 R e s u l t s f o r s e q u e n c e 2 , o r i g i n a l s i z e : b a c k g r o u n d u s i n g t h e 7 6 m o d e o f p i x e l i n t e n s i t i e s T a b l e 6 . 2 7 R e s u l t s f o r s e q u e n c e 2 . o r i g i n a l s i z e : w i t h h a l f t h e o r i g i n a l 7 7 f r a m e r a t e T a b l e 6 . 2 8 R e s u l t s f o r s e q u e n c e 2 , o r i g i n a l s i z e : w i t h o n e t h i r d t h e 7 7 o r i g i n a l f r a m e r a t e T a b l e 6 . 2 9 R e s u l t s f o r s e q u e n c e 2 , s c a l e d d o w n t o 5 6 % : b a c k g r o u n d 7 8 u s i n g t h e a v e r a g e v a l u e o f p i x e l i n t e n s i t i e s T a b l e 6 . 3 0 R e s u l t s f o r s e q u e n c e 2 . s c a l e d d o w n t o 5 6 % : b a c k g r o u n d 7 9 u s i n g t h e m e d i a n o f p i x e l i n t e n s i t i e s T a b l e 6 . 3 1 R e s u l t s f o r s e q u e n c e 2 . s c a l e d d o w n t o 5 6 ( % : b a c k g r o u n d 8 0 u s i n g t h e m o d e o f p i x e l i n t e n s i t i e s T a b l e 6 . 3 2 R e s u l t s f o r s e q u e n c e 2 , s c a l e d d o w n t o 5 6 % : w i t h h a l f t h e 8 1 o r i g i n a l f r a m e r a t e T a b l e 6 . 3 3 R e s u l t s f o r s e q u e n c e 2 , s c a l e d d o w n t o 5 6 % : w i t h o n e t h i r d 8 1 t h e o r i g i n a l f r a m e r a t e T a b l e 6 . 3 4 R e s u l t s f o r s e q u e n c e 2 , s c a l e d d o w n t o 2 5 % : b a c k g r o u n d 8 2 u s i n g t h e a v e r a g e v a l u e o f p i x e l i n t e n s i t i e s T a b l e 6 . 3 5 R e s u l t s f o r s e q u e n c e 2 , s c a l e d d o w n t o 2 5 % : b a c k g r o u n d 8 3 u s i n g t h e m e d i a n o f p i x e l i n t e n s i t i e s T a b l e 6 . 3 6 R e s u l t s f o r s e q u e n c e 2 , s c a l e d d o w n t o 2 5 % : b a c k g r o u n d 8 4 u s i n g t h e m o d e o f p i x e l i n t e n s i t i e s T a b l e 6 . 3 7 R e s u l t s f o r s e q u e n c e 2 , s c a l e d d o w n t o 2 5 % : w i t h h a l f t h e 8 5 o r i g i n a l f r a m e r a t e T a b l e 6 . 3 8 R e s u l t s l o r s e q u e n c e 2 , s c a l e d d o w n t o 2 5 % : w i t h o n e t h i r d 8 5 t h e o r i g i n a l f r a m e r a t e I -,) d[qTIJ ::> X!puaddv