COMMUNICATING DATA QUALITY IN A GIS ENVIRONMENT by Kanagaratnam Thavalingam "This thesis was submitted to the department of Earth Resources Engineering of the University of Moratuwa in partial fulfillment of the requirements for the Degree of Master of Science" Department of Earth Resources Engineering University of Moratuwa Sri Lanka October 2001 74349 Abstract The GIS database is a digital representation of the real world. Any abstract of reality will contain discrepancies from its source. With traditional methods many of the problems are visible and the skilled map analyst makes the necessary adjustments and knows how far the information can be relied upon. With a Geographic Information System the equivalent operations are not transparent (the black box effect), usually the operators are no longer so skilled and the problems are largely invisible. The digital modelling has the potential to dramatically increase both the magnitude and importance of errors in the models. The results may be used for decision making and planning despite possessing levels of uncertainty that are completely unknown and usually cannot even be guessed. That is why the accuracy analysis is one of the most important problems in the development and applications of the system. Currently there are several demands from users of data to include quality parameters in the related GIS databases. A number of researchers have done work on the derivation of data quality especially on positional or geometrical accuracies. However there has been little work done on qualitative or semantic accuracies and ways of communicating them. A major contribution toward standardizing the definition, assessment and reporting of GIS data quality has been made by the Data Set Quality Working Group of the National Committee for Digital Cartographic Data Standards. This research, provides an overview of the data quality factors that should be considered when using geographic information, and is intended to explore the possibility of generating and communicating data quality in various ways in a GIS environment. Suitable algorithms, .mainly concerning positional and attribute accuracy assessments, were adopted from relevant literature to determine the data quality parameters at different levels of abstraction, for different data types. The levels of abstraction. considered were overall accuracy parameters at coverage level and specific accuracy parameters referring to entity level. To communicate the data quality to the user different methods such as numerical, graphical and textual messages were adopted .The area for the case study is located in Kegalle district. The feasibility of the reported implementation was assessed by means of the referred case study. The results obtained with this case study were used to draw some conclusions and recommendations regarding the communication of, data quality in a GIS environment. . . T h e w o r k i n c l u d e d i n t h e t h e s i s i n p a r t o r w h o l e , h a s n o t b e e n s u b m i t t e d f o r a n y o t h e r a c a d e m i c q u a l i f i c a t i o n a t a n y i n s t i t u t i o n I O c t o b e r ' 2 0 0 1 : . '~ , \ , . . \ " ' ' \'''\"~- K . Thavaling~m C o m m u n i c a t i n g d a t a q u a l i t y i n a G I S e n v i r o n m e n t A c k n o w l e d g e m e n t s F i r s t l y , I w o u l d l i k e t o m e n t i o n a l l e f f e c t i v e s u p p o r t a n d g u i d a n c e g i v e n t h r o u g h o u t t h e r e s e a r c h , b y m y s u p e r v i s o r D r . U . G . A . P u s w e w a l a , D e p a r t m e n t o f C i v i l E n g i n e e r i n g , U n i v e r s i t y o f M o r a t u w a . I w i s h t o e x p r e s s m y a p p r e c i a t i o n t o m y f o r m e r s u p e r v i s o r D r . U . G . S e n e r a t h f o r t h e i n i t i a l g u i d a n c e g i v e n t o m e t o s t a r t t h i s r e s e a r c h p r o j e c t . T o M r . S a r a t h W e e r a w a r a n a k u l a , H e a d , D e p a r t m e n t o f E a r t h R e s o u r c e E n g i n e e r i n g . I w o u l d l i k e t o e x p r e s s m y s i n c e r e t h a n k s f o r t h e h e l p g i v e n t o m e t h r o u g h o u t t h e p e r i o d o f s t u d y a n d c o n t r i b u t e d t o t h i s t h e s i s w i t h t h e h e l p f u l h i n t s a n d s u g g e s t i o n s . T o P r o f . P . G . R . D h a r m a r a t n e , D e p a r t m e n t o f E a r t h R e s o u r c e E n g i n e e r i n g , U n i v e r s i t y o f M o r a t u w a . I w o u l d l i k e t o t h a n k s f o r t h e h e l p f u l s u g g e s t i o n s h e h a s m a d e d u r i n g t h e r e v i e w o f t h e r e s e a r c h p r o j e c t a n d t h e s i s w r i t i n g . T o t h e s t a f f a t t a c h e d t o U n i v e r s i t y o f M o r a t u w a , e s p e c i a l l y t o D e p a r t m e n t o f E a r t h R e s o u r c e E n g i n e e r i n g , I w o u l d l i k e t o e x p r e s s m y g r a t i t u d e f o r a l l t h e s u p p o r t g i v e n t o m e d u r i n g m y r e s e a r c h w o r k . M u c h o f t h e d a t a i n p u t w e r e p e r f o r m e d b y t h e G I S B r a n c h i n S u r v e y D e p a r t m e n t u n d e r t h e . . . s u p e r v i s i o n o f M r . D . N . D . H e t t i a r a c h c h i , S u p e r i n t e n d e n t o f S u r v e y s ( G I S ) ; t h e i r c o l l e c t i v e e f f o r t s a r e w a r m l y a p p r e c i a t e d . T o M r . A . D i s s a n a y a k e , S u p e r i n t e n d e n t o f S u r v e y s , K e g a l l e D i s t r i c t , a n d h i s s u r v e y o r s ; I m u s t a c k n o w l e d g e a l l h e l p g i v e n t h r o u g h t h e f i e l d d a t a c o l l e c t i o n . T o M r . S . D . P J . D a m p e g a m a , S u p e r i n t e n d e n t o f S u r v e y s ( G e o d e t i c S u r v e y s ) , I n s t i t u t e o f S u r v e y i n g a n d M a p p i n g , D i y a t a l a w a a n d h i s s u r v e y o r s a n d s t a f f , I m u s t t h a n k a l l t h e h e l p s g i v e n t h r o u g h t h e G P S o b s e r v a t i o n s a n d t h e s u p p o r t g i v e n a t a l l s t a g e s o f t h e r e s e a r c h p r o j e c t s . F o r t h e u s e f u l a n d n e c e s s a r y i n f o r m a t i o n a n d e x p l a n a t i o n r e g a r d i n g A~c/lnfo d a t a s t r u c t u r e a n d p r o g r a m m i n g , I g i v e m y t h a n k s t o M r . S . S i v a n a n t h a r a j a h , A s s t . S u p e r i n t e n d e n t o f S u r v e y s , C e n t e r f o r R e m o t e S e n s i n g . P a g e i i C o m m u n i c a t i n g d a t a q u a l i t y i n a G I S e n v i r o n m e n t F o r h e l p f u l c r i t i c i s m a n d s u g g e s t i o n s g i v e n t o m e f o r t h e i m p r o v e m e n t o f t h e r e s e a r c h , I w o u l d l i k e t o t h a n k M r . K . D . P a r a k k u m S h a n t h a , S u p e r i n t e n d e n t o f S u r v e y s ( A i r S u r v e y s ) a n d h i s w i f e M r s . S h a m i l y P a r a k k u m S h a n t h a , S u p e r i n t e n d e n t o f S u r v e y s . 1 w o u l d l i k e t o t h a n k t h e S u r v e y o r G e n e r a l a n d S u r v e y D e p a r t m e n t f o r s u p p o r t i n g t h i s r e s e a r c h a n d r e l e a s i n g t h e d i g i t a l d a t a a n d r e s o u r c e s . T h i s r e s e a r c h w a s f u n d e d b y A D B . I g r a t e f u l l y a c k n o w l e d g e t h e _ f e l l o w s h i p p r o v i d e d m e b y A D B . T h a n k y o u t o a l l c o n c e r n e d . K . T h a v a l i n g a m P a g e i i i C o m m u n i c a t i n g d a t a q u a l i t y i n a G I S e n v i r o n m e n t A b s t r a c t A c k n o w l e d g e m e n t s T a b l e o f C o n t e n t s L i s t o f A n n e x e s L i s t o f F i g u r e s L i s t o f T a b l e s C H A P T E R 1 I n t r o d u c t i o n 1 . 1 R e s e a r c h o b j e c t i v e s T a b l e o f C o n t e n t s 1 . 2 D i g i t a l S p a t i a l D a t a s e t s u s e d 1 . 3 R e s e a r c h m e t h o d o l o g y 1 . 4 T h e s i s s t r u c t u r e C H A P T E R 2 E r r o r s i n s p a t i a l d a t a b a s e c r e a t i o n 2 . 1 S o u r c e o f E r r o r s 2 . 2 D e t a i l s o f S o u r c e o f E r r o r s 2 . 2 . 1 2 . 2 . 2 2 . 2 . 3 O b v i o u s s o u r c e o f E r r o r s E r r o r s r e s u l t i n g f r o m n a t u r a l v a r i a t i o n s I o r i g i n a l m e a s u r e m e n t s E r r o r s a r i s i n g t h r o u g h p r o c e s s i n g i i i v v i i v i i i I X 1 4 4 4 5 7 7 8 8 1 0 1 1 2 . 2 . 3 . 1 N u m e r i c a l e r r o r s i n t h e c o m p u t e r 1 1 2 . 2 . 3 . 2 F a u l t s a r i s i n g t h r o u g h t o p o l o g i c a l a n a l y s e s 1 2 2 . 2 . 3 . 2 . 1 P r o b l e m s a n d e r r o r s a r i s i n g f r o m o v e r l a y b o u n d a r i e s 1 2 2 . 3 P r o b l e m i n c o m b i n i n g a l l E r r o r s 1 8 C H A P T E R 3 D a t a Q u a l i t y E v a l u a t i o n i n a G I S 1 9 3 . 1 C o m p o n e n t s o f D a t a Q u a l i t y • 1 9 3 . 2 P o s i t i o n a l A c c u r a c y 2 0 3 . 2 . 1 T h e m e t h o d s o f d e t e r m i n i n g P o s i t i o n a l A c c u r a c y 2 1 3 . 3 T h e p r o p o s e d m e t h o d o l o g y t o d e t e r m i n e t h e P o s i t i o n a l A c c u r a c y 2 6 3 . 3 . 1 I d e n t i f y C o n t r o l D a t a 2 7 3 . 3 . 2 D e t e r m i n a t i o n o f t h e S a m p l e f o r C o m p a r i s o n 2 7 3 . 3 . 3 3 . 3 . 4 3 . 3 . 5 3 . 3 . 6 D e r i v a t i o n o f s t a t i s t i c p a r a m e t e r s C h e c k i n g f o r e x i s t e n c e o f G r o s s E r r o r s ( B l u n d e r s ) H y p o t h e s i s t e s t i n g o n c o m p u t e d s t a t i s t i c s T e s t f o r t h e e x i s t e n c e o f s i g n i f i c a n c e b i a s e r r o r P a g e I V 2 8 2 9 2 9 3 0 C o m m u n i c a t i n g d a t a q u a l i t y i n a G I S e n v i r o n m e n t 3 . 3 . 7 T e s t f o r t h e p r e c i s i o n 3 1 3 . 3 . 8 C o m p u t a t i o n o f c o v a r i a n c e t o e v a l u a t e s t h e t e n d e n c y o f c o r r e l a t i o n 3 2 3 . 3 . 9 S t a t i s t i c p a r a m e t e r - R o o t M e a n S q u a r e E r r o r ( R M S E ) 3 3 3 . 4 P o s i t i o n a l A c c u r a c y a s s e s s m e n t f o r l i n e a r a n d a r e a l d a t a 3 . 4 . 1 E p s i l o n b a n d a p p r o a c h 3 . 4 . 2 D i s t a n c e b u f f e r i n g m e t h o d t o d e t e r m i n e t h e a c c u r a c y 3 . 4 . 3 " R u l e o f t h u m b " a p p r o a c h t o d e t e r m i n e t h e a c c u r a c y 3 . 5 A t t r i b u t e A c c u r a c y 3 . 5 . 1 T h e m e t h o d s o f d e t e r m i n i n g a t t r i b u t e a c c u r a c y 3 . 5 . 2 A t t r i b u t e A c c u r a c y A s s e s s m e n t - D e s c r i p t i v e L a b e l 3 . 5 . 3 A t t r i b u t e A c c u r a c y A s s e s s m e n t - N o m i n a l a t t r i b u t e s 3 . 5 . 4 A t t r i b u t e A c c u r a c y A s s e s s m e n t - I n t e r v a l a t t r i b u t e s 3 . 6 C o m p l e t e n e s s 3 . 6 . 1 D a t a c o m p l e t e n e s s a n d M o d e l c o m p l e t e n e s s 3 . 6 . 2 T e s t i n g o f c o m p l e t e n e s s 3 . 6 . 3 S o u r c e o f i n c o m p l e t e n e s s 3 . 6 . 4 E x a m p l e 3 . 7 L o g i c a l c o n s i s t e n c y 3 . 8 L i n e a g e 3 . 8 . 1 P u r p o s e o f ' L i n e a g e ' I n f o r m a t i o n C H A P T E R 4 D e t e r m i n a t i o n o f t h e D a t a Q u a l i t y o f t h e D a t a s e t s 3 3 3 3 3 5 3 7 3 8 3 8 3 9 4 0 4 6 4 6 4 8 4 8 4 9 5 0 5 0 5 3 5 3 5 7 4 . 1 P r o d u c t i o n o f t h e d a t a s e t 5 7 4 . 2 F i e l d D a t a C o l l e c t i o n 5 7 4 . 3 D e t e r m i n a t i o n o f P o s i t i o n a l A c c u r a c y 5 8 4 . 3 . 1 P o s i t i o n a l a c c u r a c y a s s e s s m e n t p r o c e d u r e s - P o i n t E n t i t y 5 8 4 . 3 . 2 P o s i t i o n a l a c c u r a c y a s s e s s m e n t p r o c e d u r e s - l i n e a r d a t a 6 0 4 . 3 . 2 . 1 R u l e o f T h u m p A p p r o a c h - l i n e . e n t i t y 6 0 4 . 3 . 2 . 2 E r r o r ( E p s i l o n ) B a n d A p p r o a c h - l i n e e n t i t y 6 1 4 . 3 . 2 . 3 P o s i t i o n a l A c c u r a c y A s s e s s m e n t - d i r e c t m e a s u r e m e n t 6 2 4 . 3 . 2 . 4 P o s i t i o n a l A c c u r a c y A s s e s s m e n t - d i s t a n c e b u f f e r i n g 6 3 4 . 3 . 3 P o s i t i o n a l a c c u r a c y a s s e s s m e n t p r o c e d u r e s - a r e a l d a t a 6 4 4 . 4 D e t e r m i n a t i o n o f A t t r i b u t e A c c u r a c y 6 5 4 . 4 . 1 A c c u r a c y o f I n t e r v a l A t t r i b u t e s 6 5 4 . 4 . 2 A c c u r a c y o f D e s c r i p t i v e ! a b e l 6 6 4 . 4 . 3 A t t r i b u t e a c c u r a c y a s s e s s m e n t f o r n o m i n a l d a t a 6 7 4 . 4 . 3 . 1 T h e p r o c e d u r e s f o r a t t r i b u t e a c c u r a c y a s s e s s m e n t 6 7 P a g e v -C o m m u n i c a t i n g d a t a q u a l i t y i n a G I S e n v i r o n m e n t 4 . 5 L i n e a g e 6 8 4 . 6 L o g i c a l c o n s i s t e n c y 6 8 4 . 7 C o m p l e t e n e s s 6 8 C H A P T E R 5 M e t h o d s o f r e p r e s e n t i n g t h e D a t a Q u a l i t y 7 1 5 . 1 M e t h o d s o f r e p r e s e n t i n g p o s i t i o n a l a c c u r a c y 7 1 5 . 2 M e t h o d s o f r e p r e s e n t i n g a t t r i b u t e a c c u r a c y 7 4 5 . 3 C o m p l e t e n e s s 7 5 5 . 4 L o g i c a l c o n s i s t e n c y 7 5 5 . 5 L i n e a g e 7 5 5 . 6 C a r t o g r a p h i c m e t h o d s f o r d a t a a n d i n f o r m a t i o n q u a l i t y r e p r e s e n t a t i o n 7 5 C H A P T E R 6 I m p l e m e n t i n g D a t a Q u a l i t y 8 0 6 . 1 T h e p r o p o s e d a p p r o a c h 8 0 6 . 2 C h o o s i n g t h e G I S 8 0 6 . 3 A c q u i r i n g d a t a s e t s 8 1 6 . 4 D e t e r m i n e a n d s t o r a g e D a t a Q u a l i t y 8 1 6 . 5 D a t a Q u a l i t y P r e s e n t a t i o n 8 1 C H A P T E R 7 C o n c l u s i o n s a n d R e c o m m e n d a t i o n s 8 2 7 . 1 C o n c l u s i o n s 8 2 7 . 2 R e c o m m e n d a t i o n s f o r f u t u r e 8 4 A n n e x e s R e f e r e n c e s P a g e V 1 X L I V C o m m u n i c a t i n g d a t a q u a l i t y i n a G I S e n v i r o n m e n t L i s t o f A n n e x e s A n n e x 1 G e n e r a l l o c a t i o n o f c a s e s t u d y a r e a I A n n e x 2 F l o w d i a g r a m I I A n n e x 3 . 1 A s s e s s m e n t o f C o m p l e t e n e s s - S h e e t N o . 5 3 / 2 3 V I I I A n n e x 3 . 2 A s s e s s m e n t o f C o m p l e t e n e s s - S h e e t N o . 5 3 / 2 4 X A n n e x 4 . 1 A s s e s s m e n t o f P o s i t i o n a l A c c u r a c y - p o i n t e n t i t y - S h e e t 5 3 / 2 3 X I I A n n e x 4 . 2 A s s e s s m e n t o f P o s i t i o n a l A c c u r a c y - p o i n t e n t i t y - S h e e t 5 3 / 2 4 X I V A n n e x 4 . 3 A s s e s s m e n t o f P o s i t i o n a l A c c u r a c y - p o i n t e n t i t y - S h e e t 5 3 / 2 3 & 2 4 X V I A n n e x 4 . 4 A s s e s s m e n t o f P o s i t i o n a l A c c u r a c y - p o i n t e n t i t y - S h e e t 5 3 X V I I I A n n e x 4 . 5 A s s e s s m e n t o f P o s i t i o n a l A c c u r a c y - l i n e e n t i t y X X A n n e x 4 . 6 A s s e s s m e n t o f P o s i t i o n a l A c c u r a c y - p o l y g o n e n t i t y X X V A n n e x 5 . 1 A s s e s s m e n t o f A t t r i b u t e A c c u r a c y - i n t e N a l d a t a - S h e e t 5 3 / 2 4 X X V I A n n e x 5 . 2 A s s e s s m e n t o f A t t r i b u t e A c c u r a c y - i n t e N a l d a t a - S h e e t 5 3 / 2 3 X X V I I I A n n e x 5 . 3 A s s e s s m e n t o f A t t r i b u t e A c c u r a c y - i n t e N a l d a t a - S h e e t 5 3 / 2 3 & 2 4 X X X A n n e x 5 . 4 A s s e s s m e n t o f A t t r i b u t e A c c u r a c y - d e s c r i p t i v e l a b e l X X X I I A n n e x 5 . 5 A s s e s s m e n t o f A t t r i b u t e A c c u r a c y - n o m i n a l d a t a X X X I I I A n n e x 6 . 1 P o s i t i o n a l A c c u r a c y R e p o r t X X X I V A n n e x 6 . 2 A t t r i b u t e A c c u r a c y R e p o r t X X X V I I I A n n e x 6 . 3 C o m p l e t e n e s s R e p o r t X L A n n e x 6 . 4 L o g i c a l c o n s i s t e n c y R e p o r t X L I A n n e x 6 . 5 L i n e a g e R e p o r t X L I I P a g e v n - C o m m u n i c a t i n g d a t a q u a l i t y i n a G I S e n v i r o n m e n t L i s t o f F i g u r e s 2 . 1 : C e n t r a l G r i d c e l l c o d i n g 1 3 2 . 2 a : D i g i t i z a t i o n 1 5 2 . 2 b : E r r o r z o n e s w i t h e p s i l o n t o l e r a n c e 1 6 2 . 3 : S p u r i o u s P o l y g o n s 1 7 2 . 4 : A C l a s s i f i c a t i o n o f e r r o r i n S p a t i a l D a t a b a s e 1 8 3 . 1 : D e t e r m i n a t i o n o f p o s i t i o n o f t h e t e l e p h o n e p o l e 2 3 3 . 2 : C o m p a r i s o n o f a n e d i t p l o t t o s o u r c e m a p 2 6 3 . 3 : E r r o r b a n d a r e a a c c o r d i n g t o c a s p a r y 3 4 3 . 4 : D i s t a n c e b u f f e r a r o u n d e n t i t i e s 3 6 3 . 5 : L i n e d e f i n e s b y f o u r C o o r d i n a t e p a i r s 3 7 5 . 1 E r r o r E l l i p s e s u s e d t o s h o w p o s i t i o n a l a c c u r a c y 7 3 5 . 2 U s e o f c u r v e s t o s h o w p o s i t i o n a l a c c u r a c y o f a l i n e 7 3 5 . 3 U s e o f b a r c h a r t s t o s h o w a t t r i b u t e a c c u r a c y 7 4 5 . 4 S i m p l e r e l i a b i l i t y d i a g r a m 7 5 5 . 5 G e o m e t r i c r e l i a b i l i t y d i a g r a m 7 6 5 . 6 L i n e a g e p r e s e n t a t i o n u s i n g q u a l i t y o v e r l a y 7 6 5 . 7 Q u a l i t y O v e r l a y i n t h e f o r m o f i s o l i n e s 7 7 5 . 8 U s i n g t h e E p s i l o n b a n d c o n c e p t s t o s h o w e r r o r i n e l e v a t i o n 7 8 5 . 9 P r e s e n t i n g t h e q u a l i t y o f i n d i v i d u a l f e a t u r e 7 / J 5 . 1 0 C r o p s u i t a b i l i t y I n f o r m a t i o n . 7 9 6 . 1 T h e P r o p o s e d A p p r o a c h 8 0 P a g e v i i i - . . C o m m u n i c a t i n g d a t a q u a l i t y i n a G I S e n v i r o n m e n t L i s t o f T a b l e s 3 . 1 R e s u l t s f r o m t h e B u f f e r / C l i p P r o n e 3 6 3 . 2 C l a s s i f i c a t i o n E r r o r M a t r i x 4 0 3 . 3 C l a s s i f i c a t i o n E r r o r M a t r i x 4 3 3 . 4 N o r m a l i z e d M a t r i x 4 4 3 . 5 C a l c u l a t i o n o f t h e K a p p a C o e f f i c i e n t 4 5 3 . 6 C o m p a r i s o n o f t h e r e s u l t s 4 6 3 . 7 F e a t u r e C o m p l e t i o n s 4 8 4 . 1 P o s i t i o n a l A c c u r a c y S u m m a r y d a t a - p o i n t e n t i t y 5 8 4 . 2 A c c u r a c y o f t h e l i n e S e g m e n t s 6 0 4 . 2 a A c c u r a c y o f r o a d s 6 1 4 . 3 A c c u r a c y o f l i n e S e g m e n t s ( E p s i l o n B a n d ) 6 1 4 . 3 a A c c u r a c y o f r o a d s ( E p s i l o n B a n d ) 6 2 4 . 4 D i r e c t C o m p a r i s o n w i t h f i e l d m e a s u r e m e n t s 6 2 4 . 5 D i s t a n c e b u f f e r i n g m e t h o d 6 3 4 . 6 E r r o r ( E p s i l o n ) B a n d A p p r o a c h - A r e a l d a t e 6 4 4 . 7 A c c u r a c y o f I n t e r v a l A t t r i b u t e s 6 6 4 . 8 C l a s s a c c u r a c y t e s t r e s u l t s 6 7 4 . 9 C o m p l e t e n e s s o f t o p o g r a p h i c d a t a 6 9 4 . 1 0 C o m p l e t e n e s s o f l a n d u s e 6 9 5 . 1 P o s i t i o n a c c u r a c y - m e t a i n f o r m a t i o n f o r a l l i n d i v i d u a l o b j e c t s 7 2 P a g e I X -