The thesis majors at: Camera surveillance systems and related works; the hand – off camera techniques in surveillance camera systems with multiple cameras; the anomaly detection techniques in video surveillance.
MINISTRY OF EDUCATION VIETNAM ACADEMY OF AND TRAINING SCIENCE AND TECHNOLOGY GRADUATE UNIVERSITY SCIENCE AND TECHNOLOGY NGÔ ĐỨC VĨNH TECHNIQUES TO PROCESS OBSERVED REGIONS AND DETECT ABNORMAL OBJECTS IN VIDEO SURVEILLANCE SYSTEMS Major: Mathematical basis for informatics Code: 62.46.01.10 SUMMARY OF MATHEMATICAL DOCTORAL THESIS Hanoi – 2016 INTRODUCTION The urgency of the thesis Nowadays, camera surveillance systems become popular and used widely in many fields With the traditional systems, video streams are observed by observers in real time, they can process as quick as problems founded Directly process video streams are quite a big problem, due to the large number of cameras with a huge of database which will lead to leave out important scenes in a video stream For these reasons, auto surveillance is the first job in video surveillance system This can help observers in controlling, observation and reduce errors Automatic systems can surveillance work without human interaction from the element (moving detection) to upper (event detection, behaviour detection) One of problems needs to solve in multi-camera surveillance system is the appearance and disappearance of an object from one camera to the other, this called finding forward camera Finding the next camera is the most important work in continuous tracking an object in a multi camera surveillance system Many projects for solving continuous tracking object when its travel pass by the camera, most of them pay attention to establishing the relation of an object in one camera and the forward camera It means that most projects, compare objects in the intersection of the observation zone of cameras in a 2D environment How to define the time and forward camera to get the continuous tracking? The researcher is working to find the answer for this question Finding forward camera requests a huge task: define time and define the next camera, transfer the object So, with the aim of strengthening the power of the system, changing camera should be at least, this is researched and given detail in chapter 2 The objectives of the thesis The thesis majors at: First: Camera surveillance systems and related works; Second: The hand – off camera techniques in surveillance camera systems with multiple cameras; Third: The anomaly detection techniques in video surveillance The new contributions of the thesis Main results of the thesis: Recommend a technique on partitioning the static observation region in a camera surveillance system on the geometric relationship between the observed regions of cameras Through the reduction of polygon edges, the recommended technique helps to reduce the time to transfer camera in the Overlapping system This technique was published in the Journal of Information Technology and Communications in 2014; Raise up a new technique to hand - off camera based on virtual line, this effect on detect the right time to change camera by calculating impact of moving object with virtual line in 3D environment Proposal technique was published at Vietnamese Journal of Science and Technology in 2013 Propose a technique to select forwarding camera This base on the movements of objects to reduce time to transition surveillance camera with the aim of improving the performance of the systems Proposal technique was presented and published in Fundamental and Applied Information Research– FAIR 2013 Raise up an abnormal detection technique based on segmentation the criteria of each route The result shows that the proposed technique could detect abnormal while the object has not finished its orbit, it means that the object still in the video This technique really helps in real time surveillance and published in Journal of Informatics and Communication 2015 Structure of the thesis The thesis includes Introduction, Summary and main chapters Chapter 1: Overview on hand-off camera and abnormal in camera surveillance systems In this chapter, we mention on the overview of the camera surveillance system and related works Chapter 2: Some techniques to hand-off camera Propose techniques to find forward camera with the aim of reducing times to choose next camera in tracking object Chapter 3: Detect abnormal based on orbital in video surveillance This chapter also gives a brief on approaches and techniques to detect abnormal in video surveillance and propose a technique to detect abnormal based on analysis the moving orbital of an object CHAPTER 1: OVERVIEW ON HAND – OFF CAMERA AND DETECT ABNORMAL IN CAMERA SURVEILLANCE SYSTEMS 1.1 Camera surveillance system In this part, the thesis present a general introduction on camera surveillance and theirs basics problems 1.2 Hand - off camera and detect abnormal Approaches to solve problems in camera surveillance: hand – off camera and detect abmormal in video surveillance are shown here 1.3 Summary and researches In this chapter, the thesis shown the overview on camera surveillance system and its ralated works Beside these, thesis also gives an introduction on some approaches in tracking object in multicameras system In this subject I pay most attention at two importance problems that have been applied in many fields: hand – off camera and detect abnormal in surveillance system CHAPTER 2: SOME TECHNIQUES TO HANDLE REGION OBSERVATION IN HAND – OFF CAMERA In this chapter, thesis presents three proposals related to: When we need to find forward camera? And How does camera get the tracking job? These proposals aim at reduce the calculation in choosing forward camera and enhance the power of the system 2.1 Introduction 2.2 Partion observation zone 2.2.1 Introduction In this part, a technique to divide the observation zone into non intersection parts in 2D environment (Fig.2b) (a) Division in 1D environment (b) Division in 2D environment Fig2.1 Some division methods 2.2.2 Intersection of two polygons Definiton 2.1 [Observation polygon] Observation polygon is the projected zone of observed area to the 2D plane Definition 2.2[Intersection point of two intersection polygons] A point is called intersection point of two polygon (A and B) if it is the intersection of an edge of polygon A and the other of B This point is neither vertex of polygon A nor B Definition 2.3[Single intersection] Having two observation polygons: A and B Intersection between A and B is called single intersection if it is convex and the rest of either A or B is a polygon (a) (b) (c) Fig2.2 Types of two polygons intersection a) Non intersection; b) Single intersection; c) Intersection Proposition 2.1 If two observation polygons A and B have a single intersection, number of intersection point can not be exceed 2.2.3 Divide observation zone of camera surveillance system 2.2.3.1 Divide intersection zone of two polygons Proposition 2.2[Divide two polygons] Let two observation polygons A and B Their intesection is single if there exist intersection points These points create an intersection edge that formed min-edges polygons (Fig.2.4) Fig2.22 Divide the intersection between two polygons 2.2.3.2 Division of observed zone in multi-cameras surveillance system In an working observation with n static cameras with information of observation zone, these observation polygons are overlapped and single intersection We divide observation zone of the system into class of observation polygon of each camera, these polygons are non-intersect PartitionTwoPolygon Function: Partition two intersecting polygons so that the edge of each polygon after separation is minimal Input: A=(A[1], A[2], , A[n]); B=(B[1], B[2], , B[m]); vertex A[i], B[j] Output: Polygon X and Y, satisfiy:𝑨 ∪ 𝑩 = 𝑿 ∪ 𝒀 in which 𝑨 ∪ 𝑿 ∩ 𝒀 = ∅; 𝑿 ⊆ 𝑨; 𝒀 ⊆ 𝑩; Pseudocode partitionTwoPolygon (A, B: polygon) {Find the Subtraction of P(P[1], P[2], , P[t]) = A\B Find intersection of each pair in A and B; P[h], P[k] (h< k< t) For i = to h 𝑋 = 𝑋 ∪ 𝑃[𝑖]; For i = k to t 𝑋 = 𝑋 ∪ 𝑃[𝑖]; Y = B – X; A = X; B = Y;} PartitionFOV Algorithm Input: Observation zone 𝑷 = {𝑷[𝟏], 𝑷[𝟐], , 𝑷[𝒏]}(with n, integer) Where, 𝑷[𝒊] = {𝑽𝟏 , 𝑽𝟐 , 𝑽𝒕 }, with vertex Vk (xk,yk) , these vertex are sorted clockwise Output: Q=(Q[1], Q[2], , Q[n]) satify: P in1 Q[i ] ; in which: 𝐐[𝐢] ∩ 𝐐[𝐣] = ∅ (∀𝒊, 𝒋 ∈ 𝟏 𝒏) and 𝐐[𝐢] ∈ 𝐏[𝐢](∀𝒊 ∈ 𝟏 𝒏) Pseudocode Add the information of observed zone of n camera: P[i] (i=1 n) Q[]={0}; Q[1]=P[1]; i=1; j=1; While (in){ i=i+1; T=P[i]; k=1; While (k ℎ𝑅 Abnormal concept In zone mornitoring by a camera, objects (human) travel along normal orbits created by certain groups (route) In this thesis, object has abnormal behaviour is an object with abnormal orbit which does not belong to any routes In the other word, this is an abnormal orbit with all the routes 20 3.3 Orbit clustering Thesis uses the rate of speed changing to cluster The clustering point is the point where the rate 𝑟𝑎𝑡𝑒(𝑣𝑖 )is pass threshold 𝜗 𝑦 𝑟𝑎𝑡𝑒(𝑣𝑖 ) = 𝑚𝑖𝑛 ( 𝑦 𝑥 𝑣𝑖𝑥 − 𝑣𝑖−1 𝑣 −𝑣 , 𝑖 𝑦 𝑖−1 ) 𝑥 𝑣𝑖−1 𝑣𝑖−1 (3.13) 𝑦 where, 𝑣𝑖𝑥 , 𝑣𝑖 the corresponding speed by direction x and y, it is the distance between two points in the same time: 𝑣𝑖𝑥 = 𝑥𝑖 − 𝑥𝑖−1 𝑦 and 𝑣𝑖 = 𝑦𝑖 − 𝑦𝑖−1 Theorem 3.1 [Abnormal detection based on the subtrajectory] Let P be the representative of route R: 𝑃 = {𝑝1 , 𝑝2 … , 𝑝𝑛 } with 𝑠𝑒𝑔 = {𝑠𝑒𝑔1 , 𝑠𝑒𝑔2 , , 𝑠𝑒𝑔𝑢 } is the segmentation point P (1 < 𝑢 < 𝑛) T* : being checked orbit If T* is defined abnormal with an orbit i(1 ≤ 𝑖 ≤ 𝑢) then T* is abnormal with all orbit 𝑙(𝑖 < 𝑙 ≤ 𝑢) 3.4 Abnoraml detection base on clustering the route In this part, the thesis proposes a phase technique in order to detect abnormalities in video surveillance base on the clustering route (figure 3.5) 21 Fig 3.5 Workflow to detect abnormal base on clustering route First pharse: Initialization Symbol: 𝑅 = {𝑅1 , 𝑅2 , … , 𝑅𝑘 } set of normal route; 𝑟𝑖 number of orbits 𝑅𝑖 with (1 ≤ 𝑖); 𝑂𝑗𝑖 is trajectories of route 𝑅𝑖 ; (1 ≤ 𝑖 ≤ 𝑘), (1 ≤ 𝑗 ≤ 𝑟𝑖 ) 𝑃 = {𝑃1 , 𝑃2 , … , 𝑃𝑘 } set of representation routes 𝑃𝑖 representation route of 𝑅𝑖 ; 𝑆𝑂𝑗𝑖 : orbit number j of representative 𝑃𝑖 ; 𝑆𝑂𝑗𝑖 = {𝑃𝑖 (𝑝1 , 𝑝2 , 𝑝𝑠𝑒𝑔𝑗 )} Step 1: Create group of orbits of the same route Step 2: Build a representative for each route Step 3: Calculate 𝑑𝑚𝑎𝑥 Threshold 𝑑𝑚𝑎𝑥 is calculate by: 𝑑𝑚𝑎𝑥 = { max {ℎ(𝑂𝑗𝑖 , 𝑃𝑖 )}} 𝑖=1 𝑘 𝑗=1 𝑟𝑖 Step 4: Segmentation representations of the route Second pharse: (3.14) 22 Detect abnormal base on the representative of route Algorithm: Abnormal Detecter Based on Sub – Trajectories of Route (ADB-STR) Input: 𝑢𝑚𝑎𝑥: number of max orbits of all route k: number of route 𝑑𝑚𝑎𝑥 : Threshold {𝑆𝑂𝑗𝑖 }(𝑖 = 𝑘); (𝑗 = 𝑢𝑚𝑎𝑥): set of orbits T*: orbit need to check Output: Abnormal or not Pseudocode: j=1; Abnormal=false; While (𝑗 ≤ 𝑢𝑚𝑎𝑥 and Abnormal=false) 𝑑 = (ℎ(𝑇 ∗ , 𝑆𝑂𝑗𝑖 )); 𝑖=1 𝑘 if (𝑑 > 𝑑max ) then Abnormal=true; j=j+1; End while; Evaluate computational complexity For each j(𝑗 ≤ 𝑢𝑚𝑎𝑥), we have to find d, the smallest similarity number of orbit T* with the other orbit of k route In general times to find d is 𝑢𝑚𝑎𝑥 × 𝑢𝑚𝑎𝑥 × 𝑘, so computational complexity of the algorithm is ADB-STR O(𝑢𝑚𝑎𝑥 × 𝑘) 3.5 Experiment To verify the proposal technique, we an experiment with the database of orbits by Piciarelli built in 2008 and data collected form camera surveillance Experiment result show that proposal 23 technique can detect abnormal event when object does not finish its orbit This really useful when applied the system in real time 3.6 Summary chapter In this chapter, thesis proposes a technique to detect abnormal base on the segmentation of representative route Proposal technique base on the affected of route to the object So, by combining similarity and segmentation of representative route, proposal can detect abnormal event when the object does not finish its orbit This was published in Journal of Information and Communication 2015 THESIS SUMMARY Research on camera surveillance usually received much attention due to its highly application In which, issue on the hand off camera to continuous tracking object when they pass by camera observation zone And one more importance problem with video surveillance detects abnormal movement The thesis expresses overview of video surveillance systems with research on how to solve the problem in finding the next observation camera and detect abnormalities in video surveillance system Based on surveys and experiments the dissertation has contributed following: Recommend a technique on partitioning the static observation region in a camera surveillance system on the geometric relationship between the observed regions of cameras Through the reduction of polygon edges, the recommended technique helps to reduce the time to transfer camera in the Overlapping system Raise up a new technique to find next camera based on virtual line, this effect on detecting the right time to change camera by 24 calculating the impact of moving objects with virtual line in a 3D environment Propose a technique to select forwarding camera This base on the movements of objects to reduce time to transition surveillance camera with the aim of improving the performance of the systems Raise up an abnormal detection technique based on segmentation the criteria of each route The result shows that the proposed technique could detect abnormal while the object has not finished its circulation, it means that the object still in the video This technique really helps in real time surveillance Further research issues: Research on detect abnormal base on orbit, using image processing with moving orbit of an object after tracking it Applied the research result on specific problems LIST OF PUBLISHING RELATED TO THESIS Ngơ Đức Vĩnh, Đỗ Năng Tồn, Hà Mạnh Toàn (2010), “Một tiếp cận phát mặt người trợ giúp camera”, Tạp chí Khoa học Công nghệ, ĐH Công nghiệp Hà Nội, ISSN 1859 – 3585, số 3.2010, tr 20 – 24 Ngơ Đức Vĩnh, Đỗ Năng Tồn (2013) “Một cách tiếp cận giải việc chuyển tiếp camera hệ thống giám sát tự động”, Tạp chí Khoa học công nghệ, Viện hàn lâm Khoa học Công nghệ Việt Nam, tập 51, số 3, 2013, tr 279 – 292 Ngơ Đức Vĩnh, Đỗ Năng Tồn (2013) “Một thuật toán lựa chọn camera hệ thống giám sát tự động” Kỷ yếu Hội nghị khoa học Quốc gia lần thứ VI: Nghiên cứu ứng dụng CNTT (FAIR 2013), tr 341 – 347 Ngô Đức Vĩnh, Đỗ Năng Toàn (2014) “Một kỹ thuật phân chia vùng quan sát camera hệ thống giám sát tự động” Chun san cơng trình nghiên cứu, phát triển ứng dụng Công nghệ thông tin Truyền thơng, Tạp chí Cơng nghệ thơng tin truyền thông, tập V – 1, số 12 (32), tr 53 - 60 Ngơ Đức Vĩnh, Đỗ Năng Tồn (2015) “Một thuật toán phát bất thường dựa vào quỹ đạo giám sát video” Chun san cơng trình nghiên cứu, phát triển ứng dụng Công nghệ thông tin Truyền thơng, Tạp chí Cơng nghệ thơng tin truyền thông, kỳ 3, tập V – 1, số 14 (34), tr.5 - 12 Duc Vinh Ngo, Nang Toan Do, Luong Anh Tuan Nguyen (2016) "Anomaly Detection in Video Surveillance: A Novel Approach Based on Sub-Trajectory", The 15th IEEE International Conference on Electronics, Information, and Communication (ICEIC 2016), Jan – 2016, pp 272-275 ... camera in tracking object Chapter 3: Detect abnormal based on orbital in video surveillance This chapter also gives a brief on approaches and techniques to detect abnormal in video surveillance and. .. TRAJECTORY IN VIDEO SURVEILLANCE In this chapter, thesis presents some approaches to detect abnormal in video surveillance Then propose a technique to detect abnormal based on the moving trajectory... Function findIntersectPolygon: find intersection point C of AB and edge of polygon P Input: P=(P[1], P[2], , P[n]); Vertex A, B Output: Point C, Intersection point of AB and an edge in P, satisfy: