ứng dụng camer trong nhận dạng và điều khiển rô bốt tự hành

NGHIEN c f l u - TRAO D6I CfiMEi^fl TRONG NH6N D$NG Vfi DIEU KHIEN RO BOT TCT H^NM (JTNG D U N G Nguyen Trpng Doanh, Ngd Vdn Dang, Nguyin Dang Tdi Vien CP khi, Trfldng Dai hpc Bdeh khpa Hd Ndi T6M TAT Trong nhieu ndm qua, chung toi da tap trung nghien cau vd phdt trien robot ta hdnh Me_ShrimpVN theo mdu Robot Shrimp III, mgt dang rdbot co khd ndng vUdt chUdng ngai vdt rat linh hoat, co khd ndng ang dung rgng thac te De robot co khd ndng ang dung cdc cong tdc khdo sdt, thdm do, ladng, cau hg cdc moi trudng dgc hqi, nguy hiem, ta phdi trang bi cho robot cdc cdm bien de ndng mUc "thong minh" cho robot Mgt thiet bi quan trgng nhdt cdc cdm bien Id camera ky thuat sg Tren cd cd cdc thuat todn xa ly hinh dnh, mdy tinhse thu hinh dnh, xd ly vd nhdn dang cdc vdt the, dinh hudng vd xdc dinh khodng cdch tiep can vat thi cho robot Thuat todn xU ly hinh dnh dadc laa chgn tren cd sd cdc thuat todn thong dung, nham xdc dinh bien dang, mdu sdc, tga vd khodng cdch den vdt the Robot Me_ShrimpVN dadc trang bi mgt Webcam ket noi vdi mdy tinh qua cong ket noi khong ddy den mdy tinh Cdc tdc dgngdieu khien sedUdc truyen tU mdy tinh den he thong dieu khien cua robot de tao cdc chuyen dgng can thiet Robot dd tii dgng tim hem dadc cdc vat the cd mdu sdc khdc khong gian Tfl khda: robot ta hdnh, dieu khien, camera so, xU ly hinh dnh, nhdn dgng MO DAU re.nhflngdetaokhdnangnhdn dang khong gian 3D thi khd phflc tap vd thfldng phai sfl dung Van de tap gide quan chp rdbdt ndi chung, hai camera dat song song Ddi vdi camera 3D thi ddc biet la nhflng loai rdbdt tfl hdnh Id mpt dieu ^ j ^ jj- ^inh la gid thdnh Nhflng mdt vdn H van cdn khd phflc tap tren the gidi Rieng d Viet , , ,- » r j^JSri ,_ f , , "^ , ? , , khac khpng kem phan phflc tap, dp la van de xu Nam, dieu de cang tro nen kho khdn vd ehfla , , , , , ,./ ^ , us , v , u J , „ ly dnh vd Iflu trfl da lieu Cpn ngfldi CO the nhan CO nhieu nghien cflu mang tinh dot phd Gide ' ' ,t quan cho rdbdt d ddy khdng chi la trang bi mpt '*'"8 cdc vat the khdng gian 3D mpt cacl chiec camera cho rdbdt, ma la tao cho nd khd dl dang, nhflng rdbdt thi khdng dl nhfl v?y Huiti nang tfl dpng quan sdt, nhan dang hinh dnh vd tfl *"h * " ^'^Vc dfldi dang sd ehfla rat nhi& thdng xfl ly cac tinh hudng Hien tai, tren the gidi cdtdn tin.mdhda dfldi dang pixel, vd mdi hinh dnh tai hai dang camera ky thuat sd trang bi chp rdbdt theo mdu sac vd dp phdn giai cd ehfla Ifldng th6ng la camera 2D vd 3D Cdc camera 2D cd gid thdnh tin can xfl ly rat Idn Q [ TAP CHf CO KHf VIET NAM • Sd (Thang nam 2012) NGHIEN cflu-TRAOD6| Nhfl vay, de xfl ly dfldc hinh dnh va tach vat the thdng qua hinh anh 2D hay 3D Trong qud dfldc thdng tin, ta can biet, tdn kha nhi^u thdi trinh robot di chuyen cac mdi trfldng dpc gian vd dung Ifldng nhd cua may tinh Mpt van hai, nguy hiem vd cfl ly Idn, thi viec truyen dfl lieu de khac khdng kem phan quan trpng la anh sang hinh anh khdng phai luc ndo cung cd the thflc Anh sdng khac thi mau sac tren cac hinh hien dflpc Hien tai, chung tdi mdi ehi trang bi anh thu dfldc cung khac Sau da xfl ly cho rdbdt tfl hdnh Me_ShrimpVN mpt Webcam hinh anh xong, xac dinh dflpc hfldng va tpa dp ddn gian de thu nhdn hinh anh 2D ket hpp vdi eua vat the can tim thi may tinh mdi cd the he thdng truyen dfl lieu khdng day tfl rdbdt den quyet dinh cho rdbdt di chuyen theo hfldng ndo may tinh vd tfl may tinh den camera Tat cd eac vd d^n dau Mdt van de nfla la he thdng Iflu trfl van de tren cd th^ dflpc giai quyet thdng qua ky thdng tin hay bd nhd Sd di eon ngfldi cd the phan thuat "Xfl ly hinh anh" ma camera thu ve de nhan biet dflpc eac vat the la chung ta da cd thdng dang ddi tflpng vd truy tim tfl ddng khdng tin Iflu nao qud trinh tiep xuc vdi gian lam vi^c cua nd Viec tim kiem ddi tfldng se moi trfldng song va qua trinh hoc tap De rdbdt trd nen hieu qua hdn, eon ngfldi khdng tdn cdng ed kha nang nhdn dang, chung ta cung phai thflc sfle tim kiem bang tay ma viec truy tim nhflng ddi hien mdt qua trinh day cho rdbdt phan biet cac tfldng mong muon lai dat xdc suat rat eao QUA TRiNH Xir LY HINH ANH Mdt he thong thu nhan hinh anh cd ban cd cau true nhfl sau: I CanPhan tich anh Thu nhan anh + Nhan dang He quy d|nh Hinh 1: So he thong thu nhdn va xfl ly hinh anh Qud trinh xfl ly hinh dnh Id mdt qud trinh khd phflc tap, nhien, cd the chia thdnh ba bifdc ca ban nhfl sau: quel tren mdy scanner Neu Id hinh dnh tfl thu camera tflong tfl, ta phdi sd hpa de bien ddi tin hieu tflong tfl sang tin hieu so trfldc chuyen sang phdn tich, xfl ly hay Iflu trfl lai + Thu nhan hinh dnh: hinh anh thdng thfldng dflpc thu nhdn qua camera Camera cd the la camera tflpng t\f (CCIR) hoac camera sd (CCD) Tuy nhien, dnh cung cd the thu nhdn tfl ve tinh qua cac bd cdm bidn (sensor) hay dnh dflpc +Phdn tich vd xa ly hinh dnh: Qud trinh ndy thflc chat bao gdm nhieu cdng doan nhd Ta chi quan tdm den hai cdng doan chinh la tdng cfldng dnh va phdt hien cdc dac tinh cua dnh '^ TAP CHl C O KHl VIET NAM *»' Sd (Thang nam 2012) NGHII^N CUfU-TRAO Ddi Trfldc het la cdng do^n tang cfldng anh de nang eao chat Iflpng hinh dnh Hinh dnh cd thi nhieu nguyen nhdn khdc 1dm suy gidm chat Iflpng nhfl: dp phan gidi cua thi^t bj thu nhdn anh, ngudn sdng hay nhilu dnh w Do v^y, cdn phdi tang cfldng vd khdi phyc dnh d^ 1dm ndi bdt mpt sd dde tin chinh eua dnh Ndi cdch khdc, Id lam cho anh gan gidng nhat vdi tr^ng thdi gdctrang thai trfldc dnh bi bi^n d^ng Cdng do^n ti^p theo Id phdt hi^n cdc dae tinh eiia hinh dnh Day la giai do^n ti^p eua qua trinh phan tieh dnh nhdm phdt hi^n nhflng dae tinh nhfl Id bien dnh , phan vung hinh dnh theo mdu sdc, dd sdng tdi phan biet vdi mdi trfldng Dp Idn eiia vat the tren khung hinh cua dnh ti 1? vdi khodng each d^n v^t th^ vd tpa dd cua vdt thi tr&n khung hinh eungti 1^ tfldng flng vdi tpa dp that cua vdt the mat phdng vudng gdc vdi tryc cua camera b Cdc bfldc cd ban di nhan dang v^t th^: + Budc 1: Ldy mdu mdu sdc eiia diem dnh (pixel) d^i dien ti^u bi^u eua vdt th^, tflc Id diim md ndm vung sai sd cdc gia tri mdu sdc (do ta cho) md cd nhi^u nhdt cdc dilm mdu cd thdnh phan mdu sdc thod man khodng gid tri De thflc hi|n phfldng phdp ndy, ta cdn viit mdt chfldng trinh +Budc nh&n dang dnh va lUu trd: Anh sau Idy gid trj thdnh phan mdu sdc, vi du, ta lay da xfl ly xong thi cdc hinh dnh da cd thi nhan thdnh phdn mdu sdc eua bdng mdu hdng, ta dat dang dflpc thdng qua bien dang vd mdu sdc Can qud bdng mdu hong trfldc dng kinh Camera mgt cfl vdo muc dich sfl dyng, ta ed the chpn ky thuat khoang cach d xac dinh nhu hnih 2, nhdn d?ing vd Iflu trfl thich hdp THUAT TOAN TIM KIEM VA DIEU KHIEN ROBOT De rdbdt cd the tim kiem dflpc mpt vat the hay mdt ngfldi nao dd, trfldc het, ta phai xdy dflng mdt thfl vien dfl Heu nhan dang ve doi tflpng can tim Khi camera thu dfldc hinh anh trdng mien lam viec eua nd, he thdng xfl ly hinh anh ket hpp vdi thfl vien dfl lieu nhan dang vat the can tim vd xac dinh tpa dp khoang each den vat ndy He thdng nhan dang cung cap thdng tin ve khoang each va tpa dp vat the cho rd bdt de di chuyen den gan vat the 3.1 Qua trinh thu thSp diit li^u va xay dflng thfl vien a Nguyen iy chung: Dfla tren ddc diem la tat ca mpi di^m dnh md td vat thi ed mau ddng nhat dd se c6 gid tri etia cae thdnh phan mdu sdc Red, Green, Blue khdng gian mdu RGB sai lech khdng dang ke so vdi gia tri cua cae phan mdu sac cua diem anh md ta moi trfldng Tfl dd, dfla vao dac diem ma ta cd the nhan d?ng dflpc bat ky mdt loai vat thi ndo ddng mau va cd mau TAP CHf CO KHi V I £ T NAM Hinh 3: Hinh anh den trang 56 (Thang nam 2012) NGHIEN CUfU-TRA0D(5l Hinh chfl n h , t mau dd cp tam la diem ta lay mau Nhfl Otren man hmh la diint Idy mdu mau sdc da dtch chuyen sp voi vi t „ ban dau Tren man hinh, ta thay hau nhfl tat ca cdc diem trin qud bdng deu thod man Vdy diem Idy mau ndy cd the' cpi Id diem dai dien mdn sdc cho vat t h J Ddng thdi, ta cung chp hien cdc gid trithdnh phan mdu sdc cfla diem nhfl sau: Hinh 4: Cac thdnh phan mdu sdc cua qua bong Ta thay, thdnh phan mdu sdc dd (Red) Id 141, Green (Xanh la cdy) Id 76, Blue (Luc) Id 84 Ddy Id gid trj chinh xdc thdnh phan mdu sdc cua diem ta vfla lay mau Ta ghi Iji de phuc vu cho qua trinh Idp trinh thfl vi^n + BUdc 2: Phdn vung dung sai gid tri cdc thdnh phan man sac cho vdt the lay mau: Ta nen phdn mien dung sai ve gid tri trung vdi mien dung sai ciia chflong trinh lay mju nhfl chflong trinh tren la ±20 DiSu ndy cd nghia Id, nhflng diem dnh nap cd cd gid tri ddng thdi: Red (141-20,141+20), Green (76-20,76+20), Blue (84-20,84+20) thi ta coi nhfl nd Id mdt di^m mdu cd tinh chat nhfl vdt the Vdi mien dung sai ta chpn chp ddi tflpng Id +20, tflc Id ta cho moi thdnh phan mdu sdc nam khodng tfl X-20 den X+20 (X=Red hoac Green hoac Blue), tflc Id mien dung sai cfla ta Id 40 vdi thdnh phan mdu sac, cdn vdi thdnh phan mdu sac Id 403=64000 Dieu tflong dflpng vdi viec ta cho 64000 kieu mdu cua diem dnh nfla thod man cung tinh chat vdi iiim dnh ta lay mau mpt sd cd ve rat Idn nhflng thflc te nd chi bang (40/256)3=0,38% so vdi tong sd mdu sac cua khdng gian mau 24 bit bieu dien tBlAfc 3: Ldp trinh ghi nhd gia tri vdo doan ma vd phan bd vflng dung sai mdu sdc Tren mdn hinh hinh 7, ta bat nut Aon vdo trdi bdng mdu hdng thi chflpng trinh se 3at gifl ddi tflpng Id trai bdng mdu hdng nhfl hinh rin, nfa ta ehfla chpn che dp tfl dpng thi toa dp t, y cfla trdi bdng tren khung hinh vd sd diem dnh :d mflc mdu sac vflng sai sd ehfla cdn thidt >hdi cap nhdt Chi ta chpn "che dp tfl dpng" hi bd timer se khdi ddng vd c^p nhdt cdc toa dp ndy mdn hinh vd khdi ddng che dp dieu khien chp robot thdng qua cdng COM - Bang cdch 1dm tflong tfl, ta cd the xdy dflng dflpc thfl vien nhdn dang rat nhieu ddi tflpng :f€ Hinh 5: Chiidng trinh nap gia tri mau vA dung sai ciia hinh anh 3.2 Thu4t toan tim kiem Khi ta da yeu cau rdbdt tim kiem mpt vat the da cd nhan dang bang hinh anh khdng gian Idm viec eiia nd thi chfldng trinh tim Idem ddac kich hoat theo thuat todn nhfl hinh Trong trfldng hdp xoay vdng ma khdng phdt hien ddi tfldng, rdbdt se tfl ddng ti^n them quang lai xday trdn d^ tim kiem vat the Rdbdt lap di lap Igii qud trinh tim kiem nhfl the cho d^n tim ddi tfldng ^ TAP C H i C O KHi VI$T NAM V Sd (Thang nam 2012) NGHIEN C J U - T R A O D O I he thdng dfl lieu, ta cd the cap nhdt trflc tiep doi tflpng thdng qua mdu sdc bang sfl ki^n click ehuOt phdi (chupt trdi chung ta dung de dieu khien niit bdm che dp khdng tfl ddng) Khi an bilu tflpng "chudt phdi nap dnh" tren mdn hinh ta co th^' sfl dyng chudt phdi click vao ddi tflpng, bilu tflpng bdt gifl se xudt hien bdt gifl Idy ddi tUpng Vd ta dn chi dd ti^ ddng thi ehfldng trinh se di^u khi^n robot hfldng theo doi tfldng, hoac klii ta ddt ddi tfldng d mdt vi tri ndo dd, robot se tti tim ki^m K^t qud Id ta cd the dat bat cfl mpt vat gi trfldc Camera chpn chi dp click chudt phdi, sau dd click chupt phdi d^ robot tiln hdnh tim kilm vat dd Qua trinh tim kiem dfldc gid thiet Id thflc hien trdng khdng gian khdng bdng phdng Neu ket hdp vdi he thdng dinh vi GPS thi ta cd the cho robot quet hinh dnh tren toan bd khu vflc can tim kiem Thdng tin dnh dau vao eiia ham thflc hien qua trinh xfl ly hinh dnh Id dnh bitmap 24 bit hoac 32 bit Ham BongLaCay (LPBITMAPINFO IpBi) cd bien dau vdo Id IpBi, kieu la LPBITMAPINFO Day la kieu thdng tin ve hinh anh bitmap arene_ dx, la chieu rdng eua dnh, arene_dy Id chieu cao cua dnh Khi tien hanh quet dnh dfldi dang tflng pixel de so sdnh mflc mdu vd tim nhflng diem thod man Sau dd, ta lay toa x Iflu vao mpt mdng mdt chieu mangx tang bien dem demx Ien, vdi toa dp y, ta cung lam tfldng tfl nhfl vdy Viec quet hinh anh dfldc thflc hien bdng vdng lap Sau khdi vdng lap thi ta da cd mdng mpt chieu Iflu gifl cdc toa dp thod man yeu cau vd can cfl vdo each quet anh suy dflpc Id diem ed toa dp (mangx[demx/2], mangy [demy/2]) Id diem cd xdc xuat la tdm diem anh eao nhat, dd, ta lay gid tri lam tam eua vat the vd hien thi toa dp x,y che dp tfl dpng Sd diem anh ed cung tinh chat vdi diem lay mau demx=demy la thdng sd cho ta biet dfldc trang thai nhan biet vdt the cua camera Khi ta ehfla cd thdng tin nhan dang TAP CHi CO KHi VIET NAM •t* 3.3 Sd dd he thdng dieu khien He thdng dieu khien eua Me_ShrimpVn dfldc xdy dflng tren ed sd may tinh cd nhan PC vd he thdng kit n6i khdng day Tin hieu hmh dnh til camera dflpc truyen v^ may tinh de xfl iy va nhan dang hinh dnh Tin hieu dieu khien dfldc grti trd lai robot thdng qua dfldng truyen khdng day va tac ddng Ien cdc mach dieu khien de thflc hien cac di chuyen can thiet M.K h pi II lUrll k^ II n • Tiul ,.i,„,n.,„, T MiihilmikhiNi loluK MfSliiimi>\"ii iVi illMl klllell Ml h Illllilrlll mihtm j n T - f Hinh 7: Sd he thong dieu khien KET QUA THI/C NGHIEM He thdng dieu khien rd bdt bang Camera da dflpc Idp dat tren rdbdt tfl hanh Me ShrimpVn da dflpc dfla vdo flng d\mg Sd (Thang nam 2012) NGHIIEN CtfU-TRA0D6l^ K^t qud cho thay rat khd quan Rdbdt cd the tien hanh tim kiem v^t the ma ta xdc dinh khdng gian lam viec eua nd vd tien gan den v^t the d mdt khoang cdch an todn xdc dinh trfldc Hinh anh qua trinh tim kiem dfldc hien thi tren man hinh dieu khien giup ta cd the quan sdt dflpc khu vflc tim kiem Ta cd the Ifla chpn cae che dp dieu khien khdc cho rdbdt thdng qua man hinh dieu khien (hinh 7) (3 chi dd dieu khi^'n tfl xa, ta cd the theo ddi tren man hinh de dieu khien rd hot tien, lui, quay phdi, quay trdi tfl ban phim hay tfl man hinh dieu khien Khi ehfldng trinh d "che dp tfl ddng" vd chon mdt eac muc trdi bdng mudn tim kiem, rdbdt se thflc hien che dp tfl quay xung quanh nd de tim kiem vdt th§' Neu tim thay qud bdng, phan mem xfl ly anh se xdc dinh tpa dp XY2 (X- phfldng ngang vudng gdc vdi true camera, Y- phfldng thing dflng vd Z Id phfldng ngang ddng true vdi true Camera) Khodng cdch D theo phfldng Z dfldc xdc dinh la dieu kien de rdbdt tien, lui hay dflng lai Trfldng hdp vat the tim kien khdng chuyen ddng va nam tren true Y thi rdbdt se tien den khodng each d va dflng lai Trfldng hpp neu tda dd x>0 thi rdbdt quay vd tien ve ben phdi chd den gia tri bang Neu x Tdi li^u tham khdo: [1] Lflong Manh Ba - Nguyin Thanh Thuy - Xfl ly dnh; 2007; NXB Khoa hpc vd Ky thudt (2] Nguyen Vin Hodng vd nhdm tdc gid ELICOM - Visual C++; [3] Johnson A.E., Herbert M - Recognizing Objects by Matching Oriented Points, IEEE CVPR1997 [4] Boguslaw Cyganek va J.Paul Siebert - An Introduction to 3D Computer Vision Techniques and Algorithms, 2009 [5] Gary Bradski vd Adrian Kaehler - Learning OpenCV Nhd xult bdn O'REILLY 2008, USA [6] Roger Y.TSAl - A Versatile Camera Calibration Technique for High- Accuracy 3D Machine Vision Metrology Using OfF-the-Shelf TV Cameras and Lenses,2001 [7] Zhang Z - A Flexible New Technique for Camera Calibration, 1998 [8] Rafael C Gonzalez, Richard Eugene Woods - Digital Image Processing, Prentice Hall, 2008 TAP CHl CO KHl V I £ T NAM • s6 (Thdng nam 2012) ... trfl lai + Thu nhan hinh dnh: hinh anh thdng thfldng dflpc thu nhdn qua camera Camera cd the la camera tflpng t\f (CCIR) hoac camera sd (CCD) Tuy nhien, dnh cung cd the thu nhdn tfl ve tinh qua cac... A Versatile Camera Calibration Technique for High- Accuracy 3D Machine Vision Metrology Using OfF-the-Shelf TV Cameras and Lenses,2001 [7] Zhang Z - A Flexible New Technique for Camera Calibration,... se xdc dinh tpa dp XY2 (X- phfldng ngang vudng gdc vdi true camera, Y- phfldng thing dflng vd Z Id phfldng ngang ddng true vdi true Camera) Khodng cdch D theo phfldng Z dfldc xdc dinh la dieu