an incremental learning algorithm based on support vector domain classifier

... An Incremental Learning Algorithm Based on Support Vector Domain Classifier YinggangZhao,QinmingHeCollegeofComputerScience,ZhejiangUniversity,Hangzhou310027,ChinaEmail:ygzl29g163.comSVDD algorithm givesus an enlightenment:whenweclassifyabinary-classdataset,ifweonlyknowpartofsample'sAbstractcategory(forexample,sampleswithcategorylabelyi=1),yettheotherpartofsample'scategoryisunknown,thenwe Incremental learning techniqueisusuallyusedtosolvecandesignnewtypeof classifier based on SVDDnamedlarge-scaleproblem.Wefirstlygaveamodifed support vector support vector domain classifier (SVDC).Thisnew classifier machine(SVM)classificationmethod ... support vector onlyneedtodescribethedatawithknowncategory,then domain classifer(SVDC),then an incremental learning obtainingthedescriptionboundaryofthisclassofdata. algorithm based on SVDCwasproposed.ThebasicideaofFinally,wecanclassifytheunknownbinary-classdatathis incremental algorithm istoobtaintheinitialtargetaccordingtotheobtainedboundary.conceptsusingSVDCduringthetrainingprocedureandthenInthispaperour incremental learning algorithm is based updatethesetargetconceptsby an updatingmodel.Difierent on SVDC,andthis algorithm ismotivatedbythefromtheexisted incremental learning approaches,inourperson -learning procedure.When learning acomplicated algorithm, themodelupdatingprocedureequalstosolveaconcept,peopleusuallyobtainainitialconceptbyusingpartquadraticprogramming(QP)problem,andtheupdatedmodelofusefulinformation,thenupdatetheobtainedconceptbystillownsthepropertyofsparssolution.Comparedwithotherutilizingnewinformation.Intermofour incremental existed incremental learningalgorithms,theinverseprocedure algorithm based on SVDC,itfirstlyutilizepartofdataofour algorithm (i.e.decreasing learning) iseasytoconduct(memoryspacepermitting),thenobtainaconcept(namelythewithoutextracomputation.Experimentresultsshowourparameterofobtaineddecisionhypersurface)bySVDC algorithm iseffectiveandfeasible. learning algorithm, finallyaccordingtotheinformationofdecisionhypersurfaceacquiredinlaststep,updatetheparameterofdecisionhypersurfacegainedinlaststeputilizingKeywords: Support Vector Machines, Support Vector Domain specializedupdatingmodelintheprocessof incremental Classifier, Incremental learning, Classification. learning, namelyupdatingtheknownconcept.Our algorithm ownsthefollowingcharacters:1.INTRODUCTION1)The incremental updatingmodelinthis algorithm Withlargeamountsofdataavailabletomachine learning hasasimilarmathematicsformcomparedwithcommunity,theneedtodesigntechniquesthatscalewellisstandardSVDC algorithm, andany algorithm usedmorecriticalthanbefore.AssomedatamaybecollectedovertoobtainthestandardSVDCcanalsobeusedtolongperiods,thereisalsoacontinuousneedtoincorporatetheobtaintheupdatingmodelofour algorithm; newdataintothepreviouslylearnedconcept. Incremental 2)Theinverseprocedureofthis algorithm, i.e.the learning techniquescansatisfytheneedforboththescalabilitydecreasing learning procedureiseasytoand incremental update.implement,thatistosaywhenweperceivethe Support vector machine(SVM)is based on statisticalgeneralizationperformancedroppedinthe learning theory,whichhasdevelopedoverlastthreedecades incremental process,wecaneasilyreturnlaststep[1,2].Ithasbeenprovenverysuccessfulinmanyapplicationswithoutextracomputation;[3,4,5,6].SVMisasupervisedbinary-class classifier, whenTheexperimentalresultsshowthe learning performancewetrainsamplesusingSVM,thecategoriesofthesamplesareofthis algorithm approachesthatofbatchtraining,andneededtobeknown.However,inmanycases,itisrarethatperformancewellinlarge-scaledatasetcomparedtootherwecanobtainthedatawiththeircategorybeknown,inotherSVDC incremental learning algorithm. words,mostoftheobtaineddata'scategoriesareunknown.InTherestofthispaperisorganizedasfollows.Insection2thissituation,traditionalSVMisn'tappropriate.wegive an introductionofSVDC,andinsection3wepresentTAXetalproposedamethodfordata domain descriptionour incremental algorithm. Experimentalandresultscalled support vector domain description(SVDD)[7],anditisconcerningtheproposed algorithm areofferedinSection4.usedtodescribedata domain anddeleteoutliers.ThekeyideaSection5collectsthemainconclusions.OfSVDDistodescribeoneclassofdatabyfindingaspherewithminimumvolume,whichcontainsthisclassofdata.Proc.5thIEEEInt.Conf. on CognitiveInformatics(ICCI'06)Y.Y.Yao,Z.Z.Shi,Y.Wang,andW.Kinsner(Eds.)801-4244-0475-4/06/$20.OO@2006IEEE802. Support Vector Domain Classifier withconstrains,==1,and0<a,<C.Wherethe2.1 Support Vector Domain Description[7]innerproducthasbeenreplacedwithkernelfunctionK(.,.),andK(.,.)isadefinitekernelsatisfyingmercerOfadatasetcontaiingNdataobjcondition,forexampleapopularchoiceistheGaussianOfadatasetcontainingNdataobjects,enl(,)=ep-xz2/2),>0fx,Z=1, ... akYkXk(I10)(13)informula(10),xkrepresents support vector, andkisFinallyweobtainthefollowingdecisionfunction:thenumberof support vector. fk(x)=sgntRk-{K(x,x)+2Ea,y,K(x,X)-ZEa,ayjy,yjK(x,ix)}Iff(x)>0,thetestedsampleiscontainedinsphere,,ESV,ESVandwelookthesamplesenclosedIspherethesame-classsgn{R21+2RklEaoy1xi+(EaciyiXi)2}objects.Otherwiseitisrejected,andwelookitastheXi,SVkxi,SVkoppositeobjects.-{K(x,x)+2Ea1yiK(x,xi)-Eaa1jy1yjK(x,xj)}xiESVxiESV3.SVDC Incremental Learning Algorithm Accordingformula(6),wesupposetheobtainedinitialsgn{ffk(x)+2RkLEaiy,x,+(aciyixi)2}parameter(sphereradius) learning withinitialtrainingsetisxicsVkxicsVkRO,andthesetof support vectorsisSVO.Theparameter(14)Fromequation(14)wecanseeitiseasytoreturnthebecomesRkinthekth incremental learning, andthesetlaststepof incremental earningwithoutextracomputation.of support vectorsbecomesSVk,andthenewdatasetinFromtheaboveanalysiswecanseeonlyconductatriflingmodification on thestandardSVDC,canitbeusedklhstepbecomesDk={(xkyk)j}l-tosolvetheupdatedmodelin incremental learning procedure.Our incremental algorithm canbedescribedasNowwesummarizeouralgorithmasfollowings:following:Step1 Learning theinitialconcept:trainingSVDCAssumewehasknownRklupdatingthecurrentusinginitialdatasetoTS,thenparameterR0ismodel~~~~~~usnSVknlnXkadaaeTSoI/hnpaaeerRmodelusingSJK,l1andnewdataset{(XiY7)}>=1obtained;WeupdatingthecurrentmodelusingthefollowingStep2Updatingthecurrentconcept:whenthenewdataareavailable,usingthemtosolveQPproblemquadraticprogramming(QP)problem:formula(11),andobtainnewconcept;ming(Rk)IRk-R112Step3Repeatingstep2untilthe incremental learning isk(Rk2_(Xk-a)'(XV-a))>XkexiDkoverwhereRk-listheradiusoflastoptimizationproblem(11),4.ExperimentsandResultswhenk=1,RoistheradiusofstandardSVDC.ItisInordertoevaluatethe learning performanceofferedbyobvious,whenRklI=0,the incremental SVDChastheour incremental algorithm, weconductedexperiment on sixdifferentdatasetstakenfromUCIMachineRepository:sameformasthestandardSVDC.WewillfoundtheBanana,Diabetes,Flare-Solar,Heart,Breast-Cancer,German.updatedmodelbythe incremental SVDCalsoownstheNotesomeofthenarenotbinary-classclassificationproblems,butwehavetransformthemtobinary-classproblembyspecialpropertyofsolutionsparsitywhichisownedbythetechnique.ExperimentparametersandDatasetareshowninstandardSVDC...

... representationwhile providing a wide range of applicationsin both analytical and computational linguistics.Rhetorical Structure Theory (Mann and Thomp-son, 1988) provides a framework to analyze andstudy ... compositionality criterion (seeSect. 3.5), we expect to see certain correlationsbetween the relation being classified and relationpatterns in either sub-tree, based on theoreticalconsiderations and ... (Marcu, 2000), which guaranteesthat only adjacent spans of text can be putin relation within an RST tree, and drasticallyreduces the size of the solution space.2.2 Support Vector MachinesAt the...

... head (e.g. PLANNING in the term RE- GIONAL NETWORK PLANNING), and expansion (e.g. REGIONAL in the term REGIONAL NETWORK) 1 This analysis allows the organisation of all the candidate terms ... shows an action (CON- STRUCTION) which can be applied to the object LINE. This definition of the context is original compared to the classical context definitions used in Informa- tion Retrieval, ... "expansion terminological context" (E- terminological context) of a NP is the set of the can- didate terms appearing in the expansion of the more complex candidate term containing...

... utterances and context. The speaker and utterance elements contain information about the characters who speak and what they said at each dialogue turn. On the other hand, context elements contain ... Salton and Buckley, 1988). Before performing the vectorization, word toke-nization was conducted. In this step, all punctua-tion marks were removed, with the exception of the question “?” and ... vector and the vector repre-sentations for each full dialogue stored in the dia-logue database are computed and used along with the utterance-level score for generating a final rank of candidate...

... Taiwanchuren@gate.sinica.edu.twAbstractThis paper describes an on- going projectconcerning with an ontological lexical re-source based on the abundant conceptualinformation grounded on Chinese ... traditional philo-logical construct of Hanzi into consideration. Byanalyzing the conceptual relations between char-acters (b) which scatter among diverse lexical re-sources, we construct an top-level ... semantic networka character-stored machine-readable lexicon and atop-level character ontology.4.1 Hanzi-grounded Lexicon and OntologyThe current lexicon contains over 5000 characters,and...

... A Mention-Synchronous Coreference Resolution Algorithm Based on theBell TreeXiaoqiang Luo and Abe IttycheriahHongyan Jing and Nanda Kambhatla and Salim Roukos1101 Kitchawan RoadYorktown ... among mentions other than the twoin question is ignored. However, models in (McCal-lum and Wellner, 2003) compute directly the probabil-ity of an entity configuration conditioned on mentions,and ... mentionsDistance token_dist how many tokens two mentions are apart (quantized)sent_dist how many sentences two mentions are apart (quantized)gap_dist how many mentions in between the two mentions...

...

... International Confer-ence on Machine Learning, pages 150–157.Susan Dumais, John Platt, David Heckerman, andMehran Sahami. 1998. Inductive learning algorithmsand representations for text categorization. ... European Conference on Machine Learning. TakuKudo and Yuji Matsumoto. 2000a. Japanese depen-dency structure analysis based on support vector ma-chines. In Proceedings of the 2000 JointSIGDAT Con-ference ... Thompson, Mary Leaine Califf, and Ray-mond J. Mooney. 1999. Active learning for naturallanguage parsing and information extraction. In Pro-ceedings of the Sixteenth International Conference on Machine...

... signal enhancement at very low Gd concentrations. This modularly designed contrast agent represents a new tool for improved monitoring and evaluation of interventions at the gene transcription level. ... synthesis and application of a GdxSc3-xN@C80-BioShuttle-conjugate referred to as Gd-cluster@-BioShuttle, which features high proton relaxation and, in comparison to the commonly used contrast ... solutions as well as Hank’s solu-tion containing 2% DMSO as a control. After 25 min-utes, the solutions were removed from the cells, then cells were washed twice with Hank’s solution and kept...

... emotion in comparison with five ones by Ortony and Johnson ( sadness, happiness, love, anger, hate)3 Emotion metaphor An investigation into the role of metaphor in description of emotion in ... manifestations than other conscious states; • they vary along a number of dimensions: intensity, type and range of intentional objects, etc. • they are reputed to be antagonists of rationality; ... gathered, analyzed and classified 92 definitions and 9 skeptical An investigation into the role of metaphor in description of emotion in poetic discourse17statements about the concept of emotion concluding...