This article was downloaded by: [Northeastern University] On: 29 December 2014, At: 23:35 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK Cybernetics and Systems: An International Journal Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/ucbs20 Trust-Based Consensus for Collaborative Ontology Building a b a Trong Hai Duong , Ngoc Thanh Nguyen , Duc Cuong Nguyen , Thi c Phuong Trang Nguyen & Ali Selamat d a International University , Vietnam National University , Ho Chi Minh , Vietnam b Wroclaw University of Technology , Wroclaw , Poland c Click for updates University of Information Technology , Vietnam National University , Ho Chi Minh , Vietnam d Universiti Teknologi Malaysia (UTM) , Johor Bahru , Malyasia Published online: 12 Mar 2014 To cite this article: Trong Hai Duong , Ngoc Thanh Nguyen , Duc Cuong Nguyen , Thi Phuong Trang Nguyen & Ali Selamat (2014) Trust-Based Consensus for Collaborative Ontology Building, Cybernetics and Systems: An International Journal, 45:2, 146-164, DOI: 10.1080/01969722.2014.874815 To link to this article: http://dx.doi.org/10.1080/01969722.2014.874815 PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content This article may be used for research, teaching, and private study purposes Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden Terms & Downloaded by [Northeastern University] at 23:35 29 December 2014 Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions Cybernetics and Systems: An International Journal, 45:146−164 Copyright © 2014 Taylor & Francis Group, LLC ISSN: 0196-9722 print/1087-6553 online DOI: 10.1080/01969722.2014.874815 Trust-Based Consensus for Collaborative Ontology Building TRONG HAI DUONG1 , NGOC THANH NGUYEN2 , DUC CUONG NGUYEN1 , THI PHUONG TRANG NGUYEN3 , and ALI SELAMAT4 Downloaded by [Northeastern University] at 23:35 29 December 2014 International University, Vietnam National University, Ho Chi Minh, Vietnam Wroclaw University of Technology, Wroclaw, Poland University of Information Technology, Vietnam National University, Ho Chi Minh, Vietnam Universiti Teknologi Malaysia (UTM), Johor Bahru, Malyasia Ontologies are widely considered to be the backbone of the Semantic Web Its importance is being recognized in a multiplicity of research fields and application areas Ontology building is crucial for the aforementioned issues The main goal of this research is to investigate an effective methodology for collaborative ontology building A trust-based consensus is proposed to support an efficient solution for conflicts among different viewpoints of participants in the collaborative ontology (CoO) building process In every cycle of the iterative collaborative process, the ontology is refined and evolved by reaching a trustbased consensus among the participants’ viewpoints The proposed method is applied for collaborative Vietnamese WordNet building The result is significant in comparison with previous approaches KEYWORDS collaborative ontology, collective ontology, ontology, ontology engineering, ontology integration INTRODUCTION The most well-known collaborative knowledge building tools such as Wikipedia and WikiMap have not yet considered consensus concourse to solve conflicts on knowledge base These tools allow everyone to participate in developing the knowledge An Open Directory Project called DMOZ is collaboratively constructing a Web catalog DMOZ is an online, open, and collaborative taxonomy Address correspondence to Ngoc Thanh Nguyen, Institute of Informatics (I-32), Wroclaw University of Technology, Wyb Wyspianskiego 27, 50-370, Wroclaw, Poland E-mail: Ngoc-Thanh.Nguyen@pwr.edu.pl Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/ucbs 146 Downloaded by [Northeastern University] at 23:35 29 December 2014 Collaborative Ontology Building 147 building project, which is commonly used as a Web directory in many Internet resources A Web directory contains a large reference library and is arranged from general to specific subjects Experts contribute to this directory and all of the resources in the directory are maintained by editors who might belong to a single large community This community will evaluate all of the submissions regarding the subject manner Ontology has become a buzzword in the Semantic Web and semantic data processing fields (Berners-Lee et al 2001), and its importance is being recognized in a multiplicity of research fields and application areas, such as knowledge engineering (Gruber 1993), database design and integration, and information retrieval and extraction (Noy and McGuiness 2001; Castillo et al 2003) Ontology is the science of what is, of the kinds and structures of objects, properties, events, processes, and relations in every area of reality (Smith and Welty 2001) For an information system, an ontology is a representation of some preexisting domain of reality that (1) reflects the properties of the objects within its domain such that there is a systematic correlation between reality and the representation itself, (2) is intelligible to a domain expert, and (3) is formalized in a manner supporting automatic information processing Ontologies play a central role in facilitating data exchange between several sources Ontology engineering has recently attracted considerable interest Most research has focused on ontology languages (Smith and Welty 2001; Cao et al 2013), inference mechanisms, ontology editing environments, and ontology integration (Doan 2004) Most ontologies are developed via an engineering-oriented method (Protg): a small group of knowledge engineers carefully builds and maintains a representation of their view of the world Maintaining such large ontologies in an engineering-oriented manner is a highly complex process: developers need to regularly merge and reconcile their modifications to ensure that the ontology captures a consistent and unified view of the reality One example of these tools is Protege (Protg), which is used by Stanford University for knowledge acquisition It provides a graphical and interactive ontology design and knowledge base development environment Ontology developers can access relevant information quickly and navigate and manipulate the ontology Currently, there are several tools oriented toward collaborative building (Tudorache et al 2008; Gabel et al 2004; Auer et al 2006; Karapiperis and Apostolou 2006; Ruiz et al 2009a,b): a consensus-building mechanism that allows a large group of people to contribute to or annotate a common ontology in a collaborative manner Tudorache et al (2008) have developed Collaborative Protege as an extension to the client server Protege Collaborative Protégé allows entire groups of developers who are collaboratively building an ontology to hold discussions, chat, and make annotations and changes as a part of the ontology development process OntoWiki in Auer et al (2006) is a Web-based ontology that focuses on an instance editor that provides only rudimentary capabilities such as the history of changes and ratings of ontology components OntoEdit in Sure (2002) is a collaborative ontology (CoO) editing environment that integrates numerous aspects of ontology engineering and allows multiple users to develop ontologies KAON (Gabel et al 2004) focuses on changes Downloaded by [Northeastern University] at 23:35 29 December 2014 148 T H Duong et al in ontology that can cause inconsistencies and proposes deriving evolution strategies in order to maintain consistencies This article aims at investigating an effective methodology for collaborative ontology building in which a trust-based consensus is proposed to support an efficient solution for conflicts among different viewpoints of participants We learned that consensus techniques are the core of collaboration The Delphi technique (Gallagher et al 1993) was applied to collaborative ontology building in order to allow an entire group to reach a consensus by sharing their understanding of ontological perspective Trust-based consensus for solutions to conflict profiles involves generation of a reconciled ontology from conflicts between participants’ versions of an ontology We applied the proposed method for Vietnamese WordNet building for demonstration and evaluation The remainder of this article is organized as follows The following section reviews related works The trust-based consensus methodology is presented next The trustbased consensus for collaborative ontology building is discussed next, followed by a description of the experiments performed by applying the proposed methods for Vietnamese WordNet building In the last section we provide our conclusions RELATED WORKS According to Holsapple and Joshi (2002), the collaborative approach phases to ontology design are as follows: • The preparatory phase, which defines the criteria for ontology design, specifies boundary conditions for the ontology, and determines standards for assessing its success • The anchoring phase, which includes development of the initial version of the ontology that will feed the next phase (evaluation phase) based on the compliance with the design criteria • The iterative improvement phase, which enhances the ontology until all participants’ viewpoints reach a consensus through a collaborative building technique In this phase, the ontology will be revised and its structure will evolve due to collaboration of the participants At each iterative improvement, the ontology is evaluated by the aforementioned standards and conditions • The application phase, which demonstrates the use of CoO by applying it in various ways In agreement with Holsapple and Joshi (2002), Karapiperis and Apostolou (2006) follow the above phases in which they start by defining the criteria for ontology design by applying the ontology building steps described in Noy and McGuiness (2001) to design the initial ontologies Their collaborative methodology for ontology building supports a team effort to iteratively revise and evolve the initial ontology until all participants’ understanding of the ontology reaches consensus The consensus is achieved through voting in a nominal group technique (NGT; Gallagher et al 1993) Downloaded by [Northeastern University] at 23:35 29 December 2014 Collaborative Ontology Building 149 The Collaborative ONTology ENgineering Tool (ContentCVS) is a system that is available for download as a Protege plugin (Ruiz et al 2009a) The tool supporting collaboration provides the means for (1) keeping track of changes in ontology versions, (2) identifying conflicts between the versions of the ontology, (3) constructing a reconciled ontology from conflicting versions and identifying errors, and (4) suggesting possible ways to repair the identified errors with minimal impact on the ontology The aforementioned methods agree that CoO involves a group of people contributing to an ontology in a specific domain of interest CoO allows an entire group to participate in the process of ontology building by reaching a consensus and usually aims at completeness CoO building involves individuals contributing to understanding of their ontological perspective, but everybody works together to build the ontology The purpose of CoO is to generate the best representative ontology from various versions of an ontology The collaborative ontology must best reflect the conflicting versions of the ontology in a compromise Therefore, the above approaches focus on human collaboration to build a common ontology Differing from the previous approaches, the main goal of our previous research (Duong and Jo 2010) was to investigate the techniques that support a solution to conflicts among different participants’ viewpoints in the CoO process A machine is considered as a leader of the collaborative group, via which conflicts among the versions of the ontology are identified and a reconciled version that best reflects the conflicting versions in a compromise is generated automatically The main contributions of this research are as follows: • We analyzed techniques supporting CoO such as ontology integration and consensus We learned that consensus techniques are the core of CoO (Holsapple and Joshi 2002; Karapiperis and Apostolou 2006; Ruiz et al 2009b) We distinguished consensus into three cases, including consensus for human collaboration, consensus for decision making (Pill 1971; Gallagher et al 1993), and consensus for a solution to a conflict profile (Nguyen 2008) The nominal group technique (Gallagher et al 1993) is applied for CoO to allow an entire group to reach a consensus by sharing their understanding of ontological perspectives Consensus for solutions to conflict profiles involves generating a reconciled ontology from participants’ conflicting versions of the ontology In every cycle of the iterative process, participants’ contributed versions tracked changes and identified conflicts automatically Unless all versions of the ontology reach consensus, the ontology is revised and evolved by a CoO algorithm (see Algorithms and in Duong and Jo 2010) • Two criteria for CoO (identity and type of concept) for core, domain, and application ontology design and eight criteria for the CoO process (inclusive, egalitarian, interactive, representative, reconcilable, trust, proof, and quality) are proposed The criteria are aimed at guiding and evaluating the CoO process • A process of CoO building was identified Different from previous approaches (Tudorache et al 2003; Karapiperis and Apostolou 2006; Ruiz et al 2009a,b), we consider machine collaboration in the CoO building process 150 T H Duong et al TRUST-BASED CONSENSUS METHODOLOGY Downloaded by [Northeastern University] at 23:35 29 December 2014 Consensus Theory Consensus is a collaborative process allowing an entire group to participate in decision making in which everyone consents to the decisions of the group (Danilowicz and Nguyen 1988; Hernes and Nguyen 2007) The goal of consensus decision making is to find common ideas and explore these issues until everyone’s viewpoint has been recognized and understood by the group Discussions leading to consensus aim to achieve mutual agreement by addressing all concerns Consensus does not require unanimity, but people must make a commitment to honest cooperation in order to make final decisions Consensus is not for individualists or people who want to dominate or coerce others; rather, discussion continues until consensus is achieved Consensus is not a process for determining whose ideas are best; rather, it is for searching together for the solution that is best for the group Everyone must agree to live with the decision There are many areas of medicine in which decision making and practice vary between clinicians Consensus is used to obtain a majority viewpoint where a range of opinions about specific issues is likely to exist Two of the methods are the Delphi and the nominal group technique (Pill 1971; Gallagher et al 1993) These have been used in a variety of health care settings Most studies in this field use a two-round procedure, with feedback between the first and second rounds, to deal with areas of disagreement NGT (Gallagher et al 1993) is a well-known method for decision making It has been used to get the final result among a group, regardless of whether it is large or small, while considering all opinions and votes from group members NGT takes into account the participants who join the discussion in order to choose the result It is successful when everybody participates and understands the manners and forms solutions or opinions on their own without influencing those around them NGT is a process where everyone is clearly involved and is aware of all of the information and the solution is obtained without excluding anyone from the discussion There are five stages to NGT: Introduction and explanation: It begins when every participant is in attendance in the meeting room and a facilitator welcomes them Then, the facilitator starts the introduction by briefly explaining what they are going to and what procedure they should follow in the meeting Private generation of ideas: In this stage, after explanation, the facilitator will distribute a paper for all participants and they must give a solution or idea to solve the given question In accordance with the rules, they are not allowed to discuss their solution with other members Sharing ideas: After generation of all of the ideas, the facilitator will ask participants to share their ideas among themselves Everyone shares their own ideas and while someone is presenting their own idea, someone else should consider and contribute another idea if they think the idea presented by the speaker is good If someone Downloaded by [Northeastern University] at 23:35 29 December 2014 Collaborative Ontology Building 151 thinks of an idea while the presenter is speaking, they can write it down In this manner, the facilitator will record the ideas presented by every participant Because this stage is only for sharing ideas, there is no discussion Group discussion: This is an important stage where everyone will be involved in the discussion about certain ideas that they not clearly understand The participants can also propose any ideas that might make an original idea stronger However, criticism or any rejection of ideas is avoided as much as possible, because this discussion needs to be fair and balanced To avoid any negativity, the discussion finishes after 30 to 45 minutes, even if every idea was not discussed during that time In addiiton, in this stage, all of the participants in the group are encouraged to give any new ideas that can be classified into categories Even when there are new ideas, original ideas from participants should not be eliminated or rejected Voting and ranking: This is the final stage where every idea related to the questions is prioritized The results are obtained after following the voting and ranking process, and participants learn the results at the end of the meeting The process is successful when the goal of the meeting is reached One of the popular consensus-building techniques is the Delphi method (Pill 1971) This method is used for normal discussion that does not need complex communication between experts, such as meeting face to face or having a meeting around a table This method can be implemented using technology such as e-mail or any other electronic technologies for communication where each question can be sent directly to every group expert Even when there is a complex problem that needs to be solved, this method can be used to find the solution by sending a series of questionnaires via multiple iterations and getting a solution (data) from experts The Delphi method is commonly used in education, to estimate forecasts, and in other fields The Delphi technique can be done in four steps: The moderator forms a group of experts that participate in the process to solve the problem However, all of the experts are unidentified A person will send a questionnaire to the participants via mail or e-mail Once the person gets the return answer from a participant, the person will analyze the results In the last step, if there is no consensus reached, a combination of previous questionnaires and results will be used as a new version of the questionnaire, and the moderator will send this new version again to the participants Step is repeated until consensus is reached or the moderator ends the process and makes a final report There are some differences between these two aforementioned methods It is well known that the Delphi method is commonly used without experts needing to meet each other In the NGT method, all participants or experts need to be in one place and working together The main point of the NGT method is that all participants are required to meet face to face in order to reach the solution It is thought that the 152 T H Duong et al NGT method will lead to every idea or opinion being strongly endorsed if experts or participants present their ideas in a formal manner in front of their peers This means that in the NGT method, consensus can be reached if there is real discussion In contrast to the Delphi methods, it is thought that without experts meeting each other and based on the views of anonymous experts, the result of consensus is more accurate Without influencing other experts, an individual expert can find the ideas and solution based on expert knowledge, so consensus results are more reliable when they are based on individual expertise Consensus theory results are in some sense consistent with those of paraconsistent logic (Nakamatsu and Abe 2014) Downloaded by [Northeastern University] at 23:35 29 December 2014 Trust-Based Consensus The current consensus theory considers conflict participants at the same level Here, the participants in a collaborative group are distinguished by their contribution measuring as a trust in Tuong et al (2013) Each participant has a value of trust denoted by in [0, 1], where = means that the system cannot trust this participant and = means that the system absolutely trusts the participant When a new participant joins to the system, his or her trust value is Const specified by the system We denote U being a set of participants The trust function t is defined as follows: t : U → [0, 1] (1) To solve a problem Pr, each participant gives knowledge through a profile P = (ei ei ) consisting of many pairs of element and its value that expresses the strength of this element In the specific case of this article, P is the set of senses and relations of one word or phrase in the ontology-based Vietnamese WordNet Element ei ∈ P is a specific sense or specific relation to other ones of this word or phrase Other participants have a right to express their agreement for each element by giving a value from to denoted by for each element ei ∈ P Each participant gives only one agreement value, but he or she can change this value later The agreement of each participant for each element is defined as follows: a : U × P → [0, 1] (2) The function a(uj , ei ) returns the value of participant uj for an element ei denoted by For example, we assume that an element ei in profile Pcan have four agreement values from four participants u1 , u2 , u3 , u4 respectively 0.4, 0.1, 1.0, and 0.9 The agreement function can be written as a(u1 , e) = 4, a(u2 , e) = 1, a(u3 , e) = 0, a(u4 , e) = Let Uei be the set of participants expressing agreement for the element ei and umax ∈ Uei be the participant whose trust value t(umax ) is the largest We propose Formula (3) to determine the strength of each element: ei = a (umax , ei ) + uei j=1 a uj , ei ∗ t(uj ) uei j=1 t(uj ) (3) Collaborative Ontology Building 153 where a uj , ei = a uj , ei − a (umax , ei ) (4) and ei is the strength of element ei ∈ P, uei is the set of participants expressing agreement for element ei , uei is the number of participants in uei , t(uj ) is the trust of participant uj ∈ uei , a (umax , ei ): is the agreement of the participant umax for the element ei , and a uj , ei is the agreement of the participant uj for the element ei (Table 1) Downloaded by [Northeastern University] at 23:35 29 December 2014 TABLE Example values of parameters, e, a, and T e1 a T a uj , ei e2 a T a uj , ei u1 u2 u3 u4 0.9 0.5 0.4 0.5 0.9 0.2 0.6 0.7 −0 −0 −0 u1 u2 u3 u4 0.9 0.9 1 0.9 0.7 0.8 0.9 0 0.1 0.1 Example Assume that participants express their agreement for e1 and e2 In the example of element e1 applying Formula (3) we have ei =09+ ∗ + (−0 4) ∗ + (−0 5) ∗ + (−0 4) ∗ = 61 (5) 08+02+06+07 The same as the example of element e2 we have ei =09+ 0∗09+0∗07+01∗08+01∗09 = 96 09+07+08+09 (6) According to Formula (3), the strength of an element ei depends not only on the values of the participants expressing their agreement for this element but also the reliability of these participants Let Eui be the set of elements in all profiles that participant ui contributed to the system When finding the consensus knowledge of a word or phrase, the system will update the trust of participant proposed this element using the following formula: (ui ) = Eui k=1 Eu i ek (7) where t(ui ): is the trust of participant ui , Eui is the number of elements in Eui , and ek is the strength of element ek ∈ Eui TRUST-BASED CONSENSUS FOR COLLABORATIVE ONTOLOGY BUILDING In the following, we briefly present the features of the proposed consensus methodology for CoO building Downloaded by [Northeastern University] at 23:35 29 December 2014 154 T H Duong et al • Phase 1—Preparatory: Instead of using a questionnaire in Delphi, we provide criteria for ontology building (Duong and Jo 2010) The criteria are aimed at guiding and evaluating the CoO building process In this phase, core ontologies are also introduced to participants • Phase 2—Contribution: Processes occur in rounds, allowing individuals to contribute or change their opinions of the current version of the ontology • Phase 3—Keeping track of changes and conflicts: The computer receives an updated version of the ontology from a participant It analyzes the updated version by keeping track of changes in the ontology and identifies conflicts between the updated version and the other ones • Phase 4—Controlled feedback: If no consensus is reached, the integration of previous versions will be used as a new version of the ontology, and the computer will show this new version of the group’s contribution (indicating to each individual their own previous response in Delphi) to each participant in the group Phase is repeated until consensus is reached Phase 1—Preparatory Criteria design We design six criteria including clarity, coherence consistency, extensibility, minimal ontological commitment, identity, and type of concept for ontology design (Duong and Jo 2010) These criteria are important in both guiding development of the ontology and evaluating the degree of its success We also provide seven criteria as composites of inclusive, egalitarian, interactive, representative, reconcilable, trust, and proof for CoO These criteria are important in guiding the development of CoO systems and assessing the quality of the CoO process Core ontology design Core ontologies are ontologies describing the most important concepts of a specific domain of interest Participants contribute to and annotate a core ontology iteratively to generate new versions of the ontology Concepts in core ontologies should permit any changes as extensions for customization without the need to revise existing definitions To develop the core ontology, we apply the following ontology-building steps described in Noy and McGuiness (2001): • • • • • • Step Determine the domain and scope of the ontology Step Consider reusing existing ontologies Step Enumerate important terms in the ontology Step Define the classes and the class hierarchy Step Define the object properties and date type properties of classes Step Define the restrictions of the data type and the object properties Collaborative Ontology Building 155 Phase 2—Contribution First, a core ontology is a changeable ontology in that the server only accepts changes to it Participants can access the server to check out the changeable version and make a copy of the version as their own target version, which they can modify at will Phase 3—Keeping Track of Changes and Conflicts Downloaded by [Northeastern University] at 23:35 29 December 2014 Keeping track of different versions and changes to an ontology Participants can commit their changes to the server at any time This allows several participants to make concurrent changes to a changeable ontology via their own target ontology Participants can share their understanding of the perspective of the target ontology, which represents the change history and the target ontology The target versions are kept in the server’s shared repository as new generated versions Identifying conflicts One well-known conflict situation is called a conflict profile, and a consensus method is an effective approach that can be used to solve this conflict In a conflict profile, there is a set of different versions of knowledge that explains the same goal or elements in the real world The consensus aims at determining a reconciled version of knowledge that best represents the given versions For example, many participants share their own knowledge to solve the same problem and there may be conflicts between their solutions In this situation, consensus is used to find the best solution in a compromise of all of the conflicted participants’ viewpoints Barthelemy and Janowitz (1991) described two classes of problems that are considered in the consensus theory as follows: Problems in which a certain and hidden structure is searched for Problems in which inconsistent data related to the same subject are unified The first class consists of problems with searching the structure of a complex or internally organized object This object can be a set of elements and the searched structure to be determined can be a distance function between these elements Data that are used to uncover this structure are usually based on empirical observations that reflect this structure but not necessarily in a precise and correct manner The second class consists of problems that arise when the same subject is represented (or voted on) in a different manner by experts (or agents or sites of a distributed system) In such a case, a particular method is desired that makes it possible to deduce one alternative from the set of given alternatives Nguyen (2008, 2009) addressed a function consensus for solving conflicts in participant opinions The following two cases can occur: The solution is independent of the opinions of the conflict participants The solution is dependent on the opinions of the conflict participants Downloaded by [Northeastern University] at 23:35 29 December 2014 156 T H Duong et al In the first case, the conflict participants not know the solution to the problem An example of this kind of conflict is different forecasts given by meteorological stations referring to the same region over a given period of time Thus, the solution is independent of the conflict content and the conflict participants In this case, their solutions have to reflect the proper solution but in an invalid and incomplete manner, so they must guess it In the second case, the reconciled solution must be constructed from the opinions of the conflict participants Voting in an election is an example In general, this case has a social or political character and a variety of participant viewpoints most often follow from differences of choice criteria or their position Here, we focus on the second case for collaborative ontology building, because participants have owning knowledge for the given problem However, instead of previous approaches for consensus considering participants at the same level, the collaborative group is flat and we consider the participants’ trust The trust is measured as Eq (7) When there are many profiles to solve a problem Pr, an element ei can appear in some profiles It will lead to the conflict profiles An example of conflict profiles for solving a problem: Pr = P1 = (e1 , 6), (e2 , 3), P2 = (e1 , 9), (e2 , 3), P3 = (e1 , 0), (e3 , 1) In the above example, element e1 appears three times with the strength values determined by Formula (3), respectively 0.6, 0.9, and 0; element e2 appears two times with the strength values 0.3 and 0.3, respectively; element e3 appears only one time with the strength value We propose an algorithm for giving consensus knowledge from profiles in the state of conflict profile based on the strength of each element The algorithm for solving the conflict profile is expressed as follows: Input: Given n profiles: P = P1 , P2 , …, Pn Output: P* the best represents to solve the problem For each Pk ∈ P For each ei ∈ Ek /*Ek : the set of elements of Pk */ Determine the umax ∈ uei with t(umax ) is the largest /* uei is the set of participants expressing agreement for ei */ ei = a (umax , ei ) + Let E = n uei j=1 a(uj ,ei )∗t (uj ) uei j=1 t uj ( ) Ei i=1 For each ek ∈ E Determine max = max( ek ) ∈ P If max ! consensus threshold then remove ei from E Create P* ← ei ∈ E 10 Return P* Collaborative Ontology Building 157 Downloaded by [Northeastern University] at 23:35 29 December 2014 Phase 4—Controlled Feedback If no consensus is reached, a reconciled ontology that is constructed from the integration of the generated versions will be used as a new version of the ontology There may be errors in the reconciled ontology For instance, we consider two versions of the concept Student : Student {Name, Date of Birth, Female} and the other Student {Name, Age, Male}, in which Date of Birth is more general than Age, so the Date of Birth with domain type of date is chosen as a property of the reconciled concept Student However, Male is an antonym of Female In this case, a reconciled property Gender has the subproperties Female and Male generated for the reconciled concept Student Therefore, the final reconciled concept Student has the structure {Name, Date of Birth, Gender} To solve this problem, we apply our previous research on ontology integration (Duong, Jason, et al 2009; Duong, Nguyen, and Jo 2009) to deal with the errors in the reconciled ontology The computer will send this new version of the group’s contribution (indicating to each individual their own previous response in Delphi) to each participant in the group Phase is repeated until consensus is reached EXPERIMENTS Evaluation Method In the experiment, we evaluate our proposed approach by applying for collaborative Vietnamese WordNet building We propose the structure of ontology-based Vietnamese WordNet improved from the ontology-based WordNet (Assem et al 2006) According to the structure, the Vietnamese WordNet is initialized by translating the English WordNet (Assem 2010; Assem et al 2006) Then the initialized version of Vietnamese WordNet (called IVW) is refined and evolved by a collaborative group of participants with various fields We compare the effectiveness of our proposed approach against Asian’s WordNet approach and our previous approach (Duong and Jo 2010) WordNet only includes four main types of content: noun, verb, adjective, and adverb They are organized into synsets that describe and represent a basic content and are connected by different kinds of relationships Ontology-based WordNet (Tuong et al 2013) has three main classes: Synset, Word, and WordSense Synset and WordSense have subclasses based on the distinction of lexical groups For Synset this means subclasses NounSynset, VerbSynset, AdjectiveSynset, and AdverbSynset For WordSense this means subclasses NounWordSense, VerbWordSense, etc Word has a subclass collocation used to represent words that have hyphens or underscores in them To develop OVW, we add a main class VWord (see Figure 1) that has a subclass V Collocation to OVW in order to store words or phrases in Vietnamese In Table we list the properties and their significance in OVW We propose three relations to enrich the structure of OVW: partOf, originalSenseOf, vietEng The partOf relation shows relation of geographic among places For example, Chua Mot Cot (Chùa_M o.ˆ t_C o.ˆ t) is a famous place in Ha Noi (Hà_N o.ˆ i) Therefore, synset Downloaded by [Northeastern University] at 23:35 29 December 2014 158 T H Duong et al FIGURE Class hierarchy of ontology-based Vietnamese WordNet (OVW) of Chùa_M o.ˆ t_C o.ˆ t, Chùa_M a.ˆ t, Nh aˆ t_Tru._Tháp has a partOf relation to synset of Hà_N o.ˆ i, Th˘a ng_Long, Thu_dô_Vi eˆt_Nam The originalSenseOf relation shows original sense of a word or phrase in Vietnamese Because Vietnamese borrows so many words from other languages, especially Chinese, the originalSenseOf relation is clearer For example, a compound word kh a´˘ c_c oˆ t_ghi_tâm in Vietnamese has the originalSenseOf relation to synset of in_sâu_vô, … because kh a´˘ c in the compound word kh a´˘ c_c oˆ t_ghi_tâm has means in_sâu_vô in Vietnamese However, a compound word thò’i_kha´˘ c has originalSenseOf relation to the synset thò’i_gian, … because kha´˘ c in thò’i_kha´˘ c means a quarter of an hour The vietEng relation shows the equivalent sense of a Vietnamese synset and an English synset For example, synset sinh_viên, … in Vietnamese has a vietEng relation to the synset student, etc., in English Experimental Results We performed a collaborative ontology-based Vietnamese WordNet building for 500 words or phrases randomly selected from IVW These words were shared with 30 participants (collaborative group) selected from Facebook (see Figure 2) We also shared the 500 words or phrase with a linguistics expert They performed the same task, collaboratively creating the sense of these 500 words In this experiment, Collaborative Ontology Building Downloaded by [Northeastern University] at 23:35 29 December 2014 TABLE 159 Relations in Ontology-Based Vietnamese WordNet Property Domain Range hyponymOf Entails similar To member Meronym Of substance Meronym Of part Meronym Of classified By Topic Synset Synset Synset Synset Synset Synset Synset Synset Synset Synset Synset Synset Synset Synset Nouns, adjectives Verbs Adjectives Nouns Nouns Nouns Nouns, adjectives, verbs classified By Usage Synset Synset classified By Region Synset Synset causes sameVerb Group As Synset Synset Synset Synset Nouns, adjectives, verbs Nouns, adjectives, verbs Verbs Verbs attribute Synset Synset derivationally Related WordSense WordSense antonymOf WordSense WordSense seeAlso WordSense WordSense participleOf WordSense WordSense adjective Pertains To Synset Synset adverb Pertains To Synset Synset gloss frame WordSense VerbWordSense Synset Synset Synset xsd:string xsd:string partOf original Sense Of vietEng Synset Synset Synset Target Meaning A is a hyponym of B B is an entailment of A B is a satellite of A B is a member meronym of A B is a substance meronym of A B is a part meronym of A A has been classified as a member of the class represented by B B is a cause of A Verb synset grouped together are similar in meaning Nouns to adjectives The adjective synset is a value of the noun synset Nouns, verbs, A is derived from B by means Adjectives, adverbs of a morphological affix Nouns, verbs, This operator specifies adjectives, adverbs antonymous words Verbs, adjectives Additional information about A can be obtained by seeing B Adjectives to verbs The adjective synset is a participle of the verb synset Adjectives to nouns An adjective synset pertains or adjectives to either the noun or adjective Adverbs to adjectives An adverb synset is derived from the adjective Synset and sentence The gloss for a synset Synset and a verb A generic sentence frame for construction pattern one or all words in a synset Nouns A is an area of B Nouns, verbs A is original sense of B Nouns, verbs, adverbs, An English word B corresadjectives ponds to a Vietnamese word A the following two aspects of the proposed approach were evaluated: collaboration vs expert and the comparison between consensus-based collaboration (Duong and Jo 2010) and trust-based consensus for collaboration (current approach) In the first aspect, the comparison of the cost and the time taken by the group and the experts is shown in Figure In Figure 3, the accuracy rates of the collaborative group and the experts are not very different This means that the collaborative group building is accurate The collaborative building requires much less time to complete the work than the experts Moreover, the cost for experts to build WordNet is very high in comparison with the collaborative approach Downloaded by [Northeastern University] at 23:35 29 December 2014 160 T H Duong et al FIGURE Collaborative group with their relationships FIGURE Comparison of the times, cost, and the accuracy rate of senses of 500 random words or phrases between group and group This current approach, CoO2, is an improvement of our previous approach (CoO1; Duong and Jo 2010) Here we make a comparison between CoO1 and CoO2 From Figure we see that CoO1 reaches consensus slower than CoO2 does The reason is that participants is far away to the consensus in CoO2 are avoided by their trust score instead of CoO1 takes into account all the participants in the consensus Here, it is postulated that the collaborative quality will increase with increasing consensus quality This was used to analyze how the collaborative process proceeds under the proposed methodology This also explains why CoO2’s result is more accurate than CoO1’s Figure presents a comparison between CoO1 and CoO2 The accuracy of the current version is higher than that of the previous version The reason here is current research considering the trust measure to evaluate participants’ contributions; therefore, the collaborative network of participants is not flat as the previous version’s Downloaded by [Northeastern University] at 23:35 29 December 2014 Collaborative Ontology Building 161 FIGURE Comparison of time reached to the consensus between CoO1 and CoO2 FIGURE The comparison of the times, costs and the accuracy rate of senses of 500 random words or phrases between CoO1 and CoO2 CONCLUSIONS This article aimed to investigate an effective methodology for collaborative ontology building in which we proposed trust-based consensus supporting collaboration We learned that consensus techniques are the core of CoO (Holsapple and Joshi 2002; Karapiperis and Apostolou 2006; Ruiz et al 2009b) The Delphi technique (Gallagher et al 1993) was applied to CoO in order to allow an entire group to reach a consensus Downloaded by [Northeastern University] at 23:35 29 December 2014 162 T H Duong et al by sharing their understanding of ontological perspective Trust-based consensus for solutions to conflict profiles involves generation of a reconciled ontology from conflicts between participants’ versions of an ontology Different from previous approaches (Tudorache et al 2008; Karapiperis and Apostolou 2006; Ruiz et al 2009a,b), we consider machine collaboration in the CoO building process In every cycle of the iterative process, participants’ contributed versions tracked changes and identified conflicts automatically Unless all versions of the ontology reach consensus, the ontology is revised and evolved by a CoO algorithm We applied the proposed method for Vietnamese WordNet building The result showed that the proposed method is more effective in comparison with previous ones in terms of both of time reduction and higher accuracy The reason for this is consideration of the trust measure to evaluate the participants’ contributions, which makes the collaborative network less flat than previous approaches This approach can be implemented and applied in different fields, like virtual organizations and e-decisional community management (Mancilla et al 2010a, 2010b) or multi-agent systems (Juszczyszyn et al 2006) It can also be useful in clustering methods for processing big data (Hong et al 2014) FUNDING This research was partially funded by the Vietnam National Foundation for Science and Technology Development (NAFOSTED) under grant number 102.01-2011.10 REFERENCES Assem, M V., A Gangemi, G Schreiber ‘‘RDF/OWL Representation of WordNet.’’ 2006 Available at: http://www.w3.org/TR/wordnet-rdf/ Assem, M V ‘‘Wordnet 3.0 in RDF.’’ 2010 Available at: http://semanticweb.cs.vu.nl/lod/ wn30/ Assem, M V., A Gangemi, and G Schreiber ‘‘RDF/OWL Representation of WordNet.’’ Available at: http://wwwhttp://www.w3.org/TR/wordnet-rdf/ Auer, S., S Dietzold, and T Riechert ‘‘OntoWikia Tool for Social, Semantic Collaboration.’’ In Proceedings of 5th International Semantic Web Conference, ISWC, Athens, GA, Springer, (May 2006), 736–749 Barthelemy, J P and M F Janowitz ‘‘A Formal Theory of Consensus.’’ SIAM Journal of Discrete Mathematics (1991): 305−22 Berners-Lee, T., J Hendler, and O Lassila ‘‘The Semantic Web.’’ Scientific American 284 (2001): 35−43 Cao, S T., L A Nguyen, A Szalas ‘‘WORL: A Nonmonotonic Rule Language for the Semantic Web.’’ Vietnam Journal of Computer Science DOI: 10.1007/s40595-013-0009-y Castillo, J A R., A Silvescu, D Caragea, J Pathak, and V G Honavar ‘‘Information Extraction and Integration from Heterogeneous, Distributed, Autonomous Information Sources a Federated Ontology Driven Query Centric Approach.’’ In Proceedings of the IEEE International Conference on Information Reuse and Integration, Information Reuse and Integration, 2003 IRI 2003, 183−91 Danilowicz, C and N T Nguyen ‘‘Consensus-Based Partitions in the Space of Ordered Partitions.’’ Pattern Recognition 21, no (1988): 269−73 Downloaded by [Northeastern University] at 23:35 29 December 2014 Collaborative Ontology Building 163 Doan, A H., J Madhavan, P Domingos, and A Halevy ‘‘Ontology Matching: A Machine Learning Approach.’’ In Handbook on Ontologies, 385−403 Springer, 2004 Duong, T H., J J Jason, N T Nguyen, and G S Jo ‘‘Complexity Analysis of Ontology Integration Methodologies: A Comparative Study.’’ Journal of Universal Computer Science 15 (2009): 877−97 Duong, T H and G S Jo ‘‘Collaborative Ontology Building by Reaching Consensus among Participants.’’ Information - An International Interdisciplinary Journal 13, no (2010): 1557−69 Duong, T H., N T Nguyen, and G S Jo ‘‘A Method for Integrating Multiple Ontologies’’ Cybernetics and Systems 40 (2009): 123−45 Gabel, T., Y Sure, and J Voelker ‘‘KAON—Ontology Management Infrastructure.’’ In SEKT Information Deliverable D3.1.1.a Institute AIFB, University of Karlsruhe, (2004) Gallagher, M., T Hares, J Spencer, C Bradshaw, and I Webb ‘‘The Nominal Group Technique: A Research Tool for General Practice?’’ Family Practice 10 (1993): 76−81 Gruber, T R ‘‘A Translation Approach to Portable Ontology Specifications’’ Knowledge Acquisition (1993): 199−220 Hernes, M and N T Nguyen ‘‘Deriving Consensus for Hierarchical Incomplete Ordered Partitions and Coverings.’’ Journal of Universal Computer Science 13, no (2007): 317−28 Holsapple, C W and K D Joshi ‘‘Collaborative Approach in Ontology Design.’’ Communications of the ACM 45 (2002): 42−47 Hong, T P., Y L Liou, S L Wang, and B Vo ‘‘Feature Selection and Replacement by Clustering Attributes.’’ Vietnam Journal of Computer Science DOI: 10.1007/s40595-013-0004-3 Juszczyszyn, K., N T Nguyen, G Kolaczek, A Grzech, A Pieczyniak, R Katarzyniak.‘‘ AgentBased Approach for Distributed Intrusion Detection System Design.’’ Lecture Notes in Computer Science 3993 (2006): 224−31 Karapiperis, S and D Apostolou ‘‘Consensus Building in Collaborative Ontology Engineering Processes.’’ Journal of Universal Knowledge Management (2006): 199−216 Mancilla, L., C Sanin, and E Szczerbicki ‘‘Using Human Behaviour to Develop KnowledgeBased Virtual Organisations.’’ Cybernetics and Systems 41, no (2010a): 577−91 Mancilla, L., C Sanin, and E Szczerbicki ‘‘Smart Knowledge Sharing Platform for e-Decisional Community.’’ Cybernetics and Systems 41 (2010b): 17−30 Nakamatsu, K and J M Abe ‘‘The Paraconsistent Process Order Control Method.’’ Vietnam Journal of Computer Science DOI: 10.1007/s40595-013-0002-5 Nguyen, N T Advanced Methods for Inconsistent Knowledge Management London: SpringerVerlag, 2008 Nguyen, N T ‘‘Processing Inconsistency of Knowledge in Determining Knowledge of a Collective.’’ Cybernetics and Systems 40, no (2009): 670−88 Noy, N and D L McGuiness Ontology Development 101: A Guide to Creating Your First Ontology Stanford Knowledge Systems Laboratory Technical Report KSL-01-05 and Stanford Medical Informatics Technical Report SMI-2001-0880, Knowledge Systems Laboratory Stanford University (2001) Pill, J ‘‘The Delphi Method: Substance, Context, a Critique and an Annotated Bibliography.’’ Socioeconomic Planning Sciences (1971): 57−71 Ruiz, E J., B C Grau, I Horrocks, and R Berlanga ‘‘Building Ontologies Collaboratively Using ContentCVS.’’ Proceedings of the 22nd International Workshop on Description Logics Oxford, UK, June 2009 (2009a) Ruiz, E J., B C Grau, I Horrocks, and R Berlanga ‘‘Conflict Detection and Resolution in Collaborative Ontology Development.’’ Technical Report 2009b Available at: http://krono.act.uji.es/people/Ernesto/contentcvs 164 T H Duong et al Downloaded by [Northeastern University] at 23:35 29 December 2014 Smith, B and C Welty ‘‘FOIS Introduction: Ontology—Towards a New Synthesis.’’ Proceedings of the International Conference on Formal Ontology in Information Systems, New York, NY (October 17–19, 2001), 3–9 Sure, Y ‘‘OntoEdit: Collaborative Ontology Development for the Semantic Web.’’ In International Semantic Web Conference 2002 (ISWC 2002), Sardinia, Italia, (June 9–12, 2002), 221–235 Tudorache, T., N F Noy, S W Tu, and M A Musen ‘‘Supporting Collaborative Ontology Development in Protg.’’ In Proceedings of 7th International Semantic Web Conference, Lecture Notes in Computer Science 17−32 Karlsruhe, Germany: Springer, 2008 Tuong, L., T H Duong, B Vo, and S Kang ‘‘Consensus for Collaborative Ontology-Based Vietnamese WordNet Building.’’ In Lecture Notes in Computer Science 7803 (2013): 499−508 ... application areas Ontology building is crucial for the aforementioned issues The main goal of this research is to investigate an effective methodology for collaborative ontology building A trust-based. .. Figure 3, the accuracy rates of the collaborative group and the experts are not very different This means that the collaborative group building is accurate The collaborative building requires much... article aimed to investigate an effective methodology for collaborative ontology building in which we proposed trust-based consensus supporting collaboration We learned that consensus techniques