Is Good Enough Really Good Enough- Does algorithmic metadata sea

Purdue University Purdue e-Pubs Charleston Library Conference Is Good Enough, Really Good Enough? Does algorithmic metadata search replace the need for discipline-oriented databases? Mark Hyer ProQuest, mark.hyer@proquest.com Helen Ivy College of Charleston, ivyh@cofc.edu Bruce Pencek Virginia Polytechnic Institute and State University, bpencek@vt.edu Sharon Jordan OSTI, jordans@osti.gov Sharon Tahirkheli American Geological Institute Follow this and additional works at: https://docs.lib.purdue.edu/charleston Part of the Library and Information Science Commons An indexed, print copy of the Proceedings is also available for purchase at: http://www.thepress.purdue.edu/series/charleston You may also be interested in the new series, Charleston Insights in Library, Archival, and Information Sciences Find out more at: http://www.thepress.purdue.edu/series/charleston-insights-library-archivaland-information-sciences Mark Hyer, Helen Ivy, Bruce Pencek, Sharon Jordan, and Sharon Tahirkheli, "Is Good Enough, Really Good Enough? Does algorithmic metadata search replace the need for discipline-oriented databases?" (2009) Proceedings of the Charleston Library Conference http://dx.doi.org/10.5703/1288284314744 This document has been made available through Purdue e-Pubs, a service of the Purdue University Libraries Please contact epubs@purdue.edu for additional information ,6*22'(128*+5($//VHH)LJ@WRWKHJUDGXDWHVWXGHQWVIDFXOW\DQGVFLHQWLVWVIURPWKHWKUHHDJHQFLHV VHUYHGE\WKHOLEUDU\7KHVXUYH\DVNHGUHVSRQGHQWVWRUDQNWKHLUFKRLFHVHOHFWWKHLUIUHTXHQF\ RIXVDJHDQGHYDOXDWHHIIHFWLYHQHVVIRUWKHVHGDWDEDVHV$6)$$TXDWLF6FLHQFHVDQG )LVKHULHV'DWDEDVH(QYLURQPHQWDO6FLHQFHV 3ROOXWLRQ0DQDJHPHQW'DWDEDVH2FHDQLF $EVWUDFWV3XE0HG&HQWUDO6FL)LQGHU6FKRODU:HERI6FLHQFH²6FLHQFH&LWDWLRQ=RRORJLFDO 5HFRUG3OXV>1RWH3XE0HG&HQWUDOZDVLQFOXGHGEHFDXVHLWZDVWKHRQO\GDWDEDVHWRSURYLGH FRYHUDJHRIVRPHQHZDUHDVRIUHVHDUFKLQWHUHVW@1H[WUHVSRQGHQWVZHUHDVNHGWROLVWXSWR WKUHHWRSLFVWKDWWKH\URXWLQHO\VHDUFKLQWKHGLVFLSOLQHVSHFLILFGDWDEDVHV5HVSRQGHQWVZHUH WKHQDVNHGDERXWWKHXVDJHDQGYDOXHRIWKHJHQHUDOGDWDEDVHV$FDGHPLF6HDUFK3UHPLHUDQG ,QIR7UDF$FDGHPLF2QH)LOHWRUHVHDUFKWKHLUOLVWHGUHVHDUFKWRSLFV)LQDOO\UHVSRQGHQWVZHUH DVNHGDERXWWKHXVDJHDQGYDOXHRIWKHPHJDPHWDGDWDGDWDEDVHV*RRJOH6FKRODUDQG 6FLHQFHJRYWRUHVHDUFKWKHLUOLVWHGUHVHDUFKWRSLFV7KHVXUYH\ZDVGHOLYHUHGE\HPDLOWR DSSUR[LPDWHO\UHVHDUFKHUVZLWKDUHVSRQVHUDWHLQFOXGLQJIDFXW\JUDGXDWH VWXGHQWVVFLHQWLVWVWHFKQLFLDQVDQGDGPLQLVWUDWLYHVWDIIFRPSOHWLQJWKHVXUYH\  6XUYH\5HVXOWVIRU6FLHQWLILF'DWDEDVHV  7KHUHVXOWVIRUIUHTXHQF\RIXVHIRUWKHVFLHQWLILFGDWDEDVHVUHWXUQHGQRVXUSULVHV'DLO\XVH± RIXVHUV:HHNO\XVH±RIXVHUV%LZHHNO\XVH±RIXVHUV0RQWKO\XVH±RI XVHUV2WKHU±RIXVHUV  7ZRTXHVWLRQVDVNHGXVHUVWRLQGLFDWHWKHLUILUVWDQGVHFRQGSUHIHUHQFHIRUWKHVFLHQWLILF GDWDEDVHV7KHWRSIRXUGDWDEDVHVZHUH$TXDWLF6FLHQFHV )LVKHULHV'DWDEDVH:HE RI6FLHQFH3XE0HG&HQWUDODQG=RRORJLFDO5HFRUG3OXV7KHWRSIRXU GDWDEDVHVIRUVHFRQGSUHIHUHQFHZHUH$TXDWLF6FLHQFHV )LVKHULHV'DWDEDVH:HERI 6FLHQFH3XE0HG&HQWUDODQG=RRORJLFDO5HFRUG3OXV  6XUYH\5HVXOWVIRU*HQHUDO'DWDEDVHV  5HVSRQGHQWVJHQHUDOO\VWXGHQWVUHSRUWHGXVLQJ$FDGHPLF6HDUFK3UHPLHUDQGRU ,QIR7UDF$FDGHPLF2QH)LOHRQDGDLO\WRPRQWKO\EDVLV2IWKRVHXVLQJWKHJHQHUDOGDWDEDVHV IHOWWKHGDWDEDVHVZHUHH[WUHPHO\XVHIXOLHWKH\IRXQGDOOWKHVRXUFHVWKH\QHHGHGDQG QHYHUXVHGWKHVFLHQWLILFGDWDEDVHVIHOWWKHGDWDEDVHVZHUHYHU\XVHIXOLHWKH\IRXQG NH\VRXUFHVWRJHWWKHPVWDUWHGIHOWWKHGDWDEDVHVZHUHXVHIXOLHWKH\IRXQGRQHRUWZR VRXUFHVWKH\FRXOGXVH7KHGDWDEDVHVZHUHQHYHUXVHGE\RIWKHUHVSRQGHQWV  6XUYH\5HVXOWVIRUWKH0HJD0HWDGDWD'DWDEDVHV  )RU*RRJOH6FKRODUDQGRU6FLHQFHJRYRIWKHUHVSRQGHQWVXVHGWKHGDWDEDVHVRQDGDLO\ WRPRQWKO\EDVLV7ZHQW\HLJKWSHUFHQWRIWKHUHVSRQGHQWVQHYHUXVHWKHVHVRXUFHV*RRJOH 6FKRODUZDVSUHIHUUHGE\RIWKRVHXVLQJWKHPHJDPHWDGDWDGDWDEDVHV  &RPSDULVRQRI6HDUFK5HVXOWVIURP)RXU'DWDEDVHV  6HOHFWLQJIRXUWRSLFVWKDWUHVHDUFKHUVURXWLQHO\VHDUFKHGDQGXVLQJ-RDQ3DUNHU¶VPHWKRGRORJ\ IRUHYDOXDWLQJELEOLRJUDSKLFGDWDEDVHRYHUODSLQPDULQHVFLHQFHDVGHWDLOHGE\KHUDUWLFOHLQWKH  6SULQJLVVXHRI,VVXHVLQ6FLHQFHDQG7HFKQRORJ\/LEUDULDQVKLSWKHOLEUDULDQGHWHUPLQHG WKHQXPEHURIXQLTXHVRXUFHVFLWHGIRUWKHSHULRGLQIRXUGDWDEDVHV7KHIRXUWRSLFV $UWLILFLDOUHHI>5HHI@&RUDOKHDOWK>&RUDO@+DUPIXODOJDOEORRPV>+$%@0DULQHJHQRPLFV>*HQ@ 7KHIRXUGDWDEDVHV$TXDWLF6FLHQFHV )LVKHULHV'DWDEDVH>$6)$@=RRORJLFDO5HFRUG3OXV >=5@:HERI6FLHQFH>:26@*RRJOH6FKRODU>*6@  )LQGLQJV  $TXDWLF6FLHQFHV )LVKHULHV'DWDEDVHDQG=RRORJLFDO5HFRUG3OXVSURYLGHGWKHPRVWXQLTXH VRXUFHVIRUWKHDSSOLHGDQGELRORJLFDOVFLHQFHV=RRORJLFDO5HFRUG3OXVSURYLGHGWKHEHVW FRYHUDJHRILQWHUQDWLRQDOSXEOLFDWLRQV*RRJOH6FKRODUSURYLGHGEHWWHUFRYHUDJHIRUKHDOWK UHODWHGWRSLFVDQGIRUWRSLFVRISRSXODULQWHUHVW*RRJOH6FKRODUGRHVFRYHUIURPWRRI WKHJUH\OLWHUDWXUHFLWHGLQ$TXDWLF6FLHQFHV )LVKHULHV'DWDEDVHDQG=RRORJLFDO5HFRUG3OXV IRUWKHIRXUVHOHFWHGWRSLFV:HERI6FLHQFH²6FLHQFH&LWDWLRQGLGSURYLGHEHWWHUMRXUQDODUWLFOH FRYHUDJHIRUVRPHWRSLFVZKLOHLWZDVHTXDOWRWKHFRYHUDJHLQ*RRJOH6FKRODUIRURWKHUWRSLFV  ,PSOLFDWLRQV  7KHDERYHILQGLQJVKDYHGLUHFWLPSOLFDWLRQVIRUEXGJHWLQJDQGIRUOLEUDU\LQVWUXFWLRQ7KHWRS WKUHHGLVFLSOLQHVSHFLILFVXEVFULSWLRQGDWDEDVHV>$TXDWLF6FLHQFHV )LVKHULHV'DWDEDVH:HE RI6FLHQFH²6FLHQFH&LWDWLRQ=RRORJLFDO5HFRUG@FRQWLQXHWREHUHTXLUHGLIDOORIWKHOLEUDU\¶V UHVHDUFKHUVDUHWRFRQGXFWFRPSOHWHOLWHUDWXUHUHYLHZVRIWKHLUUHVHDUFKWRSLFV,QVWUXFWLRQ UHJDUGLQJGDWDEDVHXVDJHFRYHUDJHQHHGVWREHWDUJHWHGWRUHVHDUFKJURXSVVRWKDWDOO UHVHDUFKHUVXQGHUVWDQGWKHYDOXHDQGLGLRV\QFUDVLHVRIGDWDEDVHVVSHFLILFIRUWKHLUUHVHDUFK ,QVWUXFWLRQIRUVWXGHQWVPXVWVWUHVVGHPRQVWUDWHWKHYDOXHDQGDGYDQWDJHVRIWKHGLVFLSOLQH VSHFLILFGDWDEDVHV  0D[LPL]LQJWKHOHDUQLQJYDOXHRIGLVFLSOLQHRULHQWHGGDWDEDVHVIRUQRQGRPDLQH[SHUWV %UXFH3HQFHN9LUJLQLD3RO\WHFKQLF,QVWLWXWHDQG6WDWH8QLYHUVLW\  ,QRXUDVVXPSWLRQVDERXWUHVHDUFKQHHGVDQGEHKDYLRUDQGLQRXUDOORFDWLRQRIUHVHDUFK UHVRXUFHVWRFRQQHFWVWXGHQWVWRVFKRODUO\LQIRUPDWLRQOLEUDULDQVDQGWHDFKLQJIDFXOW\ULVN SULYLOHJLQJDWOHDVWLPSOLFLWO\VRPHPRGHVRINQRZOHGJHDFTXLVLWLRQ±DQGWKXVVRPHXVHUVDQG VRPHNLQGVRILQIRUPDWLRQ±DWWKHH[SHQVHRIRWKHUV3HUYDVLYHLQIRUPDWLRQWHFKQRORJ\ H[DFHUEDWHVWKLVULVN:HHGXFDWRUVKDYHDSHGDJRJLFDOREOLJDWLRQWRSURYLGHXVHUVFRJQLWLYH LQIUDVWUXFWXUHVWRIUDPHWKHLUHQFRXQWHUVZLWKWHFKQRORJLFDOO\FRQVWLWXWHGLQIRUPDWLRQ 7KHORZRSSRUWXQLW\FRVWRIILQGLQJVRPHWKLQJZLWKDVHDUFKHQJLQHRQWKHIUHH:HEFRQFHDOV WKHFRJQLWLYHRYHUKHDGWKDWSHUYDVLYHLQIRUPDWLRQLPSRVHVRQUHVHDUFKHUVZKRHQWHUXQIDPLOLDU NQRZOHGJHGRPDLQV7KHLQIRUPDWLRQWHFKQRORJ\LVDWKDQGVHDUFKHQJLQHVPDNHHQRXJK VHQVHDQGTXLFNO\HQRXJKWRLQVSLUHFRQILGHQFHLQXVHUV2QWKHRWKHUKDQGIRU XQGHUJUDGXDWHXQLYHUVLW\VWXGHQWVWKHVXEVWDQWLYHFRQFHUQVDSSURDFKHVGLVFRXUVHVDQG WHFKQRORJLHVRIPRVWDFDGHPLFGLVFLSOLQHVPDNHVHQVHRQO\JUDGXDOO\0RVW86XQLYHUVLWLHV FRQFHQWUDWHWKLVVHQVHPDNLQJLQDFDGHPLFPDMRUV&RUHFXUULFXODDQGGLVWULEXWLRQ UHTXLUHPHQWVDWWHPSWWRH[WHQGWKHVHQVHPDNLQJLQWRRWKHUGLVFLSOLQHVZLWKGLIIHUHQWGHJUHHV RIVXFFHVV  ,ZULWHIURPWKHVWDQGSRLQWRIDRQHWLPHSROLWLFDOVFLHQFHSURIHVVRUZKRDVDQLQVWUXFWLRQDO OLEUDULDQDQGLQYHVWLJDWRUKDVFROODERUDWHGLQWHQVLYHO\ZLWKWHDFKLQJIDFXOW\WRZUHVWOHZLWKD IXQGDPHQWDOTXHVWLRQ+RZFDQZHLQGXFHVWXGHQWVWRRSHUDWLRQDOL]HWKHLULQIRUPDWLRQQHHGVVR WKDWWKH\FDQWKHQFKRRVHWKHDSSURSULDWHUHVRXUFHVDQGXVHDSSURSULDWHVHDUFKWHFKQLTXHVWR IXOILOOVSHFLILFUHVHDUFKWDVNV"  7KHUHDUHVHYHUDOIDFWRUVWKDWLQYLWHHYHU\RQH±VWXGHQWVEXWDOVRIDFXOW\DQGOLEUDULDQV±WREHJ WKDWTXHVWLRQ  7KHUHDG\DYDLODELOLW\RIVHDUFKDQGUHWULHYDOV\VWHPVSDUDGLJPDWLFDOO\*RRJOHUHZDUGVERWK QDwYHVHDUFKHUV±ZKRKDYHDEURDGVORSS\QRWLRQRIUHOHYDQFH±DQGGRPDLQH[SHUWV± EHFDXVHWKH\EULQJYHU\SUHFLVHQRWLRQVRIZKDWWKH\QHHGDQGZKDWZRUGVGHVFULEHLW%\WKHLU YHU\GLIIHUHQWOLJKWVERWKVXFFHHG,IERWKQDwYHXQGHUJUDGDQGH[SHUWSURIHVVRUʊ NQRZ´KRZWR ILQGWKHLQIRUPDWLRQWKH\QHHGHDFKPD\DVVXPHWKHRWKHUWKLQNVWKHVDPHZD\DQGQHLWKHU ZRXOGIHHOFRPSHOOHGWRUHWKLQNKLVKHURZQIDPLOLDUSURFHVV  7KHGLVFLSOLQDU\QDWXUHRIWKHXQLYHUVLW\FUHDWHVDQGUHLQIRUFHVFRJQLWLYHKRUL]RQVDWWKHVDPH WLPHDVLWGLIIHUHQWLDOO\DOORFDWHVLQIRUPDWLRQUHVRXUFHV$PELJXLW\LVDYH[LQJSUREOHPLQDQ\ VHDUFKLQJ7KLVLVIXUWKHUFRPSRXQGHGE\WKHRIWHQVXEWOHYDULHWLHVLQGHQRWDWLRQVDQG FRQQRWDWLRQVRIVLJQLILFDQWWHUPVDFURVVGLVFLSOLQDU\OLQHVDQGVRPHWLPHVDFURVVVXEILHOGVDQG RWKHUIDXOWOLQHVZLWKLQDGLVFLSOLQHDQGEHWZHHQDFDGHPLFDQGOD\FRPPXQLWLHVZKHQFHWKH XQGHUJUDGXDWHVFRPH  (YHQDVWKH\LQGXOJHWKHDOOWRRKXPDQLQFOLQDWLRQWRFORQHWKHPVHOYHVLQWKHLUVWXGHQWVGRPDLQ H[SHUWVRIWHQIRUJHWWKDWWKHLURZQKDELWVRIPLQGDVSURIHVVLRQDODQDO\VWVDQGUHVHDUFKHUVDUH OHDUQHGDELOLWLHV,WLVVHFRQGQDWXUHIRUDFRXSOHSURIHVVRUVFKDWWLQJDWDFRQIHUHQFHWRWXUQD SDVVLQJREVHUYDWLRQLQWRDQ"IQWHUHVWLQJTXHVWLRQ´WKHQK\SRWKHVL]HH[SODQDWLRQVDOOLQDIHZ PRPHQWV  7KHVHIDFWRUVUHIOHFWDFRPPRQUHDOLW\WKHUHDUHNQRZOHGJHLQVLGHUVDQGWKHUHDUHNQRZOHGJH RXWVLGHUVVHSDUDWHGE\DFRJQLWLYHFKDVP7KHLQVLGHUVNQRZHQRXJKLQWKHLUGRPDLQVWR UHFRJQL]HKROHVLQWKHLUNQRZOHGJHOLNH6RFUDWHVWKH\NQRZWKDWWKH\GRQRWNQRZDQG SURFHHGWRILOOWKHJDSV&RQYHUVHO\GRPDLQRXWVLGHUV¶XQGHUVWDQGLQJLVQRWGHYHORSHGHQRXJK WRDSSUHFLDWHWKHFRQWRXUVRIWKHLULJQRUDQFH±WKHNQRZQXQNQRZQVERWKVXEVWDQWLYHDQG SURFHGXUDO0RUHRYHUWKHLUVHOIFRQILGHQFHDVKHUHWRIRUHJRRGHQRXJKVHDUFKHUVPDNHVWKHP OHVVDWWHQWLYHWRVLJQVRISRWHQWLDOO\XVHIXONQRZOHGJHWKDWLVVLPSO\EH\RQGWKHLUSHUFHSWXDORU FRJQLWLYHIUDPHZRUNV±WKHLUXQNQRZQXQNQRZQV  2XUH[SHULHQFHDQGDVVHVVPHQWGDWDIURPSROLWLFDOVFLHQFHFRXUVHVDW9LUJLQLD7HFKVXJJHVW WKDWVWXGHQWVYDOXHLQVWUXFWLRQDQGH[HUFLVHVWKDWGHYHORSWKHSXUSRVLYHKDELWVRIPLQGRI LQVLGHUVDORQJZLWKWHFKQLFDOIDFLOLW\XVLQJLQIRUPDWLRQUHVRXUFHV:HKDYHGHYHORSHGDVLPSOH DQDO\WLFDOIUDPHZRUNWKDWOLEUDULDQVDQGWHDFKLQJIDFXOW\FDQXVHWRLGHQWLI\WHQGHQFLHVLQ VWXGHQWV¶UHVHDUFKEHKDYLRUVLQRUGHUWRJXLGHLQWHUYHQWLRQVWKDWUHLQIRUFHGHVLUHGLQIRUPDWLRQ EHKDYLRUVDQGUHGLUHFWLQDSSURSULDWHRQHV:HKDYHDOVRGHYHORSHGDOHDUQLQJPRGHOZKHUHE\ VWXGHQWVIROORZDUHFRPPHQGHGVHTXHQFHZKLFKZHFDOOZRUGPLQLQJIRUZRUNLQJWKURXJK GLIIHUHQWNLQGVRILQIRUPDWLRQVRXUFHVWROHDUQWKHʊ XQNQRZQXQNQRZQ´WHUPLQRORJ\RIXQIDPLOLDU GRPDLQVDQGDSSO\WKHPLQLQFUHDVLQJO\SUHFLVHVHDUFKHVWRPDVWHUHYHUPRUHLQFOXVLYH LQIRUPDWLRQHQYLURQPHQWV)RUWKHSUHVHQWGLVFXVVLRQ,IRFXVRQWKHHVVHQWLDOUROHSOD\HGE\ GLVFLSOLQHRULHQWHGGDWDEDVHV±FUXFLDOO\LQFRUSRUDWLQJDWWULEXWHVRIWUDGLWLRQDORQOLQHDEVWUDFWLQJ DQGLQGH[LQJ±LQWKDWODUJHUSURFHVV  :KHQZHNQRZZKDWNLQGRILQGRPDLQNQRZOHGJHUHVHDUFKHUVEULQJWRWKHWDVNDQGZHLGHQWLI\ WKHFXHVRUWRSLFIUDPLQJV\VWHPVWKH\FDQXVHWRQDYLJDWHNQRZOHGJHVSDFHZHFDQH[SHFW GLIIHUHQWSDWWHUQVRUWHQGHQFLHVRIVHDUFKEHKDYLRU)URPWKHVHZHFDQVHOHFWDSSURSULDWHNLQGV RILQWHUYHQWLRQIURPWKHZD\FRXUVHLQVWUXFWRUVGHVLJQDVVLJQPHQWVWRWKHUHVRXUFHVDQG WHFKQLTXHVLQVWUXFWLRQDODQGUHIHUHQFHOLEUDULDQVFRXQVHO2XUW\SRORJ\VKRXOGDOVRVHUYHDVD  UHPLQGHUWRWHDFKLQJIDFXOW\DQGOLEUDULDQVWRUHIOHFWRQWKHLURZQDVVXPSWLRQVDQGSURSHQVLWLHV DVUHVHDUFKHUV  7DEOH7HQGHQFLHVRINQRZOHGJHDFTXLVLWLRQLQOLEUDU\UHVHDUFK  5HVRXUFHFXHVIUDPHV *URXQGHG([SOLFLW   6WURQJ 5HVHDUFKHU¶V 5HFDSLWXODWHGLVFLSOLQDU\QRUPV NQRZOHGJH  EDVH :HDN 'LVFRYHU $EVWUDFW6HOIJHQHUDWHG 5HWULHYH  )RUDJHODWHUDOO\   $VH[SHUWVSURIHVVRUVOHYHUDJHWKHLUH[LVWLQJNQRZOHGJH7KH\NQRZWKHFXOWXUHRIWKHLUILHOGV± WKHMDUJRQWKHFDQRQLFDOZULWHUVWKHRXWOHWVIRUVFKRODUO\FRPPXQLFDWLRQ±DQGXVHWKHPWR VWUXFWXUHWKHLUVHDUFKIRUDQGDSSOLFDWLRQRILQIRUPDWLRQ7KH\KDYHILUPLGHDVRIZKDW LQIRUPDWLRQWKH\ZDQWDQGWKH\KDYHWKHZKHUHZLWKDOWRORRNIRULWHIILFLHQWO\$VWKHLUH[SHUWLVH GHHSHQVWKHLURSSRUWXQLW\FRVWGHFUHDVHVIRUUHWULHYLQJPRUHLQIRUPDWLRQLQWKHLUGRPDLQV)XOO WH[WVRXUFHVWKHPRUHWKHEHWWHUVXLWWKHLUQHHGVZHOO7RWKHH[WHQWWKDWVHDUFKHUVFDQEULQJ DGLVFLSOLQH¶VVHWWOHGWHUPLQRORJLFDOFRQYHQWLRQVZLWKWKHPH[SOLFLWFRQWUROOHGYRFDEXODULHVDUH OHVVVDOLHQWDQGWKHHYROYLQJVHPDQWLFZHEPD\ZHOOVDWLVI\WKHLUVHQVHRIQHHG7KHULVNLV WKDWH[SHUWV¶HVWDEOLVKHGSDUDGLJPVEHFRPHVHOIUHLQIRUFLQJZLWKLQQDUURZNQRZOHGJHKRUL]RQV SRWHQWLDOO\REOLYLRXVRIUHOHYDQWZRUNDQGXVDJHVRXWVLGHWKHLURZQ  8QGHUJUDGXDWHVWXGHQWVDUHQRWIRUWKHPRVWSDUWVWXSLG%XWWKH\DUHE\GHILQLWLRQUHODWLYHO\ LJQRUDQW:KDWHYHUWKH\PD\OHDUQLQ.WKH\FRPHWRFROOHJHZLWKDWEHVWDZHDNDQGRIWHQ GLVWRUWHGDFTXDLQWDQFHZLWKDFDGHPLFGLVFLSOLQHVFRQYHQWLRQVDQGPRUHV&RQIURQWHGZLWK DVVLJQPHQWVWKDWDVVXPHLQVLGHUSHUVSHFWLYHVWKH\JDWKHULQIRUPDWLRQWKURXJKEUXWH HPSLULFLVPIRUDJLQJIRUUHVXOWVWKDWDUHJRRGHQRXJKWRWKHPEHFDXVHWKH\GRQRWNQRZWKH PRUHH[DFWLQJVWDQGDUGVE\ZKLFKH[SHUWVWHDFKLQJWKHLUFODVVHVZLOOJUDGHWKHP2SHQHQGHG LQGLVFULPLQDWHVHDUFKLQJLQXQVWUXFWXUHGLQIRUPDWLRQVSDFHVOLNH*RRJOHPDNHVVHQVHWRWKHP KRZHYHUDFFLGHQWDOWKHLUWHFKQLTXHDSSHDUVWRXV  7KHDQWLGRWHRIFRXUVHLVWRSURYLGHVWXGHQWVH[SOLFLWRSSRUWXQLWLHVWRGLVFRYHUKRZNQRZOHGJH LVVWUXFWXUHGDQGWRVLWXDWHWKHLUQHHGZLWKLQRQHRUPRUHVWUXFWXUHV:HPXVWUHPHPEHUWKDW WKHVHDUHVWUXFWXUHVRIPHDQLQJQRWVLPSO\RILQIRUPDWLRQ,QWKLVUHVSHFWWUDGLWLRQDOOLEUDU\ GLVFRYHU\SUDFWLFHVVXFKDVSK\VLFDOFROORFDWLRQRIUHODWHGZRUNVDQGFRQWUROOHGYRFDEXODULHV PD\KDYHWKHDGYDQWDJHRYHUPRUHUHFHQWF\EHUQHWLFGLVFRYHU\WRROVSK\VLFDOREMHFWVPD\ FDUU\PRUHPHDQLQJWKDQDEVWUDFWODEHOVDQGDULJRURXVFRQWUROOHGYRFDEXODU\PLJKWDFWXDOO\EH DOHVVDEVWUDFWSRLQWRIGHSDUWXUHIRUDQDwYHXVHUWKDQDZRUGFORXGGLVWLOOHGIURPD%DEHORI  QDWXUDOODQJXDJH  3UDJPDWLFDOO\XQGHUJUDGXDWHVPXVWOHDUQWRWKLQNDQGUHVHDUFKOLNHWKHLUSURIHVVRUV7KHVH QDwYHXVHUVQHHGKHOSDQGH[SHULHQFHVWRGLVFRYHUZKHUHWKHUHOHYDQWZRUNLVGRQHZKRWKH H[SHUWVDUHDQGZKDWWHUPVWKH\XVHWRWDONDERXWWKHLUZRUN7KHFRUHUHDOLW\LQ86KLJKHU HGXFDWLRQLVWKDWERWKSURIHVVRULDOH[SHUWLVHDQGFXUULFXODFRQWLQXHWREHEXLOWDURXQGGLVFLSOLQHV 'LVFLSOLQHVVWUXFWXUHPHDQLQJQRWPHUHO\LQIRUPDWLRQ7KHXELTXLWRXVXVHRIPDMRUILHOGV FRPSOHPHQWHGE\GLVWULEXWLRQUHTXLUHPHQWVDQGFRUHFXUULFXODWRH[SRVHVWXGHQWVWRRWKHU GLVFLSOLQHVDVVXPHVWKDWVWXGHQWVFDQQRWOHDUQGLVFLSOLQDU\NQRZOHGJHRQWKHLURZQEXWUDWKHU PXVWUHFHLYHIRUPDOLQVWUXFWLRQ,WLVRGGWKHQWKDWVRPDQ\SHRSOHLQKLJKHUHGXFDWLRQVHHPWR  DVVXPHWKDWVWXGHQWVZLOOGLVFRYHURQHWKHLURZQKRZWRDFTXLUHDQGXVHLQIRUPDWLRQLQWKHZD\ GLVFLSOLQHVGHILQH*LYHQWKHWLPHFRQVWUDLQWVEXLOWLQWR86EDFFDODXUHDWHSURJUDPVDQGWKH PDJQLWXGHRIWKHFRJQLWLYHEXUGHQLWLVERWKLQHIILFLHQWDQGLUUHVSRQVLEOHQRWWRWHDFK XQGHUJUDGXDWHVWKHWRROVWKDWKHOSWKHPGLVFRYHULQIRUPDWLRQDV\HWXQNQRZQWRWKHPDVZHOODV WKRVHWKDWKHOSWKHPUHWULHYHLQIRUPDWLRQDERXWZKLFKWKH\DOUHDG\GRNQRZVRPHWKLQJ :RUGPLQLQJ  7KHZRUGPLQLQJSURFHVVGRHVQRWWU\WRRYHUFRPHWKHKDELWVDQGLQFOLQDWLRQVRIQDwYH UHVHDUFKHUV,QVWHDGLWGLUHFWVWKHPWRZKHUHWKH\FDQGRWKHPRVWJRRG$VWKHQDPH VXJJHVWVWKHSURFHVVLQYLWHVVHDUFKHUVWRPLQHUHVRXUFHVDWRQHVWDJHIRUSRWHQWLDOO\UHOHYDQW VHDUFKWHUPVWRXVHLQODWHUVWDJHVPRYLQJIURPQRQGRPDLQVHDUFKWHUPVDQGFRQFRPLWDQWO\ ORRVHVWRULHVWRLQGRPDLQWHUPVDSSOLHGWRLQGRPDLQVRXUFHV'LVFLSOLQDU\GDWDEDVHVWKDW LQFOXGHVXEMHFWWD[RQRPLHVDUHHVVHQWLDOWRROVIRUFURVVLQJWKHGRPDLQH[SHUWLVHERXQGDU\ (DFKVWDJHSUHVHQWVLQVWUXFWLRQDORSSRUWXQLWLHVWRLQWURGXFHFODVVHVWRDSSURSULDWHUHVRXUFHV DQGWRLVVXHVDERXWWKHLUVLJQLILFDQFH,GHDOO\WKHVHLQVWUXFWLRQDODQGUHIHUHQFHLQWHUYHQWLRQV LQFOXGHKDQGVRQH[HUFLVHVWKDWOHDGWRDVVHVVDEOHRXWSXWV  7KHILUVWDQGPRVWGLIILFXOWVWHSLVWROHDGVWXGHQWVYHU\UDSLGO\WRDUWLFXODWHVWRULHV±SODXVLEOH SURYLVLRQDOH[SODQDWLRQVRIZK\VRPHWKLQJLVWKHZD\LWLVEDVHGRQZKDWWKH\DOUHDG\NQRZ /D\LQJRXWWKHVWRU\LVDQDORJRXVWRDQGFDQZRUNWRJHWKHUZLWKWKHGHYHORSPHQWRIWKHRULHV DQGK\SRWKHVHVWKDWJXLGHHPSLULFDOUHVHDUFK6WXGHQWVZKRPXVWVWDWHWKHLUVWRULHVEULQJLQWR WKHRSHQZKDWWKH\PXVWORRNIRUDQGZK\  7KHHPSKDVLVXSRQWHOOLQJDVWRU\LVDFWLYHDQGSXUSRVLYHLQFRQWUDVWZLWKWKHODQJXDJHRI WRSLFVRUHYHQRIUHVHDUFKTXHVWLRQVLQYLWHVVWXGHQWVWREHSDVVLYHWRZDLWXQWLOWKH\KDYHʊ DOO WKHLQIRUPDWLRQ´WKDWZLOOP\VWHULRXVO\FRPHWRJHWKHUDWVRPHLQGHILQLWHIXWXUHWLPH,QPDQ\ UHVSHFWVWKHVWRU\WURSHFDQVHUYHDWOHDVWWRUDLVHFRQVFLRXVQHVVDERXWWKHDGHTXDF\RIWKHLU H[LVWLQJNQRZOHGJHDQGXQGHUVWDQGLQJ5HIOHFWLRQDQGGLVFXVVLRQDERXWWKHLQLWLDOVWRU\H[WHQGV WKLVVHOIDZDUHQHVVWRWKHORJLFRIWKHSRWHQWLDOH[SODQDWLRQ$UWLFXODWLQJWKHVWRU\QHFHVVDU\ OD\VRXWDUHODWLRQRIFRQFHSWV±HYLGHQFHDQGSUHPLVHV±ZKLFKDUHH[SUHVVHGLQDILUVWURXQG RISRWHQWLDOVHDUFKWHUPV  7KHILUVWVWHSRIZRUGPLQLQJSURSHULVLQIRUPDWLRQHQYLURQPHQWDOVFDQQLQJ7KLVLVDQH[SOLFLW DSSOLFDWLRQRIVWXGHQWV¶LQFOLQDWLRQVWRIRUDJHWRSOD\DURXQGZLWKVRXUFHVDQGVHDUFKHVWRVHH ZKDWKDSSHQV/LNHWKHVWRU\WKHJRDORIWKLVVWDJHLVQRWVRPXFKWREHʊ ULJKW´RUFRPSOHWHEXW UDWKHUWRIRUPLPSUHVVLRQVDERXWZKDWRWKHUVWRULHVH[LVWWKHHYLGHQFHWKH\GHSOR\DQGKRZ WKH\VTXDUHZLWKWKHLURZQSURYLVLRQDOKXQFKHV+DELWXDWHGLQWKHLUSULRUVFKRROLQJWR DVVRFLDWLQJUHVHDUFKZLWKZULWLQJPHUHUHSRUWVXQGHUJUDGXDWHVQHHGWREHVKRZQWKDW LQIRUPDWLRQLVQHVWHGLQFRQWHVWDEOHNQRZOHGJHFODLPV0RUHRYHUWKH\EHPDGHRIWKHSDUDGR[ WKDWZLGHVSUHDGDJUHHPHQWLQWKHOLWHUDWXUHRQWKHLULQLWLDOVWRU\LVDZDUQLQJIODJWRWKLQNDERXW QHZDQJOHVWKDWZLOOPDNHWKHLUSURMHFWVJHQXLQHO\WKHLURZQSHGDJRJLFDOO\UHLWHUDWLRQGRHVQRW GHYHORSDQDO\WLFDODQGV\QWKHWLFKDELWVRIPLQGDQGFRQILUPLQJWKHFRQYHQWLRQDOZLVGRPLVQRW "inWHUHVWLQJ´LQDFDGHPLFFXOWXUH  7KXVGXULQJWKHHQYLURQPHQWDOVFDQJUD\DQGLQWHUPLQDEOH*RRJOHUHVXOWVOLVWVWKH:LNLSHGLD GLVFXVVLRQWDEWKHFUDQN\UHDGHUUHSOLHVWRRQOLQHQHZVSDSHUVWRULHVWDONUDGLRIDX[WHOHYLVLRQ QHZVDUHDOOZLWKLQERXQGVDORQJZLWKFRQYHQWLRQDOEDFNJURXQGVRXUFHV6LPLODUO\DWWKLVVWDJH VHDUFKVWUDWHJLHVFDQEHUHODWLYHO\ORRVHVWXGHQWVFDQSOD\ZLWKGLIIHUHQWFRPELQDWLRQVRIWHUPV DQGH[SORUHRSHUDWRUVDQGILOWHUV7KLVLVDWLPHIRUH[SORUDWLRQZKLFKFDQEHFRPSOHPHQWHGE\ SHHUWRSHHUFRQYHUVDWLRQDQGIRUPDOLQVWUXFWLRQ  7KHUHLVHYHQDYLUWXHWRLQFRPSDWLELOLW\DQGZDFNLQHVVDPRQJWKHUHVXOWVRIWKHHQYLURQPHQWDO VFDQWKHVWXGHQWLVXQOLNHO\WREHDEOHWRUHFRQFLOHHYHU\WKLQJDQGZLOOKDYHWRWKLQNWKURXJK ZKDWSDUWVILWZLWKRQHDQRWKHUDQGZKDWPXVWEHVHWDVLGH7KHRGGVDUHWKDWWKHVWXGHQWZLOO HQFRXQWHUVROLGVRXUFHVWKDWUHODWHWRDWOHDVWVRPHSDUWRIWKHVWRU\ZKLFKZLOORIFRXUVH SURYLGHDGGLWLRQDOVHDUFKWHUPVLQFOXGLQJQDPHVRIDXWKRUVDQGDXWKRULWLHVSRVVLEO\ ELEOLRJUDSKLFUHIHUHQFHVDVZHOODVNQRZOHGJHFODLPV  ,QVWUXFWLRQDODQGUHIHUHQFHLQWHUYHQWLRQVE\WKHOLEUDU\DWWKLVVWDJHFDQDSSURSULDWHO\DGGUHVV WKHQDWXUHSXUSRVHDQGXVHRIUHIHUHQFHFROOHFWLRQVDQGUHIHUHQFHWRROVOLNHVXEMHFW HQF\FORSHGLDVEURDGJHQHUDOSXUSRVHGDWDEDVHVOLNH,QIRWUDF2QH)LOHDQG$FDGHPLF6HDUFK &RPSOHWHDQGWKHVXEMHFWEDVHGRUJDQL]DWLRQRIWKHOLEUDU\DQGLWVVHUYLFHVHJFODVVLILFDWLRQ EXWHVSHFLDOO\VXEMHFWUHVRXUFHJXLGHVDQGVXEMHFWOLEUDULDQFRQWDFW7KLVLVDOVRWKH DSSURSULDWHVWDJHIRULQVWUXFWRUVDQGRUOLEUDULDQVWRGLVFXVVGLVFLSOLQHVHYDOXDWLRQRI LQIRUPDWLRQDQGUKHWRULF  ,QWKHIDFHRILQIRUPDWLRQJOXWDQGWKHVHDUFKHU¶VOLPLWHGLQVLGHUNQRZOHGJHOHVVLVPRUH7KXV WKHFHQWUDOVWDJHLQZRUGPLQLQJLVWRXVHWKHWHUPVGHYHORSHGLQWKHSUHYLRXVVWDJHVWRVHDUFK UHSUHVHQWDWLRQVRILQIRUPDWLRQVRXUFHVFDWDORJVDEVWUDFWLQJDQGLQGH[LQJGDWDEDVHVPL[HG GDWDEDVHVZLWKVHDUFKHVOLPLWHGWRFLWDWLRQDQGDEVWUDFWILHOGV7KHEUHYLW\RIELEOLRJUDSKLF UHFRUGVDQGDEVWUDFWVWHQGVWRIRUFHSUHFLVHKLJKYDOXHZRUGVWRWKHIRUH0RUHSUHFLVHO\ KXPDQVZULWLQJDEVWUDFWVDQGDVVLJQLQJVXEMHFWKHDGLQJVFKRRVHZRUGVWKDWSUHFLVHO\DVZHOO DVFRQFLVHO\GHVFULEHWKHZRUN,QGLVFLSOLQDU\GDWDEDVHVWKHVHZLOOEHLQVLGHUWHUPVWKDWZLOO XQFRYHUDGGLWLRQDOUHFRUGVZLWKLQDQGDFURVVRWKHU$ ,GDWDEDVHVDQGWKHQXQORFNIXOOWH[W VRXUFHVLQWKHQH[WVWDJH  $QHQVHPEOHRIFKDUDFWHULVWLFVRIGLVFLSOLQDU\GDWDEDVHVEXLOWDURXQG$ ,SULQFLSOHVPDNHWKHP HVSHFLDOO\YDOXDEOHIRUEULGJLQJWKHFRJQLWLYHJDS7KHSDUWVDUHIDPLOLDUVHDUFKILHOGVDUWLFOH WLWOHVWH[WRIDEVWUDFWVDQGGHVFULSWRUV7KHFRQWHQWVRIHDFKILHOGLQDUHFRUGDUHLQWKHPVHOYHV PHDQLQJIXOFOXVWHUVRIZRUGVHDFKPRUHRUOHVVLQWHOOLJLEOHWRWKHQRQH[SHUWVHDUFKHU7DNHQ WRJHWKHUWKH\SUHVHQWVOLJKWO\GLIIHUHQWDVSHFWVRIWKHFRQWHQWRIWKHZRUGWKHUHFRUGUHSUHVHQWV (DFKLVOLNHO\WRKRRNGLIIHUHQWWHUPVIURPRUGLQDU\NH\ZRUGVHDUFKHVDOORZLQJPXOWLSOHUHVXOWV VHWVWREHFRPSDUHGDQGPDQLSXODWHG  7KHSURFHVVLVUHFXUVLYH:RUGVGLVFRYHUHGLQWKHUHFRUGVIURPRQHVHDUFKEHFRPHIRGGHUIRU IXUWKHURIWHQPRUHUHILQHGNH\ZRUGDQGVXEMHFWVHDUFKHV7KHZRUNVEHKLQGWKHUHFRUGV SURGXFHPRUHDQGPRUHLQGRPDLQVHDUFKWHUPV±DQGRIFRXUVHVXEVWDQWLYHFRQWHQW 7KXVZHXVHYDULRXVSHGDJRJLFDOVWUDWHJLHVWRJLYHVWXGHQWVH[SHULHQFHVLQUHDGLQJUHFRUGVRQ WZROHYHOV7KLQNELEOLRJUDSKLF.DEEDODK)LUVWWKHUHLVWKHIDPLOLDUOLWHUDOSURFHVVLQJRIWKH GHVFULSWLRQRIWKHZRUNWKHUHFRUGSRLQWVWR%\DSURFHVVRIPHQWDOWULDQJXODWLRQIURPWKHVH ILHOGVWKHVWXGHQWVFDQJHWDSUHWW\JRRGLGHDRIWKHZRUNDQGGHFLGHLWVDSSURSULDWHQHVVWRWKH SURMHFWDWKDQG%XWZHIXUWKHUDVNQDwYHVHDUFKHUVWRHPEUDFHWKHLUQDLYHWpDQGLQWHUURJDWH WKHUHFRUGV:KDWFRXOGWKLVXQIDPLOLDUZRUGRUFRQFHSWRUDUJXPHQWPHDQ"+RZPLJKWLW FRQQHFWWRDQRWKHURQHLQWKHUHFRUGLQRWKHUUHFRUGVDQGLQWKHVWRU\UHVHDUFKSURMHFWDWKDQG ZDQWVWRWHOO"7DNHWKDWROGIDYRULWH/&VXEMHFWKHDGLQJFRRNHU\DGGVXEGLYLVLRQVDQGVWLU EULVNO\7KDWLVZHDVNWKHVWXGHQWVWRUHIOHFWRQSRWHQWLDOLWLHVRIPHDQLQJDQGUHOHYDQFH HPERGLHGKRZHYHULPSHUIHFWO\LQDUHFRUG:HZDQWWRFXOWLYDWHLQWKHLUVHDUFKSUDFWLFHVDQ DWWLWXGHRIRSHQQHVVWRZDUGXQH[SHFWHGFRQQHFWLRQVDQGLQWHUSUHWDWLRQVWKDWUHIOHFWVWKH RSHQQHVVDQGFXULRVLW\SURIHVVLRQDOVEULQJWRWKHLUUHVHDUFKSUDFWLFHVZKHWKHUWH[WXDORU HPSLULFDODQGPDNHWKHUHVHDUFKWKHLURZQ   7KHMX[WDSRVLWLRQRIWHUPVPD\EHDVVLJQLILFDQWDVWKHLUVLPSOHGHQRWDWLRQ)RUGRPDLQ RXWVLGHUVSDUVLPRQLRXVGHVFULSWLRQVLQFRQYHQWLRQDO23$&UHFRUGVDQGGDWDEDVHVWKDW HVVHQWLDOO\GLJLWL]HROGSULQWLQGH[HVZLWKRXWDGGLQJFRQWHQWWRUHFRUGVPD\PDNHLWGLIILFXOWWR XVHPHQWDOWULDQJXODWLRQWRGHFLGHLIDZRUNLVZRUWKSXUVXLQJ%XWUHFRUGVKDYHJURZQULFKHU RYHUWKHSDVW\HDUVDQGQRZWKH\DUHPDVKHGXSZLWKVXSSOHPHQWDOLQIRUPDWLRQVXFKDV VRFLDOWDJVWKDWEULQJXQLWHFRQWHPSRUDU\DUJRWZLWKVWDEOHGHVFULSWLRQV)DFHWHGGLVSOD\V LQFUHDVHWKHQXPEHURIGDWDSRLQWVZKHUHDUHFRUGFRXOGPHVKZLWKWKHVWXGHQW¶VH[LVWLQJ XQGHUVWDQGLQJDQGJHRPHWULFDOO\LQFUHDVHWKHSRVVLELOLWLHVIRUILQGLQJFRQQHFWLRQV7RPHQWDO WULDQJXODWLRQDGGFRQQHFWLQJGRWV  6XEMHFWKHDGLQJVDUHHVSHFLDOO\YDOXDEOH:HDOONQRZWKHVKRUWFRPLQJVRIWD[RQRPLHVDQG WKHVDXULWKHLURQHGLPHQVLRQDOLW\FXOWXUDOO\DQGKLVWRULFDOO\ORDGHGODQJXDJHWKHSDLQIXO ORFXWLRQVWKHDOOWRRKXPDQLQFRQVLVWHQWDSSOLFDWLRQ$V&OD\6KLUN\VD\VRQWRORJLHVDUH RYHUUDWHG±LIXQGHUVWRRGDVDFFRXQWVRIUHDOLW\%XWWKDWGRHVQRWPHDQWKH\DUHQRWXVHIXO )RUDOORIWKDWWKH\DUHSRLQWHUVWRGLVFRYHUERXQGHGXQLYHUVHVRILQIRUPDWLRQZLWKLQWKHFDWDORJ RUGDWDEDVH$GGLWLRQDOO\PDQ\WLPHVWKH\UHYHDODWHFKQLFDOLQVLGHU¶VYRFDEXODU\WRPLQHIRU NH\ZRUGVHDUFKHVLQIXOOWH[WUHVRXUFHV  $JDLQWKHMX[WDSRVLWLRQRIZRUGVPD\EHPRUHLPSRUWDQWWKDQWKHZRUGVVHSDUDWHO\WRKHOS QDwYHVHDUFKHUVVLWXDWHWKHPVHOYHVZLWKLQGLVFLSOLQDU\VFKRODUVKLS)LHOGHGVHDUFKHVDUHNH\WR WKHSURFHVVRIPL[LQJDQGPDWFKLQJ7KHILUVWVWHSLVH[SOLFLWO\FRQWUDU\WRPDQ\VWXGHQWV¶ LQFOLQDWLRQVDQGWRWKHDGYLFHRIVRPHOLEUDULDQVNH\ZRUGVHDUFKIRUHDFKWHUPVLQJO\WRVHH KRZZHOOLWZRUNVIRURQH¶VVWRU\LQWKHFDWDORJRUGDWDEDVHDWKDQGORRNIRUDSSURSULDWHVXEMHFW WHUPVDQGRQO\WKHQFRPELQHWKHVHDUFKHV6XEMHFWWHUPVIXQFWLRQWRGHILQHFKXQNVRIUHODWHG ZRUNVNH\ZRUGVUHSUHVHQWDVSHFWVZLWKLQWKHP  7KLVFRPELQLQJDQGUHFRPELQLQJRIFKXQNVDQGDVSHFWVFDQEHGRQHPDQXDOO\LQDQ\GDWDEDVH WKDWXVHVERWKVXEMHFWDQGNH\ZRUGVHDUFKHVEXWLWLVPXFKHDVLHUZKHUHZHOOGHVLJQHGVHDUFK KLVWRU\IXQFWLRQVHJLQWKH&6$,OOXPLQDDQG2&/&)LUVW6HDUFKSODWIRUPVPDNHLWHDV\WRSXW WKHVHGLVFUHWHVHDUFKHVLQWRWKHVHDUFKKLVWRU\EXFNHWIURPWKHUHVXOWVOLVWWKHQFRPELQH VHDUFKHVZLWKDIHZFOLFNV7KHVHDUFKKLVWRU\RQWKH(EVFR+RVWSODWIRUPUHTXLUHVDQ LQFRQYHQLHQWH[WUDVWHSRSHQLQJDUHFRUGWRUXQDIUHVKVXEMHFWVHDUFKDFURVVWKHGDWDEDVH 2XUVWXGHQWVVD\WKH\ORYHWKLVWHFKQLTXHDVDZD\WROHWWKHGDWDEDVHWHOOWKHPZKDW LQIRUPDWLRQH[LVWVUDWKHUWKDQWRWU\WRWKLQNRXWFRPSUHKHQVLYHFRPELQDWLRQVRIVHDUFKWHUPVD SULRUL7KHLUELJJHVWFRPSODLQWLVQRWZLWKVHDUFKLQJ$ ,DUFKLWHFWXUHVEXWUDWKHUZLWKKRZPDQ\ VHHPLQJO\YDOXDEOHMRXUQDOVDUHQRWDYDLODEOHWKURXJKRXUOLQNUHVROYHU  $QDORJRXVWRRXUDGYLFHDERXWVHDUFKLQJWHUPVVLQJO\UDWKHUWKDQLQFRPELQDWLRQVZHWHDFK VWXGHQWVWKDWLWLVXOWLPDWHO\PRUHHIILFLHQWWRVHDUFKRQHGDWDEDVHDWDWLPHUDWKHUWKDQDFURVVD IDPLO\RUZLWKDIHGHUDWHGVHDUFKWRROV&URVVGDWDEDVHVHDUFKLQJEOXUVWKHGLIIHUHQFHVDPRQJ GLVFLSOLQHVPDNHVLWPRUHGLIILFXOWWRGLVFHUQZKDWLVLQXQIDPLOLDUGDWDEDVHVLGHQWLI\ZKDWWHUPV ZRUNDQGGLDJQRVHZK\DVHDUFKPD\IDLO&UXFLDOO\WKH\GLOXWHWKHSRZHURIVXEMHFWVHDUFKHV WRUHVROYHGDWDEDVHFRQWHQWLQWRVPDOOHUPDQDJHDEOHFKXQNV  7KHILQDOWZRVWHSVLQZRUGPLQLQJDUHVWUDLJKWIRUZDUGDSSO\WKHVHDUFKDEOHZRUGVGLVFRYHUHG KHUHWRIRUHLQIXOOWH[WVRXUFHVILUVWWKRVHWKHOLEUDU\VXEVFULEHVWRWKHQWKHIUHHZHE:H UHFRPPHQGWKDWRUGHUEHFDXVHWKHOLEUDU\UHVRXUFHVSURYLGHFXHVWRWKHLUYDOXHDORQJTXDOLW\ FRQWUROUHJDUGLQJFRQWHQWSRZHURIVHDUFKIHDWXUHVDQGFRPSOHPHQWDU\ZLWKRWKHUUHVRXUFHV /LEUDU\VXEVFULSWLRQIXOOWH[WVRXUFHVGLYLGHLQWRWKRVHWKDWDGGVXEVWDQWLDOIXOOWH[WFRYHUDJHDQG $ ,IXQFWLRQDOLW\HJ$%,,QIRUPYDULRXV(%6&2SURGXFWVDQGWKRVHZLWKOLWWOHRUQRVXEMHFW FODVVLILFDWLRQHJ&,$2/H[LV1H[LV$FDGHPLFDQGZHUHFRPPHQGWKDWVWXGHQWVXVHWKHPLQ  WKDWRUGHU7KHPL[HG$ ,IXOOWH[WGDWDEDVHVLQRXUOLEUDU\WHQGWRKDYHGLVFLSOLQDU\HPSKDVHV 2XUSXUHSOD\IXOOWH[WVXEVFULSWLRQVRXUFHVDUHPRUHOLNHO\WREHGLVWLQJXLVKHGE\JHQUH MRXUQDOLVPLQ)DFWLYDRUKLVWRULFDOQHZVSDSHUVMRXUQDOVLQ-6725WKDQGLVFLSOLQDU\IRFXV WKRXJKRXUVXEMHFWJXLGHVDQGUHIHUHQFHLQVWUXFWLRQLQWHUYHQWLRQVKHOSRXUXVHUVGHFLGHZKLFK RQHVDUHSRWHQWLDOO\DSSURSULDWHWRWKHLUVWRULHV  7KHILQDOFOHDQXSVWDJHUHVHPEOHVWKHHQYLURQPHQWDOVFDQEDFNWRWKHRSHQ:HEEXWZLWKD FOHDUHULGHDRIWKHVWRU\ZKDWLWUHTXLUHVDQGDULFKUHSHUWRLUHRIGRPDLQVLJQLILFDQWVHDUFK WHUPV%\QRZVWXGHQWV¶FRQILGHQFHLQWKHLUVHDUFKDELOLW\LVMXVWLILHGDQGWKHWDVNRIOLEUDULDQ DQGFRXUVHLQVWUXFWRULQWKDWUHJDUGLVWRSRLQWRXWOHVVIDPLOLDUSODFHVWRORRNWKURXJKVXFKDV WKHLQYLVLEOH:HEDQGFRQIHUHQFHSDSHUVHUYHUV  ,V*RRG(QRXJK5HDOO\*RRG(QRXJK"³*RYHUQPHQW5 '5HVXOWV'LVWULEXWRUWR $FFHOHUDWH6FLHQFH'LVFRYHU\´6KDURQ-RUGDQ$VVLVWDQW'LUHFWRUIRU3URJUDP ,QWHJUDWLRQ86'2(2IILFHRI6FLHQWLILF 7HFKQLFDO,QIRUPDWLRQ  :HDW'2(¶V2IILFHRI6FLHQWLILFDQG7HFKQLFDO,QIRUPDWLRQDUHIXOILOOLQJRXUPLVVLRQE\PDNLQJ JRYHUQPHQWVSRQVRUHG5 'LQIRUPDWLRQDYDLODEOHYLDRQOLQHGDWDEDVHVQRZZLWKVLQJOHTXHU\ SRUWDOLQWHUIDFHVWKDWPDNH'2(¶V5 'UHVXOWVHDVLO\DFFHVVLEOHE\PDQ\LQIRUPDWLRQVHHNHUV 7KHVHSRUWDOVLQFOXGHWKH6FLHQFH$FFHOHUDWRUZZZVFLHQFHDFFHOHUDWRUJRYZKLFK DJJUHJDWHV'2(GDWDEDVHVRI5 'LQIRUPDWLRQ6FLHQFHJRYZKLFKHQDEOHV\RXWRILQG 86VFLHQFHLQIRUPDWLRQIURPPXOWLSOHDJHQFLHVDQG:RUOG:LGH6FLHQFHRUJZKLFKILQGV JOREDOVFLHQFHLQIRUPDWLRQ7KURXJKWKHVHJDWHZD\VOLEUDULDQVUHVHDUFKHUVDQGWKHSXEOLFFDQ DFFHVVDVFLHQFHSDJHFRXQWFRPSDUDEOHWREXWQRWGXSOLFDWLYHRIWKHHQWLUHVFLHQFHFRQWHQWRI *RRJOH70  )RUSXUSRVHVRIWKLVGLVFXVVLRQZHZLOOIRFXVRQ6FLHQFHJRYZKLFKLQWHJUDWHVWKHGDWDEDVHV DQGZHEVLWHVRI86VFLHQFHDJHQFLHV6FLHQFHJRYUHVXOWVDUHXQLTXHO\GLIIHUHQWIURP *RRJOHDQG*RRJOH6FKRODU 6FLHQFHJRY6HDUFK5HVXOWV6FUHHQ6KRW    7KURXJKVSHFLDOL]HGIHGHUDWHGVHDUFKWHFKQRORJ\6FLHQFHJRYGULOOVGRZQWRVHOHFWHG GDWDEDVHVDQGZHEVLWHVLQSDUDOOHOWKHQSUHVHQWVUDQNHGVHDUFKUHVXOWVIURPRYHU GDWDEDVHVWKRXVDQGVRIDXWKRULWDWLYHVFLHQWLILFZHEVLWHVZLWKLQJRYGRPDLQVSRQVRUVKLSDQG PRUHWKDQPLOOLRQSDJHVRIVFLHQFHLQIRUPDWLRQ6FLHQFHJRYDOVRKHOSVXVHUVLGHQWLI\ GDWDEDVHVIRUPRUHDGYDQFHGVHDUFKLQJDVQHHGHG  5HFHQWO\267,FRPSLOHGVDPSOHVHDUFKHVRIVFLHQWLILFVHDUFKWHUPVW\SLFDORIZKDWD UHVHDUFKHUZRXOGORRNIRULH"aGHQRVLQHWULSKRVSKDWH´"inIOXHQ]DYDFFLQH´WUDQVSODQWSDWLHQW DQG"lDVHUFRROLQJ´:HIRXQGWKDWRIWKHWLPH6FLHQFHJRYUHVXOWVDUHXQLTXHO\GLIIHUHQW IURP*RRJOHDQG*RRJOH6FKRODUUHVXOWV6FLHQFHJRYUHVXOWVKDYHRQO\RYHUODSZLWK*RRJOH DQGDQDSSUR[LPDWHRYHUODSZLWK*RRJOH6FKRODU  6DPSOH   %HFDXVH6FLHQFHJRYXVHVDQLQWHJUDWHGVHDUFKRILQGLYLGXDOGDWDEDVHV6FLHQFHJRYXVHUV EHQHILWIURPERWKSODWIRUPV7KHUHDUHEHQHILWVWRXVLQJDQLQWHJUDWHGVHDUFKWRRODVZHOODVWR XVLQJGLVFLSOLQHRULHQWHGGDWDEDVHV$VDQLQWHJUDWHGVHDUFKWRRO6FLHQFHJRYRIIHUVXVHUVWKH RSSRUWXQLW\WRTXLFNO\VFDQWKHHQYLURQPHQWUHODWLYHWRZKDWWKH\DUHVHHNLQJILQGWKHUHOHYDQW LQIRUPDWLRQWKH\VHHNDQGGULOOGRZQIRUPRUH5HVXOWVDUHW\SLFDOO\FXUUHQWDVWKH\DUHIRXQGLQ UHDOWLPH2QFHXVHUVORFDWHWKHGDWDEDVHVWKH\QHHGWKHEHQHILWRISUHVHQWDWLRQDQGGHSWKRI LQIRUPDWLRQRIGLVFLSOLQHRULHQWHGVHDUFKFRPHLQWRSOD\ 6FLHQFHJRYDQGRWKHULQWHJUDWHGVHDUFKJDWHZD\VVXFKDV6FLHQFH$FFHOHUDWRUDQG :RUOG:LGH6FLHQFHRUJILQGLQIRUPDWLRQIRUXVHUVWKDWJHQHUDOO\FDQQRWEHIRXQGWKURXJK*RRJOH 6FKRODU*RRJOHDQGRWKHUWUDGLWLRQDOFUDZOHUV  *HR5HIDQG*RRJOH6FKRODU6RPH&RPSDULVRQV6KDURQ7DKLUNKHOL'LUHFWRU ,QIRUPDWLRQ6\VWHPV$PHULFDQ*HRORJLFDO,QVWLWXWH  $VWKHSXEOLVKHURI*HR5HIWKHSULPDU\ELEOLRJUDSKLFVHUYLFHIRUWKHJHRORJLFDOVFLHQFHV,ZDV DVNHGWRSURYLGHVRPHWKRXJKWVRQWKHGLIIHUHQFHVEHWZHHQDGLVFLSOLQHVSHFLILFELEOLRJUDSKLF VHUYLFHOLNH*HR5HIDQGDVHDUFKHQJLQHOLNH*RRJOH6FKRODUDQGORRNLQJWRWKHIXWXUHRXWOLQH VRPHSRWHQWLDOVWHSVWKDWDWUDGLWLRQDO$ ,VHUYLFHFRXOGWDNHWRHQKDQFHLWVSHUFHLYHGYDOXHWR WKHXVHU  ,QDQGDJDLQLQPLG,H[DPLQHGWKHIROORZLQJTXHVWLRQV:KDWDUHWKHFKDUDFWHULVWLFV RIDGLVFLSOLQHVSHFLILFELEOLRJUDSKLFVHUYLFHOLNH*HR5HIDQGDVHDUFKHQJLQHOLNH*RRJOH 6FKRODUŒ"$QGZKDWDUHWKHVWUHQJWKVRIHDFKW\SHRIVHUYLFH" 7KHSURFHGXUHXVHGIRUFRPSDULQJWKHWZRGLIIHUHQWVHUYLFHVLQFOXGHGH[DPLQLQJWKHZHEVLWHV DQGWHVWLQJHDFKV\VWHPZLWKWKUHHGLIIHUHQWW\SHVRIVHDUFKHV±HGLWRUDXWKRUVNQRZQLWHPDQG VXEMHFWVHDUFKHV8VLQJVRPHRIWKHVDPHVHDUFKHVLQDQGDJDLQLQDOVRDOORZHG FKDQJHVLQWKHV\VWHPVWREHREVHUYHG  6WUHQJWKV  *RRJOH6FKRODU¶VFRQWLQXRXVXSGDWLQJSURYLGHVWKHXVHUWKHRSSRUWXQLW\WRVHHUHFHQWPDWHULDOV DQGLWVEURDGFRYHUDJHRIVFLHQFHHQKDQFHVLQWHUGLVFLSOLQDU\DFFHVVWRPDWHULDOV$FFHVVLV  HDV\DQGLVVLPSOHIRUWKHXVHUWRXVH6HDUFKHVSURGXFHODUJHQXPEHUVRIKLWV$QGRIFRXUVH LWLVIUHH  *HR5HIKDVSRZHUIXOVHDUFKIHDWXUHVHQDEOLQJWKHXVHUWROLPLWVHDUFKHVWRYHU\VSHFLILF TXHULHV&RYHUDJHLVOLPLWHGWRWKHJHRORJLFVFLHQFHVHOLPLQDWLQJWKHQHHGWRKXQWWKURXJKKXJH OLVWVRIUHVXOWV$&RQWUROOHGYRFDEXODU\HQKDQFHVUHWULHYDOIRUVSHFLDOLVWVDQGWKHFRYHUDJH LQFOXGHVJUH\OLWHUDWXUHQRQ(QJOLVKODQJXDJHDQGSULQWSXEOLFDWLRQV  )XOO7H[W$FFHVV  7KHUHODWLYHVWUHQJWKRIWKHDYDLODELOLW\RIIXOOWH[WDFFHVVIURPUHVXOWVLQ*RRJOH6FKRODUDQG *HR5HIZDVLPSRVVLEOHWRDVVHVV:KLOH*RRJOH6FKRODUSURYLGHVOLQNVWR:HEEDVHG PDWHULDOVPDQ\RIWKHPDWHULDOVDUHQRWIUHHO\DYDLODEOHDQGKDYHDFFHVVFRQWUROVWKDWSUHYHQW XVHH[FHSWE\XVHUVLQLQVWLWXWLRQVWKDWOLFHQVHWKHFRQWHQW$FFHVVWRIXOOWH[WWKURXJK*HR5HILV GHSHQGHQWRQWKHXVHU¶VOLEUDU\DFFHVVV\VWHPDQGWKHXVHU¶VORFDWLRQOLEUDU\FDPSXVUHPRWH 3RWHQWLDO(QKDQFHPHQWVIRU'LVFLSOLQH6SHFLILF'DWDEDVHVOLNH*HR5HI  :KLOHLWLVFOHDUWKDWWUDGLWLRQDO$ ,6HUYLFHVOLNH*HR5HIFRQWLQXHWRKDYHYDOXHWKHUHDUHVRPH HQKDQFHPHQWVWKDWFRXOGKDYHDVLJQLILFDQWLPSDFWRQXVH0RUHIUHTXHQWXSGDWLQJRIWKH GDWDEDVHZLWKLPSURYHGFXUUHQF\FDQKHOSWRHOLPLQDWHWKHWUDGLWLRQDOODJWLPHIRU$ , GDWDEDVHV,Q*HR5HIEHJDQPDNLQJFXUUHQWUHFRUGVDYDLODEOHLQDILOHNQRZQDV*HR5HI LQ3URFHVV7KHVHFXUUHQWUHFRUGVDUHPDGHDYDLODEOHEHIRUHWKH\KDYHEHHQIXOO\HGLWHGDQG LQGH[HGIRU*HR5HI7KHUHFRUGVFRQWDLQEDVLFELEOLRJUDSKLFLQIRUPDWLRQDQGSURYLGHWKHXVHU WKHRSSRUWXQLW\WRILQGDWOHDVWDSUHOLPLQDU\UHIHUHQFHZLWKRXWZDLWLQJIRUWKHFRPSOHWHYHUVLRQRI WKHUHIHUHQFH  (QKDQFHGLQGH[LQJRIILJXUHVPDSVDQGWDEOHVFDQSURYLGHWKHXVHUZLWKDGGLWLRQDODFFHVVWR JHRORJLFDOSXEOLFDWLRQV7KHLOOXVWUDWLRQVLQWKHJHRVFLHQFHOLWHUDWXUHFDQDWWLPHVEHWKH SULPDU\IRFXVRIDSXEOLFDWLRQ7KH*HR5HIGDWDEDVHRQ3URTXHVWKDVEHHQHQKDQFHGZLWK 'HHS,QGH[LQJRILPDJHVIURPVHOHFWHGMRXUQDOV    0DSEDVHGVHDUFKDQGGLVSOD\FDQEHDWWUDFWLYHWRXVHUVDQGKDVWKHSRWHQWLDOIRUSURYLGLQJWKH XVHUZLWKDJHRJUDSKLFZD\RIFRRUGLQDWLQJUHVXOWV7KH*HR5HIGDWDEDVHRQ(QJLQHHULQJ 9LOODJHQRZSURYLGHVD*RRJOHPDSSLQJLQWHUIDFHWKDWDOORZVJHRORJLVWVWRVHHWKHJHRJUDSKLF ORFDWLRQVRIWKHLUUHVXOWV  ,PSURYLQJWKHXWLOL]DWLRQRIGLVFLSOLQHVSHFLILF&RQWUROOHG9RFDEXODU\LQUHWULHYDOFRXOGUHDS VLJQLILFDQWEHQHILWVWRXVHUV&XUUHQWLPSOHPHQWDWLRQVRI*HR5HIGRQRWPD[LPL]HWKHIXOO SRWHQWLDORID&RQWUROOHG9RFDEXODU\E\XVLQJWKHPDQ\LQWHUUHODWLRQVKLSVEHWZHHQWKH YRFDEXODU\WHUPVLQDQ\V\VWHPDWLFZD\  ,V*RRG(QRXJK5HDOO\*RRG(QRXJK"   'LVFLSOLQHVSHFLILFGDWDEDVHVFRQWLQXHWREHQHFHVVDU\IRUFRPSUHKHQVLYHUHVHDUFKDQGIRU ORFDWLQJQRQZHEEDVHGDQGIRUHLJQODQJXDJHPDWHULDOVWKHLUSRZHUIXOVHDUFKIHDWXUHVSHUPLW SUHFLVHDQGIOH[LEOHTXHULHVDQGHOLPLQDWHIDOVHUHWULHYDO