Open data, open review and open dialogue in making social sciences plausible (2)

Ope data, ope e ie a d ope dialogue i aki g so ial s ie es plausi le Qua -Hoa g Vuo g Ce t e fo I te dis ipli a y So ial Resea h Weste U i e sity Ha oi, Viet a This pape has ee pu lished : Natu e: S ie tifi Data Updates Date: De e e Pu lishe ’s URL: http:// logs atu e o /s ie tifi data/ / / /autho s- o e -ope data-ope - e ie -a d-ope -dialogue-i - aki g-so ial-s ie es-plausi le/ Ve sio e losed: e pdf * Bi liog aphi ote: Vuo g QH Ope data, ope e ie a d ope dialogue i aki g so ial s ie es plausi le Natu e: S ie tifi Data Updates, De e e A aila le f o : / / /autho s- o e -ope -data A st a t: A g o i g a a e ess of the la k of ep odu i ilit has u de i ed so iet ’s t ust a d estee i so ial s ie es I so e ases ell-k o esults e ee fa i ated o the u de l i g data has tu ed out to e eak te h i al fou datio s Ma esea he s e i estigated the plausi ilit of fi di gs i the so ial s ie es a d hu a ities A t pi al e a ple is the ste ious C iti al Mi i u Positi it Ratio F ed i kso a d Losada , hi h lai ed to sho that the e e ists su h a positi it atio a d that a i di idual’s deg ee of flou ishi g ould e p edi ted that pe so ’s atio of positi e to egati e e otio s o e ti e This atio had o e ee a ell-k o , highl i flue tial a d g eatl ad i ed ps hologi al o sta t u til it as sho B o , “okal a d F ied a to e a u fou ded, a it a a d ea i gless u e To add ess the plausi ilit p o le , I suggest that a o i atio of ope data, ope pee - e ie a d ope o u it dialogue, ould se e as a feasi le optio fo the so ial s ie es Refe e es: Aa ts AA et al doi: Ba e G Co Esti ati g the ep odu i ilit of ps hologi al s ie e S ie e; /s ie e.aa What o stitutes app op iate pee , doi: /pal o u i atio s; : Begle CG a d Ellis LM : – Bo e ie fo i te dis ipli a s : aa , esea h? Palg a e D ug de elop e t: Raise sta da ds fo p e li i al a e esea h Natu e; , doi: / a NJL, “okal AD, a d F ied a HL positi it atio A e i a Psy hologist; The o ple d a i s of ishful thi ki g: the : – , doi: /a iti al Butle D Well o e T ust lau hes ope -a ess pu lishi g e tu e Natu e Ne s; a aila le f o : Butle D doi: Gates Fou datio a / atu e ou es ope -a ess pu lishi g e tu e Natu e; Egle “J et al To a d sta da d p a ti es fo sha i g o pute : , doi: / Natu e Neu os ie e; F ed i kso BL a d Losada MF A e i a Psy hologist; Ja es ET : ode a d p og a s i : , eu os ie e Positi e affe t a d the o ple d a i s of hu a flou ishi g , doi: / X P o a ility Theo y: The Logi of S ie e Lo , UK: Ca idge U i e sit P ess Ngu e HM I e ualit i health a e osts et ee esidi g a d o - esidi g patie ts: e ide e f o Viet a I t Jou al fo E uity i Health; : , doi: /s - Nosek BA et al P o oti g a /s ie e.aa ope “ ie tifi Data Edito ial usi ess S ie tifi Data; : Va Noo de R / Ope fo a Co pa : esea h ultu e S ie e; offe s po ta le pee , doi: e ie Natu e; - /sdata : , doi: , doi: Vuo g QH “u e data o Viet a ese p ope sit to atte d pe iodi ge e al health e a i atio s S ie tifi Data; : , doi: /sdata Vuo g QH a d Ngu e TK Data i B ief; : Data o Viet a ese patie ts’ fi a ial u de s a d isk of destitutio , doi: /j.di Walsh E, Roo e M, Apple L, a d Wilki so G Ope pee e ie : a a ised o t olled t ial B itish Jou al of Psy hiat y; - ; doi: / jp http://blogs.nature.com/scientificdata/2017/12/12/authors-corner-open-data-open-review-and-open-dialogue-in-making-social-sciences-plausible/ nature.com Sitemap Scientific Data Blog Log In Register Post Previous post Author’s Corner: Revisiting the personalities of wild chimpanzees SCIENTIFIC DATA | SCIENTIFIC DATA December 12, 2017 | 2:41 pm | Posted by Varsha Khodiyar | Category: Author's Corner, Guest Posts Guest post by Quan-Hoang Vuong of Centre for Interdisciplinary Social Research, Western University Hanoi, Vietnam A growing awareness of the lack of reproducibility has undermined society’s trust and esteem in social sciences In some cases, well-known results have been fabricated or the underlying data have turned out to have weak technical foundations Many researchers have investigated the plausibility of findings in the social sciences and humanities A typical example is the mysterious Critical Minimum Positivity Ratio 2.9013 by Fredrickson and Losada (2005), which claimed to show that there exists such a positivity ratio and that “an individual’s degree of flourishing could be predicted by that person’s ratio of positive to negative emotions over time” This ratio had once been a wellknown, highly influential and greatly admired psychological “constant” until it was shown by Brown, Sokal and Friedman (2013) to be an unfounded, arbitrary and meaningless number To address the plausibility problem, I suggest that a combination of open data, open peer-review and open community dialogue, could serve as a feasible option for the social sciences Author’s Corner: Open data, open review and of QUAN HOANG VUONG 12/12/2017, 9:47 PM http://blogs.nature.com/scientificdata/2017/12/12/authors-corner-open-data-open-review-and-open-dialogue-in-making-social-sciences-plausible/ Open data Ethical standards for open data can be difficult to meet As I discuss below, my own experience of publishing with Scientific Data (Vuong 2017) showed me firsthand the issues with managing ethical concerns However, addressing these issues is not impossible, and underneath the difficulty and challenges lies the beauty of shared values and open thinking Still, psychological and financial barriers to open data remain Although it sounds simple, the practice of pursuing an open data policy has turned out to be complicated and hard to achieve, so that “nudging scientific practices toward greater openness requires complementary and coordinated efforts from all stakeholders” (Nosek et al 2015) Unfortunately, the scientific community has not always taken this seriously (Begley and Ellis 2012), forsaking the opportunity for their valuable data to stand the test of time Now with the existence of reliable data repositories such as Harvard Dataverse, Open Science Framework, Mendeley, UK Data Archive, depositing data for public use and replication has never been easier Open publishing platforms such as F1000Research even helps to securely deposit multiple data sets to their own systems without reliance on a third party’s service Scientific Data goes even further to the ideal of removing restrictions on use of open data for commercial purposes, by advocating the practice of generating economic value through commercial enterprise, and showcasing the potential benefit for society and the economy (Scientific Data Editorial, “Open for business“) A benefit to authors of sharing data with a scholarly article is the opportunity for their data to be examined by other scholars, and possibly even lead to further scientific discoveries The authors have the opportunity of having their colleagues verify both the soundness of data and robustness of the scientific process—as part of a cumulative process of uncertainty reduction (Aarts et al 2015) This is how the scientific workhorse of self-correction operates My own experiences of open data Sometimes the process of data reuse can be quite fast, as is my own experience When publishing a data article in Data in Brief, with open data sets deposited to Mendeley Data (Vuong and Nguyen, 2016), I was not sure if the data would be reused To my surprise, a researcher at the University of Pittsburgh used them to make a quality contribution to International Journal for Equity in Health (Nguyen 2017) How quickly did this all happen? My paper was published on 30 September 2016 and Nguyen’s manuscript was submitted on 15 November 2016! Publishing data that was later reused was a special experience Work value is transmitted via rigorous peer-review, to data users, i.e., other researchers, for the sake of advancing human knowledge My own questions about the data’s value were answered not just once, with the editorial and peer-review process, but twice and perhaps more than twice, with subsequent scientific publications that used the contributed data My subsequent experience while preparing data for publication at Scientific Data was even more special and unique, in the sense that meeting the most rigorous and demanding criteria boosted the sharpness, clarity, transparency, ethical values and reusability of my data Even a highly Author’s Corner: Open data, open review and of 12/12/2017, 9:47 PM http://blogs.nature.com/scientificdata/2017/12/12/authors-corner-open-data-open-review-and-open-dialogue-in-making-social-sciences-plausible/ accomplished author can find that the publishing process at Scientific Data enhances their own learning attitude and scientific exploration Open review In the course of encouraging transparency and open science for the sake of better science for all, open review has emerged to be a commendable solution This may sound strange to researchers in social sciences, where double-blind review has been predominant; however, theoretically the system has its advantages Evidence from Walsh, Rooney, Appleby and Wilkinson (2000) supports open peer-review since signed reviews are of higher quality, more courteous, and more thoughtful In addition, social values of open peer-review cannot be ignored as “Improvement of journal policies can help those values become more evident in daily practice and ultimately improve the public trust in science, and science itself,” (Nosek et al 2015) Heavyweight funders also see the value in the transparent peer-review process, public judgments, and the system’s ability to address the issues of interdisciplinary research review as raised in Bammer (2016) That is why the young platform F1000Research received substantial support not long after its debut (Van Noorden, 2013), from Wellcome Trust in 2016 (Butler 2016) and Bill & Melinda Gates Foundation in 2017 (Butler 2017) As Begley and Ellis (2012) put it, social sciences now need that open review system, which helps facilitate a transparent discovery process leading to significant social benefits Open community dialogue We refer to an enlarged notion of “dialogue” that consists of technical expert discussions about scientific methods and computer codes (Eglen et al 2017) and the research communication processes that can be made available to the community for evaluation, critique, reuse or extension (Nosek et al 2015) PubPeer has further incentivized the need for open community dialogue, as flagging a paper on the site is now perceived (incorrectly) by many as a threat This is a technical expertise community that can help fix problems, deal with statistical weaknesses (that is to improve overall quality of manuscripts), and serve the quality gatekeeping for scientific outlets PubPeer has been pushing what Eglen et al (2017) advocate, “Share the methods and computer codes.” In social sciences, their values help update and verify, “stylized facts,” in a Bayesian probabilistic world where, “an erroneous argument does not necessarily lead to a wrong conclusion,” due to Gödel’s theorem (Jaynes 2003) Nowadays, protecting trust in social sciences also means engaging in open community dialogue, which helps to safeguard robustness and improve efficiency of research methods The combination of open data, open review and open dialogue may sound simple but implementation in the real world will not be straightforward However, in view of Begley and Ellis’s (2012) statement that, “the scientific process demands the highest standards of quality, ethics and rigour,” they are worth implementing More importantly, they are feasible to work on and likely will help to restore plausibility to social sciences research Therefore, I feel it likely that the triplet of open data, open review and open dialogue will gradually emerge to become policy requirements regardless of the research funding source Author’s Corner: Open data, open review and of 12/12/2017, 9:47 PM http://blogs.nature.com/scientificdata/2017/12/12/authors-corner-open-data-open-review-and-open-dialogue-in-making-social-sciences-plausible/ References Aarts AA et al (2015) Estimating the reproducibility of psychological science Science; 349 (6251): aac4716, doi:10.1126/science.aac4716 Bammer G (2016) What constitutes appropriate peer review for interdisciplinary research? Palgrave Communications; 2: 16017, doi:10.1057/palcomms.2016.17 Begley CG and Ellis LM (2012) Drug development: Raise standards for preclinical cancer research Nature; 483 (7391): 531–533, doi:10.1038/483531a Brown NJL, Sokal AD, and Friedman HL (2013) The complex dynamics of wishful thinking: the critical positivity ratio American Psychologist; 68 (9): 801–813, doi:10.1037/a0032850 Butler D (2016) Wellcome Trust launches open-access publishing venture Nature News Butler D (2017) Gates Foundation announces open-access publishing venture Nature; 543 (7647): 599, doi:10.1038/nature.2017.21700 Eglen SJ et al (2017) Toward standard practices for sharing computer code and programs in neuroscience Nature Neuroscience; 20 (6): 770-773, doi:10.1038/nn.4550 Fredrickson BL and Losada MF (2005) Positive affect and the complex dynamics of human flourishing American Psychologist; 60 (7): 678-686, doi:10.1037/0003-066X.60.7.678 Jaynes ET (2003) Probability Theory: The Logic of Science London, UK: Cambridge University Press Nguyen HM (2017) Inequality in healthcare costs between residing and non-residing patients: evidence from Vietnam Int Journal for Equity in Health; 16 (1): 76, doi:10.1186/s12939-017-0581-3 Nosek BA et al (2015) Promoting an open research culture Science; 348 (6242): 1422-1425, doi:10.1126/science.aab2374 Scientific Data Editorial (2017) Open for business Scientific Data; 4: 170058, doi:10.1038/sdata.2017.58 Van Noorden R (2013) Company offers portable peer review Nature; 494 (7436): 161, doi:10.1038/494161a Vuong QH (2017) Survey data on Vietnamese propensity to attend periodic general health examinations Scientific Data; 4: 170142, doi:10.1038/sdata.2017.142 Vuong QH and Nguyen TK (2016) Data on Vietnamese patients’ financial burdens and risk of destitution Data in Brief; 9: 543-548, doi:10.1016/j.dib.2016.09.040 Walsh E, Rooney M, Appleby L, and Wilkinson G (2000) Open peer review: a randomised controlled trial British Journal of Psychiatry; 176 (1) 47-51; doi:10.1192/bjp.176.1.47 Author’s Corner: Open data, open review and of 12/12/2017, 9:47 PM http://blogs.nature.com/scientificdata/2017/12/12/authors-corner-open-data-open-review-and-open-dialogue-in-making-social-sciences-plausible/ Previous post Author’s Corner: Revisiting the personalities of wild chimpanzees Comments There are currently no comments You need to log in or register to comment © 2017 Macmillan Publishers Limited All Rights Reserved partner of AGORA, HINARI, OARE, INASP, ORCID, CrossRef and COUNTER Author’s Corner: Open data, open review and of 12/12/2017, 9:47 PM ... Corner: Open data, open review and of 12/12/2017, 9:47 PM http://blogs.nature.com/scientificdata/2017/12/12/authors-corner -open- data -open- review- and -open- dialogue- in- making- social- sciences- plausible/ ... http://blogs.nature.com/scientificdata/2017/12/12/authors-corner -open- data -open- review- and -open- dialogue- in- making- social- sciences- plausible/ Open data Ethical standards for open data can be difficult to meet As I discuss... suggest that a combination of open data, open peer -review and open community dialogue, could serve as a feasible option for the social sciences Author’s Corner: Open data, open review and of QUAN