Grover et al./A Critique on Wade et al IS Research Perspective About Reference Disciplines and Reference Differences: A Critique of Wade et al Varun Grover Department of Management Clemson University vgrover@clemson.edu Ramakrishna Ayyagari Department of Management Clemson University Rahul Gokhale Department of Management Clemson University Jaejoo Lim Department of Management Clemson University Abstract Two articles published in this issue (Wade et al and ours) through similar analyses reach contrasting conclusions on whether Information Systems, as a field, is evolving toward a reference discipline In this article, we provide a critique of Wade et al We first assess our different interpretations of reference discipline, and then discuss the consequences of including highly related disciplines in citation analysis Finally, we illustrate the sensitivity of Wade et al.’s results to the inclusion and exclusion of certain journals We also consider potential interpretations of second degree citations It is hoped that the arguments presented here reconcile the differences as we collectively advance thinking on the state of IS as a reference discipline The article by Wade et al (this issue) is a thoughtful and provocative piece on the maturity of the Information Systems (IS) discipline In this short paper, we engage in a closer examination of the Wade et al article, evaluating its premise and conclusions with respect to our own article in this issue Our intent is to explain and reconcile differences as we collectively advance thinking on the state of our discipline 336 Journal of the Association for Information Systems Vol No 5, pp 336-350/ May 2006 Grover et al./A Critique on Wade et al The Issue at Hand The term reference discipline, coined by Keen at the first ICIS conference, has been an important component of the vocabulary and conduct of IS scholars over the past two decades Early empirical studies such as Culnan and Swanson (1986) placed the IS discipline at the end of an intellectual food chain based on the results of a citation analysis More recently, some scholars suggest that perhaps (e.g., Baskerville and Myers, 2002; Grover et al., this issue) IS has matured to an extent where it might be serving in the role of reference discipline, while others (e.g., Benbasat and Weber, 1996; Orlikowski and Barley, 2001; Wade et al., this issue), rejecting the possibility of a twoway interaction, maintain that IS has yet to reach a stage where it can serve in that role Which is it? Sociometric analysis from similar data sets has not arrived at consistent conclusions The question of why this occurred is the issue at hand In order to examine this issue, we describe disparity in our interpretation of reference disciplines, the rationale of disciplinary systems, methodological issues, and normative guidelines Re-thinking the Concept of Reference Disciplines No agreement has ever been reached as to what criteria should be used to proclaim a discipline as a reference discipline, but basically the term has come to refer to those disciplines X that provide foundational, methodological, or other inputs to another discipline/s Y such that the state of knowledge in Y is advanced through inputs provided by X In other words if Y cites X in order to develop and advance the state of its knowledge, X becomes a reference discipline for Y, and vice versa Despite agreement at this basic level, the term reference discipline remains an imprecise concept (Westin et al., 1994) For instance, some IS scholars (see Orlikowski and Barley, 2001, or Wade et al., this issue) define it as a discipline providing extensive input to other disciplines; therefore there is presence of IS citations in any randomly selected piece of work in another discipline This implies an expectation that any work in a discipline should (ideally) cite its reference discipline A less restrictive view of a reference discipline (see Grover et al., this issue; Culnan and Swanson, 1986; Cheon et al., 1992) describes it as simply one that contributes to another discipline The distinctions between contributing and reference disciplines are articulated (Lee, 2001) and acknowledged by Wade et al (this issue) In our opinion, defining X as a reference discipline for Y based on the extent of citations provided is a matter of degree The number of inputs X needs to provide to Y in order for X to become a reference or contributing discipline cannot be determined ex-ante Thus, it is difficult to identify specifically where the reference discipline begins, and such an approach is unlikely to result in any sort of agreement in the future While a comparative analysis of the sort done by Wade et al (what we can refer to as benchmarking) may provide some additional insights and avoid the “matter of degree” problem, note that this approach assumes that all social science disciplines study phenomena of a “general kind” such that results from any one discipline are theoretically useful to another When such an approach is used to examine specialties like IS, it may result in false rejection (type II error) of the hypothesis that IS contributes to other disciplines According to Wade et al.’s definition, if the extent of citations to IS work is found to be high across any randomly selected articles in other disciplines, then that might be used Journal of the Association for Information Systems Vol No 5, pp 336-350/ May 2006 337 Grover et al./A Critique on Wade et al as criteria to judge the status of IS as a reference discipline This perspective can be useful in Management areas such as Organizational Behavior, where the concepts from one area may be applicable to another For instance, conceptual ideas such as interdependence outlined in the classic work of Pfeffer and Salancik (1978) can be easily extended to the study of control (see Emerson, 1962 for an application), mergers and acquisitions (Casciaro and Piskorski, 2005) or even human motivation When the concepts or contexts are so broadly defined, and generalizable, they tend to overlap with work in different disciplines, it is reasonable to assume that certain benefits may accrue from inter-disciplinary transfer of knowledge ceteris paribus Thus, the extent of transfer (as measured through citation analysis or other analytic techniques) may serve as a reasonable indicator of maturity Applying the same yardstick to specialties like IS, however, can be misleading IS research often focuses on topics that are not generalizable As a prominent exemplar, consider research on system analysis/design (for one specific case see Bodart et al (2001) on use of optional properties in conceptual modeling), which is an important component of the IS research agenda Unlike resource dependence, conceptual modeling is a narrowly framed research topic whose results are perhaps not amenable to work done in other disciplines Expecting that such research be used by other disciplines, in our estimation, is inappropriate, since there is no a-priori reason to expect benefits from such ‘misintegration’ We not mean to imply that specialized topics are all that populate IS research Clearly, broad topics like information, knowledge, communication, and processes have a wider audience, and such studies indeed carry the potential of impacting work in other disciplines Markus’ classic article (1983) on the study of information technology and organizations has been cited more than 200 times in disciplines outside IS, ranging from Communication, Education, and Human Resources to General Management, Sociology, and Urban Planning However, the discourses on the core of IS field strongly propels us to engage in the creation of specialized knowledge The point made here is elemental: When concepts and contexts are designed to have wide applicability, an expectation of extensive citation is appropriate However, when this is not the case, an expectation of extensive citations could be a potentially misleading approach to assess the status of a field Rather, an approach that examines “IS-related work” in other disciplines is more appropriate, since it takes into account fundamental issues such as the nature of phenomena studied within IS Contributions of IS research should not be gauged by a “random pick” of articles in another discipline, but should be examined in light of those articles or topics that have a genuine opportunity of benefiting from such integration Alternatively, we should not penalize specialized disciplines and reward disciplines with broader applicability We therefore disagree with Wade et al., and more generally with Orlikowski and Barley’s work, and suggest an alternate perspective on reference disciplines more consistent with that of Baskerville and Myers (2002) This aspect is captured by concepts of work point or discipline point IS-related work that is housed in (say) the journal Management Science reflects the work point of the field Management Science and should be the consideration set when examining the extent to which Management Science draws from IS Other articles in Management Science (e.g., “marginal conditional stochastic dominance”) should not be in the consideration set This perspective makes sense only if journals reflect their work points and are not repositories of convenience for research in other disciplines 338 Journal of the Association for Information Systems Vol No 5, pp 336-350/ May 2006 Grover et al./A Critique on Wade et al Characteristics of Disciplines Wade et al identify 14 disciplines of study including Entrepreneurship and General Management, through which an assessment of the IS discipline may be made It is not entirely clear why inclusion in the FT 40 set was the only criterion used to classify and segregate different disciplines of study Was this the only basis on which the disciplines were chosen? The discussion of reference disciplines based on citation analysis typically presumes the interdisciplinary nature of fields However, if the fields are related, the results of citation analysis will be inflated Wade et al.’s 12 disciplines (analyzed) are clearly not independent in light of this fact For instance, General Management and Strategic Management are probably highly correlated in terms of the research agenda In a study examining the question of Strategic Management’s distinctive competence, Meyer (1991: 824) notes that “true to its General Management orientation, the discipline of Strategy has consistently used firm level performance as the definitive dependent variable.” Similarly Schendel (1990) stresses the need to further Entrepreneurship research in Strategy Meyer’s and Schendel’s observations, although well informed, might still be a matter of personal opinion To more formally evaluate our argument that the program of research under different disciplines considered in the citation analysis might be similar, we examine the similarities between Strategic Management and General Management We so by mapping the similarities between fourteen of the eighteen Strategy research topics originally identified by Schendel and Hofer (1979) and consequently used by Shrivastava (1987), and Priem and Butler (2001) to those found under General Management Analysis of articles in AMR and the AMJ (two journals used to represent General Management) shows considerable overlap between General Management and Strategy research, implying that General Management is perhaps a supra set for Strategy, and possibly for Organizational Behavior Indeed a cursory analysis of topics (as seen from titles and abstracts, see also Table 1) shows overlap between topics within the same article and possible overlaps between Strategy, Entrepreneurship, and Organizational Behavior alongside General Management Therefore, if General Management, Strategy, and Organizational Behavior are split into three different disciplines, the similarity among these disciplines will overestimate the frequency of referencing each other Similarly, in the inter-disciplinary area of International Business, many researchers with doctorates in Marketing, Economics, and Finance target journals in those areas as well as in International Business, inflating the outside references number Consequently, two analyses performed in Wade et al (i.e citation analysis and selective removal citation analysis) would potentially not only have inflated the results for General Management (and other disciplines) but also raised the benchmark for comparison with the IS field The comparison serves an illustrative purpose We compare general management (GM) with strategy since it had the highest (42%) external influence in the study (see Table in Wade et al study); a research program listing was available, and this confound was the primary area where overlaps were likely to occur Journal of the Association for Information Systems Vol No 5, pp 336-350/ May 2006 339 Grover et al./A Critique on Wade et al Table Research Programs in Strategy Related to Literature in General Management Strategic Management General Management AMR # of articles AMJ # of articles Agency Theory Networks/Network theory, and Austrian economics Theory of the firm Innovation and advantage Organizational learning Contingency models 35 49 68 77 Eisenhardt, 1989, Davidson et al 2004 Jacobson, 1992; Ferrier and Smith, 1999 59 77 71 88 70 63 Steensma and Corley, 2001 Marcus, 1981; Sibbin et al., 2005 Lawrence et al., 2005; Arthur et al., 2005 Longnecker and Pringer, 1978; Mitchell et al., 1970 Strategic management processes Board of Directors General Management roles Behavioral Models and Culture; Culture and Resource selection Top management teams Managerial actions, and prescriptions 230 170 Hatch, 1993; Early and Singh, 1995 10 148* 42 184* Social responsibility Strategy formulation Environmental analysis Strategy implementation Strategy Content Formal Planning systems Strategic control Social and natural environmental issues Competitive strategy and competitiveness Environment and resources Industry structure/knowledge Human Resource Management Strategic assets and planning Strategic information systems 421 83 118 59 238 320 107 86 55 148 Entrepreneurship and new venture Multibusiness multicultural firms Others Alliances, joint ventures, and performance 53 125 Daboub and Rasheed, 1995; Boone et al., 2004 Shen and Cho, 2005; Van Preen and Janssen, 2002 Forbes and Nord, 2003; Russo and Harrison, 2005 Hult et al., 2002; Wright et al., 1995 Nehrt, 1998; Bansal and Clelland, 2004 Agarwal et al., 2004; Turner and Makhija, 2006 Wilhelm et al., 1985; Gardner, 2005; Colbert, 2004 Mcgaughey, 2002; Barkema, 2002 Post and Epsteim, 1977; Koester and Luthans 1979; Gallagher, 1974 Powell et al., 2006; Richard et al., 2004 International strategic management 126 200 Research Topic Strategic Concepts Research Program Mergers, acquisition, and diversification, and Quality Representative Authors Reuer and Lieblin, 2000 Greenhaus and Powell, 2006; Lengnick-Hall, 1996; Rindova et al., 2005; Atuahene-Gima, 2003 * Additional articles in AMR and AMJ can be classified under this label but the lower bound is presented Topics are overlapping with papers usually representing multiple topics so numbers are inflated but this fact merely reinforces either the “generic” nature of phenomena studied under Management or the considerable overlap found between GM, Strategy and other subsets of Management 340 Journal of the Association for Information Systems Vol No 5, pp 336-350/ May 2006 Grover et al./A Critique on Wade et al Empirical Issues Wade et al conclude from the results of a citation analysis that there is limited, if any, evidence supporting emergence of IS as a reference discipline Their conclusions stand in contrast to our own work, which paints a more optimistic picture of the maturity of IS Why have two studies using similar datasets arrived at contrasting opinions? Above and beyond the fundamental differences interpreting the term “reference discipline,” itself, or disciplinary overlap, we believe scope choices such as journal selection may be responsible (for divergent results), since they often underpin the results of most sociometric and scientometric techniques Measurement and measurement validity is extremely sensitive to the choice of journal basket, as exhaustively demonstrated by Chua et al (2003) In their empirical analysis of maturity, Wade et al., have made certain scope limitation choices, just as we have done in our own study This is understandable given the entangled nature of the reference discipline problem, the shortcomings inherent in sociometric techniques, and limited resources Unfortunately, as we will demonstrate, such choices can have significant implications on the results and conclusions For instance, Wade et al did not include two important journals that could easily “make the grade” as premier business journals These are Organization Science, and Decision Sciences The choice was made because data was either unavailable on ISI Web of Science or the journal did not appear in the FT 40 set By our estimate, these two journals are important knowledge sinks for IS work within the Organization Science (OS) and Management Science (MS) disciplines, respectively Our fourteen-year span of data shows that 91% of the OS discipline’s citation to IS comes from the journal Organization Science out of the set of AMJ, AMR, and OS Further, 78% of the MS discipline’s citation to IS comes from the journal Decision Sciences out of the set of Management Science and Decision Sciences Based on these figures it can reasonably be deduced that omission of these two journals had a significant impact on the results of the Wade et al study, and is perhaps the reason behind the lower citation count found in their Table (see cross-functional influence of Management areas by citation counts) Inclusion of these two journals in the Wade et al study will lead to similar results to our own To estimate the impact of adding these two journals, we counted citations from Decision Sciences and Organization Science for all articles published in the sample years 2000 and 2001 We counted citations to four journals—MISQ, ISR, JMIS, and CACM—used to represent the IS workpoint in these two journals A total of 244 citations were observed, or about 122 citations per year If we multiply the number of citations/year by 12 to extrapolate the total citations to IS from these two journals for the 12 years (1990-2001), the observed count would approximate 1464 for the four journals Excluding the citations to CACM results in 1170 citations to IS articles; in other words, around 79.9% of the external citations to IS are to MISQ, ISR, and JMIS This means that expanding the journal set used by Wade et al by including Organization Science and Decision Sciences will increase the total citation to IS articles by 42 percent (2nd column in their Table 3) Another potential issue is the inclusion of CACM in the IS journal set used by Wade et al CACM is included in IS introspective studies since it is easy to justify as an objective selection because it is included in the FT 40 set and is above a certain cutoff in journal rankings In following objectivity however, one has to be cognizant that CACM is a hybrid Journal of the Association for Information Systems Vol No 5, pp 336-350/May 2006 341 Grover et al./A Critique on Wade et al journal (Lowry et al., 2004; Peffers and Ya, 2003) In the early stages of the IS discipline, CACM was considered a research-oriented publication However, in analyzing the IS research over 1991-2001 (almost the same period as Wade et al.), Chen and Hirschheim (2004) chose ISR, MISQ and JMIS instead of CACM They argue that “CACM has changed so as to appeal more to general readers…thereby reducing the scholarly nature of its publication (page 204).” Since CACM has less chance of being cited in academic journals due to its practitioneroriented nature, inclusion of this journal might inflate the denominator for citations per article (5th column in their Table 3), and result in the underestimation of the citation count per article According to Table 3, the total number of articles published in the four IS journals is 2654 (=3479/1.311) Articles from CACM constitute a significant 62.1%, or 1648 articles, of this total while a sample investigation of citations from Decision Sciences and Organization Science reveals that citations to CACM constitute only 20.1% of total citations to IS articles Therefore, total citations to IS work without CACM can be estimated as 3953 (= 80% of 3479 + 1170 from the addition of Decision Sciences and Organization Science) Total citable IS articles without CACM (see Table 2) are then estimated at 1006 (= 38% of the authors’ citable articles, 2654), with a corresponding (citations/citable articles) ratio of 3.93 In comparison to 1.311 (5th column of the Table 3), this is a significant increase, since it is not only above the overall average (3.20), but it also clearly overtakes established disciplines like Management Science or Organizational Behavior Obviously, this result will favorably affect the ‘citations per article from other journals in same area’ and ‘citations per article from journals outside the area’ statistics in columns and of Table The inclusion of CACM might be the reason that the authors’ result showed the decreasing ratio of citations/citable article in IS (Figure in Wade et al study) However, when accumulated (see Table 2), the count of CACM articles appears to inflate the denominator of the ratio since it takes up almost Table Number of citable articles in MISQ, ISR, and JMIS Year ISR JMIS MISQ Total Accumulated* 2001 26 40 23 89 647 2000 26 38 29 93 569 1999 25 40 30 95 480 1998 26 32 24 82 405 1997 21 36 21 78 328 1996 31 35 23 89 248 1995 16 35 24 75 162 1994 21 33 23 77 83 1993 16 35 29 80 — 1992 16 35 35 86 — 1991 12 31 36 79 — 1990 20 30 33 83 — Total 256 420 330 1006 * Calculation of this followed the Wade et al.’s definition of citable articles and compensated for the three-year lag proposed in their study The exact date of the change in CACM policy is unclear But, it is safe to say that it includes a significant portion (5 or more years) of the Wade et al period With the magazine orientation, CACM publishes many more articles per issue that are shorter and less academic in tone 342 Journal of the Association for Information Systems Vol No 5, pp 336-350/May 2006 Grover et al./A Critique on Wade et al two thirds of the total IS articles each year Thus, we expect a moderate increase in accumulated citable IS articles without CACM Since the number of citations to IS has increased moderately over the years as is shown in Figure (in Wade et al.), the trend of the ratio (= the number of citations to IS/the number of accumulated citable IS articles) is expected to be horizontal or slightly decreasing rather than dramatically dropping as in Wade et al.’s study While the actual affect of CACM may be less pronounced than illustrated above, it clearly demonstrates that the journal selection of the two studies may have played a key role in producing the different conclusions Consequently, results from both studies need to be interpreted in light of this fact Making Normative Statements on Influential Research One of the eventual goals of young disciplines such as IS or International Business is to advance research streams in other disciplines Wade et al make some normative statements in order to accomplish this Of the four prescriptions provided by Wade et al., three (cross-pollination of ideas, increasing research quantity, and systems thinking) follow the direct approach of increasing generalization and visibility of IS research The other prescription refers to the second degree citations, which is based on an indirect approach to achieving the objective Though we applaud Wade et al.’s desire to be prescriptive, we believe that their suggestions should be carefully assessed With respect to the direct approach of increasing the generalization and visibility of IS research, it is important to note the pyramidal structure of academic disciplines —a “supra discipline” often being composed of a number of different “sub-disciplines.” For instance, Marketing could be defined as a supra discipline of which E-Commerce is a sub-discipline, Finance is a supra discipline of which Financial Economics is a subdiscipline When making normative statements about increasing the impact of IS research, one has to be cognizant of two issues within the supra-sub-discipline schema First, the influence of IS research will be determined by how important IS is to the subdiscipline of another discipline This in effect ties back in to our earlier argument that the term “reference discipline” needs to be re-interpreted for specialties like IS since the extent of citations to IS will be a direct function of the importance of IS to the subdiscipline Second, even when IS research is useful to the sub-discipline; it does not imply that the overall result (on the supra-discipline) will reflect favorably on IS If the sub-discipline is a small part of the supra discipline, the perspective advocated by Orlikowski and Barley (2001) might fail to detect any significant impact of IS work in nonIS disciplines We could recommend that we refocus attention from studying narrower topics such as conceptual modeling to topics that appeal to a broader academic audience, such as the effects of IT on organizational power structures In principal, this would allow coverage of more sub-disciplines within the supra-discipline However, such an approach (earlier outlined by Baskerville and Myers, 2002) carries the risk of IS losing its identity as the discipline If IS is too broad, then the field might disintegrate or lose its distinctness; as such, there is no ex-ante reason to expect other disciplines to draw upon IS in the first place, since most of what is said is already borrowed from other disciplines Note that our argument is not that generalization (see also Alter 2003a, b, c) is an unacceptable strategy However, for young fields like IS it is important that the unique value that they offer not be diffused Generalization and visibility, when described in terms of Wade et al., could be an appropriate strategy, provided that unique Journal of the Association for Information Systems Vol No 5, pp 336-350/May 2006 343 Grover et al./A Critique on Wade et al relationships can be established or changes to existing concepts and relationships can be demonstrated with the infusion of IT, as argued by Agarwal and Lucas (2005) Simply generalizing by testing existing relationships or constructs within an IT context is unlikely to be beneficial and may perhaps be counter-productive to our emergence as a reference discipline With respect to the indirect or second degree citation approach, we need to be cautious in our interpretation Conceptually, the authors note that second degree citations can be an efficient spreader of knowledge, and IS should consciously strive to increase its impact through this method Accordingly, if we consider three disciplines A, B, and C; an article at B drawing upon A is heavily cited by C, then A’s knowledge is expected to spread efficiently, and the impact is attributed to A This argument is problematic, since without knowing the type and extent of input going from A to B, it is hard to argue that A has a broader impact on C What if an article published at B cites 90% internal references and 10% to A Should the credit go to discipline A without understanding what type of contribution the discipline A made? Marketing is a good example of the above situation Practically every empirical article cites Churchill (1979) An article published on social networks may cite Churchill This social networks article may be heavily cited by other disciplines for the contribution it made to the understanding of social networks discipline In the analysis of Wade et al., the credit is given to Marketing (i.e Churchill’s article) Giving credit at the second level thus undermines the contribution made at the first level Wade et al suggest that publishing IS articles in non-IS disciplines would benefit IS research While theoretically the idea appears appealing, note that when measured through their sociometric analysis, such an approach might fail to find any influence of IS research since credit for an article published in Management Science cannot be given to (say) MIS Quarterly Thus, if we were to measure citations, it would appear that MS performs influential research when the credit should have been ideally given to IS To more precisely evaluate this argument interested readers are directed to work in control done under the banner of IS research Most studies (see, Kirsch 1996/1997/2002/2004, Choudhary and Sabherwal 2003) test relationships outlined by Ouchi (1979) within the IS context without specifying how these relationships tend to be any different from a non-IS context The uniqueness of the IS context is relegated to the background While this particular stream is still in its infancy, Orlikowski and Iacono’s (2001) finding that 24% of the research in the leading outlet Information Systems Research falls under the nominal category provides further evidence for our assertion Nominal research is where the IT artifact is essentially absent, technology is invoked for “names sake”, is incidentally referred to, and the conceptual and analytical emphasis lies elsewhere such as in topics like power that might be of broad interest to IS researchers According to these authors such research does not belong to the IS field For instance the study of power could be done entirely without any reference to a IS in which case there is apparently no a-priori reason for other disciplines to draw upon IS research Alternately, it can be shown how power is influenced through information systems Such research while distinctly IS tends to be broadly defined and generalizable in the sense that any context where power imbalance might occur due to IT, IS research can serve as a useful source of guidance on (say) how to deal with the problem This research is useful to a broad audience An additional facilitator of second degree citations is when an article published at B is not “reprocessed” at B That is an article that would be published otherwise at A has been published at B (say the article at B has 90% references from A and 10% from B) We doubt this is the case in the Ethics exemplar or in the context of the widespread second degree impact of Economics on Management We would have to conclude that economic research gets published as is or with minimal changes in management journals which belies reality 344 Journal of the Association for Information Systems Vol No 5, pp 336-350/May 2006 Grover et al./A Critique on Wade et al (thereby understating the impact of IS) Perhaps such an understatement is already reflected in their first level results Wade et al.’s argument can be reinterpreted to mean that an IS article (see footnote 5) serves as a gateway for advertising other IS research Indeed, from this lens such an approach is appropriate, but as correctly stated by the authors, it relies on extreme assumptions of influential works which are impossible to know ex-ante In sum we believe that Wade et al have identified some useful concepts such as cross pollination of ideas and use of systems thinking that can serve as powerful conduits for IS to gain visibility However, these recommendations, like the ones discussed above, may also carry caveats that need to be fully understood before strong prescriptions can be made Choosing Future Research Directions We believe that articles such as Wade, et al and Grover et al provide a useful service to the discipline Both studies go beyond rhetoric and provide data-based evidence for their position on the state of the field They both demonstrate the promise and perils of sociometric methods The contrasting conclusions yield interesting insights on definitional issues regarding reference discipline, sensitivity of sociometric methods to journal baskets and assumptions, and the necessity to carefully build prescriptions within the limitations of the methods We not wish to wade too deeply into the debate on whether the status of IS should be gauged from a broad perspective adopted by Wade et al or a narrow perspective adopted by Grover et al These perspectives have the intonation of our numerous introspective discourses on theory, core, and identity Suffice it to say that a broader perspective is appealing since it can promote the wide business impact of research done in IS (as a specialty of Management), but it also carries the risk of underestimating the true impact in case of specialties like IS The narrow perspective, on the other hand, creates non-substitutable knowledge, thereby more accurately depicting a specialty like IS; but it stands the risk of overestimating the impact of a discipline since evaluations are based only on the specialty-related work in other disciplines We believe that IS should promote specialized knowledge that may not (currently) have a broad influence in other areas, but will impact growing and increasingly important IT-related sub-areas in other disciplines Our study demonstrated that impact as we compared the reciprocity of references to and from the IS discipline and IS-related sub-disciplines in Management Science, Organization Science, Computer Science, and Marketing journals Perhaps in time, with increasing pervasiveness of IT, resilient, age-old academic institutions in other disciplines (e.g., Finance) will be compelled to study IT-related phenomena germane to their domain The two perspectives will then converge While working within the bounds of time and resources, both studies have understandably placed certain limits on the depth of analysis One such limitation is overt reliance on the quantitative aspect of references such as counts In any research study, references often serve different purposes, like motivational support, theoretical reasoning, hypotheses development, methodology support, and so on Since neither study captures the type of input provided by each reference, the results may be biased (Vessey et al., 2002) For instance, with respect to second degree citations, an influential IS article that draws 90% on IS references but develops its theoretical arguments from Journal of the Association for Information Systems Vol No 5, pp 336-350/May 2006 345 Grover et al./A Critique on Wade et al 10% non-IS references may not serve as a gateway to advertising IS research Perhaps recognition of this fact may provide more accurate depictions of IS maturity, reduce the sensitivity in results arising from journal selection and conceptual differences in different studies, and allow more efficient cross comparison of different studies Results from both studies are arguably an artifact of journal and article selection procedures, and we have shown in approximation how dramatic changes can result from revising these procedures Both studies, however, clearly agree that if IS-related work is published in the journals of other disciplines, it bodes well for the field as a reference discipline IS appears to be doing better in this regard We also believe that both of our studies potentially miss the impact of some very influential work on disciplines For instance, Wade et al present the low first-degree impact of Economics on other fields Similarly, Grover et al find weak inputs from Economics into IS (and none going the other way) However, Economics has enjoyed direct impact on many sub disciplines of Management and has even been proclaimed the queen of social sciences (Bazerman, 2005; Ferraro et al., 2005) In fact, Economics has served as a reference discipline for Finance right from its inception: there is a separate sub-discipline of Finance (Financial Economics) that draws entirely upon economic reasoning The reason for this underestimation is that many of the impacts of transaction cost economics on Management and IS (for instance) seem to come from influential books – which are ignored in both studies Or perhaps, taking the broader approach discussed earlier may result in missing the subtle role played by different disciplines in enriching each other Clarification of basic terms and a broader sociometric net may help in more objective examinations in the future Finally, both studies are in agreement that Economics does not draw upon IS Attention needs to be provided in further work on the reasons underlying this result In a recent article in the American Economic Review, Baily and Lawrence (2001) point out the impact of IT and the arrival of the new e-conomy Following the insightful suggestions made by Wade et al., perhaps it might be possible to reverse the one-way interaction between IS and Economics Similarly, Operations Management which is not considered in either study, may emerge as a knowledge sink for IS research, especially given the increasing focus on research in automated supply chains Conclusion Given the contradictory opinions from both studies, the topic of truly assessing IS influence appears to be still in its infancy Behind the sociometric methods used lie subtle assumptions regarding a field’s influence; and the consideration set of journlas can have a profound influence on the conclusions drawn Herein lies the dilemma Is one to be optimistic or pessimistic about the progress of the field? The answer to this question is an unsatisfying and ambivalent “it depends.” Each position is correct within its boundary conditions We believe IS is indeed making progress in its influence among its classical reference disciplines—but perhaps not in the pervasive sense that Wade et al imply We encourage other interested observers of our field to move this work forward It is crucial to have clear concepts and instrumentation to evaluate the state of our discipline We advocate the use of powerful sociometric tools that match clearlystated definitions, assumptions, and boundary conditions regarding the field and its influence 346 Journal of the Association for Information Systems Vol No 5, pp 336-350/May 2006 Grover et al./A Critique on Wade et al References Agarwal, R., and Lucas Jr., H.C "The Information Systems Identity Crisis: Focusing on High-Visibility and High-Impact Research.," MIS Quarterly (29:3) 2005, pp 381398 Alter, S "18 Reasons Why IT-Reliant Work Systems Should Replace "The IT Artifact" as the Core Subject Matter of the IS Field.," Communications of AIS (2003:12) 2003a, pp 365-394 Alter, S "Sidestepping the IT Artifact, Scrapping the IS Solo, and Laying Claim to "Systems in Organizations"." Communications of AIS (2003:12) 2003b, pp 494526 Alter, S "The IS Core XI Sorting Out Issues About the Core, Scope, and Identity of the IS Field.," Communications of AIS (2003:12) 2003c, pp 607-628 Baily, M.N., and Lawrence, R.Z "Do We Have a New E-conomy?," American Economic Review (91:2) 2001, pp 308-312 Barney, J.B "Is the Resource-Based 'View' A Useful Perspective for Strategic Management Research? Yes.," Academy of Management Review (26:1) 2001, p 41 Baskerville, R.L., and Myers, M.D "Information Systems as a Reference Discipline.," MIS Quarterly (26:1) 2002, pp 1-14 Bazerman, M.H "Conducting Influential Research: The Need for Prescriptive Implications.," Academy of Management Review (30:1) 2005, pp 25-31 Benbasat, I., and Weber, R "Research Commentary: Rethinking "Diversity" in Information Systems Research.," Information Systems Research (7:4) 1996, pp 389-399 Bodart, F., Patel, P., Sim, M., and Weber, R “Should Optional Properties Be Used in Conceptual Modelling? A Theory and Three Empirical Tests,” Information Systems Research (12:4) 2001, pp 384-405 Casciaro, T., and Piskorski, M.A.J "Power Imbalance, Mutual Dependence, and Constraint Absorption: A Closer Look at Resource Dependence Theory.," Administrative Science Quarterly (50:2) 2005, pp 167-199 Cheon, M J., Lee, C., and Grover, V "Research in MIS - Points of Work and Reference: A Replication and Extension of the Culnan and Swanson Study," ACM SIGMIS Database (23:2) 1992, pp 21-29 Chen, W and Hirschheim, R “A paradigmatic and methodological examination of information systems research from 1991 to 2001,” Information Systems Journal (14:3) 2004, pp.197-235 Choudhury, V., and Sabherwal, R "Portfolios of Control in Outsourced Software Development Projects.," Information Systems Research (14:3) 2003, pp 291-314 Chua, C., Cao, L., Cousins, K., and Straub, D.W "Assessing Researcher Production in Information Systems," Journal of AIS (3:6, January) 2003, pp 145-215 Churchill Jr., G.A "A Paradigm for Developing Better Measures of Marketing Constructs.," Journal of Marketing Research (16:1) 1979, pp 64-73 Davenport, T.H., and Short, J.E "The New Industrial Engineering: Information Technology and Business Process Redesign.," Sloan Management Review (31:4) 1990, pp 11-27 Emerson “Power-dependence relations.” American Sociological Review (27) 1962, pp 31–40 Ferraro, F., Pfeffer, J., and Sutton, R.I "Prescriptions are Not Enough.," Academy of Management Review (30:1) 2005, pp 32-35 Journal of the Association for Information Systems Vol No 5, pp 336-350/May 2006 347 Grover et al./A Critique on Wade et al Kirsch, L.J "The Management of Complex Tasks in Organizations: Controlling the Systems Development Process.," Organization Science: A Journal of the Institute of Management Sciences (7:1) 1996, pp 1-21 Kirsch, L.J "Portfolios of Control Modes and IS Project Management.," Information Systems Research (8:3) 1997, p 215 Kirsch, L.J., Sambamurthy, V., Ko, D.-G., and Purvis, R.L "Controlling Information Systems Development Projects: The View from the Client.," Management Science (48:4) 2002, pp 484-498 Kirsch, L.J "Deploying Common Systems Globally: The Dynamics of Control.," Information Systems Research (15:4) 2004, pp 374-395 Markus, M.L "Power, Politics, and MIS Implementation.," Communications of the ACM (26:6) 1983, pp 430-444 Meyer, A.D "What Is Strategys Distinctive Competence," Journal of Management (17:4), Dec 1991, pp 821-833 Lee, A "Editor's Comments," MIS Quarterly (25:1) 2001, pp iii-vii Lowry, P B., Romans, D., and Curtis, A "Global Journal Prestige and Supporting Disciplines: A Scientometric Study of Information Systems Journals," Journal of the Association for Information Systems (5:2) 2004, pp 29-77 Peffers, K and Ya, T "Identifying and Evaluating the Universe of Outlets for Information Systems Research: Ranking the Journals," Journal of Information Technology Theory and Application (5:1) 2003, pp 63-87 Orlikowski, W.J., and Barley, S.R "Technology and Institutions: What can Research on Information Technology and Research on Organizations Learn From Each Other?," MIS Quarterly (25:2) 2001, pp 145-165 Orlikowski, W.J., and Lacono, C.S "Research Commentary: Desperately Seeking the 'IT' in IT Research A Call to Theorizing the IT Artifact.," Information Systems Research (12:2) 2001, p 121 Ouchi, W.G "A Conceptual Framework for the Design of Organizational Control Mechanisms.," Management Science (25:9) 1979, pp 833-848 Pfeffer, J., and Salancik G R “The External Control of Organizations: A Resource Dependence Perspective,” 1978, New York: Harper and Row Schendel, D “Introduction to the special issue on corporate entrepreneurship,” Strategic Management Journal (11) 1990, pp 1-3 Schendel, D., & Hofer, C W “Strategic management: A new view of business policy and planning,” 1979, Boston: Little, Brown Shrivastava, P "Rigor and Practical Usefulness of Research in Strategic Management," Strategic Management Journal (8:1) 1987, pp 77-92 Thomas, K.W., Tymon, W.G., and Jr "Necessary Properties of Relevant Research: Lessons from Recent Criticisms of the Organizational Sciences.," Academy of Management Review (7:3) 1982, p 345 Vessey, I., Ramesh, V., and Glass, R.L "Research in Information Systems: An Empirical Study of Diversity in the Discipline and Its Journals.," Journal of Management Information Systems (19:2) 2002, pp 129-174 Westin, S M., Roy, M., and Kim, C K "Cross-Fertilizations Of Knowledge: The Case Of MIS And Its Reference Disciplines," Information Resources Management Journal, (7:2) 1994, pp 24-34 348 Journal of the Association for Information Systems Vol No 5, pp 336-350/May 2006 Grover et al./A Critique on Wade et al About the Authors Varun Grover is the William S Lee (Duke Energy) Distinguished Professor of IS in the Department of Management, Clemson University Previously he was a BP Foundation Fellow and Professor of Information Systems at the University of South Carolina Dr Grover has published extensively in the IS field, with three books and over 150 publications in refereed journals His current research focuses on the impact and effectiveness of IS at the organizational and market levels A number of recent articles have ranked him among the top five researchers based on publications in top IS journals over the past decade He is currently serving as Senior Editor for MIS Quarterly, JAIS and Database and as Associate Editor for the Journal of MIS, Journal of Operations Management, International Journal of E-Commerce, and numerous others Dr Grover has also received recognitions for his research from the Decision Sciences Institute, PriceWaterhouse Coopers, AIS and Anbar Intelligence, and is the recipient of a number of teaching awards for courses taught at the MBA and doctoral levels Ramakrishna Ayyagari is a PhD candidate in Information Systems in the Department of Management at Clemson University He holds MS and BE degrees in Electrical Engineering Broadly, his research interests include understanding the impacts of Information and Communication technologies on individuals and organizations His present work is in the areas of techno-stress, e-commerce, project management, and IT adoption Rahul Gokhale is a PhD candidate in Information Systems in the department of Management at Clemson University He holds an MS in Information Systems and BS in Electrical Engineering Previously he worked as a systems engineer with Johnson Controls His research interests lie at the intersection of social sciences and engineering He is currently exploring issues related to Control, and applications of general system theory He serves as a reviewer for Information Systems Journal and has articles under review at JAIS among others Jaejoo Lim is a PhD candidate in Information Systems at Clemson University He holds an MS in Computer Information Systems from Georgia State University and a BA from Seoul National University His research interests are in the areas of e-commerce strategy, IT investment decision, IT assimilation, and telecommunication Copyright © 2006, by the Association for Information Systems Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and full citation on the first page Copyright for components of this work owned by others than the Association for Information Systems must be honored Abstracting with credit is permitted To copy otherwise, to republish, to post on servers for commercial use, or to redistribute to lists requires prior specific permission and/or fee Request permission to publish from: AIS Administrative Office, P.O Box 2712 Atlanta, GA, 30301-2712 Attn: Reprints, or via e-mail from ais@gsu.edu Journal of the Association for Information Systems Vol No 5, pp 336-350/May 2006 349 Grover et al./A Critique on Wade et al ISSN: 1536-9323 EDITOR Kalle Lyytinen Case Western Reserve University JAIS SENIOR EDITORS Soon Ang Nanyang Technological University, Singapore Varun Grover Clemson University, USA Elena Karahanna University of Georgia, USA Izak Benbasat University of British Columbia, Canada Robert Fichman Boston College, USA Rudy Hirschheim Louisiana State University Robert Kauffman University of Minnesota, USA Juhani Iivari University of Oulu, Finland Yair Wand University of British Columbia, Canada JAIS EDITORIAL BOARD Ritu Agarwal University of Maryland Michael Barrett University of Cambridge, UK Cynthia Beath University of Texas at Austin, USA Wynne Chin University of Houston Anandhi S Bharadwaj Emory University, USA Yolande E Chan Queen's University, Canada Roger H L Chiang University of Cincinnati, USA Galletta Dennis Temple University, USA Matthew R Jones University of Cambridge, UK Chidambaram Laku University of Oklahoma, USA Ho Geun Lee Yonsei University, Korea Guy G Gable Queensland University of Technology, Australia Jae-Nam Lee Korea University, Korea Stuart E Madnick Massachusetts Institute of Technology, USA Anne Massey Indiana University, USA Ann Majchrzak University of Southern California, USA Emmanuel Monod Dauphine University, France Ryutaro Manabe Bunkyo University, Japan Ji-Ye Mao Remnin University, China Eric Monteiro Norwegian University of Science and Technology Jonathan Palmer University of William and Mary, USA B Jeffrey Parsons Memorial University of Newfoundland, Canada Sudha Ram University of Arizona, USA Nava Pliskin Ben-Gurion University of the Negev, Israel Balasubramaniam Ramesh Georgia State University, USA Ananth Srinivasan University of Auckland, New Zealand Viswanath Venkatesh University of Arkansas, USA Youngjin Yoo Case Western Reserve University, USA Jan Pries-Heje Copenhagen Business School, Denmark Suzanne Rivard Ecole des Hautes Etudes Commerciales, Canada Kar Yan Tam Hong Kong University of Science and Technology Richard T Watson University of Georgia, USA Kevin Zhu University of California at Irvine, USA Dewan Rajiv University of Rochester, USA Christina Soh Nanyang Technological University, Singapore Dov Te'eni Tel Aviv University, Israel Gillian Yeo Nanyang Business School, Singapore Ellen Christiaanse University of Amsterdam Kai H Lim Mats Lundeberg City University of Hong Kong, Stockholm School of Hong Kong Economics, Sweden Rajiv Sabherwal University of Missouri - St Louis, USA Bernard C.Y Tan National University of Singapore, Singapore Bruce Weber London Business School, UK ADMINISTRATIVE PERSONNEL Eph McLean AIS, Executive Director Georgia State University 350 J Peter Tinsley Deputy Executive Director Association for Information Systems Reagan Ramsower Publisher, JAIS Baylor University Journal of the Association for Information Systems Vol No 5, pp 336-350/May 2006 ... Negev, Israel Balasubramaniam Ramesh Georgia State University, USA Ananth Srinivasan University of Auckland, New Zealand Viswanath Venkatesh University of Arkansas, USA Youngjin Yoo Case Western... Intelligence, and is the recipient of a number of teaching awards for courses taught at the MBA and doctoral levels Ramakrishna Ayyagari is a PhD candidate in Information Systems in the Department of Management... Minnesota, USA Juhani Iivari University of Oulu, Finland Yair Wand University of British Columbia, Canada JAIS EDITORIAL BOARD Ritu Agarwal University of Maryland Michael Barrett University of Cambridge,