Introduction
The article chosen for analysis was written by an Information Systems (IS) management professor with tremendous experience in the field. It is known as “The relations of requirements uncertainty and stakeholder perception gaps to project management performance” by James Jiang, Shelly Wu, Gary Klein and Liang, T.P for the Systems and Software Journal 82(2009): 808. The paper generally seeks to revise the uncertainty framework model so as to incorporate the role of stakeholder gaps in project performance. The authors assert that requirements instability and requirements diversity are related to stakeholder perception gaps and this can be tied in with project performance. This paper will argue that the latter article is quite professional although some few flaws here and there have been identified.
Summary of the peer reviewed journal article written by Jiang et al
The authors – Jiang et al (2009) start by asserting that requirements uncertainties play a major role in determining project outcomes and two major approaches can be sought to deal with these uncertainties. One method is coming up with structures to deal with the requirements uncertainties. The problem with such a method is that it is largely reactive. The second perspective is more proactive because it seeks to identify potential requirements uncertainties in the project and possible methods of reducing these uncertainties. Furthermore, these authors identify one of the issues that can be caused by uncertainties requirements i.e. stakeholder perception gaps. Any IS project team must seek out ways of mitigating these requirements uncertainties by establishing coping mechanisms. However, those coping mechanisms never really eliminate all project risks and it is therefore safe to say that the remaining project risks or residual project risks can largely be used as a reasonable measure of project management performances. One substantive argument propelled by these writers is that users and IS experts differ in terms of what their perception gaps and this undermines project success. A look at the uncertainty model reveals that these differences amongst stakeholders are rarely taken into account by researchers. Yet some issues such as requirements instability and requirements diversity can contribute towards greater performance risk. In this regard, the authors hypothesize that there is a positive relationship between requirements instability/ requirements diversity and stakeholder perception gaps. Secondly, they assert that there is also a positive correlation between requirements instability/ requirements diversity and residual performance risk. Third, the authors claim that there is also a positive correlation between stakeholder perception and residual performance risks. The last hypothesis is that a negative correlation exists between project management performance and residual performance risk. Generally speaking, these writers are trying to modify the current uncertainty framework by including differences in perception gaps of stakeholders. (Jiang et al, 2009)
In order to test these hypotheses, the method of questionnaires was used where 500 forms were mailed to randomly selected individuals. However, only 150 respondents filled them. The tests were designed to measure requirements instability, requirements diversity, stakeholder perception gaps, horizontal coordination, residual performance risk and project management performance. At the end of it all, validity and reliability were tested using PLS. After carrying out the tests, data analyses were done and they confirmed hypotheses 1, 3 and 4. However, in hypothesis 2, only requirement diversity was affirmed but requirement instability was not. At the end of the article, the authors discuss the implications of their findings in information systems management where they assert differences in perceptions should not be the only issues considered during assessment of performance as stakeholder gaps play an important role as well. Also, it has been stated that the links between requirements instability and residual performance risks need to be acknowledged in future research.
Critical review
The overall impression one gets of this article is that it is well organized. Its format is quite professional as the writers first start by giving readers a general definition of terms, then a summary of researches carried out in the topic, then some prevailing gaps and methodology and discussion. This logical format is one of the major strengths of the paper and is probably the reason why the article was accepted as a professional journal article. Nonetheless, as one goes through the structure, one can see that the authors dedicate a substantial portion of their work on background and introduction and very little on data analysis yet this is a mechanism for explaining to readers how the authors arrived at their conclusions. (Vanier college library, 2009)
The article relies heavily on a research base. The only problem is that the researchers tend to rely on work published more than five years ago. For instance, several of the items used to measure the constructs have been derived from a book written fourteen years ago. When the authors were trying to measure requirements instability through the size of requirements fluctuation in the beginning and later phases of the work, they borrow from Nidumolu who wrote his book in 1995. (Nidumolu, 1995) It would have been more informative if the authors relied on recent work. Some of the pieces of works cited range as far back as 1981 such as Scott, W.’s “Organizations: rational, natural and open systems”. (Scott, 1981)These authors are dealing with a very dynamic topic i.e. information systems and the latter are subject to change quite frequently. Focusing on recent material would be more helpful to them than what they had chosen to do.
Despite the latter issues, one cannot ignore the fact that the article itself is quite timely. Most of the suggestions and hypotheses made in the paper were badly needed in information systems management as most IS developers focused on using reactive mechanisms to improve performance management system yet this has not necessarily yielded results. Therefore, proposing a new way of looking at the uncertainty framework model is definitely refreshing. (Chek et al, 2006)
There are numerous strengths in this article. First of all the material suggested is quite practical in information systems management. One can deduce this even from the data constructs measured e.g. requirements diversity, performance risks and performance management. All these issues have the potential to slow down project implementation, undermine usefulness of the project or increase costs. Therefore, identifying ways in which these problems can be eliminated is quite insightful. Another strength lies in the fact that the authors have given due attention to matters of reliability and validity. (Meho, 2007)To analyze these, they carried out t-tests and PLS tests and this therefore gives readers confidence in the findings of the article. Additionally, the research methodology employed is quite acceptable. They used mailed questionnaires but because these were likely to yield minimal results, then the researchers chose to send the questionnaires two more times to the respective correspondents and this is definitely a plus on their part. On top of these, the authors have acknowledged that their work was not perfect and hence the reason for proposing areas of improvement to their limitations. It is also commendable how the findings in the research have contributed to knowledge for the rest of the Information systems fraternity as the authors have affirmed that residual performance risks and requirements uncertainties are interlinked. They have also shown that dwelling on perception differences is not adequate; emphasis should also be given to stakeholder gaps. Their use of visual aids has also been satisfactory as they have included models for both the current uncertainty model an their revised version. Readers can therefore get a clear picture of what is being discussed. The authors have also presented their work in a manner that even non IT readers can understand what they are discussing. There was a thesis statement at the beginning of the paper i.e. that stakeholder gaps and requirements uncertainty are related to project management performance. Throughout the paper, emphasis has been on validating this thesis and in the end, its conclusion ties in with the thesis and the research methods. (Baline et al, 2008)
Despite all these strengths, this journal article does not lack its weaknesses. In its research methodology, the author asserts that they had chosen five hundred respondents and sent them mailed questionnaires thrice. However, the authors have not explained whether they made sure that previous respondents did not answer the questionnaire. In this regard, the authors may have based their findings on repeat responses. Aside from this, the very fact that the authors employed questionnaires as a method of data collection could be problematic because perhaps the respondents had differing explanations on some of the issues under discussion and were not satisfied with merely filling in the Likert scale or choosing from the options given in a structured interview. Also, the fact that the research mostly focused on the United States could be a problem for stakeholders in the field. It would have been more insightful or accurate if the authors widened their sample space to other parts of the world such that cultural or social factors that are distinct to the US do not interfere with the results of the paper. (Silverman, 2006) However, if this was done, then perhaps more logistics would be necessary and countries with similar information technology progress to the US should have been selected.
Also, because the participants chosen for the survey were being asked about some controversial issues such as oral communication, it is likely that various individuals interpreted the knowledge in their own ways. In other words, it is likely that some of them may interpret this question in a different manner depending on their understanding of what constitutes good oral communication and this may affect the overall outcome of the paper. When the authors were trying to measure stakeholder perception gaps as a construct, they cited a number of items that would be examined such as system capabilities, project objectives and project management performance criteria. The latter, is a general parameter and specifications should have been given on what this criteria entails. (Mondry, 2005)
In the data analysis phase, focus has been given on explaining the method of PLS test and other methods used by the authors. In fact, one can argue that this has taken up a substantial portion of the data analysis section of the paper. Instead, greater emphasis should have been given on how this test was applied to the material and not on general description of the method.
Also, the authors have gone into great detail to separate some of their hypotheses and one can argue that this may have been done so as to make the paper bulky. Instead, it would have been more acceptable if the writers had chosen to combine some of the pieces together so as to come up with fewer hypotheses. For instance, instead of separating requirements instability and requirements diversity into two hypotheses, they should have combined these into one statement. (Etter, 1993)
The writers of this article have written a well researched and well formatted article, it is has been identified that there are some flaws in it but these flaws are not adequate to warrant an alarm or to denial them a chance to publish such works. Since it is peer reviewed, then the level of rigor in the material cannot be denied; only a few matters needed to be ironed out.
Lastly, because the focus of the paper depended mostly on IS managers/ project leaders (since the latter represented about ninety two percent of the respondents while only eight percent were IS professionals) then chances are some of the responses represented the view of project leaders and not users. Some sort of mechanism should have been worked by the concerned researchers on how they could garner effective representation of both IS developers and users in this survey and this would have generated a more balanced view of the findings.
Conclusion
The article under analysis mostly focuses on the role that stakeholder perception gaps have on uncertainty requirements and hence residual performance risks which can also be tied in with project management performance. I generally agree with the authors’ assertions as the have started with a strong thesis statement and have proceeded to validate it. The article’s major strength is the scientific rigor of the work as the authors base their conclusions on mathematical and statistical relationships. However, the problem with this article is that its data analysis section is more of general description than an explanation on how methods chosen were related to the study. Also, the sampling technique should have been improved to include both users and developers of information systems.
References
Baline, J., Vucovich, L., Smith, J & AHIP. (2008). Analyzing the impact of authors’ publications. Med Lir Association Journal 96(1), 66
Chek, J., Quan. J. & Granham, B. (2006). Providing evidence on quality and impact for researchers. Journal of Quality health and research 16(3), 423
Etter, Z. (1993). Using reference questions to analyze service and collection. Academic media librarianship journal, 1(2), 66
Jiang,J., Shelly, W., Klein, G. and Liang, T. (2009). The relations of requirements uncertainty and stakeholder perception gaps to project management performance. Systems and Software Journal, 82(91), 808
Meho, L. (2007). The rise of citation analysis. Phys World journal 20(1), 32
Mondry, A. (2005). Revisiting the impact factor. Biomed digital library, 5(7), 15
Nidumolu, S. (1995). Effect of coordination & uncertainty on software project performance. Information Systems research, 6(3), 191-219
Scott, W. (1981). Organization: rational, natural and open systems. NJ: Prentice hall
Silverman, D. (2006). Interpreting qualitative data. NY: Sage
Vanier college library (2009). How to critically analyze sources of information: Vanier College website. [Online]. Web.