Abstract
People especially researchers erroneously assume that quantitative research is the most applicable in most situations. This assumption has been in existence for a long time. Such scholars are convinced that quantitative research is the most appropriate for policy making and enactment of laws especially those that are related to business operations. Other challenges that are facing the society are also assumed to be solvable through quantitative analysis.
To this end, the research by Kartiwi and MacGregor is perceived to be the solution for the problems faced by SMEs in attempts to adopt e-commerce marketing strategies. The study was conducted to identify the actual barriers facing SMEs. In this paper, the researcher critiqued the article reporting the findings of the study. It was found that the quantitative analysis had several shortcomings that impacted on the operations of SMEs as a whole.
A Critique of Quantitative Research Article
Introduction
This research was conducted to identify barriers to the adoption of electronic commerce in small and medium-sized enterprises (herein referred to as SMEs) in developed and developing countries. This is a cross-country comparison which was carried out by Mira Kartiwi and Robert MacGregor from the Australian University of Wollongong in the year 2007.
Research Summary
The research was carried out to identify barriers hindering the adoption of e-commerce among SMEs. The study assumed a quantitative analysis methodology where data and report were compiled quantitatively. Empirical data was collected from two countries representing developed as well as developing countries.
This was from Sweden and Indonesia respectively. Characteristics of these countries were compared and conclusions were made from the responses of participants from the two countries. Barriers were also identified from literature in the field and a comparison was carried out between the two countries. The research was focused on small business enterprises and their unique features which were compared to those of other big companies.
This paper is a critique of the article reporting the findings of this comparative study. In this paper, the author is going to analyze several aspects of the article. These include the appropriateness and inappropriateness (where applicable) of employing quantitative research methodology as opposed to qualitative research design or using a combination of the two.
Barriers that face e-commerce in general have been adequately analyzed by the quantitative research carried on both developed as well as developing countries. The two countries used different approaches as far as e-commerce is concerned. Therefore making use of one form of analysis (in this case quantitative analysis) can be considered as inappropriate.
This is considering that the information gleaned from numbers varies from one region to the other and more importantly some significant aspects in the two countries cannot be tested or verified using numbers (Hall, 2012). For instance the developed country would describe its barriers as technical while in the real sense there were organizational barriers also.
Therefore if research is carried out to gather some information on these two countries there is need to make use of different types of data collection methods. In many cases, quantitative analysis has been used to find solutions for the two issues specified. These are the organizational and technical issues. These two categories are different and therefore continued research based on quantitative methodology comprises the accuracy of the data that is collected (Hall, 2012).
Reports indicate that SMEs are being affected significantly since their role in the economies is reducing at a high rate than it is expected. This is seen when gross domestic product (herein referred to as GDP) stagnates or slows down drastically (Blili & Raymond, 2009). To respond to this effect, the SMEs are embracing global markets to trade their products and services, a phenomenon referred to as e-commerce.
This move has radically changed the operation of most SME business operators. Many people find e-commerce to be a more appropriate and more convenient mean of trading using computer networks. This being the case, e-commerce can be regarded as a potential business trend across the globe. Many business operators are opting for it as opposed to the traditional physical market. This is despite the fact that in e-commerce, the entrepreneurs are not in a position to meet their clients on a face to face basis or physically.
The increasing popularity of e-commerce among SMEs is evident because reports indicate those business using internet services and other forms of technology in conducting their trade perform better than their counterparts who are not using these services (Bajaj & Nag, 2009). It seems that Kartiwi & MacGregor (2007) are very much aware of this fact. This is the reason why they found it important to conduct a study in this field.
SMEs across the globe are adopting this technology to address the barriers that are hindering them from fully contributing to the growth of GDP around the world. The barriers have been widely documented after extensive studies were conducted. Kartiwi & MacGregor (2007) make their contribution to the knowledge in this field by conducting the research and documenting their findings in this article. They note SMEs and other businesses from the developing countries have not fully embraced e-commerce for various reasons.
This is despite the fact that the entrepreneurs are fully aware (most of the times at least) of the potential benefits of using e-commerce. One such reason why they have not embraced the initiative is simply because e-commerce as a form of trade is not static spatially or chronologically. Rather, it is a dynamic and fluid phenomenon that varies from one geographical location to the other and from one time to the other. SMEs and other businesses seem to have difficulties keeping track of this dynamism. This is especially so because the organizational level and structure that is supposed to incorporate e-commerce are not efficient in most of these nations (Bajaj & Nag, 2009).
Characteristics of Small and Medium Enterprises from the Perspective of the Article
Small and medium enterprises around the world have various characteristics that set them apart from other forms of businesses. It is important to analyze how Kartiwi & MacGregor (2007) address this issue in their research. This is considering that a study in this field has to acknowledge these characteristics if its contribution to the field is to be taken seriously.
In addition to this, the paper needs to examine the problems that are associated with the barriers associated with SME operations in both developed and developing nations (as far as e-commerce is concerned). It is noted that various studies have made efforts to differentiate SMEs from other forms of enterprises using the characteristics of those SMEs as the yardstick. Based on the findings of other studies conducted in the past, a critical analysis was carried out by Kartiwi & MacGregor (2007) to identify the specific features that are perceived to be unique to SMEs in both developing and developed nations.
This was aimed at justifying the quantitative approach that the scholars were planning to adopt for the study. To this end, this author notes that the study by Kartiwi & MacGregor (2007) can be considered as a significant contribution to the field of SME and e-commerce in developed and developing nations in the world. This is given the fact that the two scholars explicitly acknowledge the variation between the experiences of SMEs and other forms of business enterprises as far as e-commerce is concerned based on the characteristics (Abdullah & Bakar, 2007).
From the findings of studies in this field, it is noted that features that are unique to SMEs can be classified into two broad categories. This is the approach scholars such as Abdullah & Bakar (2007) have adopted in examining SMEs in developed and developing nations in the world. The first category touches on the external features of the business enterprises while the other category looks at the internal features or traits. The latter revolves around management and operational issues including decision making and organizational structure.
This is for example those traits touching on who is managing the SMEs, the major stakeholders in SMEs, the profile of employees in the business as well as the strategies adopted to market the products of the company. The external features of SMEs include market components and the environment within which the business operates (Abdullah & Bakar, 2007).
Given the two categories of characteristics of SMEs from the perspective of various scholars in the field, this author is of the view that the use of quantitative research methodology by Kartiwi & MacGregor (2007) is not appropriate. At least, the scholars should have combined this methodology with a qualitative approach. It is not appropriate to subject both internal and external attributes of SMEs to one type of research methodology (in this case the quantitative research methodology). The importance contribution of a qualitative methodology is lost to the study as a result.
Research Questions
Research questions are the basis on which scholars and researchers address the research problem. In other words, it is noted that by formulating and answering the research questions, researchers would have inevitably addressed the objectives of the study (or the problem identified). When questions structured to address the research problem are not sufficient enough, the purpose of the research becomes ambiguous.
This happens when the conclusions are drawn fail to address the problems identified. The same case applies to a research hypothesis. The hypothesis needs to be based on sufficient evidence to address the identified variables put in place by the researcher (Glesne & Peshkin, 2008). To this end, it is noted that the research questions and the research hypothesis are related in more than one way. Both of them need to be stated clearly by the researcher at the beginning of the study.
With regard to the study conducted by Kartiwi & MacGregor (2007), this author will try to find out how well the research question and the research hypothesis were formulated. The author will also try to identify how well the research questions and the research hypothesis were addressed in the study.
The justification for a given hypothesis is to solve the research problem identified (Glesne & Peshkin 2008). In the hypothesis, the researcher tries to come up with a tentative statement that shows the likely relationship between two or more variables. The hypothesis may be based on a question which scrutinizes the research problem. The hypothesis statement tends to melt-down the problem into something that can be tested statistically. It can also be used to falsify the information handed in after the research.
This case applies to the study conducted by Kartiwi & MacGregor (2007) touching on the adoption of e-commerce among SMEs in developed and developing nations. In such a case, the researcher may contend that the performance of SMEs in most developing countries and some developed countries are declining due to their inability to incorporate e-commerce in their operations.
After the hypothesis is formulated, scientific methods are then used to make sure that the testable hypothesis revolving around the research problem formulated can either support or falsify the assumptions made (Glesne & Peshkin, 2008).
As already indicated, the study by Kartiwi & MacGregor (2007) focused on SMEs and was a comparison of SMEs in developed and developing countries. The comparison was meant to highlight the barriers that are supposed to be tackled. However it is noted that more than four years after the completion of the study, the barriers have not yet been tackled successfully. From this angle, this author can analyze the appropriateness (or lack of it thereof) of the study that was undertaken.
As a result of the methodology that was adopted to address the research questions, the authors were unable to make predictions on the barriers facing SMEs and adoption of e-commerce at the time. This is significant given that prediction is an important aspect of quantitative research. Given that it failed to predict, it can be concluded that the methodology adopted for study (quantitative) was therefore not reliable (Patton, 2009).
A researcher who becomes fixated on one particular methodology (in this case quantitative analysis) loses focus as a result of impartialities and other personal biases dominating the results. To avert such a scenario, researchers need to adopt an array of methodologies to find out whether they can arrive at the same results using different tactics.
In other words, the researcher can duplicate the study using a different methodology and compare the results. In this case, Kartiwi & MacGregor (2007) should have duplicated the study using a qualitative methodology and see whether the results obtained using the two methodologies are the same or not.
Given that the researchers failed to do this, it is little wonder then that SMEs are still stuck as far as adoption of e-commerce is concerned. No theory was formulated from the results that were made. It is noted that in an efficient quantitative research, a hypothesis needs to be drawn, tested and then theory drawn from it (Patton, 2009). But the quantitative analysis by the two researchers failed to achieve this. It just came up with half truths and half facts that did not lead to a solid solution.
A Critique of the Research Methodology Adopted
As already alluded to in this paper, the research methodology that was employed in the case study was insufficient simply because the researchers focused on one strategy which is quantitative research. A competent research that achieves the objectives set should ideally combine both quantitative and qualitative research methodologies (Creswell, 2007).
This is what most scholars call exploratory research where the researcher starts by exploring the environment within which the research is going to take place using qualitative research methods. The results acquired are then tested using quantitative data to identify the relationship between the various variables.
The first method is meant to gather qualitative data using various strategies such as interviewing retailers and other stakeholders involved in the SMEs business operations. Such questions are best structured from a qualitative point of view (Creswell, 2007). This is for example questions touching on the factors that discourage SMEs from adopting e-commerce in their business operations. Such questions are effectively tackled from a qualitative perspective. This is for example through the use of an interview schedule as opposed to a questionnaire.
A Critique of Data Collection Methods
In this section, this author is going to examine the various data collection methods vis-à-vis the method that was used by Kartiwi & MacGregor (2007). The effectiveness of these alternatives will be addressed in the context of the method used by the two scholars.
Quantitative research has been referred by many scholars as an anguish analysis which was used to solve barriers facing SMEs sectors. The data that was collected could have been enough to solve several major problems but the data analysis method that it was subjected to made it hard to address several issues adequately as a result of an “……….inadequate point of view” (Hadjimonolis, 2010: p. 22).
Before the study the features that were seen to hinder the incorporation of technology in the SME operations were analyzed. The data was then collected based on the assumption that the SMEs have common traits. Data collection can be conceptualized the process of gathering information in an attempt to have an in-depth knowledge of the variables identified. This is by using the standard and applicable methods to answer the research questions put in place to solve the research problem (Denzin & Lincoln, 2006).
Data collection is a process that is carried out in many fields ranging from social sciences to business studies. Methods used to collect the data vary from one field to the other but the results and ethics applied are meant to remain the same (Morse, 2008). As the author of this paper had stated earlier, the focus of this section is to critique the research methodology adopted in the study by Kartiwi & MacGregor (2007). The data collection method in this research will be critically analyzed.
Adoption of e-commerce by SMEs was the major concern of this quantitative research. It is noted that the study and the results were supposed to inform recommendations on the best strategies that SMEs can use to adopt e-commerce in the developing and developed countries. However, it is noted that the data collection method adopted in this study fails to bear fruits. The researchers made use of questionnaires to collect information from the respondents. The questionnaires were structured and addressed specific sections of the research problem.
This author argues for the adoption of both qualitative and quantitative data collection methods for a study of this magnitude. For instance both quantitative and qualitative data collection methods should be employed to come up with conclusions drawn from the research.
Integrity is paramount in any research as the results obtained are supposed to address both the research problem and the challenges faced on the ground. It does not matter whether the correct instrument was used to collect quantitative data or not but results from the method can be misleading.
Cases of misleading data are evident given that the other methods (apart from quantitative research methods) were not put into consideration. The collected data has various impacts on the research variables depending on the methods used. In this case, the quantitative data collection method adds a twist to the questions meant to solve the research problem (Templeton, 2011).
In most case studies, quantitative data collection methods tend to deviate from the questions that are supposed to address the research problem. For example the quantitative analysis in this article is not sufficient to address the questions touching on the barriers facing SMEs in both developed and developing nations. Inability to address the questions adequately has hindered SMEs from expanding rapidly enough due to misleading recommendations from studies conducted in the field.
Most of the questions were formulated based on information gathered from the literature in the field highlighting on the barriers facing SME ventures (Hall, 2012). This automatically misleads other researchers who embark on extensive investigations based on the literature and findings of other studies conducted in the past. The results tend to compromise the economic situation necessary for the efficient operation of the SMEs. Another method needs to be employed to investigate the real cause of the stagnant in the growth of SMEs especially in developing nations.
Inaccurate and unreliable data collection methods employed in quantitative research analysis has negative impacts on the economy and those who are taking part in the study itself. This is given the fact that if future researchers are compelled to replicate the same study, the results will vary significantly from those made in the initial study. This means that the quantitative data collection methods adopted by Kartiwi & MacGregor (2007) has possible errors ranging from inaccurate data collection to possible misleading conclusions drawn from the analysis of the results obtained (Hall, 2012).
This author is forced to go back and critically analyze the measures put in place to maintain integrity in data collection in this study. Accurate data collection methods ensure that the results will be accurate too. However this does not mean that during the study the measures put in place to ensure integrity was upheld were not given priority.
The fact is the data collected might have been analyzed in the wrong way. For example maybe other strategies should have been employed to collect the required data. Human thinking and psychological orientation are not well catered for when quantitative research is employed in isolation.
This is given that the quality of data collected may be compromised in one way or the other and that is why it is important to apply multiple data collection methods. This is to make sure that collection of data is perfected before proceeding to the next stage. Whether Kartiwi & MacGregor (2007) did this or not is debatable.
Most quantitative studies require large samples to improve the accuracy of the data collected. In this particular research, the two scholars are faced with logistical challenges inherent in the study design as far as gathering the large sample from the universe or study population is concerned. It is noted that historically such research methodologies have been known to bear fruits but with this particular study this author feels differently. The author notes that the continued use of such research designs in most academic fields today is degrading the quality of data obtained (Hadjimonolis, 2010).
With reference to the study by Kartiwi & MacGregor (2007), the SMEs can be regarded as illustrations of a situation where quantitative data collection and analysis was applied to make conclusions. But since then, no concrete solutions have been found for all businesses across the globe. This is as far as embracing e-commerce as a mode of transacting business is concerned.
The question is why is this so? What is the problem? The author of this paper argues that with the increased rate of innovations (driven by rapid increase in use of technology), the SMEs are supposed to embrace technology and take off as far as economic development is concerned. But this is not the case especially in the developing nations.
At this juncture, the reader will agree with this author that despite all the measures and mechanisms put in place to articulate the growth of SMEs, the research by Kartiwi and MacGregor failed to meet the criteria needed to address all the problems. The approach that was adopted to analyze the barriers faced was not adequate. This is because if the approach was adequate, the barriers would have faded away. This is as opposed to improving the growth of GDP in the respective nations.
It is to be noted that each country has a preferred set of research methods and data collection instruments. Instruments vary with research methods adopted. As a result of this, quantitative methods are not preferred in data collection in the developing countries (Hadjimonolis, 2010). This however may not be the case in the developed nations.
Appropriateness of Data Used in this Research
The author will now focus on the appropriateness of the data collected using the quantitative data collection methods. The main concern here is to critically analyze the intended consumer of the data and how it was made available to them. Another concern at this juncture is the criteria that were used to evaluate the data collected from the field. For instance, how many crises were resolved or addressed in attempts to tackle barriers facing the SMEs?
The approach was to find out the number of crises resolved and duration of time required to solve them at any given point. The compilation of data was to be based on the way the problem was tackled and not the measures needed to be put in place to catapult the business enterprise to success. The researcher also sought to analyze why a significant number of businesses are still stuck (Hadjimonolis, 2010).
The analysis of the barriers faced by SMEs needs to be carried out by assessing where the barriers originate from, methods that have been used to solve the problems at one point during the crisis and any challenges that were identified at the time. This gives an insight on why the data collected may not have been sufficient enough to come up with strategies to address the problems that are experienced by SMEs. The problems faced by the SMEs were dragging the GDP of most countries down.
Another inappropriate strategy that has undermined the quantitative analysis used is the amount of the data used or in other words the size of the sample used. The author of this paper concurs that this method requires large amounts of data and the larger the size of the sample the more accurate the results.
With reference to the research done by the two scholars, generalization of the findings will automatically lead to a generalization of the problem facing SMEs. With this in mind, it is noted that the specific problems will still dominate the SME sector and as a result drag the economy behind.
In the developed countries, the findings of the study may be generalized because SMEs in these countries are easily studied or analyzed and the problems identified are solved right away. But on the other hand, the barriers facing SMEs as far as embracing e-commerce is concerned are triggered by something else which needs to be analyzed using other research methods (Templeton, 2011). This will help solve the problems accordingly without necessarily making a comparison between the two economies.
It is noted that the SMEs in developed countries can greatly benefit if the research design incorporates other data collection and data analysis methods as opposed to quantitative analysis alone. This calls for variations in data collection methods which is paramount to make conclusions which may in turn solve the problem at hand with immediate effect.
Finally, it is to be noted that the data obtained in this research is entirely quantitative. It should be noted that qualitative data gives rise to refined results that are rich in information which is necessary to embark on actual implementation of strategies as opposed to the data obtained through quantitative means.
Description and Critical Analysis of Statistical Methods Used in the Study
Quantitative research methods assume that statistical analysis can only be carried out on quantitative data. This is wrong given that several statistical analyses can be carried out using qualitative data. For instance, the use of rating of scales acquired from qualitative data collection methods will lead to more accurate results as opposed to employing quantitative research methods throughout the study.
There is a difference between qualitative and quantitative data but both are used to address problems identified in the research. If both sets of data are compared in a given research, the comparison may help researchers who adopt scientific research designs come up with a solution to a particular problem (Creswell, 2009).
Another confusing aspect of quantitative researchers is the misconception that data acquired from quantitative data collection methods is to be solely used in quantitative research. This may be true but not in all cases. Some important information can be gleaned from qualitative research design which may require qualitative data. This is for example when trying to capture attitudes and opinions of stakeholders in the SME sector. Without this, the researchers may have wasted the opportunity to obtain significant information that would have adequately addressed the SME barriers.
Exploratory strategy is another approach that many successful studies have adopted and the results are recommendable. The process involves combination of both quantitative and qualitative research methodologies. The study begins by adopting the qualitative research design which tends to explore the potential problems in a given area. The researcher then embarks on quantitative research. Findings from quantitative research alone are not sufficient to make conclusions.
The author of this paper wonders how the information touching on the retailers was obtained. For example, the open ended questions (which were sufficiently structured in a qualitative format) for retailers and also for customers were aimed at getting an insight on most of the barriers facing SME operations. The findings from these questions have not been used anywhere given that the study is entirely based on the quantitative data collected.
Discussion of the Statistical Results
The results of this study need to be discussed and analyzed extensively so that the stakeholders can implement the necessary strategies. First, the problem with quantitative research is that it has no room for criticizing the findings made once the research is completed. This is the initial stage of criticizing research findings before any other researcher does so. The research design does not allow the scholars to make recommendations for further study.
It is expected that since this is a wide area, the quantitative research would make recommendations for the benefit of future studies in the field (Weaver, 2010). This has not been addressed at all in by Kartiwi and MacGregor in their study.
The analysis did not address the relevance of the topic to identify the barriers. This is given that the research was unable to come up with relevant data to tackle all the potential barriers making SMEs not to adopt e-commerce in their operations. The researcher briefly identified the purpose of the study well in advance. However, the research did not take into account the importance of reminding the reader about the questions that were supposed to be dealt with in the study.
The purpose of the study was to identify the barriers hindering SMEs from adopting e-commerce and making it part of their business operations. But the quantitative analysis that followed failed to revisit the problem. Instead the researchers seem to have shifted their focus and the reader is led to believe that the whole process is a move to break barriers in SMEs in general. This is as opposed to breaking barriers to help SMEs adopt the e-commerce strategy (Weaver, 2010).
The research further confuses the reader by failing to state whether the aim of the study was met as it was expected or whether the findings were surprising and not as per the initial expectations. The quantitative analysis adopts a complex format in presenting the findings to the targeted consumer. The reader or the target consumer of the study needs to understand the findings and graphical representations should be recorded to make it easier for the reader.
Many researchers contend that quantitative research is accurate but at to some extent it is misleading. This is especially so when it comes to the reporting and representation of statistical data. This is considering the way the researchers present the data in a confusing order. Normally, the reader would find it easy to comprehend the data if it is presented from the most important to the least important. In other words the quantitative analysis failed to present the findings based on the level of significance criteria.
This aspect enables the reader to understand statistical significance of the findings as well as the significance of the SMEs to the economy. This quantitative analysis seems to throw words here and there just because it is approaching the problems from a scientific point of view. The researchers seem to agree that they should not logically present information for the sake of the reader (Stouthamer & Bok, 2007).
The research by Kartiwi and MacGregor is not the first one in this field. Other studies have addressed similar problems in this field and that is the reason why a literature review needs to be carried out. After the review the researchers were required to formulate the hypothesis plausibly so that the results would automatically follow the format. This research has denied the reader the opportunity to see this. The idea is to allow the reader to have access to theoretical frameworks that have been used before digging for more information.
In most quantitative case studies, the researchers do not give the reader the chance to define and predict the identified questions. The study makes use of people on the ground and this limits the findings. This is given the fact that people tend to change and it is hard to predict their behavior. This compromises the data as well as the findings that are made. Quantitative methodologies used alone can never address this problem adequately. This is without the use of other methods to address the contradictions exhibited in negative and positive findings (Creswell, 2009).
Conclusions from the Quantitative Study
The conclusions made in this study are less descriptive. This being the case, they fail to provide the implications of the research itself. The author of this paper has stated earlier that the research by the two scholars does not proceed from the general to the specific. It does not add value to the knowledge that already exists in the field. For this reason, the SMEs are not aware of the next step to take to address the problems facing them.
The conclusions did speculate on the barriers that need to be addressed in an attempt to adapt e-commerce as an alternative strategy. But the end product is not clear enough to give the reader an idea on what the researchers are talking about. A take home message is supposed to be a lucid weighty phrase to convince the reader on the need for SMEs to adopt the e-commerce strategy (Hadjimonolis, 2010). However the researchers state that there are other studies which are in progress to address other limitations associated with SMEs and adoption of e-commerce.
A clear generalization of important findings has not been made. This was meant to go beyond the sample of used in the study and into the population. Research questions are related to the references made at the end of the study. This denied the researcher the opportunity to conclude and make substantial recommendations for current and future operations of SMEs.
Again the recommendations that were made applied to any SME striving to excel at any level with or without e-commerce. But there is need for further recommendations from a qualitative point of view to provide a deeper meaning to the research question.
References
Abdullah, A., & Bakar, A. (2007). Small and medium enterprises in Asian Pacific countries. Huntington: Nova Science Publishers.
Bajaj, K., & Nag, D. (2009). E-commerce: The cutting edge of business. New Delhi: Tata.
Blili, S., & Raymond, L. (2009). Information technology: Threats and opportunities for small medium-sized enterprises. International Journal of Information Management, 13(6), 439-448.
Creswell, J. R. (2007). Research design: Qualitative and quantitative approaches. Thousand Oaks, CA: Sage Publications.
Creswell, J. W. (2009). Research design: Qualitative, quantitative and mixed methods approach. California: SAGE Publications Inc.
Denzin, N., & Lincoln, Y. (2006). Handbook of qualitative research. Thousand Oaks, CA: Sage Publications.
Glesne, C., & Peshkin, A. (2008). Becoming qualitative researchers: An introduction. White Plains: Longman Publishing.
Hadjimonolis, A. (2010). Barriers to innovation for SMEs in a small less developed country (Cyprus). Technovation, 19(9), 561-570.
Hall, C. (2012). Profile of SMEs and SME issues- Singapore: Asia-Pacific Economies.
Kartiwi, M., & MacGregor, R. C. (2007). Electronic commerce adoption barriers in SMEs in developed and developing countries: A cross-country comparison. Journal of Electronic Commerce in Organizations, 5(3), 35-51. London: McGraw-Hill.
Morse, J. (2008). Critical issues in qualitative research methods. Thousand Oaks, CA: Sage Publications.
Patton, M. (2009). Qualitative evaluation and research methods. Thousand Oaks, CA: Sage Publications.
Stouthamer, L. M., & Bok, W. (2007). Data collection and management: A practical guide. Thousand Oaks, CA: Sage Publications.
Templeton, J. (2011). The focus group: A strategic guide to organizing, conducting, and analyzing the focus group interview. Chicago, IL: Probus Publications.
Weaver, W. (2010). The Delphi forecasting method. Phi Delta Kappan, 52 (5), 267-273.