Referencing and Research Credibility

There is a general saying that no man is an island of knowledge. Why this saying? The whole of a research work can’t be carried out without making reference to external resources and other researchers’ works. This makes a research credible and an empirical one, because proven ideas would have been garnered from scholars in that particular subject area being researched.

Another important point to note is that, references (and citations) reflect the researcher’s skills and competence in carrying out research. Any form of bias is really reduced on the part of the researcher when other works are referenced.

Other points may include:

• Referencing would allow people who view the research work to verify sources and filter errors.
• The honesty of the researcher is made evident
• It will allow for the outcome of the research to be tested against credibility.

References

Reasons for Referencing. (n.d). Retrieved November 26, 2010, from the Hawker College website: http://docs.google.com/viewer?a=v&q=cache:HDUAS0-saH4J:www.hawkerc.act.edu.au/__data/assets/pdf_file/0004/62464/3_REASONS_FOR_REFERENCING.pdf+referencing+and+credibility&hl=en&gl=uk&pid=bl&srcid=ADGEESh6bWgU4hOEDuSbjkLmNlIU5w2MJMfQ11_ocGk3TLLB99P3Qe6n3DUbVDa9xrHxDUXy_mGCzu-9b0ee9u20QeOM2sEGKj0ExPn1sJsmDHLRrKFQUxd1j2RJjhmYILGst9FyI5oW&sig=AHIEtbTPoIHJbPdyhs-qGtLaSaVp5BmAgw
 
Harzing, A. (2002). Are Our Referencing Errors Undermining Our Scholarship and Credibility? The Case of Expatriate Failure Rates. Retrieved November 22, 2010, from http://business.nmsu.edu/~mhyman/M610_Articles/Harzing_Journal_of_Organizational_Behavior_2002.pdf

 

Video: Credibility Issues in Qualitative Research

After having a discussion with another student today, I have done some browsing to find a video that supports what we have laid out in this blog.

The video I have found supports many of our ideas, but also explores some additional areas of interest that this blog does not cover. For example, Negative Cases. I personally like the opening quote - "...you need to have a defence against a reader who asks, "Why should I believe you?"..." (Robson, 2007).

Credibility Issues in Qualitative Research - YouTube

Limitations of Focus Groups

There have been good points made earlier on focus groups. The following potential limitations should also be noted as well:

• There can be some dominant members in the group whose views may cover that of the rest.
• It will require expertise and competence to do the analysis of responses from a focus group.
• Adopting a focus group may limit the number of people to carry out the research on.
• There may be no separate view; one view usually depends on another in the guise to avoid confronting other members.
• It will take a trained and capable moderator to facilitate a focus group.
• The researcher's bias in analysing viewpoints cannot be overlooked.

All of these points can influence the credibility of research results.


References
Focus Groups - How to Run Them. Retrieved November 22, 2010 from http://www.webcredible.co.uk/user-friendly-resources/web-usability/focus-groups.shtml
Focus Groups - Advantages and Limitations. Retrieved November 22, 2010 from http://www.frontend.com/design/focus-groups-advantages-limitations.html

Research Question

When analysing a particular subject the research question also needs to be correctly specified for the research to be credible. If the research deviates from the original question then that research is not going to be relevant to the point.

Several aspects need to be considered when approaching and deciding on a suitable research question, the purpose of the research needs to be determined. Why is the research being done? What is the research trying to achieve? Without knowing why the research is being done or what the research is trying to achieve then the research cannot be correctly completed to compliment the research question. If the results of the research are completely irrelevant to the point of the study then it can be stated that the research completed is not credible for the point it was intended for.

Research needs to be framed correctly depending on what it is that is being researched. Each study is unique from the next, so when deciding whether to base research to return quantitative or qualititative data would depend on the subject at hand. Some studies will benefit from a well prepared fixed research that will return statistical data that can be easily interpreted and discussed. But for other studies a more flexible approach may be more suitable to better prove the point of the study.

Web Research: Who can we trust?

As discussed previously in a previous blog post “Web Research and Focus Groups” on 17th Nov, research is done in many forms - but noticeably over the past few years, web research has risen sharply. This can be attributed to the relative ease and speed of using the internet as a tool for research. This does not limit to just using the internet for primary, but using the internet to find sources of information for secondary research.

Primary Research

Businesses have been extracting information from users over the internet for many years (Amazon being a prime example) but more relevant to research specifically - sites such as Toluna, MyVoice and YouGov.

The given examples show two “paid opinion surveys” and the UK Government “opinion portal” which is also paid. The first two are likely have participants swayed by the prospect of being paid, whereas the latter has the same incentives and risks but may be more likely to draw out real issues of the people.

The issue with this is that the information gathered might be a half-truth from the majority of participants, so whilst still proving useful to companies it is not safe to base academic research on any information from these sites.

Secondary Research

To find information on the internet that is reliable is extremely hard to do from standard search engines such as Google or Bing. Instead alternatives such as Google Scholar and CiteULike are available that search academic sites, but for standard research Wikipedia is the most known for trying to alter the unspecific and incorrect information on the internet, requiring sources and correct documentation.

Stanford University have begun the Web Credibility Project which poses the following questions:

- What causes people to believe (or not believe) what they find on the Web?

- What strategies do users employ in evaluating the credibility of online sources?

- What contextual and design factors influence these assessments and strategies?

- How and why are credibility evaluation processes on the Web different from those made in face-to-face human interaction, or in other offline contexts?

There is more reading available to these questions and theories on how to solve the problems. However, in my eyes I believe there will be no way to decipher valid research on the internet as it is too vast. Much alike I discussed in my first post, it is again up to the researchers judgement to put the information into context; is this for good or bad?

Resources

Stanford Web Credibility Project - Stanford University - http://credibility.stanford.edu/research.html

Metzger, M. (2007). Making sense of credibility on the web: Models for evaluating online information and recommendations for future research. Journal of the American Society for Information Science and Technology.

http://onlinelibrary.wiley.com/doi/10.1002/asi.20672/full

Flanagin, A. J., & Metzger, M. J. (2007). The role of site features, user attributes, and information verification behaviors on the perceived credibility of Web-based information. New Media & Society.

http://nms.sagepub.com/content/9/2/319.full.pdf+html

Research Credibility

There are many factors:

- Range and Variance of participants

- Background of individual eg. social status, marital status, wealth

- Target Audience

- Inducement

- Age of Data

- Researchers personal position and status (professional or academic) resulting in Bias

- Social view on topic.

These influence:

- Relevance

- Quality

- Accuracy

- Completeness.

Case Study Example of Research Credibility

Currently I have been examining my case study, which is looking at customer loyalty, online and in-store shopping using primary research taken from a survey and personal evaluation.

Whilst the survey appears to have sensible measurable questions, I believe the credibility to be weak as it draws all 80 participants from University academic staff or students in the Computing department or on a Computing-related course; so whilst the information from that pool might be correct, it does not accurately reflect a wide variety of people and is focusing on a specific target audience only.

The personal evaluation also draws cause for concern as it is the researchers own personal opinion of how well the site performs; even though they have graded it against criteria, surely what their deems as easy or hard to do on the site is pre-determined by their own experiences with such sites - and considering it is their project they possibly have much more knowledge than an average user.

Application of Case Study Example

The main research contains issues regarding how it is gathered, namely the sources from which it is gathered. From this one might suggest that the researcher needs to widen their participant field in order to obtain more applicable data to the real world and create an accurate measurement. I agree that this would need to be done, but I also believe that as mentioned in a previous blog post on the 16th November, that they need to consider alternative ways to gather the information required. For example, focus groups of various target audiences would provide a broader spectrum to examine, and combined with survey results would provide a much more complete grounding for analysis.

The latter of the above example supports the viewpoint of my colleague made in a previous blog post on the 11th November about research credibility being subjective and how the context of the findings can be twisted to a personal viewpoint – a viewpoint which may have been set before the project was in motion. Taking this a step further, surely all research will have a level of bias towards the researchers individual beliefs and aims; because as I learnt today in a Intermediate Systems Design and Analysis lecture, humans must pass their own judgement to provide context on data. Judgement will never be unbiased.

Triangulation

The Credibility of research can be strengthened using a number of methods.  Shipman(1988)
recommends that a researcher should go beyond the traditional concerns for reliability, validity and generalizability when considering the trustworthiness of research.  He also suggests that asking the question “Is there sufficient detail on the way the evidence is produced for the credibility of the research to be assessed?”

In Real World Research Robson(Robson, 2002) also suggests  that there are strategies for dealing with threats to validity, a key mechanism used to support the credibility of research.  Triangulation of data involves the collection of material from more than one source.  Another approach could be the combining of quantitative and qualitative data to support a hypothesis.  The use of triangulation in this way can help to reduce the threat to the validity of research.

The reliability and testability of research also improves the credibility of the findings.  For a project to be testable, rigorous record keeping and fact reporting must take place so that the exact environment can be recreated for the purpose of testing out the theory.  Any failure in this method will result in the loss of credibility to the researcher and any of the facts within the report will be questionable.

ROBSON, C. (2002) Real world research : a resource for social scientists and practitioner-researchers,Oxford, Blackwell.

SHIPMAN, M. D. (1988) The limitations of social research, London, Longman.

http://www.enquirylearning.net/ELU/Issues/Research/Res1Ch3.html