Module 06 - Home Page

Assessing Causality, Standards for the Quality of Conclusions, Testing and/or Confirming Study Findings

 I.    Assessing Causality (A Discussion)

Throughout the course I have inferred that qualitative studies may not be used in a credible manner to prove causation.  In most cases this is true because qualitative research methods don't meet the usual stringent requirements of true experimental research.  However, qualitative research methods may be used to assess causality.  In many studies this is a necessity since one of the reasons for conducting qualitative research is to explain a phenomenon.  Just the act of explaining a phenomenon will lead to to a position where you may be required to assess causality.  What is the value of an explanation if we can't discuss causality?

When we explain a phenomenon we try to describe it in a manner which is intelligible.  We may explain something in various ways.  For instance, we can provide information or descriptions, justify an action or belief, provide a reason, support a claim, or make a causal statement.  We are not trying to provide a scientific explanation.  We are just trying to explain why things happened and provide support that may justify the explanation. 

 In qualitative research we develop information by using various methods of inquiry.  Information is developed as we ask questions, review data, confront key persons, and develop findings within a conceptual framework.

The problem that we have with assessing causality in qualitative research evolves around the information we receive from people (the study's subject). Each person has their own perception of what is occurring in the setting and why things have developed as they have over the years, months, or days. Furthermore, some people are more valuable than others because they have more relevant information and an institutional memory about an organization or case that other persons may lack.  Some individuals and can relate to the different constructs that compose the conceptual framework of the study much better than others.  Other people are more valuable to the researcher because they are very close to the activity related to the phenomena under study.  Still others maybe more valuable because they may be able to provide insight that supervisors or other key personnel are unable to provide.

Can you understand the problem confronting a researcher when an explanation of the phenomena and an assessment of causality is required for the proposed study. We need to know "How can we determine causality?  

One method is to examined the information and facts that could lead us to a conclusion of causation.  Factors associated with the study that can assist us in assessing causality are:

• Strength of association
• Consistency
• Specificity
• Temporality
• Plausibility
• Coherence
• Experiment

In any case, the causes of causality are many and the researcher must be able to process potentially complex causes.  Let's consider some of the causes of the particular event.

Causal Networks

One assumption we can make is that the causes of any particular event or phenomena are always multiple.  Furthermore, we can state with confidence that most causes are not only multiple but are also conjunctural. This means that the multiple causes of any event combine and affect each other, as well as, the "effects" or the phenomenon under study. Some researchers believe that the effects of multiple causes are not the same in all contacts, and different combinations of causes can turn out to have similar effects. According to the Solomon (1991), this means that researchers have to consider cause and effect in terms of a network.  

A causal network is a display of the most important independent and dependent variables in the study. The relationships among the variables are indicated with arrows. In a causal network diagram, the relationships plotted are directional. It allows the researcher to determine which factors exert an influence on other factors, which factors are stronger than others, and which factors are more central to the issue at hand.  For a causal network display to be useful it must be accompanied by an associated analytic text. 

The researcher is charged with assembling fragments of information in the form of field notes, summaries written by field researchers, pattern codes developed by the researchers and any other remarks or data developed in the course of the study.

Once the researcher has assembled these fragments of information, they must be connected by the researcher into a logical chain of evidence. The researcher is on the verge of assessing causality for a particular phenomena.

Pattern Codes

In a paragraph or two above, pattern codes were mentioned.  A pattern code or pattern coding, is a device for summarizing segments of information or data. Pattern coding is a way of grouping these summaries into a number of sets, themes, or constructs. The important functions of pattern coding are as follows:

• It reduces large amounts of data into smaller analytic units.
• It helps to move the researcher into analysis during data collection.  This is very important in most qualitative studies because it allows continuous evaluation of the case, and thereby, creating an opportunity for the researcher to change the direction of the inquiry when needed.  Therefore, the researcher has the ability to change the research question and research objectives as needed to support the research purpose.
• Pattern  coding also allows the researchers to gain a better understanding of local incidence and interactions between the study's constructs and other variables.
• For multiple case studies, a lays the foundation for cross-case analysis by combining common sets and/or constructs.

Qualitative research methods can be used effectively to assess causality.  It is an important research tool that can give a close-up look into a significant problem. It can identify mechanisms that may be responsible for and provide answers for establishing the root cause of a problem.

II.   Standards for the Quality of Conclusions

When a scientist is ahead of his times, it is often through misunderstanding of current, rather than intuition of future truth. In science there is never any error so gross that it won’t one day, from some perspective, appear prophetic.

Jean Rostand (1894–1977), French biologist, writer. Pensées d’un Biologiste (1939; repr. in The Substance of Man,"A Biologist’s Thoughts," ch. 7, 1962).
 

Standards for the Quality of Conclusions

One of the problems for establishing credibility for qualitative studies is its subjective nature.  Probably the most important figure that can influence a qualitative study is the researcher.  The question is "What are the standards for the quality of conclusions reached in qualitative studies?"  The following is a brief summary of standards used to assess the quality of a study's conclusions.

• Objectivity /Confirmability 

  This standard is concerned that the study's conclusions are dependent on the study's subject and the conditions of the inquiry rather than on the researcher.  This can be assessed, in part, by determining if the study's general methods and procedures were described explicitly and in detail; that we can determine the actual sequence of how dated were collected, processed and displayed for drawing a specific conclusion; that the conclusions are linked with the display data; that the researcher has left an audit trail; and that the researcher has been explicit and as self-aware as possible about personal feelings, values and biases that may affect the study.

   

• Reliability/Dependability/Auditability

   This standard is concerned with whether the process of the study is consistent and stable throughout the period the research was conducted. This standard can be assessed, in part, by determining if the research questions were clear; the researchers role and status was explicitly described; the analytic constructs were clearly specified and the data was collected across the full range of settings, times, and respondents as indicated by the research questions or objectives.

   

• Internal/Validity/Credibility/Authenticity

  This standard is concerned with truth and whether the account and descriptions in the study "ring true."  This standard also answers the question of a bias study. This standard can be assessed, in part, by determining if the study was comprehensive, all alternatives to possible conclusions were adequately discussed, and, among other questions, was negative evidence sought by the researchers.

   

• External Validity/Transferability

  This standard is concerned with the ability to generalize the study findings. This standard can be assessed, in part, by determining if the sample of persons, settings, and processes were described enough to permit comparisons with other samples.  A few significant questions that may be asked to assess this standard are:

  • Does the study examined possible threats to generalizability?
  • Is the sample adequate enough to encourage broader application of the study findings?
  • Does the study indicate or direct one to other settings where the findings may be tested?
  • Have the findings been replicated in other studies?
  • Are the findings consistent with the experience of others who have reviewed the study?

There are probably other standards that can be used to assess the quality and accuracy of qualitative studies; however, if a study can withstand the rigors required to meet the above the standards for the quality of its conclusions we can be reasonably assured that the study was sound and credible.

III.  Testing and/or Confirming Study Findings

What is the good of drawing conclusions from experience? I don’t deny we sometimes draw the right conclusions, but don’t we just as often draw the wrong ones?

G. C. Lichtenberg (1742–99), German physicist, philosopher. Aphorisms,"Notebook F," aph. 123 (written 1765–99; tr. by R. J. Hollingdale, 1990).
 

Testing and/or Confirming Study Findings

One of the most important tasks that should be considered for most qualitative studies is the examination or check of the subject study to confirm its findings.  Many of the methods used to confirm a study's findings have already been discussed in the sections above; however, this topic is important enough with qualitative studies to warrant a more detailed discussion.

The following is a brief summary of a few of the tactics or methods used to confirm a study's findings:

• All facets of the study should be checked for representativeness

            Quite often researchers will depend too much on elite informants and/or persons who make themselves readily accessible.  This can lead to a sampling of non-representative informants.  Just as important is that the researcher does not depend too much on generalizing from non-representative events or drawing inferences from non-representative processes.  It should be noted that although sampling techniques are not as exact and well defined as in quantitative research, the researcher must conduct the study in an unbiased manner and at all times seek to have a representative sample of the participants or informants.

• Triangulating

            Was there an effort to on the part of the researcher to support a finding by showing that independent measures of it agree with it and do not contradict it?

• Weighing the Evidence

          Was the data collected scene or reported first hand, from field workers or informants who were trusted, collected in an informal setting and when the researcher was alone with the field worker or informants? 

• The Meaning of Outliers and Extreme Cases

Outliers and extreme cases can be used in verifying and confirming conclusions.   The reason is that the researcher can use a careful inquiry into determining the cause and/or reason for the existence of the outliers and extreme cases.  While gathering data on the outliers and extreme cases the researcher has an opportunity to gather more information about the situations that are central to the study's research question and objectives.

As you can imagine from this discussion, qualitative studies are not as simple as many people think.  To develop and conduct a very credible, unbiased, and insightful study requires that the researcher possess outstanding organizational skills, as well as, communication and people skills.  It is also important that the researcher has a clear understanding of their own biases and values and they have taken the proper precautions to ensure that their personal feelings do not affect a study's outcome.  After all if someone wants to discredit a qualitative research the weakest link is always the RESEARCHER.