Case Study Notes

A case study is an empirical enquiry that investigates a contemporary phenomenon in depth and within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident. The case study inquiry copes with the technically distinctive situation in which there will be many more variables of interest than data points, and as one result relies on multiple sources of evidence, with data needing to converge in a triangulating fashion, and as another result benefits from the prior development of theoretical propositions to guide data collection and analysis.

Performing a case study entails:

Literature Review Posing research questions formal and explicit procedures Protecting against threats to validity Maintaining “chain of evidence” Investigating and testing rival explanations

In advance of doing a case study:

Define the “case” being studied Determine relevant data to be collected What to do with data once collected

Decide: Exploratory, Descriptive or Explanatory research purposes?

How to choose a research method given a research question type:

How, Why: Experiment iff controllable events, else history or case study Who, What, Where, How Many, How Much: Survey, Archival Analysis

Case study applicability: “How” or “Why” questions, in-depth explanation of social phenomena, contemporary events, little or no control over events.
Well-formed research questions = Who, What, When, Where, Why, How
Choose research method based on RQ’s, then reformulate RQ’s to match the research method.
Chapter 2: Research Design

Components of research design:

Study’s questions Propositions Units of analysis Logic linking data to propositions Criteria for interpreting the findings

Research Design Quality (items covered in detail below)

Construct Validity Correct operational measure for concept being studied Use multiple sources of evidence Establish chain of evidence Key informants review case study report drafts Internal Validity Causal relationships established Pattern matching Explanation building Address rival explanations Logic models External Validity Generalizing to what domain Theory and replication logic Literal replication across cases = same procedure -> same results Theoretical replication across cases = different procedure -> predictably different results Reliability Reproducible Case study protocol Case study database

Keep in mind: Multiple cases != sampling. Different logical basis for generalization.
Chapter 3

A good case study investigator:

Ask good questions Good listener Adaptive and flexible Firm grasp of issues being studied Unbiased by preconceived notions

Human Subjects Protection:

Informed consent: Informed as to nature of study, formal record of volunteerism Protection from harm, avoidance of deception Protection of privacy and confidentiality Protection of especially vulnerable groups

As pretraining, review:

Why the study is being done What evidence is being sought What variations can be anticipated What would constitute supportive or contrary evidence for any proposition

Case study protocol: Per case instructions for the case study investigator.

Project goals, case study issues, readings Field procedures: Access & contact info for case site, procedural reminders Sufficient resources when in the field Procedure for calling for assistance Clear schedule of data collection activities Provisions for changes in mood and motivation of investigator Case study questions to keep investigator on track, linked to potential sources of information Levels of question targets: Specific interviewees Individual case bibliography, methodology.

After data collection, write descriptive data section (even before analysis).

Identify case study subjects unless ethical concerns preclude it. Anonymizing is lots of work.

Have informants review drafts.

Exemplary case studies:

Individual cases are of general public interest Underlying issues are important Case study is complete. Boundaries between phenomenon and context are clear. Investigator pursued all evidence, especially that would support rival explanations. Case study not cut short by resource or other constraints. Consideration of alternative perspectives Display sufficient evidence so that reader can come to own conclusions just from the evidence. Composed in an engaging manner.

Overall Impressions

Yin spends a lot of effort justifying the validity of case studies, but I think it really comes down to the research questions. Case studies can help answer “how” or “why” questions but I don’t trust them for “whether” or other types of questions. Case studies seem to be used in policy to evaluate programs and decisions. I don’t think this is valid. I think more solid empirical methods are needed to evaluate whether a program is achieving its goals. Case studies should be used to understand how and why (or why not). Case studies are a lot of work. Yin says this a few times, actually, claiming that case studies are the most difficult of the research methods I’m not sure how well my intended topic matches the case study method. Several of my questions seem better addressed by surveys. The book was remarkably dry. Unnecessarily so in my opinion. It’s an instructional tutorial, and I don’t see why it needs to be written in a formal academic style. I want to talk to someone who is good at case studies to find out what transferable skills I can expect to learn by pouring myself into this. The work/reward ratio for this thing seems pretty bad to me. I need to find some good extrinsic motivation.

Opportunities for Case Studies

Reading the book made me think of several possible interesting case study topics, one not software engineering.

The decision by the Athletic Centre to institute women’s only hours. The decision in IBM’s Toronto Lab to split the DB2 development team into New Development and Continuing Engineering teams. More generally, case studies of how real engineers made decisions on what components, vendors, libraries and languages to use for projects.

My New To-Do List

This book dramatically expanded my known to-do list for my thesis. In currently-planned order:

Finish preliminary literature review reading. Write literature review. Rewrite research questions to be well-formed. Prune research questions that are not suitable for a case study. Write an short explanation of the case study research method and a defense of its validity, targeted at a software engineering researcher. That is, rewrite the above summary in a way suitable for inclusion in my thesis. Show the subset of the total case study method I intend to use for my own study. Justify. Write full case study design: Research questions Propositions Rival propositions Case study protocol (dependent on sources of evidence). Description of sources of evidence I intend to collect Description of analysis I intend to perform Chain of evidence for analysis and sources of evidence In parallel with (7), list 20-30 candidate cases. Solicit their interest in participation, filter as needed. I doubt I’ll have a lot of interest; I bet my case selection will be driven by whomever agrees to participate. Some ideal candidates off the top of my head: Unspace Shopify Mozilla Freshbooks Rypple Refresh Microsoft Google Red Hat EMC Big five banks Basie RIM Learnhub Seek preliminary IRB for aforementioned design (or real IRB if skipping 10-12) Lightweight preliminary pilot Revise protocol and design Full IRB for revised design Collect data Analyse data Revise and update literature review Edit