Foundations of Knowledge

FOUNDATIONS OF KNOWLEDGE
BECOMING A CRITICAL READER OF QUANTITATIVE RESEARCH

RE-SEARCH
methodical investigation to seek answers that involve explanation and understanding Positivism – falsification and replication all research has flaws limited resources & ‘the least worst option’ is research ‘fit for purpose?’ research is presented as if ‘fit for purpose’ facts, findings and critical reading

Tim Hartford – but there are more Ben Goldacre (Guardian), Seife (Proofiness)

WHY BE A CRITICAL READER?
 The seduction & authority of numbers  “Proofiness” (Siefe 2010)
     the dark arts of mathematical deception Potemkin numbers dis-estimation fruit packing & cherry picking

WHY BE A CRITICAL READER?
numbers can clarify but also confuse or misdirect?
Examples 1. Fish oil mothers depression and child intelligence
2. School-age drinking and social networking sites http://www.straightstatistics.org/article/seeing-doubleover-school-age-drinking-wales http://www.straightstatistics.org/article/fishing-significance

3. Improvements in re-offending

http://www.straightstatistics.org/article/bent-statisticsgoing-straight

? Some possibly deliberately falsify?

e.g. Sir Cyril Burt’s study of intelligence in twins & 2 tiered education system & 11+

BE A CRITICAL READER…..
Look for potential flaws in the statistical analysis

But do not assume that all studies are equal (research design)
   Randomised control trails (experiments) Longitudinal studies One-time cross sectional surveys

e.g. Fish oil mothers depression and child intelligence http://www.straightstatistics.org/article/fishingsignificance

Several reports * The Times. The Daily Telegraph, the Guardian, the Daily Mail, The Sun, the Independent and the Daily Mirror • Lancet published 2007 • eat less than 340g fish a week & children in lowest quartile One report only in British Newspapers (daily telegraph) * randomised control trail *Journal of the American Medical Association, October 2010

CRITICAL READING (NOT JUST BEING NEGATIVE)
What are the arguments? Where do they come from? What data have they got to support them?  plus data transparency
What is the data? Is it fit for purpose? i.e. measured well? How was the data collected? i.e. who is it from (sample) What is the research design & analysis – it is reported in detail? Can you evaluate it? • What is not reported? • • • •
Note: You will need to demonstrate transparency in your dissertation studies – so start practicing and evaluating what others studies do.

QUANTITATIVE METHODS
Usually associated with philosophy of positivism Associated with ‘survey’, standardised information, & large scale (sample)

OFFICE OF NATIONAL STATISTICS (ONS) SURVEYS

PHILOSOPHY-OF-SCIENCE ISSUES
• Positivism
– There is a world ‘out there’ that exists prior to, and independent of the research and the researcher – It is possible to discover, know or find out something about this world (through certain types of research practice) – It is possible to discover causal relationships in social phenomena – Theories about the world must be tested using evidence in order for them to be accepted or trusted

CAN RESEARCHERS BE ‘NEUTRAL’?
• to be as objective and value-free as possible. • to find out how the world really is, value judgements about whether these findings are good or bad is for others (you) to debate and decide • researchers need to ensure their research has
– Validity – we need to be as sure as possible that a research method actually measures (without bias) what it claims to measure – Reliability – we need to ensure and check that results are stable and consistent

QUANTITATIVE METHODS
Descriptive or Explanatory Sometimes exploratory

Expressed numerically & analysed statistically

BUT - Is a number always a number?

COUNTING NUMERICALLY
• How old are you? 18-24 years 25-30years 31-35 years 36-40 years 41-50 years 51-60 years
 How many cigarettes

did you smoke yesterday?
    

None 1-4 cigarettes 5-10 cigarettes 11-20 cigarettes 20+ cigarettes

COUNTING NUMERICALLY
• How old are you? 28 (please specify)
 How many cigarettes did you

smoke yesterday? (please specify)

4

DATA
Facts and figures collected, statistically manipulated and reported Also known as ‘variables’ or ‘questions in survey’ or ‘observation’

Cross sectional or time series
Statistically manipulated to produce the research findings

DATA
A critical reader needs to evaluate the data that has been collected The GIGO problem with quant methods

DATA
Garbage in – garbage out
The key is Measurement and Research Design
• Research design begins (and ends) with the research questions • RQ’s drive the measurement of data, data collection and data analysis • even if the question in the questionnaire is measured well poor research design will result in poor, inaccurate or off the wall data

SOME QUESTIONS?
 What is the average cost of a wedding?  Is marriage getting more popular?  Where do people celebrate their marriages?  What do guests like about weddings?  Why do married men get paid 20% more than single men?  How many people watched the royal wedding?

BUT WHAT HAS BEEN MEASURED?
Measurement and content validity Measuring how many people watched the royal wedding? The marriage premium & proxy variables Is wages a good way to measure productivity? Sampling and Zombie statistics

DATA – LEVEL OF MEASUREMENT
• • • • How to know which statistical tools can be used? Levels of measurement Classifies the character of data Four possible levels of measurement
• • • • Nominal Ordinal Internal Ratio

NOMINAL (OR CATEGORICAL) DATA
has values which have no numerical value classify data into categories this process involves labelling categories and then counting frequencies of occurrence there is no order or sequence in the values of nominal variables values must be mutually exclusive can be dichotomous or have several values
• Gender, occupation

ORDINAL DATA
• values whose order is significant, but on which no meaningful arithmetic-like operations can be performed.
• greatly dislike > moderately dislike, but • indifferent / moderately like = ? • quite useful for subjective assessment of 'quality’ and ‘preferences’

INTERVAL DATA
• An ordinal variable with the additional property that the magnitudes of the differences between two values are meaningful. • Thus the order of data is known as well as the precise numeric distance between data points
• Analyze the actual percentage scores of the essays (assuming they are given by the instructor). • Time 8 PM > 6AM but 10 PM * 2 hrs = ?

RATIO DATA
• A variable with the features of interval variable and, additionally, whose any two values have meaningful ratio, making the operations of multiplication and division meaningful.

Type of Variable

Is there a true zero point?

Are distances between categories equal?

Can the categories be ranked or ordered?

Ratio
Interval

Yes
No

Yes
Yes

Yes
Yes

Ordinal
Nominal

No
No

No
No

Yes
No

Summary –we will come back to this levels of measurement determine appropriate analysis

increasing level of sophistication

Discrete (Non-metric) Non-parametric statistics small sample sizes large sample sizes

Continuous (Metric) Parametric statistics

non-numerical/ categorical non-numerical no rank order mutually exclusive no equal intervals ordered data no fixed zero no equal intervals

Nominal

Ordinal

Interval numerical & ordered data no fixed zero point equal intervals

Ratio

numerical & ordered data fixed & known zero point

IF YOU ENJOYED THIS, TRY THESE…
BBC Radio 4 programme “More or Less” http://www.bbc.co.uk/programmes/b006qshd Royal Statistics Society videos http://scijourntraining.wordpress.com/2011/07/11/behind -the-numbers-video/

XKCD - A webcomic of romance,sarcasm, math, and language http://xkcd.com/