Pyb210 Week 3 Research Design Assignment
Research Design Assignment
Jack Jamieson
N8851093
PYB210
Research Design
For a potential researcher, choosing the correct research design is essential in the battle to control variables. Because without controlling for variables, a researcher is not able to make inferences about cause/effect relationships. Correlation does not imply causation is the all-important rule that any psychology undergraduate would be able to tell you. However, some research designs can be used in the absence of some variable control measures and still implying correlational relationships.
The research design used in this situation is a Quasi-experimental design due to the distinctive lack of randomisation in the different groups of the experiment. Quasi-experimental …show more content…
research shares many of the characteristics of true experimental research but lacks the randomisation of sampling, this means that the researcher cannot be completely certain about the equivalence of the groups. This non-equivalence of the groups will eventually cause problems with the reliability of the dependent variable (Christensen, Johnson, & Turner, 2014, p. 272).
The non-equivalence of the groups is due to the type of sampling used; convenience sampling. Convenience sampling is a form of non-random sampling that involves the researcher choosing research participants based on their accessibility to the researcher rather than their representativeness within the target population. Convenience sampling does not condemn a study though, Wilkinson (1999, p. 595) noted that a convenience sampling study may be strengthened by “explicit comparison of sample characteristics with a defined population”. As the researcher is studying the ability of the young and old on a computer-based interface it is important to remember the distinct difference between the two groups in this specific task-set.
Independent and Dependent Variables
In any study, it is important to identify the variables that are being manipulated to understand that the study is focusing on the correct aspect of the relationship. In this experiment the variable that is being manipulated, also known as the independent variable, is the age of the participants in the study. Once identifying the independent variable we must now figure out the operationalisation of said variable, which in this case is done through the application of the computer-based interfaces of three varieties. The testing method of the three different interfaces (text-based, icon-based, and text/icon-based) is the means by which we test the independent and dependent variable.
The variable that is being tested in this study, also known as the dependent variable, is the time taken and errors made on the three tests being used.
The differences between the scores of the young group vs the old group is how the researcher will draw conclusions about causal relationship of the independent and dependent variables.
Strengths and Weaknesses of the Measurement of Dependent Variable
The strength of the variables used are mainly dependent on the testing procedures that are used to test these variables. In this case, the independent variable is tested in two separate conditions, once with the younger groups and once with the older groups. This means that the testing itself between the two groups should be the same. However, the fact is that the young and old groups have been again split into three separate types of tasks (word-processing, spreadsheet, and database tasks) and tested with three conditions within these groups.
The splitting of these groups into the different types of computer-based tasks allows for more detail into whether the nature of the task is indicative of the difficulty of the task.
Conversely, each of the groups taking the task are assigned in three separate conditions (text-based, icon-based, and text/icon-based) meaning that there may be an issue of internal reliability within the testing measures.
Due to the test being taken in three different ways by each participant, it raises the question of whether the participant would learn how to take the testing procedure more effectively throughout the course of the measurement and effect results through their memory of the testing. The researcher arranged the tests text-based first, followed by text/icon-based and lastly icon-based. This arrangement of the tests is so that the participants understand the icon’s function because they have already been exposed to the text options. Though it may be that this actually threatens the reliability of the dependent variable. This ‘memory effect’ would also have to be accounted for when correlating the scores of each participant over the course of the three conditions of their testing. The reason this problem is such a threat to the internal reliability of the test is there is no way to accurately measure the amount that each participant will improve through test-retest reliability correlations.
Another weakness of the measurement of variables that is present within this study is the conditions of the testing procedure itself. The 75 students were testing by the researcher in groups at a university and the 75 aged-care participants were tested individually at the aged-care facility. This presents a problem with the internal reliability of the results because the groups were tested under different conditions. This difference in testing would also effect the results of the dependent
variable.
Extraneous Variables
Extraneous variables are simply variables that influence the dependent variable without being manipulated by the researcher. It is extremely important for a researcher to identify and define any extraneous variables within a study because you may only conclude that the independent variable caused the difference in the dependent variable if no other variables differed across the groups (Harrington, 2011).
The first extraneous variable that must be examined is the difference in ability to complete specific computer-based tasks. As each person may have differing levels of computer skills specific to each task, three different tasks have been used to control against participant’s previous computing skill-sets.
The other obvious extraneous variable that must be taken into account in this research is the participant’s current ability to complete general computer-based tasks. Research done at Eindhoven University of Technology (Rama, Ridder, & Bouma, 2001) explains that a generational gulf exists in general familiarity with software-style interfaces. This gulf points toward an overall dominance of younger generations having more generalised experience with such tasks. This predisposition toward a difference in the two age group’s ability to complete computer-based tasks creates another aspect that would influence the dependent variable.
Controlling Extraneous Variables
As mentioned previously, the extraneous variable of pre-existing computing skill-sets has been combatted by the use of three different computing tasks.
However, another large extraneous variable has been neglected and would most certainly affect the dependent variable. The general generational differences in familiarity with computing tasks would give the younger generation an advantage and jeopardises the overall standardisation of the study. Due to the participants being chosen by non-randomised methods, there is no way to control against having distinct outliers in either group effect the overall dependent variable correlation.
Weaknesses in the Research Design/ Internal Validity
The research study has a number of problems that all based around the internal validity of the results. Firstly, the non-randomised selection of participants means it is harder for the researcher to infer a causal relationship between age (IV) and the use of icons, text and text/icons without controlling for a number of variables.
Secondly, because the tests are being repeated three times with the different conditions (icon, text, and text/icon) it raises the problem of the ‘memory effect’ playing a part in the results of each participant gradually getting better over the course of the testing, effecting results.
The test themselves also pose a threat to the reliability of results because they were performed in different settings between the two groups.
And finally, the research evidence suggests that the younger age group would have a natural advantage in the tasks due to a generational experience gulf in computing tasks between age groups.
These are all issues with the consistency of the measurement always occurring in the same way and testing the same thing each time. Due to these issues with internal reliability, the study would not be deemed valid.
Possible Improvements to the Study
As mentioned above, the issues with the study all relate back to the internal validity of the measurement procedure. So in order for the study to be deemed reliable and by extension valid, the researcher would need to change a number of the measurement procedures to be more standardised across groups.
Firstly, random sampling from populations of the target age groups would provide a much more representative sample because each person has equal probability to be chosen (Christensen, Johnson, & Turner, p. 145).
Next, the repeated tests leaves open the possibility of the participant learning how to complete the task better each time, and could have been controlled by slightly changing the test’s layout when testing each of the three conditions (text, icon and text/icon).
As mentioned previously, the measurement procedures themselves absolutely must be standardised to be able to draw conclusions from results (Holmes, McDonald, Jones, Ozdemir, & Graham, 2010). So the testing procedure must be changed in order for all participants to be tested under the same conditions.
And finally, the researcher has also neglected to factor in the large gulf in the differences in ability to complete computing tasks between generations. This factor would have to be analysed with regard to current research highlighting younger generation’s heightened ability and controlled through in-depth statistical analysis which cannot be used unless the samples are randomised.
If these problems with the study were rectified, then the researcher would be able to infer a causal relationship between the independent variable and dependent variable of age and ability to complete computing tasks effectively.
Works Cited
Christensen, Johnson, & Turner. (2014). Research Methods, Design and Analysis. Sydney: Pearson.
Harrington, M. (2011). The Design of Experiments in Neuroscience. California: Sage Publications.
Holmes, C., McDonald, F., Jones, M., Ozdemir, V., & Graham, J. E. (2010). Standardization and Omics Science: Technical and Social Dimensions Are Inseparable and Demand Symmetrical Study. OMICS : a Journal of Integrative Biology, 327-332.
Rama, M. D., Ridder, H. d., & Bouma, H. (2001). Technology generation and Age in using layered user interfaces. Gerontechnology, 34.
Wilkinson, L. (1999). Statistical Methods in Psychology Journals: Guidelines and Explanations. American Psychologist, 595.
Jack Jamieson
N8851093
PYB210
Research Design
For a potential researcher, choosing the correct research design is essential in the battle to control variables. Because without controlling for variables, a researcher is not able to make inferences about cause/effect relationships. Correlation does not imply causation is the all-important rule that any psychology undergraduate would be able to tell you. However, some research designs can be used in the absence of some variable control measures and still implying correlational relationships.
The research design used in this situation is a Quasi-experimental design due to the distinctive lack of randomisation in the different groups of the experiment. Quasi-experimental …show more content…
research shares many of the characteristics of true experimental research but lacks the randomisation of sampling, this means that the researcher cannot be completely certain about the equivalence of the groups. This non-equivalence of the groups will eventually cause problems with the reliability of the dependent variable (Christensen, Johnson, & Turner, 2014, p. 272).
The non-equivalence of the groups is due to the type of sampling used; convenience sampling. Convenience sampling is a form of non-random sampling that involves the researcher choosing research participants based on their accessibility to the researcher rather than their representativeness within the target population. Convenience sampling does not condemn a study though, Wilkinson (1999, p. 595) noted that a convenience sampling study may be strengthened by “explicit comparison of sample characteristics with a defined population”. As the researcher is studying the ability of the young and old on a computer-based interface it is important to remember the distinct difference between the two groups in this specific task-set.
Independent and Dependent Variables
In any study, it is important to identify the variables that are being manipulated to understand that the study is focusing on the correct aspect of the relationship. In this experiment the variable that is being manipulated, also known as the independent variable, is the age of the participants in the study. Once identifying the independent variable we must now figure out the operationalisation of said variable, which in this case is done through the application of the computer-based interfaces of three varieties. The testing method of the three different interfaces (text-based, icon-based, and text/icon-based) is the means by which we test the independent and dependent variable.
The variable that is being tested in this study, also known as the dependent variable, is the time taken and errors made on the three tests being used.
The differences between the scores of the young group vs the old group is how the researcher will draw conclusions about causal relationship of the independent and dependent variables.
Strengths and Weaknesses of the Measurement of Dependent Variable
The strength of the variables used are mainly dependent on the testing procedures that are used to test these variables. In this case, the independent variable is tested in two separate conditions, once with the younger groups and once with the older groups. This means that the testing itself between the two groups should be the same. However, the fact is that the young and old groups have been again split into three separate types of tasks (word-processing, spreadsheet, and database tasks) and tested with three conditions within these groups.
The splitting of these groups into the different types of computer-based tasks allows for more detail into whether the nature of the task is indicative of the difficulty of the task.
Conversely, each of the groups taking the task are assigned in three separate conditions (text-based, icon-based, and text/icon-based) meaning that there may be an issue of internal reliability within the testing measures.
Due to the test being taken in three different ways by each participant, it raises the question of whether the participant would learn how to take the testing procedure more effectively throughout the course of the measurement and effect results through their memory of the testing. The researcher arranged the tests text-based first, followed by text/icon-based and lastly icon-based. This arrangement of the tests is so that the participants understand the icon’s function because they have already been exposed to the text options. Though it may be that this actually threatens the reliability of the dependent variable. This ‘memory effect’ would also have to be accounted for when correlating the scores of each participant over the course of the three conditions of their testing. The reason this problem is such a threat to the internal reliability of the test is there is no way to accurately measure the amount that each participant will improve through test-retest reliability correlations.
Another weakness of the measurement of variables that is present within this study is the conditions of the testing procedure itself. The 75 students were testing by the researcher in groups at a university and the 75 aged-care participants were tested individually at the aged-care facility. This presents a problem with the internal reliability of the results because the groups were tested under different conditions. This difference in testing would also effect the results of the dependent
variable.
Extraneous Variables
Extraneous variables are simply variables that influence the dependent variable without being manipulated by the researcher. It is extremely important for a researcher to identify and define any extraneous variables within a study because you may only conclude that the independent variable caused the difference in the dependent variable if no other variables differed across the groups (Harrington, 2011).
The first extraneous variable that must be examined is the difference in ability to complete specific computer-based tasks. As each person may have differing levels of computer skills specific to each task, three different tasks have been used to control against participant’s previous computing skill-sets.
The other obvious extraneous variable that must be taken into account in this research is the participant’s current ability to complete general computer-based tasks. Research done at Eindhoven University of Technology (Rama, Ridder, & Bouma, 2001) explains that a generational gulf exists in general familiarity with software-style interfaces. This gulf points toward an overall dominance of younger generations having more generalised experience with such tasks. This predisposition toward a difference in the two age group’s ability to complete computer-based tasks creates another aspect that would influence the dependent variable.
Controlling Extraneous Variables
As mentioned previously, the extraneous variable of pre-existing computing skill-sets has been combatted by the use of three different computing tasks.
However, another large extraneous variable has been neglected and would most certainly affect the dependent variable. The general generational differences in familiarity with computing tasks would give the younger generation an advantage and jeopardises the overall standardisation of the study. Due to the participants being chosen by non-randomised methods, there is no way to control against having distinct outliers in either group effect the overall dependent variable correlation.
Weaknesses in the Research Design/ Internal Validity
The research study has a number of problems that all based around the internal validity of the results. Firstly, the non-randomised selection of participants means it is harder for the researcher to infer a causal relationship between age (IV) and the use of icons, text and text/icons without controlling for a number of variables.
Secondly, because the tests are being repeated three times with the different conditions (icon, text, and text/icon) it raises the problem of the ‘memory effect’ playing a part in the results of each participant gradually getting better over the course of the testing, effecting results.
The test themselves also pose a threat to the reliability of results because they were performed in different settings between the two groups.
And finally, the research evidence suggests that the younger age group would have a natural advantage in the tasks due to a generational experience gulf in computing tasks between age groups.
These are all issues with the consistency of the measurement always occurring in the same way and testing the same thing each time. Due to these issues with internal reliability, the study would not be deemed valid.
Possible Improvements to the Study
As mentioned above, the issues with the study all relate back to the internal validity of the measurement procedure. So in order for the study to be deemed reliable and by extension valid, the researcher would need to change a number of the measurement procedures to be more standardised across groups.
Firstly, random sampling from populations of the target age groups would provide a much more representative sample because each person has equal probability to be chosen (Christensen, Johnson, & Turner, p. 145).
Next, the repeated tests leaves open the possibility of the participant learning how to complete the task better each time, and could have been controlled by slightly changing the test’s layout when testing each of the three conditions (text, icon and text/icon).
As mentioned previously, the measurement procedures themselves absolutely must be standardised to be able to draw conclusions from results (Holmes, McDonald, Jones, Ozdemir, & Graham, 2010). So the testing procedure must be changed in order for all participants to be tested under the same conditions.
And finally, the researcher has also neglected to factor in the large gulf in the differences in ability to complete computing tasks between generations. This factor would have to be analysed with regard to current research highlighting younger generation’s heightened ability and controlled through in-depth statistical analysis which cannot be used unless the samples are randomised.
If these problems with the study were rectified, then the researcher would be able to infer a causal relationship between the independent variable and dependent variable of age and ability to complete computing tasks effectively.
Works Cited
Christensen, Johnson, & Turner. (2014). Research Methods, Design and Analysis. Sydney: Pearson.
Harrington, M. (2011). The Design of Experiments in Neuroscience. California: Sage Publications.
Holmes, C., McDonald, F., Jones, M., Ozdemir, V., & Graham, J. E. (2010). Standardization and Omics Science: Technical and Social Dimensions Are Inseparable and Demand Symmetrical Study. OMICS : a Journal of Integrative Biology, 327-332.
Rama, M. D., Ridder, H. d., & Bouma, H. (2001). Technology generation and Age in using layered user interfaces. Gerontechnology, 34.
Wilkinson, L. (1999). Statistical Methods in Psychology Journals: Guidelines and Explanations. American Psychologist, 595.