This is a group task completed with your research group. This group task is to answer a research question by designing and carrying out a small vignette experiment. However, each group member is responsible for writing and submitting their own report.
A list of possible research questions is provided below. You can either choose an option from this list, or, if you are feeling adventurous, you are free to modify these questions or come up with one of your own.
Detailed guidance on how to carry out the project is provided on the second page of this document. In addition, a template for the report is provided on the Moodle page (in the same place you found this document).
Possible research questions
Crime and punishment
1. Do people feel more negatively about black or minority ethnic suspects than white suspects?
2. Do people feel more negatively about young suspects than older suspects?
3. Do people feel more negatively about immigrant suspects than native suspects?
Note that it’s up to you which type of crime you want to focus on, and how you interpret/measure ‘feel more negatively about’.
Feminism & gender discrimination
4. Are female candidates more or less likely than male candidates to be evaluated as suitable for certain types of job?
5. Are news media articles viewed as less authoritative if they are written by female journalists than if they are written by male journalists?
Note that for Q4 it’s up to you what type of job you want to focus on, and also how you want to measure ‘suitability’. For Q5, it’s up to you what type of article you want to focus on and how you measure ‘authoritativeness’.
Vignette experiment – detailed guidance
We will discuss how vignette experiments work in the Week 22 lecture and seminar. However, here
is a brief overview.
A vignette study involves two components:
1. The vignette: this is a short text description of some person or scenario.
2. Survey questions: These are questions tapping into the concepts you want to measure (for example, negative feelings about a person described in the vignette, or the person described in the vignette’s suitability for a particular job).
For example, let’s say you are interested in the following research question:
Do people consider heterosexual stalking to be a more or less serious crime depending on the gender of the perpetrator?
You hypothesise that, given the same stalking behaviour, people will consider stalking to be a more serious crime if it is perpetrated by a man against a women than if it is perpetrated by a woman against a man. A vignette experiment is a great way to test this hypothesis.
Your first step would be to write the vignette describing the scenario. For example, you might write:
[JANE] is a doctor in a large hospital where [JOHN], another doctor, was recently hired. Shortly after meeting [JANE], [JOHN] became interested in pursuing a relationship with [HER]. [JOHN] called [JANE] on the phone, but [SHE] told [HIM] [SHE] was not interested in a relationship. Since then, [JOHN] has called [JANE] several times a week, often late at night, to keep asking [HER] out. [JOHN] has also sent flowers and other gifts to [JANE]’s house. Recently, [JANE] thinks that she has seen [JOHN] outside her house. 1
The square brackets highlight the details in the vignette which indicate the gender of the victim and perpetrator. In this version, the perpetrator is male and the victim is female. For the purposes of your experiment, a second version of the vignette will swap this around (see below).
The next step would be to write some survey questions to tap into your respondents’ perceptions of the seriousness of the stalking behaviour. For example, you might want to ask them questions like:
1. Is [JOHN] stalking [JANE]?
2. Should [JANE] report [JOHN]’s behaviour to her managers at work?
3. Should [JANE] report [JOHN]’s behaviour to the police?
4. On a scale of 1 to 5, where 1 is ‘not at all concerned’ and 5 is ‘extremely concerned’, how concerned do you think [JANE] should be about [HER] personal safety?
People’s responses to these questions tell you about how seriously they take the perpetrator’s behaviour. For example, people who consider the behaviour to constitute stalking clearly consider the perpetrator’s behaviour to be more serious than those who do not consider it to be stalking.
Similarly, people who think the victim should contact the police clearly consider the behaviour to be more serious than those who don’t think the victim should contact the police.
The key to the experiment is that you should create two different versions of the vignette. In one version, the perpetrator is male – indicated by a male first name (John) and male pronouns (he/him) – and the victim is female – indicated by a female first name (Jane) and female pronouns (she/her).
In the second version, the genders are flipped – the perpetrator becomes female and the victim becomes male.
When you distribute your survey, half of your participants (Group A) should get the version of the vignette where the perpetrator is male and the victim is female. The other half (Group B) should get the version where the perpetrator is female and the victim is male.
Once all of your respondents have read the vignette and completed the survey questions, you can compare Group A and Group B’s responses. Your hypothesis predicts that people in Group A should perceive the stalking to be more serious than the people in Group B.
It is crucial that the only difference between the two vignettes is the thing you are interested in testing (in this case the gender of the perpetrator and victim). Everything else should stay the same in both versions. The whole point of an experiment is that you can be confident that X (in this case, the gender of the perpetrator and victim) caused Y (judgements of the seriousness of the stalking). If something else is different between the two vignettes – in this example, this could be the nature of the perpetrator’s behaviour, or the way the perpetrator and victim know each other – then this could be the explanation for why Group A’s responses differ from Group B’s.
Conducting your experiment
Once you’ve decided on your research question and written up your vignettes and survey questions, you then have to figure out how to get people to fill it in. There are two main ways you could do this:
1. You could print everything out and ask people to fill it in in person – for example, by walking around campus with a clipboard and asking people you see.
2. You could try to recruit people over the internet – for example, using social media or email lists you have access to. In which case you could deliver the survey over the internet by either emailing a word document to participants for them to fill out and send back, or by using an online survey tool like SurveyMonkey.
Remember that you are looking to compare two different groups of people, so you need to make sure you have enough people in each group. You should aim to get between 30 and 40 completed surveys in total (though don’t worry too much if you end up with fewer than this). It’s probably best if the people you recruit are not other students from SO341 – because they’ll know exactly what you are trying to do and this might bias your results.
If you are conducting the survey in person, you should try not to interview more than one person at once. If you interview multiple people at the same time, then you risk them discussing the answers and this could also bias your results.
Another important element of good experimental design is randomisation. In the above example, we should randomly assign who gets which version of the vignette. The alternative would be for us to purposefully select who gets the male perpetrator/female victim version and who gets the female perpetrator/male victim version. If we do this, we run the risk of accidentally (or intentionally) picking certain types of people to be in Group A and certain types of people to be in Group B. If we do this, it means that these differences could be the explanation for any differences between these group’s responses on the stalking questions. For example, if we observe that Group A think that the perpetrator’s behaviour is more serious than Group B, it could be because of the effect of the vignette, but it could also be just because we picked a certain type of person to be in Group A. Randomisation solves this problem.
If you are using pen and paper or sending people word documents, randomisation is easy. All you have to do is randomly choose which version of the survey each person gets (for example, by flipping a coin).
If you are using something like SurveyMonkey and posting a link somewhere (e.g. on Facebook), then things get a bit trickier. A good option in this case would be to put up the male perpetrator/female victim version of the survey for a while. Then, when enough people have participated, take this version down and replace it with the female erpetrator/male victim version (keeping the URL the same). This is not quite random (people who complete the survey early might be different to people who complete it late), but it’s close enough for our purposes.
The qualitative element
In addition to the elements described above, this project should also include a qualitative element. You have three options to choose from in terms of how you integrate qualitative methods into your project:
When conducting research involving people, you have to get their informed consent. This means you need a page at the beginning of your survey explaining:
You need to make sure you store people’s data securely. That means:
Summary: carrying out your project
A successful project should contain the following elements: