This is a group task completed with your research group. This group task is to answer a research question by designing and carrying out a small vignette experiment. However, each group member is responsible for writing and submitting their own report.

A list of possible research questions is provided below. You can either choose an option from this list, or, if you are feeling adventurous, you are free to modify these questions or come up with one of your own.

Detailed guidance on how to carry out the project is provided on the second page of this document. In addition, a template for the report is provided on the Moodle page (in the same place you found this document).

Possible research questions

Crime and punishment

1. Do people feel more negatively about black or minority ethnic suspects than white suspects?
2. Do people feel more negatively about young suspects than older suspects?
3. Do people feel more negatively about immigrant suspects than native suspects?

Note that it’s up to you which type of crime you want to focus on, and how you interpret/measure ‘feel more negatively about’.

Feminism & gender discrimination

4. Are female candidates more or less likely than male candidates to be evaluated as suitable for certain types of job?
5. Are news media articles viewed as less authoritative if they are written by female journalists than if they are written by male journalists?

Note that for Q4 it’s up to you what type of job you want to focus on, and also how you want to measure ‘suitability’. For Q5, it’s up to you what type of article you want to focus on and how you measure ‘authoritativeness’.

Vignette experiment – detailed guidance

We will discuss how vignette experiments work in the Week 22 lecture and seminar. However, here
is a brief overview.

A vignette study involves two components:

1. The vignette: this is a short text description of some person or scenario.
2. Survey questions: These are questions tapping into the concepts you want to measure (for example, negative feelings about a person described in the vignette, or the person described in the vignette’s suitability for a particular job).

For example, let’s say you are interested in the following research question:

Do people consider heterosexual stalking to be a more or less serious crime depending on the gender of the perpetrator?

You hypothesise that, given the same stalking behaviour, people will consider stalking to be a more serious crime if it is perpetrated by a man against a women than if it is perpetrated by a woman against a man. A vignette experiment is a great way to test this hypothesis.

Your first step would be to write the vignette describing the scenario. For example, you might write:

[JANE] is a doctor in a large hospital where [JOHN], another doctor, was recently hired. Shortly after meeting [JANE], [JOHN] became interested in pursuing a relationship with [HER]. [JOHN] called [JANE] on the phone, but [SHE] told [HIM] [SHE] was not interested in a relationship. Since then, [JOHN] has called [JANE] several times a week, often late at night, to keep asking [HER] out. [JOHN] has also sent flowers and other gifts to [JANE]’s house. Recently, [JANE] thinks that she has seen [JOHN] outside her house. 1

The square brackets highlight the details in the vignette which indicate the gender of the victim and perpetrator. In this version, the perpetrator is male and the victim is female. For the purposes of your experiment, a second version of the vignette will swap this around (see below).

The next step would be to write some survey questions to tap into your respondents’ perceptions of the seriousness of the stalking behaviour. For example, you might want to ask them questions like:

1. Is [JOHN] stalking [JANE]?
2. Should [JANE] report [JOHN]’s behaviour to her managers at work?
3. Should [JANE] report [JOHN]’s behaviour to the police?
4. On a scale of 1 to 5, where 1 is ‘not at all concerned’ and 5 is ‘extremely concerned’, how concerned do you think [JANE] should be about [HER] personal safety?

People’s responses to these questions tell you about how seriously they take the perpetrator’s behaviour. For example, people who consider the behaviour to constitute stalking clearly consider the perpetrator’s behaviour to be more serious than those who do not consider it to be stalking.

Similarly, people who think the victim should contact the police clearly consider the behaviour to be more serious than those who don’t think the victim should contact the police.

The key to the experiment is that you should create two different versions of the vignette. In one version, the perpetrator is male – indicated by a male first name (John) and male pronouns (he/him) – and the victim is female – indicated by a female first name (Jane) and female pronouns (she/her).

In the second version, the genders are flipped – the perpetrator becomes female and the victim becomes male.
When you distribute your survey, half of your participants (Group A) should get the version of the vignette where the perpetrator is male and the victim is female. The other half (Group B) should get the version where the perpetrator is female and the victim is male.

Once all of your respondents have read the vignette and completed the survey questions, you can compare Group A and Group B’s responses. Your hypothesis predicts that people in Group A should perceive the stalking to be more serious than the people in Group B.

It is crucial that the only difference between the two vignettes is the thing you are interested in testing (in this case the gender of the perpetrator and victim). Everything else should stay the same in both versions. The whole point of an experiment is that you can be confident that X (in this case, the gender of the perpetrator and victim) caused Y (judgements of the seriousness of the stalking). If something else is different between the two vignettes – in this example, this could be the nature of the perpetrator’s behaviour, or the way the perpetrator and victim know each other – then this could be the explanation for why Group A’s responses differ from Group B’s.

Conducting your experiment

Once you’ve decided on your research question and written up your vignettes and survey questions, you then have to figure out how to get people to fill it in. There are two main ways you could do this:

1. You could print everything out and ask people to fill it in in person – for example, by walking around campus with a clipboard and asking people you see.
2. You could try to recruit people over the internet – for example, using social media or email lists you have access to. In which case you could deliver the survey over the internet by either emailing a word document to participants for them to fill out and send back, or by using an online survey tool like SurveyMonkey.

Remember that you are looking to compare two different groups of people, so you need to make sure you have enough people in each group. You should aim to get between 30 and 40 completed surveys in total (though don’t worry too much if you end up with fewer than this). It’s probably best if the people you recruit are not other students from SO341 – because they’ll know exactly what you are trying to do and this might bias your results.

If you are conducting the survey in person, you should try not to interview more than one person at once. If you interview multiple people at the same time, then you risk them discussing the answers and this could also bias your results.

Randomisation

Another important element of good experimental design is randomisation. In the above example, we should randomly assign who gets which version of the vignette. The alternative would be for us to purposefully select who gets the male perpetrator/female victim version and who gets the female perpetrator/male victim version. If we do this, we run the risk of accidentally (or intentionally) picking certain types of people to be in Group A and certain types of people to be in Group B. If we do this, it means that these differences could be the explanation for any differences between these group’s responses on the stalking questions. For example, if we observe that Group A think that the perpetrator’s behaviour is more serious than Group B, it could be because of the effect of the vignette, but it could also be just because we picked a certain type of person to be in Group A. Randomisation solves this problem.

If you are using pen and paper or sending people word documents, randomisation is easy. All you have to do is randomly choose which version of the survey each person gets (for example, by flipping a coin).

If you are using something like SurveyMonkey and posting a link somewhere (e.g. on Facebook), then things get a bit trickier. A good option in this case would be to put up the male perpetrator/female victim version of the survey for a while. Then, when enough people have participated, take this version down and replace it with the female  erpetrator/male victim version (keeping the URL the same). This is not quite random (people who complete the survey early might be different to people who complete it late), but it’s close enough for our purposes.

The qualitative element

In addition to the elements described above, this project should also include a qualitative element. You have three options to choose from in terms of how you integrate qualitative methods into your project:

1. A focus group to help you refine either your survey questions or your vignette (or both). Writing good vignettes and good survey questions is difficult. One good way to integrate qualitative methods would therefore be to use a focus group to ‘workshop’ these elements of your project. Provide the group with example vignettes and ask them about their impressions of the person/scenario. Do they take away what you intended when you wrote the vignette? Or are they picking up on things you did not intend?
   You can do something similar with the survey questions. Ask them to give an answer and then ask them to explain in detail what they a) understood the question to mean, and b) what they intended their answer to mean. Do people seem to be interpreting your questions and response options in the way you intended?
2. An open-ended question in your survey. Add an open-ended question to your survey. For example, in the above stalking example, I could ask people ‘How would you feel if you were the victim?’. People’s open-ended responses might provide an interesting insight beyond the specific questions I asked.
3. Follow-up interviews. Follow up with a few people who participated in your experiment. Ask them what their thought process was when they were answering the survey questions. For example, in the above stalking example, I might ask people ‘Why did you say that the victim should/should not contact the police?’. People’s responses might corroborate your interpretation of your results (for example, that the respondent felt that male on female stalking was not that serious and therefore shouldn’t be reported to the police). Or they might offer an additional insight (for example, perhaps the respondent felt that male on female stalking was very serious, but that it shouldn’t be reported to the police because they wouldn’t take it seriously).

Ethics (IMPORTANT!)

Informed consent

When conducting research involving people, you have to get their informed consent. This means you need a page at the beginning of your survey explaining:

• Who you are (and how the respondent can contact you)
• The purpose of the research (including broadly what the survey is about)
• That they don’t have to participate if they don’t want to, and that they can withdraw from the survey any time they want, without penalty
• This page should also include a space where they can sign to say they agree to participate (if you are delivering the survey on paper), or, if you are delivering the survey electronically, a box they can tick or a button they can press.
• An example of an informed consent page is given on the Moodle page. Feel free to copy and paste this text.

Data protection

You need to make sure you store people’s data securely. That means:

1. Not storing people’s names or any other identifying details (like email addresses) unless you have to (for example, because you want to conduct a follow-up interview with them).
2. If you do store people’s names or other details, do not store them in the same place as their data (survey or interview responses).
3. Store all digital data in a password-protected place that only you have access to. Store all physical data (e.g. paper surveys) in a locked container that only you have access to.
4. Destroy all the data after you are sure you don’t need it anymore (i.e. after the marks are out and you’re sure you won’t have to resit anything).

Summary: carrying out your project

A successful project should contain the following elements:

1. A clear research question and hypothesis.
2. A well-designed vignette. Make sure you are clear about what you are changing between the two versions of the vignette.
3. Well-designed survey questions. Make sure you are tapping into what you really want to measure.
4. Clearly explained results. How did the people who received the different vignettes differ in their responses to your survey questions? Make sure you use the appropriate statistics (averages or percentages).
5. A qualitative element that clearly enhances the project. Choose one of the three options described above and use it well.