The ABC of a Decent Experiment For Graduate-level Hard-working Individuals

Koji Miwa    Last updated 01/25/2020

  • It is ideal for graduate students to complete the steps 1 to 5 by the end of the first year. 
  • Foreseeing that you need to use an experiment programming software and R, start studying these early.
  • If you are stuck, wondering about the problem alone may not present you a solution.
    • Try to read one paper a day. Do lots of readings, and you may be able to connect dots as time goes by.
    • Ask your graduate student colleague for help. Please also help your colleagues with your abundance mentality.
    • You can also make an apppointment with your supervisor. Check our Google Calendar [Koji’s Psycholinguistics Lab]  for the availability of the lab and your supervisor. Then, simply propose a date/time for the meeting.
  • If you have something to be read and approved by your collaborator (or supervisor), please give him/her at least 2 weeks.

  1. Check your academic background
  2. Find a critical “why” and ask primitive questions
    • It is a good practice to think about many things in life more …. and more and wonder why things are as they are.
    • Always distinguish facts from opinions (and judge whether those opinions accompany credible supporting evidence).
    • It is important to find time alone to wonder many things and juggle ideas in your mind.
    • Broaden your perspective by reading papers on many different topics (do not say “this paper is not relevant to my study without reading it).
  3. Do literature reviews
    • Narrow down your focus and identify one topic that you are motivated to study further.
    • It is particularly important to read papers published in peer-reviewed international journals. 
    • Reading 100 relevant papers should give you a good idea about the topic of your interest.
    • Note that literature reviews will continue till the end.
  4. Narrow down the topic and design a study with a testable hypothesis
    • At this point, “why” should become “whether," “which," "how (much)," or “when” for a quantiative investigation.
    • It often helps to compare two or more things (e.g., two different groups of participants, two different tasks).
    • Do not make your study design more complex than necessary.
  5. Organize your team
    • Note that it is nearly impossible for typical students to carry out a decent psycholinguistic study all by themselves. It is extremely important that you find a collaborator (most likely your supervisor) before you move on.
    • If you have a collaborator(s), at this point, it is recommended that you discuss the authorship issue with them to avoid any future conflict.
    • Whoever contributes significantly to your project should be a co-author. Whoever provides only minor contribution should not be a co-author (but should be acknowledged instead). Do not let other people work hard "free of charge."
  6. Prepare your experiment
    • Program your experiment with an experiment building software.
    • Make sure to study the ethical guidelines stated in the Declaration of Helsinki (World Medical Association, 1964).
    • Test yourself first to check (1) whether the experiment is doable, (2) whether the experiment script is error-free, and (3) how long the experiment lasts. Do NOT let other people do what you yourself do not want to do.
    • Prepare research ethics documents.
      1. Information sheet (English example, Japanese example)
      2. Consent form (English example, Japanese example)
      3. Debriefing sheet (English example)
    • Obtain ethics review application forms from here and submit it to the Graduate School of Humanities Ethics Board together with (1) an information sheet, (2) a consent form.
    • If you want your experiment cost reimbursed, also make sure to submit the following documents to your supervisor (so that he can complete a 謝金支出伺 form) before you start your experiment: (1) an information sheet, (2) a consent form, and (3) an experiment ad.
  7. Recruit participants
    • Pay attention to your target population and sample appropriate participants (do NOT carelessly recruit your “friends”)
    • Make a one-click-sign-up page in a recruiting website (Example).
    • Make an experiment ad and advertise your experiment (Example.pptx). Making the recruitment URL shorter or generating a QR code may help.
    • Reserve time slots for your experiment on our Google Calendar [Koji’s Psycholinguistics Lab] . If there are multiple experiments going on, be considerate of others.
    • When participants sign up, indicate on our Google Calendar [Koji’s Psycholinguistics Lab] that the time slot is formally taken.
  8. Collect data
    • If necessary, do a pilot study. If not necessary, then don’t waste time and money.
    • Remind yourself that clean data are vital to the success of your study (Note: statistics does not improve the data quality).
    • Before the experiment
      1. Come to the experiment room early, at least 15 minutes before the experiment.
      2. Turn on the experiment devices you are going to use.
      3. Place the information sheet/ the consent form and a bottle of water on the desk.
      4. Place the “実験中 (Experiment in Progress)” board outside the room.
    • During the experiment
      1. Keep track of the participants’ info (i.e., who participated, when they participated, what happened during the session).
      2. Ask the participant to read the information sheet and sign the consent form.
      3. Make sure to give the participant an opportunity to ask questions.
      4. Treat participants with respect and pay attention to the participant’s physical condition.
      5. Pay attention to all the details so that you can collect clean data.
    • After the experiment
      1. Ask the participant to fill out the receipt with their complete address.
      2. Briefly explain the purpose of the experiment to the participant.
      3. Give the debriefing sheet to the participant.
    • Once you finish collecting all the data, submit a "被験者謝金支給調書” sheet to your supervisor.
  9. Complete the method section
    • Write the method section of your manuscript first (or you will never be able to write it).
  10. Analyse the data and interpret the results
  11. Write up a manuscript

 Last updated 02/16/2020                                  © Koji Miwa 2015-2020 All Rights Reserved