The ABC of a Decent Experiment For Graduate-level Hard-working Individuals

Koji Miwa    Last updated 01/27/2019


  • It is ideal for graduate students to complete the steps 1 to 5 by the end of the first year. 
  • Foreseeing that you need to use an experiment programming software and R, start studying these early.
  • If you are stuck, wondering about the problem alone may not present you a solution.
    • Try to read one paper a day. Do lots of readings, and you may be able to connect dots in the
    • Ask your graduate student colleague for help. Please also help your colleagues with your abundance mentality.
    • You can also make an apppointment with your supervisor. Check our Google Calendar [Koji’s Psycholinguistics Lab]  for the availability of the lab and your supervisor. Then, simply propose a date/time for the meeting.
  • If you have something to be read and approved by your collaborators (or supervisor), please give them at least 2 weeks.

  1. Check your academic background
  2. Find a big “why” and ask primitive questions
    • It is a good practice to think about many things in life more …. and more and wonder why things are as they are.
    • Always distinguish facts from opinions (and judge whether those opinions accompany credible supporting evidence).
    • It is important to find time alone to wonder many things and juggle ideas in your mind.
    • Broaden your perspective by reading papers on many different topics (do not say “this paper is not relevant to my study without reading it).
  3. Do literature reviews
    • It is particularly important to read papers published in peer-reviewed international journals. 
    • Reading 100 relevant papers should give you a good idea about the topic of your interest.
    • Note that literature reviews will continue till the end.
  4. Narrow down the topic and design a study with a testable hypothesis
    • At this point, “why” should become “whether," “which," "how (much)," or “when” for a quantiative investigation.
    • It often helps to compare two or more things (e.g., two different groups of participants, two different tasks).
    • Do not make your study design more complex than necessary.
    • If you have a collaborator (or a supervisor), at this point, it is recommended that you discuss the authorship issue with them to avoid any future conflict (see Gaffey, 2015, for APA’s suggestion to determine authorship).
  5. Prepare your experiment
  6. Recruit participants
    • Pay attention to your target population and sample appropriate participants (do NOT carelessly recruit your “friends”)
    • Make a one-click-sign-up page in a recruiting website (Example).
    • Make an experiment ad and advertise your experiment (Example.pptx). Making the recruitment URL shorter or generating a QR code may help.
    • Reserve time slots for your experiment on our Google Calendar [Koji’s Psycholinguistics Lab] . If there are multiple experiments going on, be considerate of others.
    • When participants sign up, indicate on our Google Calendar [Koji’s Psycholinguistics Lab] that the time slot is formally taken.
  7. Collect data
    • If necessary, do a pilot study. It not necessary, then don’t.
    • Remind yourself that clean data are vital to the success of your study (Note: statistics does not improve the data quality).
    • Before the experiment
      1. Come to the experiment room early, at least 15 minutes before the experiment.
      2. Turn on the experiment devices you are going to use.
      3. Place the information sheet/ the consent form and a bottle of water on the desk.
      4. Place the “実験中 (Experiment in Progress)” board outside the room.
    • During the experiment
      1. Keep track of the participants’ info (i.e., who participated, when they participated, what happened during the session).
      2. Ask the participant to read the information sheet and sign the consent form.
      3. Make sure to give the participant an opportunity to ask questions.
      4. Treat participants with respect and pay attention to the participant’s physical condition.
      5. Pay attention to all the details so that you can collect clean data.
    • After the experiment
      1. Ask the participant to fill out the receipt with their complete address.
      2. Briefly explain the purpose of the experiment to the participant.
      3. Give the debriefing sheet to the participant.
    • Once you finish collecting all the data, submit a "被験者謝金支給調書” sheet to your supervisor.
  8. Complete the method section
    • Write the method section of your manuscript first (or you will never be able to write it).
  9. Analyse the data and interpret the results
  10. Write up a manuscript
    • Do not try to write everything all at once, after a long period of procrastination, because most likely you cannot!
    • Follow the APA Style and write professionally, foreseeing a future publication(Example).
    • It is also recommended to present your study at a (legitimate) conference. Stay away from predatory conferences.
    • When you submit your manuscript for publication, stay away from predatory journals (Beall’s List).
    • Useful resources:

 Last updated 07/16/2019                                  © Koji Miwa 2015-2019 All Rights Reserved