The ABC of a Decent Experiment For Graduate-level Hard-working Individuals


Koji Miwa

Last updated  09/01/2024


     I believe that this is, at the moment, my best recipe for graduate students’ experimental projects. Foreseeing that you need to use an experiment programming software and R, start studying these early. If you are stuck, wondering about the problem alone may not present you with a solution. Ask your graduate student colleague for help. Please also help your colleagues with your abundance mentality. You can also make an appointment with your supervisor. Check our Google Calendar [Koji’s Psycholinguistics Lab]  for the availability of the lab and your supervisor. Then, simply propose a date/time for the meeting.


1. Set your goal

  • It is recommended that you begin with the end in mind. When do you want to graduate? What do you want to achieve before you graduate? What do you want to do after you graduate? You can ask yourself several questions like these to fully understand what you want to do.
  • In academia, you might also want to ask yourself how you want to understand things (i.e., in the future, when can you say you finally understand a given topic?)


2. Take care of your and others' mental health

  • First and foremost, it is extremely important that you maintain your mental health because your academic life is “done” when you are mentally “done.” Please support your mental health as much as you keep your physical health. 
  • If you start feeling depressed or burnt out, visiting a mental clinic might be a good idea. If you do not know which clinic you should visit, I recommend that you visit Hirokojidori Mental Clinic (https://www.hirokoji-mental.com/).
  • If you also have some room to care about other people, please think about the mental health of your supervisor and graduate student colleagues. They are humans, and they have their own life. They do not enjoy your verbal abuses and illogical complaints.
  • Note that I am not suggesting that you should be optimistic. Quite the other round.


3. Study the basics


4. Participate in weekly meetings

  • Talk to your supervisor every week regardless of whether you have something to discuss with him. Always participate in weekly meetings and report your progress (or no progress).
  • After weekly meetings, submit your meeting note: https://forms.gle/5ktyKpiN2cUikZwb9


5. Find a critical “why” and ask simple questions

  • Always distinguish facts from opinions (and judge whether those opinions accompany credible supporting evidence).
  • It is a good practice to think about many things in life more ….and wonder why things are as they are and how certain ideas can be tested.
  • It is sometimes important to find time alone to wonder many things and juggle ideas in the mind.
  • Broaden your perspective by reading papers on many different topics (do not say “this paper is not relevant to my study" without reading it).


6. Do literature reviews

  • Narrow down your focus and identify one topic that you are motivated to study further.
  • It is particularly important to read papers published in peer-reviewed international journals. 
  • Reading 100 relevant papers should give you a good idea about the topic of your interest. 
  • Note that literature reviews will continue till the end.


7. Narrow down the topic and design a study with a testable hypothesis

  • At this point, “why” should become “whether," “which," "how (much)," or “when” for a quantitative investigation.
  • It often helps to compare two or more things (e.g., two different groups of participants, two different tasks).
  • Make sure that your hypothesis is motivated by a past study/studies (preferably a model) instead of anecdotal evidence. 
  • Filling a gap is important, but it is better to provide a more scholastic motivation.
  • Do not make your study design more complex than necessary.


8. Organize your team

  • Note that it is nearly impossible for typical humanities students to carry out a decent experimental project alone. Therefore, you must find a collaborator with a decent experience (most likely your supervisor) before you move on.
  • If you have a collaborator(s), before you start the project, it is recommended that you discuss the authorship issue with them to avoid future conflicts.
  • Whoever contributes significantly to your project should be a co-author. Whoever provides only a minor contribution should not be a co-author but should be acknowledged instead. Do not let other people work hard for you "free of charge” (which is ghost authorship). Do not give an authorship to those who did not contribute significantly (which is gift authorship).
  • Pay attention to the APA authorship guidelines. Be very careful when helping your graduate student colleagues (i.e., non-lab members) for their projects to a significant extent unless you are a co-author. It is not your responsibility to educate your graduate student colleagues. You might want to allocate your time to the projects you are officially working on. You cannot be responsible for the consequence of your colleagues' experiments/analyses either (if you can, you should be a co-author).
  • In case your research interest changes or you find it impossible to work with your supervisor, find a new supervisor and submit a 指導教員変更願 to the administration office. Changing your supervisor will not end your life, but it will help you start a new (hopefully better) life.


9. Prepare your experiment


10. Research ethics application

  • Make sure to study the ethical guidelines stated in the Declaration of Helsinki (World Medical Association, 1964).
  • Also, read the local rules of the Graduate School of Humanities. 
  • Test yourself first to check (1) whether the experiment is doable, (2) whether the experiment script is error-free, and (3) how long the experiment lasts. Do NOT let other people do what you do not want to do yourself.
  • Prepare research ethics documents.
    1. Information sheet (English example, Japanese example)
    2. Consent form (English example, Japanese example)
    3. Debriefing sheet (English example)
  • Obtain ethics review application forms from here and submit them to the Graduate School of Humanities Ethics Board together with (1) an information sheet, (2) a consent form.
  • If you want your experiment cost reimbursed, also make sure to submit the following documents to your supervisor (so that he can complete a 謝金支出伺 form) before you start your experiment: (1) an information sheet, (2) a consent form, and (3) an experiment ad.


11. Recruit participants

  • Before you start recruiting participants, make sure that you become the first participant in your experiment. Go through the whole experiment by yourself and see whether it is doable and whether the data are properly collected.
  • Pay attention to your target population and sample an appropriate number of participants.
  • Make a one-click sign-up page on a recruiting website (Example). Doodle.com is useful.
  • Make an experiment ad and advertise your experiment (Example.pptx). Making the recruitment URL shorter or generating a QR code may help.
  • Reserve time slots for your experiment on our Google Calendar [Koji’s Psycholinguistics Lab] . If there are multiple experiments going on, be considerate of others.
  • When participants sign up, indicate on our Google Calendar [Koji’s Psycholinguistics Lab] that the time slot is formally taken.


12. Collect data

  • If necessary, conduct a pilot study to see whether your experiment design is appropriate. Note that the purpose of the pilot study is not to test a hypothesis.
  • Remind yourself that clean data are vital to the success of your study (Note: statistics does not improve the data quality).
  • Before the experiment
    1. Come to the experiment room early, at least 10 minutes before the experiment.
    2. Turn on the experiment devices you are going to use.
    3. Place the information sheet/ the consent form and a bottle of water on the desk.
    4. Place the “実験中 (Experiment in Progress)” board outside the room.
  • During the experiment
    1. Keep track of the participants’ info (i.e., who participated, when they participated, what happened during the session).
    2. Ask the participant to read the information sheet and sign the consent form.
    3. Make sure to give the participant an opportunity to ask questions.
    4. Treat participants with respect and pay attention to the participant’s physical condition.
    5. Pay attention to all the details so that you can collect clean data.
  • After the experiment
    1. Give an Amazon gift card (1000 yen / hour) to participants right after each experiment. Ask the participant to fill out the receipt with their complete address.
    2. Briefly explain the purpose of the experiment to the participant.
    3. Give the debriefing sheet to the participant.
  • Once you finish collecting all the data, submit a "被験者謝金支給調書” sheet to your supervisor.


13. Complete the method section

  • Write the method section of your manuscript first (or you will never be able to write it).


14. Analyse the data and interpret the results


15. Write up your manuscript

  • Do not make your introduction too long (10 pages should be sufficient). It is not recommended that you devote a paragraph to summarize a previous study one by one.
  • Do not copy and paste texts directly from other researchers’ work (i.e., plagiarism) or from your own work (self-plagiarism). Plagiarism is a serious research misconduct for which you may be suspended or expelled. Before you submit your paper, you might want to use a plagiarism checker.
  • Do not try to write everything all at once, after a long period of procrastination, because most likely you cannot!
  • If you have something to be edited or approved by your supervisor, it is a good idea to present a deadline (give him 2 weeks).
  • Follow the APA Style and write professionally, foreseeing a future publication (Example). 
  • Follow the APA journal manuscript preparation guidelines
  • It is also recommended to present your study at a (legitimate) conference. Stay away from predatory conferences.
  • When you submit your manuscript for publication, stay away from predatory journals (Beall’s List).
  • Useful resources:


16. Submit your manuscript/ thesis

  • Make sure that you discuss with your collaborators regarding where to publish the study.
  • Make sure to allow your collaborators to read the manuscript.
  • In your master’s thesis or doctoral dissertation, you can include acknowledgment, but you do not have to (you can just thank whoever helped you individually).
Last updated 10/06/2024          © Koji Miwa 2015-2024 All Rights Reserved