Esra Ataman, Ozan Can Çağlar, Bilal Kırkıcı

7411

Dilbilim Araştırmalarında Açık Bilim

Daha açık, şeffaf, tekrarlanabilir ve yeniden üretilebilir çalışmalar yürütmeye dikkat çeken açık bilimle ilgili tartışmalar, son yıllarda küresel alanyazında görünürlük kazanmaktadır. Bu tartışmalar, dilbilim alanında da kendine yer bulmuş, fakat Türkiye’de bu alanda açık bilime dair belirgin bir alanyazın henüz oluşmamıştır. Bu doğrultuda, bu makale tekrarlama krizi, bilimsel sahtekarlık ve yayın yanlılığı gibi açık bilime zemin hazırlayan gelişmelerin güncel bir özetini sunarak, şeffaflık, tekrarlanabilirlik ve yeniden üretilebilirlik gibi ilkelerin dilbilim araştırmalarında nasıl vücut bulabileceğini irdelemiştir. Makalede gizli esneklik ve sakıncalı araştırma uygulamaları sorunlarına dilbilim araştırmaları özelinde odaklanılmış ve bu sorunların ciddiyetine yönelik bir farkındalığın oluşturulması amaçlanmıştır. Bu sorunlara karşı alanyazında önerilmiş dört temel çözüm önerisi (ön kayıt, yeniden üretilebilir iş akışı, ön baskı ve veri/materyal paylaşımı) artı ve eksileriyle tartışılarak, dil araştırmacılarının dikkatine sunulmuştur. Araştırmacıların bu çözüm önerilerini kendi alan ve koşullarını düşünerek değerlendirmeleri ve olabildiğince uygulamaları, dilbilim araştırmalarında şeffaflık, tekrarlanabilirlik ve yeniden üretilebilirliği artırmaya katkı sağlayacaktır.

Open Science in Linguistic Research

Discussions about open science, which draws attention to conducting more open, transparent, replicable, and reproducible studies, have been making an appearance in the global literature in recent years. These discussions have also found their place in linguistics, though there is no well- established literature on open science in this field in Turkey yet. Accordingly, by presenting an up-to-date summary of developments that pave the way for open science, such as the replication crisis, scientific fraud, and publication bias, this article examines how related principles such as transparency, replicability, and reproducibility can manifest themselves in linguistic research. In the article, the problems of hidden flexibility and questionable research practices are discussed within the scope of linguistic research with the aim of raising awareness about the seriousness of these problems. Four basic solutions suggested in the literature to avoid these problems (pre-registration, reproducible workflow, preprint, and data/material sharing) are presented together with their pros and cons and are brought to the language researchers' attention. It is possible to increase transparency, replicability, and reproducibility in linguistic research when researchers evaluate these proposed solutions in the literature by considering their own fields and conditions and apply them to the extent possible.
PDF

___

  • Aschwanden, C. (2019). We’re all “p-hacking” now. Wired. https://www.wired.com/story/were-all-p-hacking-now/ Aygen, G. (2012). Morpho-syntactic variation and methodology. Dil ve Edebiyat Dergisi, 8(1), 1-14.
  • Berez-Kroeker, A. L., Gawne, L., Kung, S. S., Kelly, B. F., Heston, T., Holton, G., Pulsifer, P., Beaver, D. I., Chelliah, S., Dubinsky, S., Meier, R. P., Thieberger, N., Rice, K., & Woodbury, A. C. (2018). Reproducible research in linguistics: A position statement on data citation and attribution in our field. Linguistics, 56(1), 1-18. https://doi.org/10.1515/ling-2017-0032
  • Bowers, J. (2011). Six steps to a better relationship with your future self. The Political Methodologist, 18(2), 2-8.
  • Camerer, C. F., Dreber, A., Forsell, E., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Almenberg, J., Altmejd, A., Chan, T., Heikensten, E., Holzmeister, F., Imai, T., Isaksson, S., Nave, G., Pfeiffer, T., Razen, M., & Wu, H. (2016). Evaluating replicability of laboratory experiments in economics. Science, 351(6280), 1433-1436. https://doi.org/10.1126/science.aaf0918
  • Center for Open Science. (2020). Preregistration. https://www.cos.io/initiatives/prereg
  • Chambers, C. (2017). The seven deadly sins of psychology. Princeton University Press; JSTOR. https://doi.org/10.2307/j.ctvc779w5
  • Chládková, K., & Šimáčková, Š. (2021). Distributional learning of speech sounds: An exploratory study into the effects of prior language experience. Language Learning, 71(1), 131-161. https://doi.org/10.1111/lang.12432
  • Christensen, G., Freese, J., & Miguel, E. (2019). Transparent and reproducible social science research (1. bs). University of California Press; JSTOR. https://doi.org/10.2307/j.ctvpb3xkg
  • Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49(12), 997-1003. https://doi.org/10.1037/0003-066X.49.12.997
  • Coretta, S. (2020). Open Science in phonetics and phonology [Preprint]. Open Science Framework. https://doi.org/10.31219/osf.io/4dz5t
  • Crüwell, S. (2019). Seven easy steps to open science: An annotated reading list. Zeitschrift Für Psychologie, 227(4), 237-248. https://doi.org/10.1027/2151-2604/a000387
  • Devezer, B., Navarro, D. J., Vandekerckhove, J., & Ozge Buzbas, E. (2021). The case for formal methodology in scientific reform. Royal Society Open Science, 8(3), 200805. https://doi.org/10.1098/rsos.200805
  • Dirnagl, U. (2020). Preregistration of exploratory research: Learning from the golden age of discovery. PLOS Biology, 18(3), e3000690. https://doi.org/10.1371/journal.pbio.3000690
  • European Organization For Nuclear Research & OpenAIRE. (2013). Zenodo. CERN. https://doi.org/10.25495/7GXK-RD71
  • Ferguson, C. J., & Heene, M. (2012). A vast graveyard of undead theories: Publication bias and psychological science’s aversion to the null. Perspectives on Psychological Science, 7(6), 555-561. https://doi.org/10.1177/1745691612459059
  • Franco, A., Malhotra, N., & Simonovits, G. (2014). Publication bias in the social sciences: Unlocking the file drawer. Science, 345(6203), 1502-1505. https://doi.org/10.1126/science.1255484
  • Gelman, A., & Loken, E. (2019). The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. 17.
  • Haven, T., & Van Grootel, Dr. L. (2019). Preregistering qualitative research. Accountability in Research, 26(3), 229-244. https://doi.org/10.1080/08989621.2019.1580147
  • Havron, N., Bergmann, C., & Tsuji, S. (2020). Preregistration in infant research—A primer. Infancy, 25(5), 734-754. https://doi.org/10.1111/infa.12353
  • Hoekstra, R., & Vazire, S. (2020). Hoekstra & Vazire (2020), Intellectual humility is central to science [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/edh2s
  • Hui, B., & Huntley, E. (2020). Embracing open science in applied linguistics: Graduate students’ perspectives [Ön Baskı]. Open Science Framework. https://doi.org/10.31219/osf.io/vmtza
  • Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124
  • John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23(5), 524-532. https://doi.org/10.1177/0956797611430953
  • Kahraman, H., & Kırkıcı, B. (2021). Letter transpositions and morphemic boundaries in the second language processing of derived words: An exploratory study of individual differences. Applied Psycholinguistics, 42(2), 417-446. https://doi.org/10.1017/S0142716420000673
  • Kathawalla, U.-K., Silverstein, P., & Syed, M. (2021). Easing into open science: A guide for graduate students and their advisors. Collabra: Psychology, 7(18684). https://doi.org/10.1525/collabra.18684
  • Kerr, N. L. (1998). HARKing: Hypothesizing after the results are known: Personality and Social Psychology Review. https://doi.org/10.1207/s15327957pspr0203_4
  • King, G. (2007). An introduction to the dataverse network as an infrastructure for data sharing. Sociological Methods and Research, 36, 173-199.
  • Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., Cemalcilar, Z., Chandler, J., Cheong, W., Davis, W. E., Devos, T., Eisner, M., Frankowska, N., Furrow, D., Galliani, E. M., ... Nosek, B. A. (2014). Investigating variation in replicability: A “Many Labs” Replication Project. Social Psychology, 45(3), 142-152. https://doi.org/10.1027/1864-9335/a000178
  • Lindsay, D. S., Simons, D. J., & Lilienfeld, S. O. (2016). Research preregistration 101. APS Observer, 29(10). https://www.psychologicalscience.org/observer/research- preregistration-101
  • Marsden, E., Mackey, Alison, & Plonsky, Luke. (2016). The IRIS Repository: Advancing research practice and methodology. Içinde Advancing Methodology and Practice: The IRIS Repository of Instruments for Research into Second Languages (ss. 1-21). Routledge. https://www.routledge.com/Advancing-Methodology-and-Practice-The- IRIS-Repository-of-Instruments-for/Mackey-Marsden/p/book/9780415833646
  • Marsden, E., Morgan-Short, K., Thompson, S., & Abugaber, D. (2018). Replication in second language research: Narrative and systematic reviews and recommendations for the field: Replication in Second Language Research. Language Learning, 68(2), 321- 391. https://doi.org/10.1111/lang.12286
  • Marsden, E., Thompson, S., & Plonsky, L. (2017). Open science in second language acquisition research: The IRIS repository of research materials and data. SHS Web of Conferences, 38, 00013. https://doi.org/10.1051/shsconf/20173800013
  • Mertzen, D., Lago, S., & Vasishth, S. (2020). The benefits of preregistration for hypothesis-driven bilingualism research [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/nm3eg
  • Mertzen, D., Lago, S., & Vasishth, S. (2021). The benefits of preregistration for hypothesis-driven bilingualism research. Bilingualism: Language and Cognition, 1- 6. https://doi.org/10.1017/S1366728921000031
  • Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., Glennerster, R., Green, D. P., Humphreys, M., Imbens, G., Laitin, D., Madon, T., Nelson, L., Nosek, B. A., Petersen, M., Sedlmayr, R., Simmons, J. P., Simonsohn, U., & Van der Laan, M. (2014). Promoting transparency in social science research. Science, 343(6166), 30-31. https://doi.org/10.1126/science.1245317
  • Moshontz, H., Binion, G., Walton, H., Brown, B. T., & Syed, M. (2021). A guide to posting and managing preprints. Advances in Methods and Practices in Psychological Science, 4(2), 25152459211019948. https://doi.org/10.1177/25152459211019948
  • Mudrak, Ben. (2020). What are preprints, and how do they benefit authors? | AJE. https://www.aje.com/arc/benefits-of-preprints-for-researchers/
  • Munafò, M., & Neill, J. (2016). Null is beautiful: On the importance of publishing null results. Journal of Psychopharmacology, 30(7), 585-585. https://doi.org/10.1177/0269881116638813
  • Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600-2606. https://doi.org/10.1073/pnas.1708274114
  • Olsson-Collentine, A., van Assen, M. A. L. M., & Hartgerink, C. H. J. (2019). The prevalence of marginally significant results in psychology over time. Psychological Science, 30(4), 576-586. https://doi.org/10.1177/0956797619830326
  • Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716-aac4716. https://doi.org/10.1126/science.aac4716
  • Roettger, T. B. (2021). Preregistration in experimental linguistics: Applications, challenges, and limitations. Linguistics, 0(0), 000010151520190048. https://doi.org/10.1515/ling-2019-0048
  • Roettger, T. B., & Baer-Henney, D. (2018). Toward a replication culture: Speech production research in the classroom. PsyArXiv. https://doi.org/10.31234/osf.io/q9t7c
  • Roettger, T., Winter, B., & Baayen, H. (2019). Emergent data analysis in phonetic sciences: Towards pluralism and reproducibility. Journal of Phonetics, 73, 1-7. https://doi.org/10.1016/j.wocn.2018.12.001
  • Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638-641. https://doi.org/10.1037/0033-2909.86.3.638
  • Schönbrodt, F. (2019). Training students for the Open Science future. Nature Human Behaviour, 3(10), 1031-1031. https://doi.org/10.1038/s41562-019-0726-z
  • Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359-1366. https://doi.org/10.1177/0956797611417632
  • Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2013). P-Curve: A key to the file drawer (SSRN Scholarly Paper ID 2256237). Social Science Research Network. https://papers.ssrn.com/abstract=2256237
  • Spellman, B. A., Gilbert, E. A., & Corker, K. S. (2018). Open Science. Stevens’ Handbook of Experimental Psychology and Cognitive Neuroscience içinde (ss. 1-47). American Cancer Society. https://doi.org/10.1002/9781119170174.epcn519
  • Szollosi, A., Kellen, D., Navarro, D. J., Shiffrin, R., Rooij, I. van, Zandt, T. V., & Donkin, C. (2020). Is preregistration worthwhile? Trends in Cognitive Sciences, 24(2), 94-95. https://doi.org/10.1016/j.tics.2019.11.009
  • The Retraction Watch Leaderboard. (2015, Haziran 16). Retraction watch. https://retractionwatch.com/the-retraction-watch-leaderboard/ United Nations Educational, Scientific and Cultural Organization (UNESCO). (2016).
  • The Future of scientific advice to the United Nations: A summary report to the Secretary-General of the United Nations from the Scientific Advisory Board.
  • Vasishth, S., Mertzen, D., Jäger, L. A., & Gelman, A. (2018). The statistical significance filter leads to overoptimistic expectations of replicability. Journal of Memory and Language, 103, 151-175. https://doi.org/10.1016/j.jml.2018.07.004
  • Vazire, S. (2019). A toast to the error detectors. Nature, 577(7788), 9-9. https://doi.org/10.1038/d41586-019-03909-2
  • Wicherts, J. M., Bakker, M., & Molenaar, D. (2011). Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical Results. PLOS ONE, 6(11), e26828. https://doi.org/10.1371/journal.pone.0026828
  • Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016). Degrees of freedom in planning, running, analyzing, and reporting psychological studies: A checklist to avoid p-hacking. Frontiers in Psychology, 7. https://doi.org/10.3389/fpsyg.2016.01832
Dilbilim Araştırmaları Dergisi-Cover
  • ISSN: 1300-8552
  • Yayın Aralığı: Yılda 2 Sayı
  • Başlangıç: 1990
  • Yayıncı: Dilbilim Derneği
Arşiv
Sayıdaki Diğer Makaleler

Türkçede görme algı eylemleri ve kesinlik derecelenmesi: Görünmek ve gözükmek

Zeynep ERK EMEKSİZ

Türk İşaret Dilindeki Olumsuzluk Stratejilerinin Derlem-Tabanlı Tipolojisi

Bahtiyar Makaroğlu

Dilbilim Araştırmalarında Açık Bilim

Esra Ataman, Ozan Can Çağlar, Bilal Kırkıcı

Değişken Altbiçimlenmeye Çizgilesellik-dışı Engeller

Ömer Demirok