Design vs Report: How Research Design Choices Predict Quality of Scientific Papers in Paper Writing Training
DOI:
https://doi.org/10.54849/monas.v7i2.279Keywords:
Scientific Papers, Research design, Research quality, KTI TrainingAbstract
This study examines the alignment between planned research designs and final Scientific Paper (KTI) reports produced by participants of basic-level KTI training (N = 160 participants; 80 teams) during 2023–2024, and tests whether design elements and participants’ substantive understanding predict paper quality and presentation. We coded concordance between planned and implemented elements (problem formulation, data source, analysis method) and analyzed associations using Fisher’s exact tests, logistic regression for binary outcomes, and linear regression for presentation scores. Results show that 44.8% of teams produced KTIs fully concordant with their designs; 32.5% of teams moved from descriptive designs to include inferential analysis in final KTIs. Concordance of analysis method (design to implementation) was the only design element that significantly predicted substantive-quality scores (p = 0.029). Participants’ substantive-understanding scores predicted presentation quality (β = X, p = 0.013). We discuss implications for training curricula and recommend embedding method-focused mentoring and assessment checkpoints.
References
Abutabenjeh, S., & Jaradat, R. (2018). Clarification of research design, research methods, and research methodology: A guide for public administration researchers and practitioners. Teaching Public Administration, 36(3), 237–258. https://doi.org/10.1177/0144739418775787
Adeoye, M. A. (2024). Mastering the basics: A guide to research methodology for effective writing and publication. Chalim Journal of Teaching and Learning, 4(1), 30–41. https://doi.org/10.31538/cjotl.v4i1.1345
Breakwell, G. M. (2023). Choosing a research design. In A. L. Nichols & J. Edlund (Eds.), The Cambridge handbook of research methods and statistics for the social and behavioral sciences: Volume 1: Building a program of research (pp. 85–102). Cambridge University Press. https://doi.org/10.1017/9781009010054.006
Diaz Gonçalves, T. (2023). On the inconsistency between practice and reporting in science: The genesis of scientific articles. Social Epistemology, 37(5), 684–697. https://doi.org/10.1080/02691728.2023.2209536
Dong, Y. (2023). Descriptive statistics and its applications. Journal of Advanced Mathematics and Statistics, 47, 16–23.
Fadila, A., Rahman, J., Kamaliah, N., Sukoco, S. H., Digital, T., Ilmiah, P., & Digital, L. (2024). Transformasi digital Widyaiswara: Analisis interaksi publikasi ilmiah dengan pemanfaatan teknologi digital. Jurnal Teknologi Pendidikan, 18(1), 45–57. https://doi.org/10.38075/tp.v18i1.400
Jorge Iván Pérez-Rave, J. C. C.-M., & García-Espinosa, F. (2019). Inferential analysis is an advanced analysis used for decision-making/conclusions on a larger data set. Journal of Property Research, 1(1), 1–10.
Kamaliah, N., Fadila, A., Rahman, J., Puspitasari, D., & Rahayu, D. I. (2024). Coaching collaboration training strategy for improving the quality of scientific writing. Journal of Innovative Educational Research, 5(2), 252–260. https://doi.org/10.46843/jiecr.v5i2.1407
Khusainova, R. M., Shilova, Z. V., & Curteva, O. V. (2022). Selection of appropriate statistical methods for research results processing. International Electronic Journal of Mathematics Education, 11(1), 303–315. https://doi.org/10.29333/iejme/334
Kholija Sitompul, S., & Anditasari, A. W. (2022). Challenges in writing academic research: An exploration of master’s students’ experiences. Getsempena English Education Journal, 9(2), 136–148. https://ejournal.bbg.ac.id/geej
Kwak, S. G., & Kim, J. H. (2017). Cornerstone of modern statistics. Korean Journal of Anesthesiology, 70(2), 144–156. https://doi.org/10.4097/kjae.2017.70.2.144
LeBel, E. P., McCarthy, R. J., Earp, B. D., Elson, M., & Vanpaemel, W. (2018). A unified framework to quantify the credibility of scientific findings. Advances in Methods and Practices in Psychological Science, 1(3), 389–402. https://doi.org/10.1177/2515245918787489
Mahat, D., Neupane, D., & Shrestha, S. (2024). Quantitative research design and sample trends: A systematic examination of emerging paradigms and best practices. Cognizance Journal of Multidisciplinary Studies, 4(2), 20–27. https://doi.org/10.47760/cognizance.2024.v04i02.002
Marshall, G., & Jonker, L. (2010). Inferential statistics primer. Unpublished manuscript.
Mascha, E. J., & Vetter, T. R. (2018). Significance, errors, power, and sample size: The blocking and tackling of statistics. Anesthesia & Analgesia, 126(2), 691–698. https://doi.org/10.1213/ANE.0000000000002741
Morris, R. (2019). Experiential learning—a systematic review and revision of Kolb’s model. Interactive Learning Environments, 28(8), 1066–1081. https://doi.org/10.1080/10494820.2019.1570279
Musterman, M., & Placeholder, P. (2018). Applications of inferential statistical methods in library and information science: What is so different about neuroenhancement? Data and Information Management, 2(2), 103–120. https://doi.org/10.2478/dim-2018-0007
Newbury, J. (2011). A place for theoretical inconsistency. International Journal of Qualitative Methods, 10(4), 335–347. https://doi.org/10.1177/160940691101000403
Ragab, M. A., & Arisha, A. (2017). Research methodology in business: A starter’s guide. Management and Organizational Studies, 5(1), 1–10. https://doi.org/10.5430/mos.v5n1p1
Rahman, M. M., Tabash, M. I., Salamzadeh, A., Abduli, S., & Rahaman, M. S. (2022). Sampling techniques (probability) for quantitative social science researchers: A conceptual guidelines with examples. SEEU Review, 17(1), 42–51. https://doi.org/10.2478/seeur-2022-0023
Ramsey, M. H. (1998). Sampling as a source of measurement uncertainty: Techniques for quantification and comparison with analytical sources. Journal of Analytical Atomic Spectrometry, 13(2), 97–104. https://doi.org/10.1039/a706815h
Sawers, A., & Hahn, M. E. (2013). Gradual training reduces practice difficulty while preserving motor learning of a novel locomotor task. Human Movement Science, 32(4), 605–617. https://doi.org/10.1016/j.humov.2013.01.002
Sharma, L. R., Jha, S., Koirala, R., Aryal, U., & Bhattarai, T. (2023). Navigating the research landscape: A guide to the selection of the right research design. International Research Journal of MMC, 4(1), 64–78. https://doi.org/10.3126/irjmmc.v4i1.51863
Sürücü, L. (2020). Business & management studies. Business and Management Horizons, 3(1), 2694–2726.
Susanti, E., Ladjin, N., & Qadrini, L. (2021). Buku ajar statistika untuk perguruan tinggi. Penerbit Adab.
Tobi, H., & Kampen, J. K. (2018). Research design: The methodology for interdisciplinary research framework. Quality & Quantity, 52(3), 1209–1225. https://doi.org/10.1007/s11135-017-0513-8
Zhao, A., & Ding, P. (2022). Regression-based causal inference with factorial experiments: Estimands, model specifications and design-based properties. Biometrika, 109(3), 799–815. https://doi.org/10.1093/biomet/asab051
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Naily Kamaliah, Alpha Fadila Juliana Rahman, Sutrisno Heru Sukoco, Dwi Indah Rahayu
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Authors who publish in this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- The journal allows the authors to hold the copyright without restrictions and to retain publishing rights without restrictions.
- Authors can enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
