Review of 'Enabling interdisciplinary research capacity for sustainable development: Self- evaluation of the Blue Communities project in the UK and Southeast Asia'

This review focuses on the preprint entitled ‘Enabling interdisciplinary research capacity for sustainable development: Self-evaluation of the Blue Communities project in the UK and Southeast Asia’, authored by Fiona Culhane, Victoria Cheung and Melanie Austen [1].

Firstly, I commend the authors for meticulously reviewing my feedback [2] and the other reviewer’s [3]. They have earnestly engaged with points they deem useful and necessary, significantly enhancing the quality of research dissemination by appropriately revising the paper. Notably, the analysis section shows a noticeable improvement from the previous version, rendering it even more valuable.

Below, I present my thoughts on the revised version [1]. I conducted the review while adhering to the ‘General Factors Ratings’ outlined in the journal’s peer-review guidelines. I took note of each aspect: (a) Level of importance, (b) Level of validity, (c) Level of completeness and (d) Level of comprehensibility. To illustrate the relationship between each point raised and its respective aspects, I have indicated the corresponding aspect (a–d) at the end of each comment. I have not classified these into ‘Major’ or ‘Minor’ categories.

I acknowledge that there might be points in this review where the authors might not necessarily agree. If there are instances due to my misunderstanding or misinterpretation, there is no need to consider those. The decision on whether the authors should address my suggestions (and, if addressed, whether it is done appropriately) will be left to the overseeing editor unless specifically requested otherwise. Even if the overseeing editor deems the current version acceptable for publication without further revision, I have no objection.

A) Comments on the study’s validity and ‘Discussion’

1. It is noted that responses were received from 56 out of approximately 115 individuals (line 165). It would be beneficial to understand the profile of the non-respondents, including information such as their country/region and sector involvement (academic, NGO, governmental, etc.). As this survey targets project participants, this information should be reasonably ascertainable. This clarification is crucial for discussing the validity of conclusions drawn from the responses of these 56 individuals. For instance, if the other approximately 55 non-respondents hold significantly different perceptions or inclinations, it could influence the assessment of the BC project, consequently affecting broader implications and discussions in line with those changes. Non-response is inevitable and typical in this type of survey research, but if there are aspects of sampling bias that can be discussed from the profile of non-respondents, they should be addressed. [a, b, c]
2. Regarding the discussion on the ‘limitations’ in Section 4.3, further elaboration on the potential impacts or biases arising from these limitations on the outcomes and analyses of this study would be beneficial. [a, b, c]
3. Regarding lines 613–614, the authors have not specifically detailed the ‘key lessons’ they have gleaned and how they intend to ‘modify’ or ‘enhance’ their approach. This point is crucial in this kind of survey research, and a more specific discussion would be beneficial. A structured subsection like ‘Lessons learned and implications for future projects’ could aid clarity. Related discussions are also found around lines 545–546. [a, c, d]
4. There needs to be more clarity regarding the relationship between the ‘Discussion’ (Section 4) and the ‘Conclusion’ (Section 4.4). For instance, the content discussed towards the end of the ‘Conclusion’ section, highlighting issues within the current academic system, is not entirely evident if it directly derives, with persuasive evidence, from the survey conducted or is a kind of ‘Discussion’ segment. It would benefit from further consideration. [b, c, d]
5. In line with the previous point and echoing comments [2] on the Version 1 preprint, it is essential to consistently acknowledge the limited scope of insights derived from the responses of these 56 individuals. The scope of conclusions drawn from this data is inherently limited, whether deductively or inductively. Caution must be exercised in drawing conclusions regarding the success or evaluation of the BC project, let alone in making definitive assertions about broader aspects like the nature of ‘interdisciplinary research’ or the ‘academic system’. Exhaustive discussion on internal and external validity is crucial to elevate the paper’s quality. Ensuring a robust discussion and adequately substantiating these aspects within the paper is essential to presenting a compelling argument. [b]
6. As demonstrated in the current manuscript, combining quantitative and qualitative approaches is commendable and recommended. These two approaches should complement each other judiciously to address their respective limitations. While Section 4 shows instances where this combination is aptly done, there still seems to be an inclination towards an episode-driven discussion overall. Presumably, the authors aim to introduce discourse heard from participants and prior studies as corroboration or support after establishing quantitative evidence from the survey. At times, this intent might not be entirely clear. In this regard, two points are noted:
   • The frequent use of ‘most’ (respondents/markers/levels/parameters), echoing previous review comments [2], could benefit from specifying figures (‘how many out of how many’ and/or ‘%’). Alternatively, referencing a corresponding table might aid in maintaining the link between the quantitative analysis and qualitative discussion more appropriately. Especially with ‘most respondents’, it is crucial to scrutinise whether it genuinely applies to the population under consideration for deriving various conclusions and implications. [c, d]
   • Additionally, there appear to be instances where individual anecdotes or specific participant remarks more or less determine the overall project evaluation (success) or situation, not necessarily aligning effectively with the quantitative analysis results (typical instances being lines 473–479). Reassessing the appropriateness of such descriptions is advisable. Distinguishing between success experiences or lessons learned at an individual respondent level (obtained from personal responses) and those pertinent to the project as a whole (derived or inferred from statistical approaches) while effectively integrating them in discussions would be advantageous. [b, c]

B) Comments on the presentation of analysis results

1. The arrangement of the bar graphs in Figures 1 to 5 needs more clarity in their sequence from top to bottom. Being clear about the order of the bars is fundamental. Usually, this information is discernible from the figures but is unclear in the current manuscript. [d]
2. A minor point regarding Figures 3 (b) and 4 (b) is that reconsidering the horizontal bar graph domain might be advisable. If my understanding of this bar graph is correct, the graph scale extends to a theoretically unattainable upper limit, which might not be scientifically reasonable. [d]
3. Figures 7 to 9 offer a clearer and significantly enhanced visual representation, marking the most noticeable improvement compared to the Version 1 preprint. To enhance the clarity of these diagrams, the following point should be noted. Line 157 mentions the ‘trend line’, referring to two line segments in each panel of Figures 7 to 9. However, it is not evident what the slope and intercept of these lines signify. An explanation of this point would help readers to understand how these relate to the displayed ‘R’ values. (Note that the ‘R’ value does not represent the slope of the ‘trend line’.) [d]
4. Although unnecessary to include in the paper, as previously suggested in the review [2] of the Version 1 preprint, the overlaying histograms for Southeast Asia and the UK corresponding to Figures 7 to 9 for each variable could provide useful insights. If not already attempted, it is worth exploring as it would reveal additional insights not visible from scatterplots.
5. The authors employ Fisher’s exact test regarding hypothesis testing. Note that Fisher’s exact test is valid when both row and column totals are fixed by design. However, considering the conclusions and discussions the authors aim to draw, it appears they seek to generalise beyond the 56 respondents’ group. Providing clarification on the intended ‘population’ for drawing conclusions and assessing whether the use of Fisher’s exact test remains sufficiently appropriate would be beneficial. [b, c]
6. Lines 311–313, 347–349 and 383–386 conclude that ‘[…] improved from a lower success or skill level to achieve the same success or skill level that UK respondents/teams/institutions started the project with’. However, this assessment might not be entirely suitable. From the questionnaire, it appears respondents from Southeast Asia evaluated changes in their success or skill levels primarily concerning their own past, not necessarily making direct comparisons with the success or skill levels of UK respondents. [b]

C) Miscellaneous

1. The data and its metadata on the open repository appear to remain unchanged. It would be advisable to ensure their update coincides with the publication of the latest version. [c, d]
2. Accruing experience in critically analysing research from diverse perspectives is crucial in academia and society. This belief aligns with the asserted importance of interdisciplinary capacity building highlighted in the authors’ manuscript. I hope that an open platform like UCL Open: Environment, where diverse opinions are exchanged, and various perspectives are considered, fosters a culture that promotes interdisciplinary collaboration. I desire the authors’ paper to be appropriately published and contribute to the advancement of social sciences and better academia-society relationships as envisioned by the journal.

Keisuke Okamura

Washington D.C., USA

17^th December 2023

References

[1] Culhane, Fiona E. and Cheung, Victoria and Austen, Melanie (2023). Enabling interdisciplinary research capacity for sustainable development: Self-evaluation of the Blue Communities project in the UK and Southeast Asia, 2017-2021. https://doi.org/10.14324/111.444/000189.v2

[2] Keisuke Okamura (2023). Review of ‘Growing interdisciplinary research capacity for sustainable development: Self-reported evaluation’. https://doi.org/10.14293/S2199-1006.1.SOR-SOCSCI.APE1TG.v1.RRZRYX

[3] Carla-Leanne Washbourne (2023). Review of ‘Growing interdisciplinary research capacity for sustainable development: Self-reported evaluation’. https://doi.org/10.14293/S2199-1006.1.SOR-SOCSCI.AHPMPZ.v1.RMKJUG