Loading...
Thumbnail Image
Publication

GrantCheck-an AI Solution for Guiding Grant Language to New Policy Requirements: Development Study

Shi, Qiming
Oztekin, Asil
Matthew, George
Bortle, Jeffrey
Jenkins, Hayden
Wong, Steven Koon
Langlois, Paul
Zaki, Anaheed
Coleman, Brian
Luzuriaga, Katherine
... show 1 more
Embargo Expiration Date
Abstract

BACKGROUND: Academic institutions face increasing challenges in grant writing due to evolving federal and state policies that restrict the use of specific language. Manual review processes are labor-intensive and may delay submissions, highlighting the need for scalable, secure solutions that ensure compliance without compromising scientific integrity.

OBJECTIVE: This study aimed to develop a secure, artificial intelligence-powered tool that assists researchers in writing grants consistent with evolving state and federal policy requirements.

METHODS: GrantCheck (University of Massachusetts Chan Medical School) was built on a private Amazon Web Services virtual private cloud, integrating a rule-based natural language processing engine with large language models accessed via Amazon Bedrock. A hybrid pipeline detects flagged terms and generates alternative phrasing, with validation steps to prevent hallucinations. A secure web-based front end enables document upload and report retrieval. Usability was assessed using the System Usability Scale.

RESULTS: GrantCheck achieved high performance in detecting and recommending alternatives for sensitive terms, with a precision of 1.00, recall of 0.73, and an F-score of 0.84-outperforming general-purpose models including GPT-4o (OpenAI; F=0.43), Deepseek R1 (High-Flyer; F=0.40), Llama 3.1 (Meta AI; F=0.27), Gemini 2.5 Flash (Google; F=0.58), and even Gemini 2.5 Pro (Google; F=0.72). Usability testing among 25 faculty and staff yielded a mean System Usability Scale score of 85.9 (SD 13.4), indicating high user satisfaction and strong workflow integration.

CONCLUSIONS: GrantCheck demonstrates the feasibility of deploying institutionally hosted, artificial intelligence-driven systems to support compliant and researcher-friendly grant writing. Beyond administrative efficiency, such systems can indirectly safeguard public health research continuity by minimizing grant delays and funding losses caused by language-related policy changes. By maintaining compliance without suppressing scientific rigor or inclusivity, GrantCheck helps protect the pipeline of research that advances biomedical discovery, health equity, and patient outcomes. This capability is particularly relevant for proposals in sensitive domains-such as social determinants of health, behavioral medicine, and community-based research-that are most vulnerable to evolving policy restrictions. As a proof-of-concept development study, our implementation is tailored to one institution's policy environment and security infrastructure, and findings should be interpreted as preliminary rather than universally generalizable.

Source

Shi Q, Oztekin A, Matthew G, Bortle J, Jenkins H, Wong SK, Langlois P, Zaki A, Coleman B, Luzuriaga K, Zai AH. GrantCheck-an AI Solution for Guiding Grant Language to New Policy Requirements: Development Study. JMIR Form Res. 2025 Nov 27;9:e79038. doi: 10.2196/79038. PMID: 41308189.

Year of Medical School at Time of Visit
Sponsors
Dates of Travel
DOI
10.2196/79038
PubMed ID
41308189
Other Identifiers
Notes
Funding and Acknowledgements
Corresponding Author
Related Resources
Related Resources
Repository Citation
Rights
©Qiming Shi, Asil Oztekin, George Matthew, Jeffrey Bortle, Hayden Jenkins, Steven (Koon) Wong, Paul Langlois, Anaheed Zaki, Brian Coleman, Katherine Luzuriaga, Adrian H Zai. Originally published in JMIR Formative Research (https://formative.jmir.org), 27.Nov.2025. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.