Policies on Generative AI, LLMs, and Related Tools

The advent of generative AI, large language models, and related technologies have created many opportunities for our community, both in support of new kinds of research and in helping improve writing. Along with the opportunities come some challenges, particularly regarding the appropriate use of these technologies.

Using generative AI to support writing papers

The ACM Policy on Authorship indicates that

Rather than attempt to limit the use of Artificial Intelligence (AI) to conduct research or report on the results of that research by placing expectations on authors to disclose all uses of large language models in their Works, this updated Policy attempts to set clear expectations for their responsible use, as follows:

1. When using Artificial Intelligence to conduct research, including the design and methodology of the research project, creation and selection of data sources, designing experiments, generation and collection of data, coding, implementing models, running simulations, data analysis, testing, validating results, deploying software, archiving data and code for reproducibility, or any other aspects of the research lifecycle that are directly relevant to the conclusions of the research underlying the Work, the specific use(s) of AI tools must be described in detail in the methods section of the Work. This includes the creation of artifacts that are directly relevant to the conclusions of the research, such as code, datasets, and charts or figures that rely on the AI tools.

2. When using Artificial Intelligence to assist with writing an ACM submission, ACM no longer requires the disclosure of information regarding the use of AI (as distinct from AI used in the conduct of the research itself, addressed in item 1 above).

All named authors on an ACM submission will be held responsible and accountable for any problematic content contained in the submission regardless of the source of that problematic content:

1. In the event content integrity issues stemming from the use of AI during authorship are identified prior to publication or posting in the ACM Digital Library, ACM reserves the right to reject submissions in their entirety and impose additional penalties.

2. In the event content integrity issues stemming from the use of AI during authorship are identified after publication or posting in the ACM Digital Library, ACM reserves the right to retract the published Work in its entirety.

We expect authors of SIGCSE TS papers to abide by these—and all—ACM guidelines. In particular, we note that the presence of “hallucinated references” will be considered grounds for desk rejection.

Generative AI as a research and teaching tool

Many SIGCSE TS authors are exploring possible roles of generative AI in their research and teaching. In some cases, questions have been raised about the appropriateness of using copyrighted work, such as problems from a course’s web site or a service like HackerRank. Since submitting copyrighted material to an LLM may make that material available as training data, and since the products of an LLM trained on such materials may be considered derivative works, authors of papers that involve the use of copyrighted works as input to or training data for LLMs must obtain explicit permission from the copyright holders to use the works. Authors must disclose such permission in the acknowledgements section of the paper.

We expect that reviewers who encounter situations in which an LLM or other tool is trained on copyrighted (or otherwise sensitive) data will verify that the acknowledgements section indicates that permission was sought and obtained.

Generative AI in the review process

Reviewers must not submit papers to LLMs, plagiarism detectors, summarizers, or other such tools. As the ACM Peer Review Policy indicates,

Reviewers may not upload confidential submissions, technical approaches described by authors in their submissions, or any information about the authors into any system managed by a third party, including LLMs, that does not promise to maintain the confidentiality of that information by reviewers, since the storage, indexing, learning, and utilization of such submissions may violate the author’s right to confidentiality.

Obviously, reviewers may not use generative AI tools to write their reviews. However, since “writing helpers”, broadly defined, are now incorporated within most major editors and word processors, reviewers may take advantage of such tools to polish the writing in their reviews.

Similarly, APCs may not use generative AI tools to synthesize the reviews and discussion into a metareview, but may use tools to improve the writing or structure of the metareview.

Checking citations

While we do not expect our reviewers to check every reference, we ask that reviewers pay additional attention to references cited. Reviewers want to consider checking some references in each paper they review.