Artifact Evaluation: Tips for Authors

A number of software research conferences such as ICSE, ISSTA, PLDI, POPL, OOPSLA, SOSP, and USENIX Security incorporate an artifact evaluation (AE) process: authors of (conditionally) accepted papers can optionally submit their tools, code, data, and scripts for independent validation of the claims in their paper by an artifact evaluation committee (AEC). Papers with accepted artifacts get stamped with one or more badges. Personally, I’m a big fan of the AE process, as it promotes reproducible and reusable research. But what makes a good artifact? Although there exist many great resources for writing good papers, as well as for writing good rebuttals, I haven’t found anything similar for submitting artifacts for evaluation. This is my attempt at filling the gap.