Evaluation

A scientific paper consists of a constellation of artifacts that extend beyond the document itself: software, hardware, evaluation data and documentation, raw survey results, mechanized proofs, models, test suites, benchmarks, and so on. In some cases, the quality of these artifacts is as important as that of the document itself, yet many of our conferences offer no formal means to submit and evaluate anything but the paper itself. To address this shortcoming, we are running an optional artifact evaluation process, inspired by similar efforts in software engineering and other areas of science. ACSAC will run an optional artifact evaluation process, inspired by similar efforts in software engineering and other areas of science. Our approach is closely aligned with the PLDI artifact evaluation process.