Pioneering work undertaken in 2006 by a team including two current members of the Australian National University (ANU) Research School of Computer Science has been recognised for its significant influence in the programming languages community.

Professor Steve Blackburn and Professor Tony Hosking from the ANU, along with former ANU PhD students Robin Garner and Daniel Frampton collaborated with researchers from eight institutions to author The DaCapo benchmarks: Java benchmarking development and analysis paper in 2006. 

The paper has been awarded the Object Oriented Programming, Systems, Languages and Applications (OOPSLA) Most Influential Paper of 2006 Award by the Association for Computing Machinery’s Special Interest Group on Programming Languages (ACM SIGPLAN) on Wednesday 2 November 2016.

The paper outlines a description of best practices for using benchmarks to evaluate Java system performance, as well as an approach for evaluating whether a suite of benchmark applications is appropriate and effective.

“This paper set the standard for the benchmarking of Java,” said Professor Blackburn.

“The work is distinguished in that the benchmarks have become a standard used by both academics and industry. They are ‘real world’ - representative of everyday applications, such as search engines. They are also open source programs, allowing researchers to study the code” added Professor Hosking.

The paper has received over 1000 citations, making it among the most cited papers in the 31-year history of OOPSLA.

“The paper enabled a quantum shift in the way these systems were evaluated; it enabled the evaluation of new ideas and systems in a much more standardised way.”

Professor Blackburn acknowledged that developing new methodology on a large scale, and managing people across eight institutions was no mean feat for the DaCapo team, but their hard work paid off.

“Many thousands of people hours went into this paper – as a group, I think we were a model for how to undertake large-scale collaborative research across institutions.”

“What resulted was a culture change within the programming languages community; we changed an out-dated way of thinking.”

Both professors view the award as a career highlight.

“We did work that not a lot of people wanted to do. It was really gratifying that we got such a big response – this paper has facilitated lots of amazing research,” said Professor Hosking.

“This paper received lots of traction and credibility; it is fantastic to see that it has been used by industry and academia alike and is being recognised for the impact it has had over the past decade."

In addition to the Most Influential Paper of 2006 Award, Professor Hosking also received a Distinguished Paper Award at the 2016 OOPSLA Conference. The paper, Hybrid STM/HTM for Nested Transactions on OpenJDK, was a collaboration between ANU, Purdue University, and the University of Massachusetts.