September 25, 2018, Madrid, Spain

Third International Workshop on
Dynamic Software Documentation (DySDoc3)

Hosted by the IEEE International Conference on Software Maintenance and Evolution (ICSME 2018)

The DySDoc3 workshop will host the First Software Documentation Generation Challenge (DocGen).


Documentation is an essential component of the software development process, yet efficiently writing and maintaining high-quality documentation is extremely challenging. Automatically-generated On-Demand Developer Documentation (OD3) has the potential to make information on software more accessible to developers and cheaper to produce.

The third version of the DySDoc workshop (following DySDoc1 and DySDoc2) will host the First Software Documentation Generation Challenge (DocGen), which aims to spur research on automation of software documentation generation. Participation in the challenge involves developing a prototype tool and entering it in a competition. The competition will feature a live, juried tool evaluation, presentations of the best approaches, and a poster session. The extended summaries describing the entries will be published in the ICSME proceedings. The best entries will be recognized with awards.


Call for submissions (PDF)

The challenge is to build an automated system that can create, on-demand, reference documentation for a Java class. The documentation should address, as best as possible, the information needs of an experienced programmer who is considering using the class as part of a software development task. The documentation will provide information helpful to understand the design of the class and how to use it, among others. Examples of the types of information that the document could integrate, include: key methods, usage constraints, usage examples, the role of the class in applicable design patterns, known issues and limitations, etc.

An entry to the 2018 DocGen Challenge is a program that takes as input the fully qualified name of a class from the Apache POI project and generates an output that contains the reference documentation for the input class in html format.

The target system for this challenge is the Apache POI project, version 3.17-final (r31807825, commit 219dff00e6, on Sept. 8, 2017). The source code of the project can be found on Github. The project's main contributors are aware of the challenge and willing to provide information to participants if needed.

The document generator may rely on the following (and any other) sources of information:

We provide a preprocessed dataset of some the above information, which includes the following:

  • All the commits of the project.
  • All the issues of the project.
  • The call graph of project methods and classes.
  • Inheritance tree of the project classes.
  • Stack Overflow posts.

Contestants are free to use these preprocessed data sets, develop their own tools to process the original information found in the repositories, or use a combination of both. The solution can involve human-produced intermediate data and annotations. However, any manual input to the generation process must be clearly detailed. Entries that focus on one aspect of the reference documentation are explicitly encouraged, in particular if they are novel and innovative.


Interested teams of participants can enter the challenge in one of two categories: challenge or exhibition. The Challenge category will only accept entries that address the specific 2018 DocGen Challenge. The summaries of the challenge entries will be published in the ICSME proceedings. The Exhibition category will accept any demonstration of documentation generation technology, for any programming language, technology, or types of documents. Entries in the Exhibition category will be invited to present at the workshop, but will not be evaluated as part of the competition. The purpose of the exhibition category is to showcase state of the art in documentation generation tools, which may inspire the following edition of the DocGen Challenge.

Participation in the 2018 DocGen Challenge involves two phases: the qualification and the competition. All participants in both categories must qualify. All qualifying entries will be invited to the DySDoc workshop.


To qualify for participation in the challenge or exhibition, participants must submit a 2-page description of the key idea(s) behind their document generator. The description should contain the following information:

  • Authors
  • Data sources used
  • Brief overview of technique(s) used
  • Preliminary results, including an HTML link to:
    • for the challenge entries only - the generated reference documentation for class XSSFWorkbook
    • for the exhibition entries only - samples of generated documentation produced by the tool

All submissions must be formatted according to the ICSME Formatting Instructions and must be submitted through EasyChair.

Each submission will be evaluated by the program committee.


On the day of the workshop, the qualified challenge entries will be evaluated live. A list of target class names will be provided to the competitors, who will use their tools to generate the respective reference documentation. For each input class, the reference documentation must be generated in less than 2 minutes. The generated documents will be evaluated by a panel of judges and by the workshop attendants based on:

  • Comprehesiveness: does the information provided cover different specific needs and variety of usage scenarios for the given class?
  • Correctness: is the information provided consistent with the source code and does not present contradictions?
  • Conciseness: is the information provided succint, relevant, and non-redundant?
  • Originality: does the information provided present new and original content?
  • Usefulness: does the information provided address common information needs and is critical for understanding the given class?
  • Presentation: is the information provided easy to read, understand, and navigate?

The best entries will receive awards in several categories, such as: best tool, best presentation, people's choice, etc.


On the day of the workshop, the qualified exhibition entries will be presented to the audience, not as part of the challenge. See details in the program.

Important dates

July 20, 2018 July 21, 2018 - Qualifier submissions

July 27, 2018 - Notifications of qualifiers

August 3, 2018 - Camera-ready version for accepted challenge entries

September 25, 2018 - DySDoc3 workshop


The workshop registration is free, generously provided by ICSME 2018. At least one author of each qualified entry must register, attend the workshop, and present the entry.

Note that the ICSME early bird registration ends on July 15, 2018!

DysDoc3 attendance is free for ICSME, SCAM and VISSOFT participants, but it is mandatory to specify attendance in the registration form.

The DysDoc participants who will not attend ICSME/VISSOFT/SCAM must notify the DysDoc organizers via e-mail that they will participate by July 28, 2018


Session 1: Welcome and Overview

General co-chairs


Session 2: Live competition

Laura Moreno & David Shepherd Challenge co-hosts


Each team will have 15 minutes to:

- Introduce themselves and the tool

- Generate the class documentation

- Present the results

- Answer judges questions


Session 3: Lightning presentations

Neil Ernst & Marco Gerosa Program co-chairs


- Challenge and exhibition presentations

- Informal demos and posters


Session 4: Awards and closing

General co-chairs

Competition entries

- Automatically Redocumenting Source Code with Method and Class Stereotypes (Challenge winners - Usefulness category)
Drew Guarnera, Michael L. Collard, Natalia Dragan, Jonathan I. Maletic, Christian Newman, and Michael Decker

- Enriching API Documentation by Relevant API Methods Recommendation Based on Version History
Yuu Arimatsu, Yoshiya Ishida, Kunihiro Noda, and Takashi Kobayashi

- Catalogen: Generating Catalogs of Code Examples Collected from OSS (Challenge runner-ups - Usefulness category)
Daiki Takata, Abdulaziz Alhefdhi, Maipradit Rungroj, Hideaki Hata, Hoa Khanh Dam, Takashi Ishio, and Kenichi Matsumoto

- A Timeline Summarization of Code Changes
Michael J. Decker, Christian D. Newman, Michael L. Collard, Drew T. Guarnera, and Jonathan I. Maletic

- Automatic Generation of API Documentations for Open-Source Projects (Challenge runner-ups - Comprehesiveness category, and Audience Award winners)
Xin Peng, Yifan Zhao, Mingwei Liu, Fengyi Zhang, Yang Liu, Xin Wang, and Zhenchang Xing

- Generating an Interactive View of Dynamic Aspects of API Usage Examples
Yoshiya Ishida, Yuu Arimatsu, Lyu Kaixie, Go Takagi, Kunihiro Noda, and Takashi Kobayashi

- DynaDoc: Automated On-demand Context-Specific Documentation (Challenge winners - Comprehesiveness category)
Ahmed Tamrawi, Sharwan Ram, Payas Awadhutkar, Benjamin Holland, Ganesh Ram Santhanam, and Suresh Kothari

- Automatically Generating Natural Language Documentation for Methods
Christian Newman, Natalia Dragan, Michael L. Collard, Jonathan Maletic, Michael Decker, Drew Guarnera, and Nahla Abid

Exhibition entries

- Software Explanation Composer
Alex Tao, Mahsa Roodbari, Truong Ho Quang, Rodi Jolak and Michel R.V Chaudron

- WikifyDocs: Addressing Ambiguity through Definitions in API Documentation (Audience Award runner-ups)
Andrea Chamorro and Laura Moreno

- Towards an Automatic Extraction of Documentation from Unit Tests
Mathieu Nassif and Martin Robillard

- AutoDoc: Software Functional Documentation based on Stack Overflow Data
Zixiao Zhu, Yanzhen Zou and Bing Xie


General co-chairs


Martin Robillard McGill University, Canada


Andrian Marcus The University of Texas at Dallas, USA


Christoph Treude University of Adelaide, Australia


Michele Lanza Università della Svizzera italiana, Switzerland

Program co-chairs


Neil Ernst University of Victoria, Canada


Marco Gerosa Northern Arizona University, USA

Challenge co-hosts


Laura Moreno Colorado State University, USA


David Shepherd ABB Corporate Research, USA

Data masters


Sarah Nadi University of Alberta, Canada


Shinpei Hayashi Tokyo Institute of Technology, Japan


Hideaki Hata Nara Institute of Science and Technology, Japan


Oscar Chaparro The University of Texas at Dallas, USA


James Clause University of Delaware, USA

Web and publicity


Takashi Kobayashi Tokyo Institute of Technology, Japan


Neil Ernst University of Victoria, Canada


Oscar Chaparro The University of Texas at Dallas, USA

Program committee

  • Christoph Treude
  • David Shepherd
  • Martin Robillard
  • Sarah Nadi
  • Laura Moreno
  • Andrian Marcus
  • Michele Lanza
  • Takashi Kobayashi
  • Shinpei Hayashi
  • Hideaki Hata
  • Marco Gerosa
  • Neil Ernst
  • James Clause
  • Oscar Chaparro

Panel of judges


Margaret-Anne Storey University of Victoria, Canada


Michael W. Godfrey University of Waterloo, Canada


Michel R. V. Chaudron University of Gothenburg, Sweden


DySDoc3 will take place at the conference room of the IMDEA Software Institute (Address: UPM Campus Montegancedo, 28223, Pozuelo de Alarcón, Madrid, Spain).

The IMDEA Software Institute will host the ICSME pre-reception, together with the ICSME tool demos and posters, right after DySDoc3.

Buses will take DySDoc3 participants to venue from the city center.

Bus pick-up instructions: we will meet at 8:45 am local time (Sept. 25), at the entrance of the Círculo de Bellas Artes (Address: Calle del Marqués de Casa Riera, 2). From there, we will walk to the bus pick-up place (6-min walk).

Likewise, in the evening, buses will bring the participants back to the city center. More details will come shortly.