Hosted by the IEEE International Conference on Software Maintenance and Evolution (ICSME 2018)
The DySDoc3 workshop will host the First Software Documentation Generation Challenge (DocGen).
Documentation is an essential component of the software development process, yet efficiently writing and maintaining high-quality documentation is extremely challenging. Automatically-generated On-Demand Developer Documentation (OD3) has the potential to make information on software more accessible to developers and cheaper to produce.
The third version of the DySDoc workshop (following DySDoc1 and DySDoc2) will host the First Software Documentation Generation Challenge (DocGen), which aims to spur research on automation of software documentation generation. Participation in the challenge involves developing a prototype tool and entering it in a competition. The competition will feature a live, juried tool evaluation, presentations of the best approaches, and a poster session. The extended summaries describing the entries will be published in the ICSME proceedings. The best entries will be recognized with awards.
The challenge is to build an automated system that can create, on-demand, reference documentation for a Java class. The documentation should address, as best as possible, the information needs of an experienced programmer who is considering using the class as part of a software development task. The documentation will provide information helpful to understand the design of the class and how to use it, among others. Examples of the types of information that the document could integrate, include: key methods, usage constraints, usage examples, the role of the class in applicable design patterns, known issues and limitations, etc.
An entry to the 2018 DocGen Challenge is a program that takes as input the fully qualified name of a class from the Apache POI project and generates an output that contains the reference documentation for the input class in html format.
The target system for this challenge is the Apache POI project, version 3.17-final (r31807825, commit 219dff00e6, on Sept. 8, 2017). The source code of the project can be found on Github. The project's main contributors are aware of the challenge and willing to provide information to participants if needed.
The document generator may rely on the following (and any other) sources of information:
We provide a preprocessed dataset of some the above information, which includes the following:
Contestants are free to use these preprocessed data sets, develop their own tools to process the original information found in the repositories, or use a combination of both. The solution can involve human-produced intermediate data and annotations. However, any manual input to the generation process must be clearly detailed. Entries that focus on one aspect of the reference documentation are explicitly encouraged, in particular if they are novel and innovative.
Interested teams of participants can enter the challenge in one of two categories: challenge or exhibition. The Challenge category will only accept entries that address the specific 2018 DocGen Challenge. The summaries of the challenge entries will be published in the ICSME proceedings. The Exhibition category will accept any demonstration of documentation generation technology, for any programming language, technology, or types of documents. Entries in the Exhibition category will be invited to present at the workshop, but will not be evaluated as part of the competition. The purpose of the exhibition category is to showcase state of the art in documentation generation tools, which may inspire the following edition of the DocGen Challenge.
Participation in the 2018 DocGen Challenge involves two phases: the qualification and the competition. All participants in both categories must qualify. All qualifying entries will be invited to the DySDoc workshop.
To qualify for participation in the challenge or exhibition, participants must submit a 2-page description of the key idea(s) behind their document generator. The description should contain the following information:
All submissions must be formatted according to the ICSME Formatting Instructions and must be submitted through EasyChair.
Each submission will be evaluated by the program committee.
On the day of the workshop, the qualified challenge entries will be evaluated live. A list of target class names will be provided to the competitors, who will use their tools to generate the respective reference documentation. For each input class, the reference documentation must be generated in less than 2 minutes. The generated documents will be evaluated by a panel of judges and by the workshop attendants based on:
The best entries will receive awards in several categories, such as: best tool, best presentation, people's choice, etc.
On the day of the workshop, the qualified exhibition entries will be presented to the audience, not as part of the challenge. See details in the program.
July 20, 2018 July 21, 2018 - Qualifier submissions
July 27, 2018 - Notifications of qualifiers
August 3, 2018 - Camera-ready version for accepted challenge entries
September 25, 2018 - DySDoc3 workshop
The workshop registration is free, generously provided by ICSME 2018. At least one author of each qualified entry must register, attend the workshop, and present the entry.
Note that the ICSME early bird registration ends on July 15, 2018!
DysDoc3 attendance is free for ICSME, SCAM and VISSOFT participants, but it is mandatory to specify attendance in the registration form.
The DysDoc participants who will not attend ICSME/VISSOFT/SCAM must notify the DysDoc organizers via e-mail that they will participate by July 28, 2018
- Automatically Redocumenting Source Code with Method and Class Stereotypes (Challenge winners - Usefulness category) Drew Guarnera, Michael L. Collard, Natalia Dragan, Jonathan I. Maletic, Christian Newman, and Michael Decker
- Enriching API Documentation by Relevant API Methods Recommendation Based on Version History Yuu Arimatsu, Yoshiya Ishida, Kunihiro Noda, and Takashi Kobayashi
- Catalogen: Generating Catalogs of Code Examples Collected from OSS (Challenge runner-ups - Usefulness category) Daiki Takata, Abdulaziz Alhefdhi, Maipradit Rungroj, Hideaki Hata, Hoa Khanh Dam, Takashi Ishio, and Kenichi Matsumoto
- A Timeline Summarization of Code Changes Michael J. Decker, Christian D. Newman, Michael L. Collard, Drew T. Guarnera, and Jonathan I. Maletic
- Automatic Generation of API Documentations for Open-Source Projects (Challenge runner-ups - Comprehesiveness category, and Audience Award winners) Xin Peng, Yifan Zhao, Mingwei Liu, Fengyi Zhang, Yang Liu, Xin Wang, and Zhenchang Xing
- Generating an Interactive View of Dynamic Aspects of API Usage Examples Yoshiya Ishida, Yuu Arimatsu, Lyu Kaixie, Go Takagi, Kunihiro Noda, and Takashi Kobayashi
- DynaDoc: Automated On-demand Context-Specific Documentation (Challenge winners - Comprehesiveness category) Ahmed Tamrawi, Sharwan Ram, Payas Awadhutkar, Benjamin Holland, Ganesh Ram Santhanam, and Suresh Kothari
- Automatically Generating Natural Language Documentation for Methods Christian Newman, Natalia Dragan, Michael L. Collard, Jonathan Maletic, Michael Decker, Drew Guarnera, and Nahla Abid
- Software Explanation Composer Alex Tao, Mahsa Roodbari, Truong Ho Quang, Rodi Jolak and Michel R.V Chaudron
- WikifyDocs: Addressing Ambiguity through Definitions in API Documentation (Audience Award runner-ups) Andrea Chamorro and Laura Moreno
- Towards an Automatic Extraction of Documentation from Unit Tests Mathieu Nassif and Martin Robillard
- AutoDoc: Software Functional Documentation based on Stack Overflow Data Zixiao Zhu, Yanzhen Zou and Bing Xie
DySDoc3 will take place at the conference room of the IMDEA Software Institute (Address: UPM Campus Montegancedo, 28223, Pozuelo de Alarcón, Madrid, Spain).
The IMDEA Software Institute will host the ICSME pre-reception, together with the ICSME tool demos and posters, right after DySDoc3.
Buses will take DySDoc3 participants to venue from the city center.
Bus pick-up instructions: we will meet at 8:45 am local time (Sept. 25), at the entrance of the Círculo de Bellas Artes (Address: Calle del Marqués de Casa Riera, 2). From there, we will walk to the bus pick-up place (6-min walk).
Likewise, in the evening, buses will bring the participants back to the city center. More details will come shortly.