Design and development of web crawlers along with a virtual judge software application

Abstract

The ICPC (International Collegiate Programming Contest) club in NTU (Nanyang Technological University) hosts programming contests among their members to gauge their skills before selecting the best performing members for competitions.

The holding of a contest requires the ICPC mentors to refer the members to problems from external online judges that will allow the members to submit solutions. Subsequently, the mentors will manually consolidate the scores from the answers received. Additionally, there will be a consistent turnover of members and without a centralized management application to manage the ICPC members and teams, the tabulation of the scores and team management is often inefficient as it is time consuming and tedious. Therefore, the utilization of Virtual Judge was employed to alleviate these issues. However, due to the inadequacy of the application, there is a need to develop a new Virtual Judge from scratch.

The web application developed in this project, Virtual judge 2.0, aims to solve the issues faced with hosting programming contests and team managements issues faced by the ICPC mentors in NTU. Virtual Judge 2.0 employs web crawlers that were designed and developed in this project, to source for problems hosted by external online judges; the problems are then subsequently used for its contests. This project is managed by applying the software engineering and architectural techniques taught in the Software Engineering (CZ2006), Advanced Software Engineering (CZ3002) and Software Systems Analysis and Design (CZ3003) modules by NTU, and explores the use of a functional programming language to develop the application.

This report contains - requirements elicitation, software architectural design, database design, implementation details, and testing involved in the project. This project has been developed according to the elicited requirements, deployed onto a virtual desktop in NTU, and tested by the ICPC mentors and has yield positive feedbacks. As this is a deployed web application that will be used continually after project submission, it is recommended to perform updates regularly and to add more features that will help the ICPC mentors and members.