Battle of the Chatbots

February 8, 2017

Register Now

AI Games is an event for Artificial Intelligent platforms from all over the world to compete and determine which has the best ability to exhibit intelligent behavior. Each participant will prepare their system for the competition which will take place during AI Games panel on September 21st, 2017. All AI natural language question answering platforms are welcome to enter and compete.

Description of Competition

Phase A:

  • The judges will provide each contestant with a specific training set and database to train their system.
  • Each participant will be given a timeframe of 1 week (or as determined by judges) for training.

Phase B:

  • The participants will be evaluated by the judge based on specific metrics.
  • Official decisions made by the Judge will be based on testing protocols, procedures, guidelines, and rules set forth in the Competition.
  • If any of the competitions result in a tie, then the Judges shall use a predetermined and agreed upon mechanism to settle the tie.
  • Announcer will provide results during the Award Ceremony that takes place on September 21st, 2017.

Items that will be measured during testing of AI Games:

Duration of training: Each contestant will be provided with an agreed upon time frame to train their system. As the training is completed, a notification must be sent to the judge. The contestant that’s capable of training their platform quickest wins this round (Phase A).

Accuracy: Will be determined during testing (Phase B) of the competition. This will be measured based on the system’s accuracy in responses (lead by judges). The most accurate system gains point towards their advantage. Judges will do a comparison of responses for each platform.

Ease of Use: This will be measured based on how easy and user friendly the judge found the system to be. Ease of use referring to training, adding dialogs and other items per judge’s discretion.   

Evolution: Measured by ease which the system evolves and learns as new questions/requests arise. The platform that demonstrates to learn more quickly, gains points.

Timeline for the competition:

August 7th – August 25th – Submission period
September 4th – September 15th – Training period
September 21st– Competition
September 21st – Winners announced

Prizes to be unveiled on Sept 4th

The evaluation metric used for scoring and ranking will be displayed on the Competition Website

Agenda for Competition

*Time is in Eastern Savings Time

9:00am – 9:50am:
Opening Remarks. Competitor introductions and description of platforms.
10:00am – 10:50am:
Panel – AI Revolutions and Disruptions
11:00am – 11:30am:
Overview of AI Games terms and rules
11:30am – 12:30pm:
AI Tournament
12:30pm – 1:30pm:
Panel to discuss results and Award Ceremony
1:30pm – 2:00 pm:
Closing remarks

Guidelines for Chatbot Competition

Participants must do the following to compete:

  • Register for AI Games through the following link here
  • 1 registration per company
  • Enter your AI platform during Submission Period along with a video/demo
  • Overview of technology
  • All submissions must be in english
  • Complete onboarding of system with provided training set and variables.
  • Winning participants will be required to provide a complete written description of their methodology

By entering a Submission, you represent and warrant that all information you enter on the Website is true and complete to the best of your knowledge and that your submission:

  • is your own original work, or is used by permission, in which case full and proper credit and identify is given and the third party contributions are clearly identified within your Submission
  • does not violate or infringe upon the patent rights, industrial design rights, copyrights, trademarks, rights of privacy, publicity or other intellectual property or other rights of any person or entity
  • does not contain malicious code, such as viruses, timebombs, cancelbots, worms, Trojan horses or other potentially harmful programs or other material or information
  • does not and will not violate any applicable law, statute, ordinance, rule or regulation