ARC 1 was originally released in November 2019, and was the subject of a public Kaggle competition with a $20,000 prize in early 2020. The contest ended with the best entrant solving 20.6% of test tasks. Three years later, in 2023, the state-of-the-art on ARC 1 is around 30%.
ARC 1 featured 1,000 tasks split between 400 generally easier training tasks, 400 validation tasks, and 200 tasks forming the private test set. Half of the private test set was used as the test set in the original Kaggle competition.
ARC 2 is an ambitious extension of ARC that aims to address certain weak points of ARC 1. While every task in ARC 1 was unique, ARC 1 wasn't as diverse as it ideally would be: some types of tasks (e.g., "fill the empty patch in this patterned grid") were reused more than once and were occasionally reused across different dataset splits.
ARC 2 is intended to be larger and more diverse than ARC 1, with no specific task type used more than once. ARC 2 will consist of tasks created by a huge community and with a great diversity.
While ARC 1 was the work of a single person, ARC 2 is intended to be the work of many: to ensure scale and task diversity, we are opening up the task-creation process to the crowd. You can create new tasks using the public ARC editor and submit them for inclusion in ARC 2. We look forward to as many contributions as possible and appreciate your submissions. ARC 2 is hosted by François Chollet and Lab42. If you have any questions, feel free to contact us any time.
Once the ARC 2 set is created, we will run a competition on ARC 2. The goal of the competition is to develop an AI that is able to solve over 90% of the tasks in the private test set, without advance knowledge of the tasks. We intend to run the competition until the goal is achieved. This time not only will there be prize money, but we will also engrave the winners' names for eternity (or almost), and the winners will be invited on a memorable trip to Davos, Switzerland.
The competition is hosted by François Chollet and Lab42. In case of any questions, please feel free to contact us at any time.
Home
Playground
Task Editor