In a groundbreaking move, the Epica Awards, renowned for three decades as the premier creative accolade evaluated by journalists, introduced "AIJE," the Artificial Intelligence Jury Experiment—the initiative aimed to explore the capabilities of AI in scrutinising and comprehending creative concepts.
Conducted alongside the 2023 competition, the experiment's findings were kept distinct from the prominent awards, which enlisted over 150 human journalists as jurors. Mark Tungate, Editorial Director at Epica Awards, highlighted the experiment's lighthearted yet thought-provoking nature, responding to the pressing issue of AI in the creative domain.
The AI evaluation process focused on text descriptions provided by entrants, utilising a standardisation tool to distil ideas into concise formats. Nicolas Huvé, the creator of AIJE, emphasised the democratic nature of text-based assessment, aligning with the concept of an 'elevator pitch.'
Utilising the GPT-4 Turbo API, the AI processed entries alongside the Epica Awards' scoring scale, ensuring alignment with human judgment criteria. Scores and justifications were generated through 80 iterations, with results averaged using the interquartile range method to eliminate outliers.
The AIJE experiment showcased a modest correlation (correlation coefficient ~0.25) with human voting patterns. Notably, AI scores averaged 7.45, surpassing human scores at 6.60, highlighting a fundamental difference in evaluation approaches.
Huvé acknowledged the AI's tendency to be more easily impressed, contrasting the critical analysis of journalists who identified originality more profoundly. Despite this, AIJE demonstrated impartiality, free from human biases influencing scores.
Huvé cited a specific example, 'The X-Tinction Timeline' by McCann Worldgroup Germany, where AI perceived a powerful and market-leading campaign, while a human juror offered a nuanced perspective.
The experiment provides valuable insights into AI's potential role in assessing creativity. Future iterations will expand to more categories and incorporate visual elements, allowing AIJE to interpret entire case study videos. Huvé emphasised the commitment to a general AI approach, aligning with the evolving landscape of the creative industry.
For the 2024 Epica Awards, entrants automatically qualify for participation in the next AIJE iteration, promising continued exploration of AI's role in shaping the future of creative judgment.
