Subscribe
بالعربي
Home » Abu Dhabi’s Technology Innovation Institute launches new AI challenge on crowdsourced human labeling methods

Abu Dhabi’s Technology Innovation Institute launches new AI challenge on crowdsourced human labeling methods

by Hadeer Elhadary

 The Technology Innovation Institute (TII), the applied research pillar of Abu Dhabi’s Advanced Technology Research Council (ATRC), has launched The CrowdLabel Challenge: Crowdsourcing LLM Feedback, an initiative focused on building new innovative methods for passive human labeling.

Large language models (LLMs) utilize Reinforcement Learning from Human Feedback (RLHF) to align intelligent agents with human preferences; however, employing human labelers can be expensive. The goal of this challenge is to identify a passive human labeling approach where the act of labeling a data sample is seamlessly integrated into another process or operation, allowing the human operator to perform the task without consciously realizing they are labeling data. Passive alignment methods can lower costs and be built into other tasks or daily activities like bot detectors.

TII has created an alignment samples database for use by LLM experts and researchers to give scalable and reliable feedback to their models. In this challenge, TII is looking for approaches to present these alignment datasets to broad and diverse audiences to label hundreds of thousands of data samples at scale and at a low cost.

The proposed solutions will be competitive if (i) they can show adaptability to multiple languages, but must be feasible in at least two languages, and (ii) have multimodal content of varying label types, (iii) on varying topics. The submissions will be evaluated based on the novelty of the approach, cost effectiveness, and quantity and quality of labels crowdsourced during an experimental launch. TII welcomes individuals, teams, startups, research institutes, and university students from around the world to propose and develop passive human labeling methods. These methods should enable human users to annotate data samples seamlessly, without necessarily realizing they are completing a labeling task.

The total award pool is $50,000, including $30,000 prize for first place. Submissions to the Developing Innovative Passive Human Labeling Methods for LLM Alignment Challenge must be received by February 28th, 2025.

You may also like