Microsoft has announced a new collaboration with Be My Eyes, a move designed to infuse high-quality, disability-representative data into its AI training systems. This partnership is a crucial step in Microsoft’s ongoing commitment to developing inclusive technology, specifically targeting the 340 million people globally who are blind or have low vision.
AI models rely on extensive data to function effectively, yet datasets often underrepresent or miscategorize disability. Recent research from Microsoft highlighted that disability-related objects, such as braille devices, were less frequently included in popular image-text datasets, leading to a significant drop in accuracy when recognizing these objects. This ‘disability data desert,’ as it’s referred to, can limit a technology’s effectiveness, reinforce stereotypes, and exacerbate bias. The company believes that inclusive data, built through partnerships like this, is paramount to creating inclusive AI.
Transparency and user control are central to this data-sharing agreement. Be My Eyes will provide video datasets that capture unique objects, lighting, and framing to realistically represent the experiences of blind and low vision users. Personal information will be removed from the metadata, and Be My Eyes is committed to providing a transparent process that includes options for users to opt out of data sharing. Microsoft will then use the data to improve the precision and accuracy of scene understanding and descriptions, aiming to make AI applications more useful for the blind and low vision community.
This announcement builds on a pre-existing partnership that began in 2017 when the Be My Eyes app was integrated into the Disability Answer Desk to offer improved technical support. Last year, Microsoft pioneered the use of ‘Be My AI’ to solve technical support issues. Additionally, Be My Eyes has launched a Windows app now available in the Microsoft Store.
Microsoft recognizes Responsible AI as inclusive AI. The company has been focused on how generative AI impacts disabled people, guided by the Microsoft Responsible AI framework. This framework incorporates fairness, reliability and safety, privacy and security, transparency, accountability, and inclusiveness, including accessibility. AI innovations have the potential to reduce cognitive load, through methods such as summarization, automated meeting notes, and detailed image descriptions.
In addition, speech recognition is another avenue of investment. Since 2022, Microsoft has been collaborating with the University of Illinois Urbana-Champaign (UIUC), along with Apple, Google, Meta, and Amazon, on the Speech Accessibility Project. UIUC is working with nonprofit organizations to collect speech data to improve recognition for individuals with diverse speech patterns including those with ALS and other speech disabilities. This data helps AI systems better understand speech diversity and will help scenarios where this technology can bring benefits, including improving accuracy in mainstream speech platforms, such as Azure Speech.
Microsoft is dedicated to building inclusive AI that adequately represents all users while also protecting marginalized members of society from potential bias. The company believes accessible technology to be a fundamental right ready to unlock opportunities across society, and generative AI is a powerful tool to do just that, when designed inclusively and responsibly.
Microsoft is committed to transparency and data protection for all individuals. Today’s collaboration with Be My Eyes is a strong measure of this commitment. Microsoft will continue to seek out partnerships across the industry and with communities to build a more inclusive future.
