1794571413-logo-v2
SIGN IN

Helping AI Understand the Real World

1750128659-英文logo带背景

Posted at 14小时 ago

On November 7, Google DevFest 2026 was held at Shanghai Jeurong Hotel . MagicHub.com and Magic Data joined the event on-site to meet and exchange ideas with developers.

1. The MagicHub Ambassador Program: Bring Your Ideas to the Table

At the event, many attendees showed interest in our Ambassador Program.

Previously, supported by the MagicHub Foundation, we launched this initiative to invite developers, researchers, and industry professionals to participate in community building, help more people discover and use high-quality datasets, and create opportunities for labs and enterprises to collaborate on dataset development.

We welcome everyone to bring their specific needs and ideas to explore new collaborations, whether it's jointly creating and developing datasets, using existing datasets to evaluate model capabilities, or exploring the design and release of benchmarks for specific tasks......

2. Discovering MagicHub: Starting with "What Is a Dataset?"

At our booth, we also met many people who were interested in open-source communities but were not yet familiar with datasets.

Take a speech dataset, for example. It typically contains audio recordings collected and organized for specific purposes, along with corresponding information such as transcriptions, speaker labels, and timestamps. Depending on their needs, developers can use these datasets to train models, fine-tune them, or evaluate their performance.

To address the challenges of data scarcity and difficulty in finding suitable datasets, Magic Data launched the MagicHub.com open-source community in April 2021, aiming to provide AI developers with free access to high-quality datasets.

So far, the community has released nearly 300 open-source datasets covering 50 languages and dialects, including Mandarin, Chinese dialects, and other languages. These datasets support a range of tasks, such as speech recognition, speaker identification, speech synthesis, and LLM fine-tuning. The resources are organized by language, task type, and other categories to help developers find datasets suited to their projects.

With so much public data already out there, why keep building new datasets?

Projects often have specific requirements: Do the languages and accents match? Are the recording environments close to real-world usage scenarios? Are the annotations complete? Do they capture the interaction patterns the model needs to learn?

By continuously making datasets available and organizing them, we hope to reduce the time developers spend finding, filtering, and processing data, allowing them to get started with their experiments more quickly.

3. Full-Duplex Data: How to Preserve the Details of Natural Conversation?

Full-duplex speech is becoming an important area of research and development in real-time AI interaction. Recently, Boson AI, co-founded by Mu Li, announced Higgs Realtime Duplex, further exploring interaction capabilities such as listening while speaking, handling interruptions, and executing tasks in the background. As models begin to learn when to respond, when to wait, and when to take a turn, speech data that preserves authentic interaction is also becoming an important foundation for research.

How full-duplex speech data is collected was another topic that attracted interest at our booth.

In everyday conversation, we may already respond with "Right, I've run into that too" before the other person has finished speaking. A pause does not necessarily mean that a speaker has finished, and a brief "uh-huh" may indicate that someone is listening or hesitating. Laughter, interruptions, and overlapping speech all play a role in how conversations unfold.

These subtle dynamics must be preserved during data collection and curation.

Take the open-source full-duplex conversation datasets on MagicHub.com as an example. Participants are preferably people who already know each other and are invited to talk freely about topics they are interested in. There are no scripted lines or strict turn-taking rules, allowing backchannels, interruptions, and laughter to occur naturally.

During recording, the two participants are in separate rooms and record their voices independently, while preserving the timing relationship between the two audio tracks. When both speakers talk at the same time, their speech can be found on their respective tracks. The accompanying transcripts and timestamps also record who said what and when.

Compared with conversations recorded sentence by sentence with strictly alternating turns and then stitched together, this approach preserves more of the natural rhythm of interaction: when a backchannel occurs during the other speaker's turn, whether a pause signals the end of a turn, and how the two speakers continue talking after an interruption.

This provides models with examples of how to take turns, wait, respond with backchannels, and handle interruptions, helping support more natural voice interaction.

4. Open Mic: Using Data to Address Specific Challenges

During the Open Mic session, Steven from the MagicHub open-source community shared our thoughts on speech data and community building.

The data a speech model needs is closely related to the problem it is designed to solve. Full-duplex interaction requires data that captures pauses, backchannels, and interruptions. In the context of in-vehicle speech, models also need to handle accents, noise, and speakers changing what they say mid-sentence. In a word, a dataset's scenario coverage and annotation methods influence how effectively it can support model development.

Thank you to everyone who stopped by our booth—we look forward to keeping the conversation going well beyond the event.

Our conversations in Shanghai have come to a close for now, but we look forward to continuing the discussion on full-duplex speech in Sydney.

05. See You in Sydney This December! The Full-Duplex Voice AI Meetup Is Coming Soon

This December, alongside NeurIPS 2026 in Sydney, MagicHub will host the Full-Duplex Voice AI Meetup!

We plan to bring together researchers, open-source framework creators, and developers working on full-duplex speech for an in-depth discussion of real-time voice interaction:

  • Open-Source Framework Sharing:

Creators of open-source full-duplex frameworks such as Sophon-Realtime will share their technical explorations and practical experience.

  • Data Methodology Sharing:

The Magic Data team will share their experience with full-duplex data annotation and multilingual dataset development.

  • Technical Discussions:

Discussions will focus on key challenges such as turn-taking control, interruption detection, paralinguistic modeling, and cross-lingual transfer.

  • Live Demos & Developer Networking:

Live demonstrations of real-time voice interaction technologies, creating opportunities for exchange and collaboration within the open-source community.

Registration is now open!

Fill out the registration form below to sign up for the Full-Duplex Voice AI Meetup.

👉 Registration Link:

https://wj.qq.com/s2/28695398/fzn1/

See you in Sydney this December!

Related Datasets

Datasets Download Rank

ASR-RAMC-BigCCSC: A Chinese Conversational Speech Corpus
Multi-Modal Driver Behaviors Dataset for DMS
ASR-SCCantDuSC: A Scripted Chinese Cantonese (Canton) Daily-use Speech Corpus
ASR-CCantCSC: A Chinese Cantonese (Canton) Conversational Speech Corpus
ASR-SCSichDiaDuSC: A Scripted Chinese Sichuan Dialect Daily-use Speech Corpus
ASR-SCCantCabSC: A Scripted Chinese Cantonese (Canton) Cabin Speech Corpus
ASR-CShhiDiaCSC: A Chinese Shanghai Dialect Conversational Speech Corpus
ASR-EgArbCSC: An Egyptian Arabic Conversational Speech Corpus
ASR-CSichDiaCSC: A Chinese Sichuan Dialect Conversational Speech Corpus