LP-MusicDialog: Intent-Grounded Dialogue Generation

Task Generate large-scale, human-like conversations for training conversational music retrieval systems.

Goal Address the scarcity of public multi-turn music discovery data while covering diverse listener intents and musical attributes.

Method LP-MusicDialog derives taxonomies of user intents, system actions, and musical attributes from human dialogues, constructs coherent attribute sequences through cascading music database filtering, and uses an LLM to generate the corresponding utterances. The work releases a public synthetic dataset of about 288,000 dialogues linked to more than 390,000 tracks.

TalkPlay: Generative Multimodal Retrieval

Task Generate both a music recommendation and a natural-language response from a multi-turn dialogue.

Goal Unify retrieval and dialogue generation in one model while preserving long conversational context.

Method TalkPlay reformulates recommendation as next-token generation. Its multimodal music tokenizer converts playlist signals, tags, metadata, lyrics, and audio into discrete music tokens added to the LLM vocabulary, allowing one model to generate retrievable music identifiers and conversational responses in a single end-to-end process. The paper also introduces TalkPlayData 1, a synthetic multi-turn conversation dataset for training this unified system.

Task Create realistic training and evaluation conversations for multimodal music recommendation.

Goal Provide diverse conversations grounded in listener profiles, discovery goals, and multimodal music information.

Method TalkPlayData 2 is generated by an agentic pipeline with four specialized multimodal LLM roles: Profile, Goal, Listener, and Recommender. Their asymmetric information access simulates a realistic discovery process instead of scripting the full trajectory in advance.

Task Choose and execute the retrieval operations needed to satisfy each conversational request.

Goal Move beyond a single retrieval method toward a flexible, production-oriented recommendation pipeline.

Method TalkPlay-Tools introduces LLM tool calling as a unified retrieval-and-reranking controller. The LLM plans tool types, execution order, and arguments across structured filtering, lexical search, multimodal similarity, personalization, and generative retrieval, then uses the retrieved tracks to produce a grounded response.