Natural language prompts are changing how we ask for music recommendations. Users want to say things like, "Recommend some old classics for slow dancing?". But traditional LLMs often just generate song titles, which has drawbacks like needing extra steps to find the actual track and being inefficient.

Introducing Text2Tracks from Spotify! This novel research tackles prompt-based music recommendation using generative retrieval. Instead of generating titles, Text2Tracks is trained to directly output relevant track IDs based on your text prompt.

A critical finding is that how you represent the track IDs makes a huge difference. Using semantic IDs derived from collaborative filtering embeddings proved most effective, significantly outperforming older methods like using artist and track names. This approach boosts effectiveness (48% increase in Hits@10) and efficiency (7.5x fewer decoding steps).

While developing effective ID strategies was a key challenge explored, Text2Tracks ultimately outperforms traditional retrieval methods, making it a powerful new model particularly suited for conversational recommendation scenarios.

Paper link: https://arxiv.org/pdf/2503.24193

Generate 200 words description for my podcast. The description should convey the novelty, limitation and applications. Include organization name. Description should be helpful, relevant. Include paper link at the bottom

Explore MEGABYTE from Meta AI, a novel multi-scale transformer architecture designed to tackle the challenge of modelling sequences of over one million bytes. Traditional large transformer decoders scale poorly to such lengths due to the quadratic cost of self-attention and the expense of large feedforward layers per position, limiting their application to long sequences like high-resolution images or books.

MEGABYTE addresses this by segmenting sequences into patches, employing a large global model to process relationships between patches and a smaller local model for prediction within patches. This design leads to significant advantages, including sub-quadratic self-attention cost, the ability to use much larger feedforward layers for the same computational budget, and improved parallelism during generation. Crucially, MEGABYTE enables tokenization-free autoregressive sequence modelling at scale, simplifying processing and offering an alternative to methods that can lose information or require language-specific heuristics.

The architecture demonstrates strong performance across various domains, competing with subword models on long context language modelling, achieving state-of-the-art density estimation on ImageNet, and effectively modelling audio from raw files. While promising, the current experiments are conducted at a scale below the largest state-of-the-art language models, indicating that future work is needed to fully explore scaling MEGABYTE to even larger models and datasets.

Learn how MEGABYTE is advancing the frontier of efficient, large-scale sequence modelling.

[https://proceedings.neurips.cc/paper_files/paper/2023/file/f8f78f8043f35890181a824e53a57134-Paper-Conference.pdf]