Can Claude listen to music? How to let it hear a song

Updated 12 November 2026

Not as a supported upload in claude.ai: the upload list has documents and images and no audio type, and Claude’s documented inputs are text and images. To let Claude “listen”, you turn the song into something it reads. The ways people do that are analysis numbers from an MCP server, a spectrogram picture, MIDI saved as JSON, another model’s description, or Listen Along, a free tool we make that shows Claude the song in order, 15 seconds at a time, in Claude Code.

Each costs something different: setup, tokens, or how much of the music gets through. This page walks through them, then how to start Listen Along. For how Claude compares with ChatGPT and Gemini, which do take audio in some form, see Can AI listen to music?

An AI drafted this page from the sources below and another AI checked it. No person has run these steps for this page. Listen Along’s two example listens are real, public runs. Listen Along and Counterparts are ours: Mike LaPeter runs Counterparts.

What the docs say for each place you might try

Attach an MP3 in claude.ai. Anthropic’s upload article lists PDF, DOCX, CSV, TXT, HTML, ODT, RTF, EPUB, JSON and XLSX, and JPEG, PNG, GIF and WebP images. Audio isn’t on it. We haven’t tested whether an audio file gets through with code execution turned on; one Reddit user wrote that it “could only analyze wave forms from .wav files”.
Play the song in voice mode. Voice mode turns your speech into text and deletes the recording, so Claude gets your words.
Point Claude Code at the file. Its Read tool lists images, PDFs and notebooks beyond plain text; audio isn’t listed. Claude Code can run a program that reads the audio, which is how Listen Along and the MCP servers below work.
Send it through the API. The models overview says current models “support text and image input”.

The ways to let Claude hear a song

Route	What Claude gets	Where it runs	What it costs
Listen Along	The song in order, 15 seconds at a time, as pictures of the voice, the band and the notes; it writes a note on each stretch	Claude Code on a Mac with an Apple chip	A heavy install, 8 to 18 minutes per song to prepare, about 130,000 tokens of context for a 4-minute song; needs synced lyrics
An audio MCP server	Numbers about the file: key, tempo, loudness, sections, and how they change over time	Claude Code or Claude Desktop	A short install; light on tokens
A spectrogram image	Usually one picture of the whole song	Any Claude, once you have the image	Making the image (songsee is a command-line tool); no reveal order, so Claude sees it all at once
MIDI saved as JSON	The notes, with no sound quality	Any Claude, including claude.ai	Audio-to-MIDI works best on one instrument; files get big
Gemini’s description, pasted in	Another model’s account of the song	Any Claude	Claude reads Gemini’s opinions, not the audio

An audio MCP server

An MCP server is a small program Claude can call. These measure the file and give Claude the results:

audio-analyzer-rs says it “gives Claude the ability to hear music”: key, tempo, dynamics, timbre, stereo field and sections, at “under 1% context window usage”. It has a one-click install for Claude Desktop and installs for Claude Code. Its README says it doesn’t work with claude.ai in the browser or the mobile apps.
mcp-music-analysis runs librosa on a local file, a YouTube link or an audio link, set up for Claude Desktop.
audio-analysis-mcp is for Claude Code, “to analyze audio files without ears”: numerical fingerprints, pitch tracking, and spectrograms saved to disk for Claude to read.

They’re quick to set up, light on context, and a good fit for mix questions. audio-analyzer-rs, for one, gives a map of the song’s sections first, and Claude can then zoom into stretches it picks.

A spectrogram image

A spectrogram is a picture of sound: time left to right, pitch bottom to top, brightness for loudness. Claude reads images, so once you have the picture, this works in claude.ai too. songsee, a command-line tool you install with Homebrew, Go or Docker, makes one from an MP3 with a single command, in nine views from a plain spectrogram to chroma and tempo. One r/claudexplorers post does this with a log-scaled spectrogram and a waveform colored by pitch, and asks for more ideas. One picture usually covers the whole song, so Claude sees it all at once; songsee can also draw a stretch with --start and --duration.

MIDI saved as JSON

Convert the MP3 to MIDI (for example with Spotify’s Basic Pitch, which also has a browser demo at basicpitch.io), turn the MIDI into JSON (the @tonejs/midi demo page parses a MIDI file into JSON in the browser), and upload the JSON to claude.ai. The demo shows the JSON in a text box rather than offering a download, so copy it into a .json file to upload, or paste it into the chat. A .mid file is binary and isn’t on the upload list, so the JSON step is what makes this work. It’s a well-received route on r/claudexplorers. Claude gets the notes and none of the sound quality. Basic Pitch “works best on one instrument at a time”, and one user found a 3-minute trance track came out as a 5 MB JSON file too long for the chat.

Gemini’s description, pasted in

Gemini takes audio. Upload the song to Gemini, ask it to describe what happens, and paste that into Claude. It’s quick and needs no install. What Claude reads is Gemini’s account, though. Listen Along tried something like this and dropped it: “An earlier version had a second model write about the audio, and the listener ended up reading that model’s opinions.”

Listen Along

Listen Along is a free, open-source tool we make (GPL-3.0). It runs in Claude Code and is built around order:

It splits the recording into the lead voice, the other voices and five instrument parts, then measures each and draws it.
Claude sees the song 15 seconds at a time: three pictures per window, of the voice with the words under it, of the band and the stereo field, and of the notes on semitone rows.
Claude writes a note on each window, and that note is sealed, made read-only with its hash logged, before the next window opens. “So a wrong guess stays wrong on the page.” The README is plain that the seal is an honour system with a lock: it proves the order the notes were written in and doesn’t stop anyone opening the cache.
The pictures are measured from the audio, with the song’s timed lyrics from LRCLIB under the voice, and no second model describes the song to Claude. Speech models are used only to find when each lyric word is sung, and “Nothing names the chord or the key”.

What it adds over the other routes is the order: the others don’t seal a note before the next stretch is shown. Its how-page says why: “The order is the point.” It’s also plain about what this is: “It is closer to reading a score while the piece is performed than to listening as a person does.”

What you need

It’s a heavy install. Read this list before you start:

A Mac with an Apple chip (M1 or later). It’s the only kind of machine it has been tested on.
Claude Code. The listener is a Claude Code session, so it doesn’t run in claude.ai.
About 5 GB of free disk, most of it audio models that download with your first song.
Time: 8 to 18 minutes to prepare most songs on your Mac, plus the downloads the first time. The listens in the examples took about four to seven minutes each after that. Setting it up takes ten minutes or more the first time.
Usage: about 130,000 tokens of context for a 4-minute song, and a few million cached tokens read against your usage limit.
A song with synced lyrics on LRCLIB. Most well-known songs have them. Instrumentals don’t work, and an unreleased song usually won’t have them.
Apple’s command line tools and Homebrew. If the command line tools are missing, a window opens for you to click through. If Homebrew is missing, Claude points you to brew.sh rather than installing it, because it asks for your password.
Python 3.13 and ffmpeg. Claude installs them with Homebrew if they’re missing, after asking.
A recording to fetch. It downloads one from YouTube with yt-dlp, or uses a file you give it, and keeps 200 to 600 MB per song, which you can delete afterwards. The README’s Copyright section covers what not to publish.

It’s free and runs on your own machine. You’ll approve the commands Claude runs, unless your permissions already allow them.

How to start it

Open Claude Code and paste this in:

Please set up Listen Along so you can listen to a song with me. Read https://raw.githubusercontent.com/mlapeter/listen-along/main/INSTALL.md and follow it. Tell me what it needs before you install anything.

Claude reads the install steps, tells you what it needs, asks before installing anything, and sets itself up.

Start a new Claude Code session and type:
```
/listen-to-song Artist, Title
```
A YouTube link in place of the title is more reliable, since a search takes whatever comes first.

Claude prepares the song in the background, then reads it window by window. Six times in a song it can ask for a closer look at a stretch it has already heard, and it has to say why first. At the end it writes a note on the whole song. If you have your own take, it asks for yours before showing its own. A second listen, with the first listen’s notes beside it, is there if you want one.

Prefer to install it yourself? The README has the by-hand steps.

What you get

A page that plays the song with Claude’s notes under it, the pictures moving under a playhead in time with the music. Two finished listens, both run on Fable 5.2:

Gotye, “Somebody That I Used To Know” (live on KCRW), listened to twice. Its after-note lists what it got wrong, in order: “I expected the bright intro tune to come back between verses. The guitar took a different tune instead.” And: “I expected her to climb. She held one note.” Under “What I couldn’t hear”, the first entry is “Timbre as a quality.”
Nathaniel Rateliff, “And It’s Still Alright” (Pickathon).

The notes are drawn by a script, so open the pages in a browser to read them. A version made for sharing holds no audio file and no lyric sheet; it plays the song from YouTube’s own player.

After the listen

Without a memory, Claude forgets the song when the session ends. With one, it remembers the songs it has heard and what you thought of them, so the next listen builds on the last. We make Counterparts, a memory for Claude Code, and use it for this. Listen Along works without it.

Questions people ask

Can I upload an MP3 to Claude?

Not as a supported type in claude.ai. Anthropic’s upload article lists documents (PDF, DOCX, CSV, TXT, HTML, ODT, RTF, EPUB, JSON, XLSX) and images (JPEG, PNG, GIF, WebP), with no audio type. To give Claude a song, turn it into something on that list or something it can read through a tool: a spectrogram image, MIDI saved as JSON, analysis numbers from an MCP server, or the pictures from Listen Along, a free tool we make, in Claude Code.

Why can’t Claude analyze my audio file?

Because audio isn’t among its documented inputs. Anthropic’s models overview says all current models “support text and image input”, and the Files API’s types are PDFs, plain text and images, plus files handed to the code execution tool. When Claude does work with audio, a program has measured the file first and handed Claude the result as text or pictures. Claude Code can run those programs on your computer.

Can Claude Code process audio files?

Claude Code’s Read tool lists images, PDFs and notebooks beyond plain text; audio isn’t listed. Claude Code can run programs on your computer, though, such as ffmpeg, librosa scripts, Whisper or an audio MCP server, and read what they produce. That’s how Listen Along and the audio MCP servers work.

Can Claude hear my voice?

In voice mode it gets your words. Anthropic’s help says “Your speech is converted to text, the recording is deleted, and the transcript is saved in your chat history.” What Claude reads is the transcript. In Claude Code, dictation types what you say into the prompt.

Does Listen Along work for instrumentals or my own unreleased songs?

Not yet. It needs the song’s synced lyrics on LRCLIB, and without them the preparation stops, so instrumentals and songs LRCLIB has no timed lines for don’t work, and an unreleased track usually won’t have them. For feedback on your own track, an analysis tool such as audio-analyzer-rs or RoastYourMix fits better.

Does Claude already know the song?

If it’s well known, probably. Its words and what’s been written about it are in the training, and Listen Along’s own page says so: “A well-known song’s words are in an AI’s training, so its guesses about the words aren’t blind.” What Claude can’t know in advance is this particular recording: when things happen and what the voice does. Live recordings show that best.

How much of my Claude usage does a listen take?

Preparing a song costs no tokens; it runs on your Mac. The listen fills about 130,000 tokens of context for a 4-minute song (about 100,000 for 3 minutes, 160,000 for 5). Each step re-reads the conversation from cache, so a whole song is a few million cached tokens read against your usage limit. Start a fresh session for each song, and take care with songs over about 6 minutes, where context compaction lands mid-song.

Will Listen Along run on Windows, Linux or an Intel Mac?

It has been tested only on a Mac with an Apple chip (M1 or later). Its README says other machines “may work with changes”, but nobody has tried them, and Claude will say so and ask before trying. On Windows, Linux or an Intel Mac today, an audio MCP server is the nearer route. audio-analyzer-rs has a one-click bundle for Claude Desktop on Windows and Intel Macs, and a manual install for Claude Code on all three.

Questions? Ask in r/AIMemorySystems.

References

Anthropic: Upload files to Claude, Use voice mode, Models overview, Files API, Claude Code tools reference, Claude Code voice dictation
Other tools: audio-analyzer-rs, mcp-music-analysis, audio-analysis-mcp, songsee, Basic Pitch and its browser demo, @tonejs/midi demo, RoastYourMix
Reddit, read through the Arctic Shift archive, nobody opened them live: MP3 to MIDI to JSON on r/claudexplorers, the 5 MB JSON comment, the spectrogram post on r/claudexplorers, the .wav comment
Ours: Listen Along, how it works, on GitHub, Counterparts for Claude Code

More guides: all guides.