Cue: skip song by song inside a DJ mix.

A DJ mix is one long file. If a song you hate comes on you scrub blind; if one you love comes on you cannot find it again. Cue finds the song starts and turns them into chapters that a phone, a car stereo and any chapter-aware player already understand.

My role
Independent builder & product direction
Pieces
iPhone player (SwiftUI) · detection engine (Python, open source)
Status
Engine public; the app runs on my phone, not yet on the App Store

What I found

  1. Scored blind against 190 DJ-published boundaries in eight public mixes, fingerprints plus the transition detector found 99 within 10 seconds; fingerprint offsets alone found 62.

  2. Within 30 seconds, the figures were 144 with the detector and 131 with fingerprints alone.

  3. Human annotators disagree by about 9 seconds; cut-style mixes (dancehall, hardstyle, jungle) score at that human level, while long beat-matched house blends land around 25 seconds median.

  4. Multimodal language models placed boundaries no better than a novelty curve and invented titles; three times more minute-resolution training labels gave no clear gain.

The central idea

The hard part is not playback. It is deciding where a song starts inside a beat-matched blend, and being honest about how often that decision lands within ten seconds of a human’s.

Three iPhone screens of Cue: the mixtape library with a mix ready to listen, the same mix playing, and the app icon on the home screen.
Cue in the iOS simulator: the library and player, now playing, and the app icon. Stills from the UI-test recordings.
  1. 01Add a mix
  2. 02Read or detect the tracklist
  3. 03Place each song start
  4. 04Write chapters into the file
  5. 05Skip with the car controls
From a mix file to a skippable mixtape · simplified from the current implementation
Contents
  1. What I found
  2. One file, many songs
  3. The player
  4. Finding the starts
  5. Measured against DJ-published timestamps
  6. What it looks like
  7. What is bundled, and what I am not claiming
  8. Sources & scope

One file, many songs

Mix platforms show a tracklist but will not let you skip, because licensing treats the mix as a single unit. On a phone or in a car that leaves two bad options: scrub blind, or listen through. Every song in a mix has a start; the file just does not know where.

Cue’s bet is that the start times are the product. Once a mix carries them as ordinary MP4 chapters, the rest is playback that iOS already does well: background audio, lock-screen titles, and the media buttons on a steering wheel.

The player

The app is native SwiftUI for iOS 17 and later. Audio plays locally through AVPlayer with background playback, Now Playing information on the lock screen, and the standard Bluetooth media commands. There is no hosted service and no account. Song starts come from three places, in order of preference: chapters already embedded in the file, a timestamped tracklist pasted into the app, or marking them by hand while listening.

The car controls are the reason the app exists, so they are deterministic and tested. A single Next goes to the next marked song; two Next presses within 0.4 seconds go to the next mixtape; holding seek-forward for at least 0.65 seconds also goes to the next mixtape; Previous returns to the start of the current song, or to the previous song when already near the start. Cue uses the normal iOS media-command mechanism, so it cannot see raw Bluetooth presses or reprogram a stereo; double presses work only where the stereo sends two commands.

Car controls · from the app’s README
Input delivered by iOSCue action
One Next commandNext marked song, after a 0.4-second double-press window
Two Next commands within 0.4 sNext mixtape, from 0:00
Seek-forward held for at least 0.65 sNext mixtape, once
Previous commandCurrent song start, or the previous song when near the start
Play / PauseNormal playback control

Finding the starts

Detection runs on a Mac, not on the phone, through the open-source engine. It reads a tracklist when one exists: a text file beside the mix, or the page description when you hand it a URL. Timestamps in the tracklist are used as they are. Otherwise it combines three kinds of anchor. Audio fingerprinting says which song is playing and roughly when it was first heard; the provider is pluggable, with a licensed service for app use and an unofficial client kept for personal research only. A small transition detector trained on the public DJ Mix Dataset places the exact cut just before that. And free 30-second preview clips of the tracklist’s titles, aligned into the mix with tempo-tolerant chroma DTW, anchor songs when no fingerprint service is available.

A resolver combines the anchors, and the output is a plain m4a with MP4 chapters plus title, artist, date and genre tags. It is CPU only, about a minute per hour of audio on a laptop.

Measured against DJ-published timestamps

The engine is scored blind against timestamps DJs published for eight public mixes, 190 boundaries in all, boundaries only. Within 10 seconds, fingerprint offsets alone find 62 of 190; fingerprints plus the transition detector find 99. Within 30 seconds the figures are 131 and 144. Published disagreement between human annotators on DJ-mix boundaries is about 9 seconds, so on cut-style mixes (dancehall, hardstyle, jungle) the engine sits at the level of a human; long beat-matched house blends land in the right neighbourhood, around 25 seconds median, and are honestly ambiguous even for people.

Three things did not help, and are recorded as such: multimodal language models listening to the audio placed boundaries no better than a novelty curve and invented titles; snapping to a beat grid changed nothing against DJ-typed timestamps; retraining the detector on three times more minute-resolution labels gave no clear gain, because label precision is the limit.

What it looks like

Fifty-seven seconds of the app: adding a mix, the detected song starts, and skipping through them. Recorded on the phone; sound starts only when you press play.

What is bundled, and what I am not claiming

The proof-of-concept build ships two complete mixes from an artist’s freely downloadable catalogue, converted to 128 kb/s AAC, with original artwork and source links preserved. A free download is not a general license to redistribute recordings in an App Store release; those examples would be removed before any public build. The app installs on my own phone with a free development profile. There is no App Store listing, no user study and no adoption claim.

What exists is the engine, its accuracy tables, and a player that does the job in my car. The open question is whether DJs would publish their own cue sheets so that listeners get the artist’s starts rather than a detector’s guess; that is a distribution question, not a technical one.

The open-source engine: song-start detection for DJ mixes, with the accuracy tables and a demo of the app.

Visit Cue (opens in a new tab)

Questions about this project? Get in touch.

sipratt@gmail.com