Information and Code Resources

Good General Resources

• musicinformationretrieval.com
  • Great source of low level audio/MIDI processing techniques.
• Sonification Design
  • Cool examples of data sonification.

Articles to Read

• Sonification 101: How to Convert Data Into Music with Python
• Visualizing Audio Data and Performing Feature Extraction
• Audio Deep Learning Made Simple (Part 1)
• Maelzel Documentation

Libraries for creating music with raw Python code

• Sardine - GitHub
  • For controlling MIDI devices and using "Python as an instrument"
• pygmu - GitHub
  • Declarative music generation by writing code to define raw notes, properties, etc.
• HANS - GitHub
  • Experimental music generator.

Libraries for modifying audio files

• bilthoven - GitHub
  • Possible syntax/framework for defining custom audio transformations to wav files.

Libraries for creating or modifying MIDI data

• expremigen - GitHub
  • Generation of "expressive" MIDI, composed of "phrases."
• braid - GitHub
  • For building MIDI sequences and polyrhythms
  • Cons
    • Seems to mainly work by sending MIDI data to a channel, not outputting a file.
• Numula - GitHub
  • Custom syntax you can apply to MIDI to add "nuance" to MIDI
  • Pros
    • Adds human-like variance to MIDI notes
  • Challenges to use
    • Requires pre-defining a string with custom syntax to apply to MIDI data
  • Possible experiments
    • Generate syntax randomly with LLM
    • Write function to build syntax based on structure of MIDI + some parameters
• Procedural_MIDI - GitHub
  • Script to take chord MIDI and generate semi-random melodies.
  • Custom script that can be organized and parameterized.
• Magenta Studio - Magenta
  • Ableton plugin for AI-based MIDI generation and modification
• ddpy - GitHub
  • Has methods for converting pandas data frame w/ number sequences into MIDI

Libraries for getting or creating sounds

• py_modular - GitHub
  • Construct simple modular synthesis pipelines.

Models

• AudioLDM - GitHub
  • Text to audio model for generating 5-second audio clips.
  • Run on replicate

Various Other Libraries

• musicntwrk - GitHub
  • Library for music theory analysis (eg. pitch class set classification).
• polyrhythm - GitHub
  • For creating polyrhythm data.

Creating Audio Embeddings

• torchvggish
• audio-embedding (with wav2vec 2.0)
• openl3
• panns_inference
• Embedding clustering and analysis

Possible Methods for generating music data using LLMs

• Music notation methods
  • LilyPond - best choice for creating note sequences and sheet music.
  • Music21 tinyNotation - optional choice after LilyPond.
  • ABC notation - optional choice but syntax is arbitrary.
  • ChordPro - Good for notes + lyrics.
  • Drum Tablature (Drum Tabs) - Notation for drum rhythms and patterns.
• SuperCollider3 one line sound generation
  • sc-140
  • Label all lines with description of code and description of generated sound.
  • Use as "in context learning" examples.
  • Ask to generate different types of sounds.