Oliver has published widely, and on twentieth-century writers, artists and composers particularly.

His essays and reviews have appeared in the Guardian, Spectator, Times Literary Supplement, London Review of Books, Literary Review, The Art Newspaper, BBC Music Magazine, and Gramophone.

The biographer of composer Michael Tippett, he has contributed articles on Tippett's life and work to several essay collections and to peer-reviewed scholarly journals such as Musical Quarterly and Tempo. His programme and liner notes accompany most recent performances and recordings of Tippett's work.

Recent essays and reviews

On Ivor Gurney (£), London Review of Books, September 2022
"Grace Notes: Michael Tippett" (£) Times Literary Supplement, July 2020
"Priaulx Rainier: Fearless and Pioneering Composer" Engelsberg Ideas, August 2021

Review of Tell Me Good Things by James Runcie (£) - Spectator, Jan 2023
Review of Barbara Hepworth by Eleanor Clayton - Guardian, May 2021
Review of Spirit of Place by Susan Owens - Guardian, December 2020
Review of As If By Chance by David Lan - Guardian, March 2020
Review of Counterpoint by Philip Kennicott (£) - Spectator, March 2020
Review of The Life & Loves of E. Nesbit (£) - Spectator, October 2019
Review of The Letters of Cole Porter - Literary Review, October 2019

Other Articles / Book Chapters

- 'Design for Living: David Myerscough Jones',
Random Spectacular at The Fry Art Gallery (Norwich: St Jude's, 2020)

- 'Tippett, Eliot, and Madame Sosostris',
in The Edinburgh Companion to Literature and Music, ed. Delia da Sousa
(Edinburgh: Edinburgh University Press, 2019)

- 'Sculpting with Sound: Priaulx Rainier',
The Strad, 127 (May 2016), 41-44

- 'Darkness Declares the Light: Tippett's String Quartets',
The Strad, 127 (March 2016), 38-42

- 'Michael Tippett and Morley College',
in Morley: 125th Anniversary Portrait, ed. Janet Sacks
(London: Third Millennium Publishing, 2015)

- 'An unknown bird in the early Faber nest', co-written with Jim McCue, Notes and Queries, 61 (1), March 2014

- 'The Clarion Airs of Michael Tippett and Christopher Fry',
Musical Quarterly, 97 (2014), 616-660.

- The Orchestral Music of Michael Tippett – review",
Cercles, 34 (July 2014).

- 'Tippett and Eliot', Tempo, 67 (266), October 2013, 28-53

- "Cambridge Companion to Michael Tippett – review",
Tempo 67 (265), July 2013, 88-92

CD Liner Notes

-Tippett, The Midsummer Marriage (LPO Live)
- Tippett, Complete String Quartets (Wigmore Live CD)
- Tippett, Complete Symphonies (Hyperion)

Introductions to scores

- Tippett, Symphony in B flat (Schott music, 2018)
- Tippett, arr. Manning, Time Past, Time Present, Time Future (Schott music, 2015)