Songs of Praise, the debut record from South London post punk titans Shame, couldn’t have received a better response on its release in January. Lauded for their political, sometimes aggressive and always lively punk-blues, Shame are riding high on their success, but trying to keep level headed. “We try to walk the tightrope between praising ourselves and degrading ourselves, because we don't want to lean too far either side. We're just enjoying the flame while it flickers!” says singer and lyricist Charlie Steen.

Steen is full of ideas, quick to laugh and with an anecdote fit for every occasion. So it’s unsurprising that he’s a lyricist, eager to put a story to everything and with the sharp wit required to make every story enticing. For Steen, the key to a good song is a gripping narrative, Bob Dylan and Squeeze rank highly on his list of music’s best storytellers and are among many of the musicians he first heard via his parents’ record collection.

He has stories behind every one of these songs, from the trips in his girlfriend’s car sound-tracked by Bruce Springsteen, harrowing memories of a panic-riddled tour through Germany to the now triggering ‘Jubilation’ by Norma Tanega and a track he heard just a few days ago, that he’s certain he’ll remember as the song which marks this very point in his life.

“The importance and power of all of these songs is that when we're on the road and touring, we're able to be brought back to a memory just through one song,” Steen says, as this list continues to be added to.

1. “Show Me Heaven” by Maria McKee

“I picked this one because it's technically the first song I ever heard! I was born to this song. It was playing when I was born at St George's hospital, which is near to where I still live. I was in my Mum's stomach for ten months - I was a month late, overcooked!

“It was probably just Magic Radio station playing it or something like that, but I thought I'd start this off with the way I came into the world. It was a huge hit, really near the top of the charts and it’s just a great pop song.

“My Mum told me the story when I was quite young and unfortunately my memory doesn't span back to minute one of my life! Sometimes I get mixed up about which song it was. I’ve definitely told the story wrong in the past, but ‘Show Me Heaven’ is definitely the song I was born to.”

2. “Masters of War” by Bob Dylan

“I grew up with great music always being played in the house. My Mum was really into Bob Dylan, The Beatles and Joni Mitchell and my Dad was more into Django Reinhardt and Elvis Presley. So I've always listened to Bob Dylan, right from when I can remember.

“Our guitarist Sean has been my best mate since we were eight. We were driving in his Dad's car, I can't remember where we were going, but this song played and it was the first time that I felt like I'd really properly listened to it, the lyrics had a real impact on me. Not to sound really cheesy, but it was one of those moments where you stop what you're doing completely and just immerse yourself in what he's speaking about.

“I love the fact that the instrumental to it is very simple and very repetitive; it almost reminds me of The Fall, it's more about telling the story and drilling that in and being able to form an emotional and accurate picture. That's one of his greatest talents, Dylan has so many songs that I could've picked, but 'Masters of War' is the one that resonates with me the most.

“I’ve always been attracted to any lyricist who can paint a picture, tell a story and really engage with the audience in a way that a lot of songwriters might not be able to do. People like Tom Waits, Bob Dylan, even the Beatles, Roxy Music - that song ‘In Every Dream Home a Heartache’ - Nick Cave as well. And as much as I love music, lyrics have always held more of an importance to me.

“Me and Sean would be at his house from the age of eight, playing guitar and ripping off lyrics from Tom Waits songs. When we started I just wanted to be able to tell a story, because it's such a hard thing to be able to do. A lot of our lyrics are social observations; they're just a conversation with myself, trying to understand an issue or a subject that I'm particularly interested in.

“When we started as a band and I was given the responsibility of doing the lyrics I realised that you can't just write another love song or a song about unrequited love, or all these different issues that have been mastered hundreds of thousands of times by so many other people. I always thought that there's a lot of stuff to talk about, so why not try and take an interest in something else? If you want to write a love song, you've got a lot of competition. If you're writing about a different subject matter, you're picking a slightly easier route in that sense, but a more interesting route.”

3. “Up the Junction” by Squeeze

“To be honest I'm not a serious fan of Squeeze, I don't know all of their stuff that well but I always remember this song being played when I was a kid and in terms of storytelling, it was my favourite.

“Even though it wasn't in the time of Britpop, I still think it's the best Britpop song ever made. It completely captures a moment in time but can still relate to the modern day. I'm from South London and Clapham is near-ish to where I live. It's just a great fucking song and it's got a great story that tells something so true. At that time people like The Specials, with ‘A Message to You Rudy’ captured that as well. When you're listening to ‘Up the Junction’ for just for a moment you can feel slightly proud to be British and then it sort of fades away pretty quickly.

“It has one of the best lines out of any song: "I beg for some forgiveness, but begging's not my business." The rhymes are great. There are no questions that need to be asked about this song, it's so well explained.”

4. “These Boots Are Made For Walkin’” by Nancy Sinatra

“Again, this is a song I remember from my childhood, my Dad loves 'These Boots Are Made For Walkin’' and he'd play it all the time. It's such good pop song and everyone loves it - there's no way you can avoid it, it brings me back to that period of my youth.

"It's hard to focus on the full gravity of each of these songs, but this is just a song that immediately clicked with me, that I'll always remember and that you can always put on in any situation you're in, just to cheer you up a little bit. It has that funk ecstasy.

“I was stuck between picking this or ‘Stuck in the Middle With You’, both of those songs bring me back to an era that I wasn't born in, but they create a picture of what the world used to be. You can imagine your Mum singing it and you can imagine your kids singing it. ‘These Boots are Made for Walking’ is an untouchable song.”

5. “Human Fly” by The Cramps

“Do you remember iPod classics? My Mum loves The Cramps, she would let me play her iPod and this song was on there.

“It was the first song I tried to rip off lyrics to. When I was nine or something like that, I had basically 'written' the exact same song but changed around a few words. It has that sort of depth to it that can transport you, it's like a motion of power that almost gives you strength. This might be sounding pretty fucking stupid!

“But the lyrics in this are fucking amazing, “I got 96 tears and 96 eyes!" What a lyric! And The Cramps are a band I fucking love. That band did so much, they did a tour of only mental hospitals – you can watch footage of it on YouTube and its just chaos. They're such an important influence that you can't ever get away from.

“Lux Interior created his own identification with the way he sang and the way he spoke. It might have been derivative of another singer, but I don't know who it was. He was a Southerner, wasn't he? They've got a song that's like "The hottest thing from the north to come out of the south", he's got a real distinction. When you hear those songs you immediately know it's him and you can't imagine it's too much of a façade.

“It's hard to say few words and have a big impact, but he does that in a few songs with short lines and rhymes that immediately click and connect with you. He has a great confidence in his voice that a lot of people don't have. He sounds like he doesn't care whether you hate it or whether you love it. A lot of their songs are pure moments of freedom.”

6. “Colour Television” by Eddy Current Suppression Ring

“When we were 17 and we were doing our first demos at Dropout Studios in Camberwell, which is where we practise now, a guy called Jason, who owns the studio, played us the album Primary Colours. From that moment we were starting the band and being exposed to all these different bands and types of music we'd never heard before. Even though we knew The Fall, and The Stooges etc., Eddy Current Suppression Ring were a group who immediately had a huge impact.

“They gave us a drive as to what we wanted to achieve musically and we loved how simple and powerful their music is, especially their lyrics. You forge a special bond whenever you meet anyone else who knows that band, because they never really got big in the UK, Europe or America. They just had this cult following. I've met bands from around the world that were really impacted by this band.

“I like the mystery behind them. They formed other bands like Total Control but there isn't a lot of live footage or interviews from which you can learn about them, apart from through what other people can tell you. I think the fact that they were from Australia – the other side of the world! – it’s something that's so appealing to me; it feels so far-fetched and so out of my reach in terms of ever being able to see them play live.

“I know people who have met them and they just really don't give a shit about trying to appeal to anyone. They've got a lot of humour too, which is something we always want to have and something I think is an important element to any band.

“’Colour Television’ was the first song I remember hearing off that album and I just loved how simple it was. It's one of those driving songs, not in terms of something you want to be driving a car to, but the kind of song that sparks a bit of energy into you. Maybe you do wanna be driving a car to it, I don't know, I can't drive!”

7. “Devils & Dust” by Bruce Springsteen

“Everyone knows who Bruce Springsteen is, but the time I heard this song was when I properly got into him. When I first started going out with my girlfriend just over two years ago she only had CDs in her car and this was one we always used to listen to, because she fucking loves Bruce. This one isn't a driving song, but a driven-around song! I'm always in the passenger seat.

“It's not the most obvious Springsteen choice but that album Devils & Dust has so many fucking great tracks on it. The majority of songs that I've picked revolve around telling stories, that's just something I'm attracted to and this song's the same.

“The music is beautiful. Bruce Springsteen is everything that's good about America. He's symbolic, he's someone who works really hard and has secured his place in history because he's got his style – you hear one of his songs and you know it's him, whether that's because of his voice or the music, it doesn't matter. He's like Prince: he's creates his own… not genre, but identity and that's so important. I've read his book and it's quite funny. I love ‘Born in the USA’ and I love satire, I think it's one of the best forms of humour, and you couldn't accomplish more than he did in that song, in playing the best joke on a country and it working.

“I was in Berlin a little while ago for a press trip and my girlfriend came over. We were in the bar at like four in the morning and they played this whole album. It was one of those moments where it's all a bit weird, because it isn't the obvious choice. It wasn't predictable.”

8. “Jubilations” by Norma Tanega

“We were doing a tour of Germany, I was having panic attacks and on the brink of mental and physical collapse. Our tour manager broke the aux cord in our van and he only had this song - ‘Jubilation’ - on his phone, it was all he'd play, there was one time when he played it seven times in a row.

“If you can imagine when you're at your bleakest hour and you're staring out of the window at some anonymous stretch of German motorway, it's dark and cold, you haven't eaten and you're throwing up fifteen times a day… that song immediately brings me back to that period of madness. The song didn't help at all, it almost drove me further down the hole, but it's important and I can see the humour in it now.

“We probably heard this song about fifty times over a few days. If you listen to the song you can imagine being transported to that place of hell. It’s funny, because of how happy and upbeat the song is, but every time I listen to it a bit of my soul breaks off more and more. I've got a trigger now.

“Norma Tanega was Dusty Springfield's lover for a period. I don't know much about her but the whole story beyond it seems interesting.”

9. “Spinning Away” by John Cale and Brian Eno

“I had my eighteenth birthday at The Queen's Head, where we used to practise, and I was given John Cale's autobiography by our drummer Charlie’s Dad. It's really fucking quick to read but a really unique book in terms of how big it is and how it has a lot of drawings, cartoons and paintings. It's a really interesting way to do an autobiography.

“I only heard this song for the first time two days ago and it means a lot to me in this moment. It's another funny time; I was helping my girlfriend move out of her house, so I’m at the end of an era with that and it's before I go away for a long time. So it's a song that I'll always remember for this kind of in-between phase. That whole idea of ‘Spinning Away’ is humorous to the situation that I'm currently in, in terms of the amount of touring that we're about to be doing.

“I just fucking love John Cale. I don't know a lot of Brian Eno's stuff – I know his music from when he was in Roxy Music, but I don't know his solo stuff that well. I think the other songs that I’ve chosen are ones that people might already be aware of, but this one feels quite different, or at least it's new and different to me. I need to give Eno’s solo stuff more of a listen – this track definitely grabbed my attention.

“I thought I'd end this on something that I'm listening to at the moment, and that I know in the future will resonate with me in the same way that all the other songs that we've been speaking about have.”

Songs of Praise is out now