So this is an incredibly big record for me. Rid of Me is a record that she wrote in a couple of weeks, which also, as a writer, really kind of distressed me that someone could be that good in such a short amount of time. She made it with Steve Albini [Editor’s note: This interview was recorded prior to Albini’s passing.] I never heard a record of a three piece that’s been more in tune. Was it Robert Ellis [and Steve Vaughan]? The expression between the three of them to take very raw — basic’s the wrong word because it’s really elevated — but there’s something base about it and raw, but it takes all three of them to make it work. The dynamics of each of the three members. It’s not just a PJ Harvey record. It’s actually the three of them.

I must have played this record to death. When I went to make Sixteen Stone, I remember all I wanted to do was make every song sound more like Rid of Me. While I was in the studio, going, “How do we get it to be more like this?” I’ve never done that with anyone, but I did it with her.

I’ve got my first guitar off of her, because she played a Jazzmaster at that show [I was at] where she was shouted at from the crowd. I hadn’t seen that and I was like, “Super cool guitar.” So when I got a record deal, I went to this store called Vintage and Rare in London and in the window, they had the same Jazzmaster, same as Polly was playing. So I was like, “Oh my god, I’m one step closer of a thousand steps.

Essential Track: “Man-Size.” What I liked about it is that when she wants to get heavy and aggressive, she’s more aggressive than anyone. She can do white chalk and can be sort of very poetic and lilting, but when she wants to give it…