I've mentioned before about peering into the corners of photos to see new things. I tend to think of Hertford Street as a solid block of tall buildings but as the 1850 map and this snippet of a picture show, there were originally breaks in the street front and smaller buildings. The two storey building below fronted a ribbon manufactory and the 3 storey building was the Mechanics Institute. This makes sense of Neil's photo from post 309 with its two storey buildings at the back of the Red Lion yard, which were next door to the Institute.
Just a guess but the Institute building might have survived as the building in this post card by CliffB (post 335), just to the left of the Queens Hotel.
This morning I have confirmed that it is the right location and on the Fire Insurance maps 1897 was the Coventry Savings Bank at number 69.