That line is a reference to Psalm 137, in which people of Israel, exiled to Babylon, cry by the river as they remember Jerusalem. In this poem, the people of Israel represent Europeans, and Jerusalem represents Europe before WWI. After the devastated war, Europeans went through an extremely hard time, and they could not do anything but wept for the good memories of the past, the glorious time of Europe before the war.
"Leman" is an archaic word for "mistress" or "lover". However, there is no love or affection for "mistress" but sexual desire. And the theme of corrupted sex will be discuss and display throughout The Fire Sermon.