What is the largest perennial herbaceous plant? My guess would be some kind of banana or bamboo.
As mentioned above, the largest perennial herbaceous plant is indeed the banana. Whilst the main reference to this ("Yes, we have more bananas" - an article in the Royal Horticultural Society Journal from May 2002) has been removed from their website it would be I'm sure possible to order should you need to.
This summary of the banana mentions that the pseudo-stem of the banana plant grows to 6-7.6m in height - I haven't found anything taller when looking around so it doesn't generate any reason to doubt the removed article.
Regarding the comments on Bamboo, there are 7 genera containing bamboo species. All of these (Arthrostylidium Rupr, Bambusa Schreb, Chusquea Kunth, Dendrocalamus Nees, Phyllostachys Siebold & Zucc., Pseudosasa Makino and Sasa Makino & Shibata) are described as having persistent woody stems so don't meet your herbaceous requirement.
The tallest banana species is Musa Ingens (15m) which grows in the forest of Papua New Guinea and Indonesia. It also claimed as the the world's largest herbaceous plant. http://www.thestatworld.com/2015/12/musa-ingens-the-tallest-banana-plant-in-world.html