In my experience (in common with the experience of everyone I've talked to who could be considered an expert on the subject), taking old wood and using that as a scion when grafting new trees rejuvenates them, and they grow as new trees.
I'll take apple trees as an example. As you can see from the table here, there is a distinct age after which the tree begins to decline. Now, that's not concrete (I've seen some apple trees last over 100 years (didn't plant those myself btw)), but generally, an apple tree will lose vigor and productivity before it reaches 50 years of age. This tree's 'immune system' also goes down, and it becomes susceptible to many diseases. You'll rarely see such a tree, as orchardists generally replace them rather fast.
But to the point of your question. When you will graft an apple tree, you will generally bud graft it, as described here. That article (which is spot-on) says:
Select a healthy branch of this year's growth from your desired variety of tree. Look for a plump leaf bud about halfway down the branch. Leaf buds are close to the branch where fruiting buds tend to stand out more.
Note that it says nothing about the tree's age, simply stating that the wood must be healthy. That's because old wood/young wood produce the same result in the new trees. I was using a 40 year old razor russet to graft onto all the Malling V rootstocks I had for that variety. And of course they grew as expected, as a young sapling should. I've heard of a nearby nursery taking budwood from a 100 year old tree that still looked good.
Additionally, you can use a rootstock taken from an old plant, and also a scion from an old plant, and the new plant created will grow as a young sapling. Here's how the rootstocks I use are created: You cut down a tree at ground level, and it sends out a lot of shoots. You pile soil over it, but allow the ends of the shoots to stick through. These shoots will root into the soil, and can be cut at the base, and planted out for grafting later. This way, you can have 1000's of identical (cloned) rootstocks. You can also get rootstocks from old trees. Here's a diagram of the process.
About herbaceous perennials, I've found (to some people's surprise) the same thing. The scion grows as a young plant again once grafted. I found this out because I graft heirloom tomatoes onto hybrid tomato rootstocks (the latter are seedlings). Usually, the scions will be from seedlings, but at times I take the scions from a large mature plant.
Just so you know, an indeterminate tomato plant will, once mature, flower and fruit until frost. A determinate tomato variety will produce a large crop all at once, and then die back (this is the kind preferred by owners of mechanised (machine harvested) fields).
In my tests, you could take the scion from an old determinate plant that was through fruiting, right before it started dying back, and successfully use that for grafting the new plants. These new plants' growth was identical to the growth of grafted plants whose scions came from seedlings, and they began to flower and produce fruit at the same time. Of course, actual seedlings did not have the vigor of the grafted plants, and were behind in growth.
Off-topic for your question, but possibly still of interest, I've found that cuttings and layers (not scions) taken from old plants (both woody and herbaceous), and rooted, would grow as young plants again.
