There is no one answer of "how little". Each terrestrial plant depending upon its type has some minimum below and a maximum above which it could not survive over a sustained time at that level. So it's not just "how little" but "what plant" and "for how long" until it dies, rather than just going dormant or simply not growing. But the question doesn't involve carbon dioxide alone, there are other conditions (amount of sunlight, temperature, humidity, annual rainfall, soil quality, air quality) impacting any lower and upper limits. Deriving some answer for some plant in a closed system like a tube in a lab is one thing. In nature, the conditions leading to some extreme carbon dioxide level (like either 50 ppmv or 5,000 ppmv) the conditions themselves just to get there likely would kill a plant before even reaching the level of carbon dioxide.
It's rather immaterial to the question this assumption that there is some problem in current atmospheric concentrations of carbon dioxide. Problem to what, how and why? Yes, carbon dioxide levels rose in 2017 by about 2.48 ppmv, to 407.54 ppmv. https://www.esrl.noaa.gov/gmd/ccgg/trends/global.html There's also a current 0.79°C anomaly to go along with that 408 ppmv level. https://www.ncdc.noaa.gov/cag/global/time-series/globe/land_ocean/12/3/1880-2018 Assuming some sort of a direct cause/effect relationship to the planetary average temperatures solely attributable to this rise in carbon dioxide levels, still asks the question, is that all a problem? Trying to answer that is another issue entirely though, and also rather immaterial to the question of how little is too little, since it's going up not down.
We can scientifically - by observation and experimentation - establish both that increasing atmospheric concentrations of carbon dioxide are happening, and that carbon dioxide absorbs and emits in the infrared. We can also show the global average sampled near-surface temperature and trend are up. However, like questions of what level of carbon dioxide is not enough for plants, what we can't observe or experiment upon - on the actual Earth's planetary weather systems - is maintaining everything else and seeing what occurs with a given amount of one or more greenhouse gases that is higher or lower than the current amount. We can only experiment in non-Earth conditions or model scenarios on computers. Simulation, given certain other assumptions. This makes it difficult to do a lot of things, including calculating in the atmosphere when there's not enough carbon dioxide for plants. In large part, this is because it's not possible to control most of the hydrosphere, or the sun, or the many other factors impacting plants and how well or badly they fare.
That said, there is only one period in Earth's history where the conditions are like they are now, and that is now. In the existing conditions, it seems unlikely that - regardless of how much or little warming it causes - there is any way to reduce the amount of carbon dioxide in the atmosphere in any realistic timeframe to something lower than it was in 1880 (proxy, ice cores, 282.9 ppmv https://sealevel.info/co2_and_ch4.html)
A question like "at what level of carbon dioxide in the Earth's atmosphere do plants start to die" is a tricky one, because the conditions required to get to 250 (or 200 or 150 or 100 or 50 or 0) in the atmosphere would most likely require an atmosphere that would have lost much of the greenhouse effect to get there. Essentially plants (and just about every animal that would miss eating them) would be hit by extreme cold before they were hit by carbon dioxide starvation.
Although the best answer to these sorts of questions might be found in the ice core proxies, in that during the last million years or so, the atmosphere has not been lower than about 180 ppmv https://carboncycle.princeton.edu/research/atmospheric
In a controllable laboratory environment, different plants have different levels of carbon dioxide at which they won't grow, go dormant, or die. These are created conditions though, artificial, unnatural. On the Earth, in the system that actually exists, those levels of any of the greenhouse gases appear unreachable in a working biosphere anywhere near like the one we have. There are too many variables to just consider carbon dioxide, because it can only be experimented upon in a closed system that is unlike reality.