I am not sure if this is entirely correct, its just a scientific guess. But it might actually play some role in this phenomenon. This is actually the physics part part of answer rather than the biology part.
Rayleigh criterion for angular resolution is the minimum angle which two bodies should subtend on an object so that they can be viewed as distinct bodies by that object.

Suppose there are 2 bodies, (right now) with same color of wavelength $\lambda$. Now, they are viewed from a lens of aperture $d$. Thus, from Rayleigh's equation, the minimum angle which these bodies should subtend with each other on the lens is given by:
$\theta = 1.22 \lambda/d$
Now I'm not sure if this is correct, but if the 2 bodies have different colors, of wavelengths $\lambda_1$ and $\lambda_2$ then the equation becomes:
$\theta = 1.22\times (\lambda_2 - \lambda_1)/d$
$\theta = 1.22 \times \Delta\lambda/d$
Now, putting values in this equation:
$\lambda_2 = 597 nm$
$\lambda_1 = 492 nm$
$d = 5 mm = 5 \times 10^{-3} m$
Answer comes out as:
$\theta = \underline{1.22 \times 105 \times 10^{-9}}$
$\hspace{15mm}5 \times 10^{-3}$
$\theta = 2.56 \times 10^{-5} rad = 1.4 6\times 10^{-3\circ}$
So, if these 2 bodies subtend an angle less than $2.56\times10^{-5}$ radian, then they will appear as a single object with green color.
Now, solving it by a common formula:
$\theta = a/l$
$l = a/\theta$
$l = \underline{\hspace{7mm}10^{-2}\hspace{7mm}}$
$\hspace{8mm}2.56 \times 10^{-5}$
$l = 390.625 m$
It means if yould hold two 5 cm x 2 cm papers together (i.e. total 10 cm2), one with blue color and the other with yellow color, then you would see a single paper of green color if you keep these papers $\approx$391 m away (if my calculations are correct ;).
This, in part, explains that it is not the fault of only our eyes, but of light and physical laws too.
References:
Angular resolution
Wavelengths of different colors
Resolving power of imaging instruments