When gas exchange occurs during respiration, the pressure of oxygen in alveoli is around 105 mmHg, whereas in the blood vessels in close contact with alveoli is 40 mmHg. For carbon dioxide the values are 40mmHg in alveoli, and 45mmHg in vessels. So the difference in pressure for oxygen is 65 and for carbon dioxide 5. The fact that the pressure difference required for oxygen diffusion to occur is greater than that of carbon would mean that is diffuses with more difficulty. But how is that possible since the oxygen molecule is smaller and non-polar, so would technically pass easier through a hydrophobic membrane?
After diffusion the partial pressure of both O2 and CO2 in arteries become almost equal to the atmospheric values(100mmHg and 40mmHg), regardless of partial pressures in veins. Rate of diffusion through membrane is roughly the same, it's the solubility that is different, as CO2 is about 22 times more soluble than O2 (it can be stored is water as HCO3-). However, oxygen still can easily enter and stay inside the blood vessel because it's bound to hemoglobin inside the erythrocyte. Hence there is no significant difference between the diffusion rates.