It may seem counterintuitive that deoxyribonucleic acid has nitrogenous bases. Nonetheless, nucleic acids (thus including RNA) were called that way because the phosphate backbone (linked by phosphodiester groups) is a derivative of phosphoric acid.
Note that the only difference between these two groups is that in the case of the phosphodiester, two O-H bonds are replaced by O-C bonds. This make the last remaining hydrogen atom quite acidic (can quickly be identified by how the conjugate base exhibits resonance). In fact, it is so acidic that at physiological pH, the phosphodiester group is deprotonated, giving that nucleotide a negative charge. This property is does have a profound effect on DNA's structure and "behavior" in a cell, and is often exploited in genetics, such as in gel electrophoresis.
An interesting fact is that this acidity of the phosphodiester group actually competes with the basicity of the nitrogenous bases. However, since most often DNA is double-stranded, individuals bases will hydrogen-bond with its partner to make base pairs, which stabilizes and thus reduces this basicity.
TLDR: Although different acids of different molecular composition can act in very different ways, the overall acid-base chemistry is seen in nucleic acids.