T cells in a graft see both non-self peptides associated with the non-self HLA molecules, and can also see the non-self HLA molecules regardless of peptide. (MHC is a broader term than HLA, and I will use MHC here.)
Surface expression of CD4 or CD8 is commonly used to identify T-cell subsets that recognize antigen presented by class II MHC or class I MHC, respectively. This holds true for T cells that respond to allogeneic MHC molecules that are directly recognized as foreign, as well as peptides from allogeneic MHC molecules that are indirectly presented by self MHC molecules.
--Contrasting Alloreactive CD4+ and CD8+ T Cells: There's More to It Than MHC Restriction
Recall that as T cells mature in the thymus, they undergo negative selection such that those T cells that spontaneously recognize self MHC molecules are deleted, but there's also positive selection that ensures the T cells can recognize MHC features in general. That leaves many T cells that could spontaneously recognize slightly different MHC molecules (i.e. non-self) that are not deleted. That means that in a transplant, there are a large number of T cells that can recognize the non-self MHC in a non-peptide-dependent manner. There's also a much smaller number that can interact with the non-self-MHC but only in a peptide-dependent manner; this is just by chance, in that that particular T cell receptor happened to have the shape that sees that non-self-MHC and the peptide.
As for autoimmunity, that's all peptide-dependent, as far as we know, barring some of the really horrendous infant-onset autoimmunities that result from improper T cell maturation in the thymus.