I had worked on a similar problem and I am telling my experience.
First, you have to consider the fact that a mature miRNA can arise from multiple genomic locations i.e. many pre-miRNAs can give rise to same mature miRNA.
As you said there are products of pre-miRNA cleavage with heterogeneity at their ends i.e. there are sequences that do not exactly match the annotated mature region. While products with 3' heterogeneity can be called isoforms, products with 5' heterogeneity are likely to have different targets as the seed sequence is disrupted and hence they are technically different miRNAs (by definition).
Now, the best technique to analyse isoforms is small RNA sequencing (NGS). qPCR would not be a good technique as primers cannot always differentiate highly similar sequences (worsened by the fact that miRNAs are too small). A NGS library is a set of DNA sequences that you obtain from shearing the RNA/DNA followed by adapter ligation (reverse transcription for RNA). Since the adapter sequence is known, the sequencing reaction does not require you to know the sequence of the products. However, you would need to know the reference genome for finding novel miRNAs. This is primarily due to the fact that it is very difficult to sequence pre-miRNAs (low abundance and hairpin structure). You can identify new small RNAs very easily but cannot say that they are miRNAs unless you can map them to a stem-loop precursor (pre-miRNA).
You can very well discover new reads and people have discovered a lot of new miRNAs using NGS. I can tell you the methodology I followed to find atypical products of the same pre-miRNA. I for simplicity I call the 5' heterogeneous products as badseeds (because they have disrupted seed sequence) and 3' heterogeneous products as isomiRs. I used a methodology that is similar to that of a well known program – miRdeep.
- Get the pre-miRNA sequences from miRbase
- Get mature miRNA sequences from miRbase
- Use bowtie-1 to build an index from the pre-miRNA sequences
- Remove adapter from your fastq reads and remove low quality reads. There are many tools to do that. You can try Trimmomatic.
- Map mature miRNA sequences to pre-miRNA index using bowtie to find the exact location of mature miRNA in pre-miRNA
- Map reads to pre-miRNA region. Filter reads that do not map in a pre-defined window around mature region.
- Classify the reads as badseed or isomiR
Now, isomiRs and reads that perfectly map to mature miRNA region can be considered as contributors to total miRNA expression. However, badseeds should not be considered as an isoform. You can likewise allow a certain level of mismatch and count these reads. Sequences with mismatch in seed region would again be badseeds. You can additionally have scoring schemes for your reads.
There are not many studies on such analysis. I did not get any publishable result with our data but the methodology is reasonably correct. You can go ahead an apply it for your study.