Skip to content

Add reference confidence i.e. GVCF output for Mutect2 - #5312

Merged
ldgauthier merged 1 commit into
masterfrom
ldg_mutectRefConf
Jan 4, 2019
Merged

ldgauthier merged 1 commit into
masterfrom
ldg_mutectRefConf

Conversation

@ldgauthier

Copy link
Copy Markdown

Since we're using Mutect2 for mitochondrial calling, we want some reference confidence representation for joint calling.

I did my best, but further refactoring suggestions appreciated. Tests to follow.

@davidbenjamin can you take a look at the LODs in the integration test results? I'm not entirely surprised that at the same depth, the variant LOD is higher than the reference LOD. I'm not sure that the NON_REF LOD at variant sites is coming out right though. Is there an effective negative LOD asymptote?

@ldgauthier

Copy link
Copy Markdown
Author

Actually, the GVCFWriter-related code doesn't need to be reviewed until after Tom's PR (#5311) goes in because I'm going to have a heck of a rebase. But the Mutect2 part shouldn't change.

@davidbenjamin davidbenjamin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I think I've gone through this pretty thoroughly. One more request in addition to the line-by-line comments would be a paragraph or two in mutect.{tex,pdf} describing the definition of the M2 reference confidence TLOD.

*/
public static List<ReadPileup> getPileupsOverReference(final Haplotype refHaplotype,
final Collection<Haplotype> calledHaplotypes,
final SimpleInterval paddedReferenceLoc,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

paddedReferenceLoc is not used.

public static List<ReadPileup> getPileupsOverReference(final Haplotype refHaplotype,
final Collection<Haplotype> calledHaplotypes,
final SimpleInterval paddedReferenceLoc,
final AssemblyRegion activeRegion,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You only use this for its getHeader(), so it could be clearer to replace this with a SAMFileHeader argument.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, the only place this method is called the activeRegionSpan is just activeRegion.getSpan(), so you could dispense with that argument and just pass the active region.

final Collection<Haplotype> calledHaplotypes,
final SimpleInterval paddedReferenceLoc,
final AssemblyRegion activeRegion,
final SimpleInterval activeRegionSpan,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename to refSpan?

/**
* Get a list of pileups that span the entire active region span, in order, one for each position
*/
public static List<ReadPileup> getPileupsOverReference(final Haplotype refHaplotype,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't seem like you need refHaplotype or calledHaplotypes.

* Creates a new ref-vs-alt result indicating the genotype likelihood vector capacity.
*/
SomaticRefVsAnyResult() {
lods = new PerAlleleCollection<>(PerAlleleCollection.Type.REF_AND_ALT);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In SomaticReferenceConfidenceModel the lods are set to the output of SomaticGenotypingEngine, which is Type.ALT_ONLY.

nonRefLikelihood = QualityUtils.qualToErrorProbLog10(qual) + MathUtils.LOG10_ONE_THIRD;
result.refDepth++;
}
readLikelihoods.sampleMatrix(0).set(0, i, nonRefLikelihood);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're calculating nonRefLikelihood why are you setting allele 0 and not 1 here?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I'm having trouble wrapping my head around the variable allele fraction. There's something that surprises me about the asymmetry of your model.

1 Q10 alt and 758 refs is -2.8 against the NON-REF (if I change the calculations as you suggest)
1 ref and 818 alts is 2596 for the ALT

If the statistic is a log odds then I would expect the alt versus ref LOD to be the negative of the ref versus alt LOD but those nearly complementary pileups have drastically different LODs. Either my assumption of symmetry (LOD of the flipped comparison being negative of the original) is wrong or there's something I don't understand about the priors.

@davidbenjamin davidbenjamin Oct 31, 2018 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The -2.8 isn't really against the NON-REF, per se. It's -2.8 for the NON-REF, that is to say, very weak evidence that the NON-REF allele might exist.

The somatic likelihoods model doesn't assign a LOD to the reference allele. In fact, it doesn't compare the reference allele to the alt alleles directly. Rather, it's the log odds of excluding each alt allele. That is, your calculations mean that

  • 1 alt and 758 refs: the likelihood of no variation (only the ref is present) is 10^2.8 times as great as the likelihood for both the ref and alt to be present. This makes sense -- the model is penalized for the greater complexity but adding in the alt allele doesn't do any huge harm.
  • 1 ref and 818 alts: the likelihood of no variation is 10^2596 times smaller than the likelihood for the ref and alt to be present. This also makes sense because without the alt allele there are 818 reads that would have to be explained as sequencing error.

In the case of 1 alt and 758 refs if we calculated the probability of just the alt allele and no ref we would also get something preposterous like 10^2596.

I could calculate the ref allele's LOD to restore symmetry if you think the M2 GVCF needs it.


@Override
public void addGenotypeData(final ReferenceConfidenceResult result, final GenotypeBuilder gb) {
gb.attribute(GATKVCFConstants.TUMOR_LOD_KEY, ((SomaticRefVsAnyResult)result).lods.get(Allele.NON_REF_ALLELE));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, this means that the reference confidence "score" is the TLOD of the non-ref allele, which is a little strange because this score increases as the evidence for a non-ref allele increases. Also, if the math is being done right then the integration test vcf values (and the default TLOD bands) of 1000 or more would imply extremely good evidence of variation in the reference confidence blocks. I think some of my math comments above might help.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to have the output be logically equivalent to the germline GQ, which is the Phred-scaled probability that the genotype call was wrong. As someone who's looked at a lot of GVCFs, it's nice to skim down the GQ column and see that everything is going well because all the numbers are on the large side, meaning the calls are all confident whether each position is variant or not. Maybe I need a different INFO key for the ref blocks because the number I'm getting out of my ref conf model is the LOD between the call (reference) and anything that's not the call (non-ref). REFLOD?

@ldgauthier

Copy link
Copy Markdown
Author

Sorry I forgot to look at any of the Javadoc. You can just bounce it back next time.

@ldgauthier

Copy link
Copy Markdown
Author

@davidbenjamin LODs are fixed, but I'm not super happy with them. They fluctuate a lot, making for a big GVCF. How easy would it be to modify the likelihood calculation to integrate over all AFs greater than some threshold of interest? I'm hoping that would produce more stability. Just to give you an idea of the fluctuation, below are some lines from my integration test VCF where I block anything less than -2 and between -2 and 0. I tried a few other thresholds, but it's just a lot of variability. I have a hunch it has to do with the minimum base quality in the pileup.

#CHROM  POS     ID      REF     ALT     QUAL    FILTER  INFO    FORMAT  NA12878
chrM    1       .       G       <NON_REF>       .       .       END=4   GT:DP:MIN_DP:TLOD       0/0:67:48:-1.958e+00
chrM    5       .       A       <NON_REF>       .       .       END=5   GT:DP:MIN_DP:TLOD       0/0:107:107:-2.033e+00
chrM    6       .       C       <NON_REF>       .       .       END=6   GT:DP:MIN_DP:TLOD       0/0:123:123:-1.603e+00
chrM    7       .       A       <NON_REF>       .       .       END=8   GT:DP:MIN_DP:TLOD       0/0:135:135:-2.138e+00
chrM    9       .       G       <NON_REF>       .       .       END=9   GT:DP:MIN_DP:TLOD       0/0:138:138:-1.975e+00
chrM    10      .       T       <NON_REF>       .       .       END=13  GT:DP:MIN_DP:TLOD       0/0:178:154:-2.226e+00
chrM    14      .       T       <NON_REF>       .       .       END=15  GT:DP:MIN_DP:TLOD       0/0:208:205:-1.974e+00
chrM    16      .       A       <NON_REF>       .       .       END=23  GT:DP:MIN_DP:TLOD       0/0:259:218:-2.424e+00
chrM    24      .       A       <NON_REF>       .       .       END=25  GT:DP:MIN_DP:TLOD       0/0:312:310:-8.945e-01
chrM    26      .       C       <NON_REF>       .       .       END=26  GT:DP:MIN_DP:TLOD       0/0:317:317:-2.509e+00
chrM    27      .       C       <NON_REF>       .       .       END=27  GT:DP:MIN_DP:TLOD       0/0:335:335:-1.962e+00
chrM    28      .       A       <NON_REF>       .       .       END=50  GT:DP:MIN_DP:TLOD       0/0:492:343:-2.821e+00
chrM    51      .       T       <NON_REF>       .       .       END=51  GT:DP:MIN_DP:TLOD       0/0:700:700:-3.808e-01
chrM    52      .       T       <NON_REF>       .       .       END=63  GT:DP:MIN_DP:TLOD       0/0:822:722:-2.943e+00
chrM    64      .       C       <NON_REF>       .       .       END=64  GT:DP:MIN_DP:TLOD       0/0:909:909:-1.492e+00
chrM    65      .       T       <NON_REF>       .       .       END=86  GT:DP:MIN_DP:TLOD       0/0:1064:938:-3.065e+00
chrM    87      .       A       C,<NON_REF>     .       .       DP=942;ECNT=8;POP_AF=5.000e-08,5.000e-08;TLOD=-2.463e+00,-2.668e+00     GT:AD:AF:DP:F1R2:F2R1:MBQ:MFRL:MMQ:MPOS:ORIGINAL_CONTIG_MISMATCH:SA_MAP_AF:SA_POST_PROB 0/1/2:921,6,0:1.676e-03,1.073e-03:927:464,3,0:457,3,0:30,10,0:377,8284,0:60,0:27,0:6:0.010,0.010,6.472e-03:3.936e-04,0.010,0.989
chrM    88      .       T       <NON_REF>       .       .       END=151 GT:DP:MIN_DP:TLOD       0/0:1212:946:-3.167e+00

@davidbenjamin

Copy link
Copy Markdown
Contributor

@ldgauthier I could imagine incorporating an AF threshold as follows: it would be pretty easy to have the somatic likelihoods engine emit the Dirichlet posterior on AF. This is already calculated internally -- it's the quantity alleleFractionsPosterior in log10Evidence -- we would just have to not throw it away. Given this, we could do the integral to compute how much of this posterior is above some allele fraction threshold for each allele. For the general multiallelic case, it goes like this:

  • Suppose the Dirichlet posterior on AFs is x, y, z. Then the marginal for, WLOG, the first alt allele is Beta(y, x + z).
  • The weight of this posterior that falls above a threshold f is the integral from 0 to f of this Beta density. This integral is called the incomplete Beta function and it's implemented in apache commons.
  • We can multiply the LOD by this weight (add the log10 of this weight to the TLOD) to get an AF -thresholded TLOD.

@ldgauthier ldgauthier changed the title Add reference confidence i.e. GVCF output for Mutect2 (do not merge) Add reference confidence i.e. GVCF output for Mutect2 Dec 14, 2018
@ldgauthier

Copy link
Copy Markdown
Author

For the record, I'm not thrilled with what the dynamic pruning has done to the MT integration test output compared to the old --min-pruning 5 argument, but for some reason the coverage in that bam ramps up pretty slowly, so it's not representative of our WGS data. But I'd like to see some more analysis of the dynamic pruning for exomes before this goes into our exome pipeline in the future.

@davidbenjamin

Copy link
Copy Markdown
Contributor

@meganshand has found that the new pruning tends to shift the TLOD and thus requires a few parameter defaults to change. I hope I'm not oversimplifying too badly.

@davidbenjamin

Copy link
Copy Markdown
Contributor

But I'd like to see some more analysis of the dynamic pruning for exomes before this goes into our exome pipeline in the future.

@ldgauthier Do you mean somatic exomes or mitochondrial reads from germline exomes?

@ldgauthier

Copy link
Copy Markdown
Author

@davidbenjamin I added the AF thresholding and limited it to just the NON_REF allele. There's a mildly awkward check in the integration test for it. If you can formulate an elegant unit test that makes for nice round numbers I'm happy to add it, but that's beyond me.

I lost the new Twist exome, but once I can get ahold of it I'll make a GVCF and see what happens.

@ldgauthier

Copy link
Copy Markdown
Author

Nope, not ready. I've got a TLOD=-Infinity I need to deal with.

@ldgauthier

Copy link
Copy Markdown
Author

Maybe TLOD = -Inf isn't so unreasonable. If the depth is ~4000 and you're looking for variants of at least 1% AF then you can be pretty darn sure there isn't one.

@davidbenjamin

Copy link
Copy Markdown
Contributor

It sounds reasonable to me.

@codecov-io

codecov-io commented Dec 21, 2018 •

Copy link
Copy Markdown

Codecov Report

Merging #5312 into master will decrease coverage by 0.011%.
The diff coverage is 84.483%.

@@               Coverage Diff               @@
##              master     #5312       +/-   ##
===============================================
- Coverage     87.083%   87.072%   -0.011%     
- Complexity     31250     31334       +84     
===============================================
  Files           1915      1926       +11     
  Lines         144178    144527      +349     
  Branches       15910     15984       +74     
===============================================
+ Hits          125554    125842      +288     
- Misses         12839     12862       +23     
- Partials        5785      5823       +38
Impacted Files Coverage Δ Complexity Δ
...otypecaller/HaplotypeCallerArgumentCollection.java 100% <ø> (ø) 3 <0> (ø) ⬇️
...er/engine/spark/datasources/VariantsSparkSink.java 89.655% <ø> (ø) 9 <0> (ø) ⬇️
...stitute/hellbender/tools/HaplotypeCallerSpark.java 82.759% <ø> (ø) 20 <0> (ø) ⬇️
...roadinstitute/hellbender/utils/read/ReadUtils.java 79.812% <ø> (ø) 204 <0> (ø) ⬇️
...ls/variant/writers/GVCFBlockCombiningIterator.java 100% <ø> (ø) 1 <0> (ø) ⬇️
...e/hellbender/utils/variant/GATKVCFHeaderLines.java 95.833% <100%> (+0.025%) 10 <0> (ø) ⬇️
...walkers/haplotypecaller/HaplotypeCallerEngine.java 78.201% <100%> (-0.224%) 73 <0> (-3)
...ers/haplotypecaller/ReferenceConfidenceResult.java 100% <100%> (ø) 3 <3> (?)
...kers/haplotypecaller/ReferenceConfidenceModel.java 92.391% <100%> (+0.01%) 69 <2> (-7) ⬇️
.../tools/walkers/haplotypecaller/RefVsAnyResult.java 100% <100%> (ø) 2 <2> (?)
... and 47 more

@ldgauthier

Copy link
Copy Markdown
Author

@davidbenjamin ready for re-review.

I'm going to do a little performance benchmarking in the meantime. It takes ~40min to call the whole contig for my 4000X bams, which isn't terrible, but it isn't great. Anecdotally it seems like the AF thresholding slowed things down, but I'll collect some numbers.

@ldgauthier

Copy link
Copy Markdown
Author

Wow, it's 5 minutes not in GVCF mode. :-( AF thresholding doesn't seem to have made a difference, but apparently evaluating the likelihoods model for every single position on the MT contig is time consuming. Who would have guessed?

I will do a comparison over some autosomal territory after the break. This incarnation of GVCF mode may or may not be reasonable outside of MT calling. Maybe we should have a leaner ref conf likelihoods calculation like HaplotypeCaller has. I will brainstorm on that.

@davidbenjamin

Copy link
Copy Markdown
Contributor

I wouldn't be surprised if I could optimize the implementation of the likelihoods model quite a bit. Just having a slightly looser threshold for convergence, for example.

@ldgauthier

Copy link
Copy Markdown
Author

I'll open a ticket for ref conf performance optimization, but I'm keen to get a "draft" of the MT joint calling pipeline into 4.1.

@davidbenjamin

Copy link
Copy Markdown
Contributor

Sounds good.

…2 to enable mitochondrial joint calling

Also a fix for ValidateVariants gvcf mode;  updated invalid test resources
@ldgauthier
ldgauthier merged commit 46b26eb into master Jan 4, 2019
@ldgauthier
ldgauthier deleted the ldg_mutectRefConf branch January 4, 2019 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants