Original Authors: Belinda Phipson, Anna Trigos, Matt Ritchie, Maria Doyle, Harriet Dashnow, Charity Law Based on the course RNAseq analysis in R delivered on May 11/12th 2016

Resources and data files

This material has been created using the following resources:
http://www.statsci.org/smyth/pubs/QLedgeRPreprint.pdf [@Lun2016]
http://monashbioinformaticsplatform.github.io/RNAseq-DE-analysis-with-R/99-RNAseq_DE_analysis_with_R.html

Data files downloaded from:
ftp://ftp.ncbi.nlm.nih.gov/geo/series/GSE60nnn/GSE60450/suppl/GSE60450_Lactation-GenewiseCounts.txt.gz http://bioinf.wehi.edu.au/software/MSigDB/mouse_c2_v5.rdata http://bioinf.wehi.edu.au/software/MSigDB/mouse_H_v5.rdata

Data files:
sampleinfo.txt
GSE60450_Lactation-GenewiseCounts.txt
mouse_c2_v5.rdata
mouse_H_v5.rdata

Data files available from: https://figshare.com/s/1d788fd384d33e913a2a You should download these files and place them in your /data directory.

Differential expression with edgeR

Now that we are happy that we have normalised the data and that the quality looks good, we can continue to testing for differentially expressed genes. There are a number of packages to analyse RNA-Seq data. Most people use DESEQ2 or edgeR. We will use edgeR for the rest of this practical.

**First make sure we have all the objects and libraries loaded*

library(edgeR)
library(limma)
library(Glimma)
library(gplots)
library(org.Mm.eg.db)
load("Robjects/preprocessing.Rdata")

Recap of pre-processing

The previous section walked-through the pre-processing and transformation of the count data. Here, for completeness, we list the minimal steps required to process the data prior to differential expression analysis.

## Read the counts from the downloaded data
seqdata <- read.delim("data/GSE60450_Lactation-GenewiseCounts.txt", stringsAsFactors = FALSE)
#
# Remove first two columns from seqdata

countdata <- seqdata[,-(1:2)]

# Store EntrezGeneID as rownames
rownames(countdata) <- seqdata[,1]
countdata
colnames(countdata) <- substr(colnames(countdata), 1, 7)
countdata
## Calculate the Counts Per Million measure
myCPM <- cpm(countdata)
## Identify genes with at least 0.5 cpm in at least 2 samples
thresh <- myCPM > 0.5
keep <- rowSums(thresh) >= 2
# Subset the rows of countdata to keep the more highly expressed genes
counts.keep <- countdata[keep,]
## Convert to an edgeR object
dgeObj <- DGEList(counts.keep)
## Perform TMM normalisation
dgeObj <- calcNormFactors(dgeObj)
## Obtain corrected sample information
sampleinfo <- read.delim("data/SampleInfo_Corrected.txt")
group <- paste(sampleinfo$CellType,sampleinfo$Status,sep=".")
group

Create the design matrix

First we need to create a design matrix for the groups, as we have seen in the linear models lecture. We have two variables, status and cell type. We will fit two models under two assumptions; no interaction and interaction of these two factors.

Let’s start with the model with only main effects, that is no interaction. The main assumption here is that the effect of the status is the same in both type of cells.

# Create the two variables
group <- as.character(group)
type <- sapply(strsplit(group, ".", fixed=T), function(x) x[1])
status <- sapply(strsplit(group, ".", fixed=T), function(x) x[2])
# Specify a design matrix with an intercept term
design <- model.matrix(~ type + status)
design

Data exploration

An MDS plot shows distances, in terms of biological coefficient of variation (BCV), between samples. What do you think of the quality of the data? Can you anticipate if the interaction term will be important?

plotMDS(dgeObj, labels=group, cex=0.75, xlim=c(-4, 5))

Estimating the dispersion

The common dispersion estimates the overall BCV of the dataset, averaged over all genes:

dgeObj <- estimateCommonDisp(dgeObj)

Then we estimate gene-wise dispersion estimates, allowing a possible trend with averge count size:

dgeObj <- estimateGLMTrendedDisp(dgeObj)
dgeObj <- estimateTagwiseDisp(dgeObj)

Plot the estimated dispersions:

plotBCV(dgeObj)

Testing for differential expression

First, we fit genewise glms:

# Fit the linear model
fit <- glmFit(dgeObj, design)
names(fit)
head(coef(fit))

Conduct likelihood ratio tests for luminal vs basal and show the top genes:

lrt.BvsL <- glmLRT(fit, coef=2)
topTags(lrt.BvsL)

Challenge

Conduct likelihood ratio tests for virgin vs lactate and show the top genes.

Contrasts

Suppose we want to find differentially expressed genes between pregnant and virgin. We don’t have a parameter that explicitly will allow us to test that hypothesis. We need to build a contrast:

PvsV <- makeContrasts(statuspregnant-statusvirgin, levels=design)
lrt.pVsV <- glmLRT(fit, contrast=PvsV)
topTags(lrt.pVsV)

Challenge

1.Fit a model with interaction: What is the rationale to include the interaction (What assumption are you relaxing?) 2. Is the number of replicates good enough to include the interaction? 3. Is the interaction needed in the model?

Solution

Testing relative to a threshold

Statistical tests can sometimes find small log-fold changes to be highly significant. In many biological applications, small fold-change differences may not be interesting, in which case we would like to find “significant” DE genes that also have a minimum fold-change. The glmTreat function to test for differential expression relative to a fold-change threshold that we apply.

In this example, we will say that we are not interesting in any gene that has a fold-change less than 1.5, so we set the threshold to log2(1.5).

trt.BvsL <- glmTreat(fit, coef = 2, lfc = log2(1.5))
topTags(trt.BvsL)

Here, the top DE genes have large fold-changes anyway, but observe the difference in p-values between this TREAT test and the standard LRT above.

Challenge

  1. Experiment with different lfc thresholds in glmTreat. How do these changes affect the results and any conclusions you might draw?

Solution

save(lrt.BvsL, dgeObj, group,file="Robjects/DE.Rdata")

Alternative: limma analysis

If the sequencing depth is reasonably consistent across the RNA samples, then the simplest and most robust approach to differential exis to use limma-trend. This approach will usually work well if the ratio of the largest library size to the smallest is not more than about 3-fold.

limma-trend

In the limma-trend approach, the counts are converted to logCPM values using edgeR’s cpm function:

logCPM <- cpm(dgeObj, log = TRUE, prior.count = 3)

The prior count is used here to damp down the variances of logarithms of low counts. The logCPM values can then be used in any standard limma pipeline, using the trend=TRUE argument when running eBayes. For example:

lmfit <- lmFit(logCPM, design)
lmfit <- eBayes(lmfit, trend=TRUE)
topTable(lmfit, coef = 2)

How does this gene ranking compare to that obtained with edgeR?

voom

When the library sizes are quite variable between samples, then the voom approach is theoretically more powerful than limma-trend. In this approach, the voom transformation is applied to the normalized and filtered DGEList object:

v <- voom(dgeObj, design, plot = TRUE)

After this, the usual limma pipelines for differential expression can be applied, for example:

vfit <- lmFit(v, design)
vfit <- eBayes(vfit)
topTable(vfit, coef = 2)

Or, to give more weight to fold-changes in the ranking, one could use a treat approach as introduced earlier:

tvfit <- treat(vfit, lfc = log2(1.5))
topTreat(tvfit, coef = 2)

Challenge

  1. Compare the results obtained with edgeR, treat, limma-trend and limma-voom. How do they differ? Are these differences important?

Solution

LS0tCnRpdGxlOiAiUk5BLXNlcSBkaWZmZXJlbnRpYWwgZXhwcmVzc2lvbiBhbmFseXNpcyBpbiBSIgphdXRob3I6ICJEYXZpcyBNY0NhcnRoeSwgU3RlcGhhbmUgQmFsbGVyZWF1LCBNYXJrIER1bm5pbmcsIE9zY2FyIFJ1ZWRhLCBBc2hsZXkgU2F3bGUiCmRhdGU6ICdgciBmb3JtYXQoU3lzLnRpbWUoKSwgIkxhc3QgbW9kaWZpZWQ6ICVkICViICVZIilgJwpvdXRwdXQ6CiAgaHRtbF9ub3RlYm9vazoKICAgIHRvYzogeWVzCiAgICB0b2NfZmxvYXQ6IHllcwogIGh0bWxfZG9jdW1lbnQ6CiAgICB0b2M6IHllcwogICAgdG9jX2Zsb2F0OiB5ZXMKbWludXRlczogMzAwCnN1YnRpdGxlOiBVc2luZyBlZGdlUiBhbmQgbGltbWEKbGF5b3V0OiBwYWdlCi0tLQoKYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9CmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gVFJVRSkKYGBgCgoqKk9yaWdpbmFsIEF1dGhvcnM6IEJlbGluZGEgUGhpcHNvbiwgQW5uYSBUcmlnb3MsIE1hdHQgUml0Y2hpZSwgTWFyaWEgRG95bGUsIEhhcnJpZXQgRGFzaG5vdywgQ2hhcml0eSBMYXcqKgpCYXNlZCBvbiB0aGUgY291cnNlIFtSTkFzZXEgYW5hbHlzaXMgaW4gUl0oaHR0cDovL2NvbWJpbmUtYXVzdHJhbGlhLmdpdGh1Yi5pby8yMDE2LTA1LTExLVJOQXNlcS8pIGRlbGl2ZXJlZCBvbiBNYXkgMTEvMTJ0aCAyMDE2CgojIyBSZXNvdXJjZXMgYW5kIGRhdGEgZmlsZXMKClRoaXMgbWF0ZXJpYWwgaGFzIGJlZW4gY3JlYXRlZCB1c2luZyB0aGUgZm9sbG93aW5nIHJlc291cmNlczogIApodHRwOi8vd3d3LnN0YXRzY2kub3JnL3NteXRoL3B1YnMvUUxlZGdlUlByZXByaW50LnBkZiBbQEx1bjIwMTZdICAKaHR0cDovL21vbmFzaGJpb2luZm9ybWF0aWNzcGxhdGZvcm0uZ2l0aHViLmlvL1JOQXNlcS1ERS1hbmFseXNpcy13aXRoLVIvOTktUk5Bc2VxX0RFX2FuYWx5c2lzX3dpdGhfUi5odG1sICAKCkRhdGEgZmlsZXMgZG93bmxvYWRlZCBmcm9tOiAgCmZ0cDovL2Z0cC5uY2JpLm5sbS5uaWguZ292L2dlby9zZXJpZXMvR1NFNjBubm4vR1NFNjA0NTAvc3VwcGwvR1NFNjA0NTBfTGFjdGF0aW9uLUdlbmV3aXNlQ291bnRzLnR4dC5negpodHRwOi8vYmlvaW5mLndlaGkuZWR1LmF1L3NvZnR3YXJlL01TaWdEQi9tb3VzZV9jMl92NS5yZGF0YQpodHRwOi8vYmlvaW5mLndlaGkuZWR1LmF1L3NvZnR3YXJlL01TaWdEQi9tb3VzZV9IX3Y1LnJkYXRhCgpEYXRhIGZpbGVzOiAgCnNhbXBsZWluZm8udHh0ICAKR1NFNjA0NTBfTGFjdGF0aW9uLUdlbmV3aXNlQ291bnRzLnR4dCAgCm1vdXNlX2MyX3Y1LnJkYXRhICAKbW91c2VfSF92NS5yZGF0YQoKRGF0YSBmaWxlcyBhdmFpbGFibGUgZnJvbTogW2h0dHBzOi8vZmlnc2hhcmUuY29tL3MvMWQ3ODhmZDM4NGQzM2U5MTNhMmFdKGh0dHBzOi8vZmlnc2hhcmUuY29tL3MvMWQ3ODhmZDM4NGQzM2U5MTNhMmEpCllvdSBzaG91bGQgZG93bmxvYWQgdGhlc2UgZmlsZXMgYW5kIHBsYWNlIHRoZW0gaW4geW91ciBgL2RhdGFgIGRpcmVjdG9yeS4KCiMjIERpZmZlcmVudGlhbCBleHByZXNzaW9uIHdpdGggZWRnZVIKCk5vdyB0aGF0IHdlIGFyZSBoYXBweSB0aGF0IHdlIGhhdmUgbm9ybWFsaXNlZCB0aGUgZGF0YSBhbmQgdGhhdCB0aGUgcXVhbGl0eSBsb29rcyBnb29kLCB3ZSBjYW4gY29udGludWUgdG8gdGVzdGluZyBmb3IgZGlmZmVyZW50aWFsbHkgZXhwcmVzc2VkIGdlbmVzLiBUaGVyZSBhcmUgYSBudW1iZXIgb2YgcGFja2FnZXMgdG8gYW5hbHlzZSBSTkEtU2VxIGRhdGEuIE1vc3QgcGVvcGxlIHVzZSBERVNFUTIgb3IgZWRnZVIuIFdlIHdpbGwgdXNlIGVkZ2VSIGZvciB0aGUgcmVzdCBvZiB0aGlzIHByYWN0aWNhbC4KCioqRmlyc3QgbWFrZSBzdXJlIHdlIGhhdmUgYWxsIHRoZSBvYmplY3RzIGFuZCBsaWJyYXJpZXMgbG9hZGVkKgoKYGBge3J9CmxpYnJhcnkoZWRnZVIpCmxpYnJhcnkobGltbWEpCmxpYnJhcnkoR2xpbW1hKQpsaWJyYXJ5KGdwbG90cykKbGlicmFyeShvcmcuTW0uZWcuZGIpCmxvYWQoIlJvYmplY3RzL3ByZXByb2Nlc3NpbmcuUmRhdGEiKQpgYGAKCgojIyMgUmVjYXAgb2YgcHJlLXByb2Nlc3NpbmcKClRoZSBwcmV2aW91cyBzZWN0aW9uIHdhbGtlZC10aHJvdWdoIHRoZSBwcmUtcHJvY2Vzc2luZyBhbmQgdHJhbnNmb3JtYXRpb24gb2YgdGhlIGNvdW50IGRhdGEuIEhlcmUsIGZvciBjb21wbGV0ZW5lc3MsIHdlIGxpc3QgdGhlIG1pbmltYWwgc3RlcHMgcmVxdWlyZWQgdG8gcHJvY2VzcyB0aGUgZGF0YSBwcmlvciB0byBkaWZmZXJlbnRpYWwgZXhwcmVzc2lvbiBhbmFseXNpcy4KCmBgYHtyIGV2YWw9RkFMU0V9CiMjIFJlYWQgdGhlIGNvdW50cyBmcm9tIHRoZSBkb3dubG9hZGVkIGRhdGEKc2VxZGF0YSA8LSByZWFkLmRlbGltKCJkYXRhL0dTRTYwNDUwX0xhY3RhdGlvbi1HZW5ld2lzZUNvdW50cy50eHQiLCBzdHJpbmdzQXNGYWN0b3JzID0gRkFMU0UpCiMKIyBSZW1vdmUgZmlyc3QgdHdvIGNvbHVtbnMgZnJvbSBzZXFkYXRhCgpjb3VudGRhdGEgPC0gc2VxZGF0YVssLSgxOjIpXQoKIyBTdG9yZSBFbnRyZXpHZW5lSUQgYXMgcm93bmFtZXMKcm93bmFtZXMoY291bnRkYXRhKSA8LSBzZXFkYXRhWywxXQpjb3VudGRhdGEKY29sbmFtZXMoY291bnRkYXRhKSA8LSBzdWJzdHIoY29sbmFtZXMoY291bnRkYXRhKSwgMSwgNykKY291bnRkYXRhCiMjIENhbGN1bGF0ZSB0aGUgQ291bnRzIFBlciBNaWxsaW9uIG1lYXN1cmUKbXlDUE0gPC0gY3BtKGNvdW50ZGF0YSkKIyMgSWRlbnRpZnkgZ2VuZXMgd2l0aCBhdCBsZWFzdCAwLjUgY3BtIGluIGF0IGxlYXN0IDIgc2FtcGxlcwp0aHJlc2ggPC0gbXlDUE0gPiAwLjUKa2VlcCA8LSByb3dTdW1zKHRocmVzaCkgPj0gMgojIFN1YnNldCB0aGUgcm93cyBvZiBjb3VudGRhdGEgdG8ga2VlcCB0aGUgbW9yZSBoaWdobHkgZXhwcmVzc2VkIGdlbmVzCmNvdW50cy5rZWVwIDwtIGNvdW50ZGF0YVtrZWVwLF0KIyMgQ29udmVydCB0byBhbiBlZGdlUiBvYmplY3QKZGdlT2JqIDwtIERHRUxpc3QoY291bnRzLmtlZXApCiMjIFBlcmZvcm0gVE1NIG5vcm1hbGlzYXRpb24KZGdlT2JqIDwtIGNhbGNOb3JtRmFjdG9ycyhkZ2VPYmopCiMjIE9idGFpbiBjb3JyZWN0ZWQgc2FtcGxlIGluZm9ybWF0aW9uCnNhbXBsZWluZm8gPC0gcmVhZC5kZWxpbSgiZGF0YS9TYW1wbGVJbmZvX0NvcnJlY3RlZC50eHQiKQpncm91cCA8LSBwYXN0ZShzYW1wbGVpbmZvJENlbGxUeXBlLHNhbXBsZWluZm8kU3RhdHVzLHNlcD0iLiIpCmdyb3VwCmBgYAoKCiMjIyBDcmVhdGUgdGhlIGRlc2lnbiBtYXRyaXgKCkZpcnN0IHdlIG5lZWQgdG8gY3JlYXRlIGEgZGVzaWduIG1hdHJpeCBmb3IgdGhlIGdyb3VwcywgYXMgd2UgaGF2ZSBzZWVuIGluIHRoZSBsaW5lYXIgbW9kZWxzIGxlY3R1cmUuIApXZSBoYXZlIHR3byB2YXJpYWJsZXMsIHN0YXR1cyBhbmQgY2VsbCB0eXBlLiBXZSB3aWxsIGZpdCB0d28gbW9kZWxzIHVuZGVyIHR3byBhc3N1bXB0aW9uczsgbm8gaW50ZXJhY3Rpb24gYW5kIGludGVyYWN0aW9uIG9mIHRoZXNlIHR3byBmYWN0b3JzLiAKCkxldCdzIHN0YXJ0IHdpdGggdGhlIG1vZGVsIHdpdGggb25seSBtYWluIGVmZmVjdHMsIHRoYXQgaXMgbm8gaW50ZXJhY3Rpb24uIFRoZSBtYWluIGFzc3VtcHRpb24gaGVyZSBpcyB0aGF0IHRoZSBlZmZlY3Qgb2YgdGhlIHN0YXR1cyBpcyB0aGUgc2FtZSBpbiBib3RoIHR5cGUgb2YgY2VsbHMuCgpgYGB7cn0KIyBDcmVhdGUgdGhlIHR3byB2YXJpYWJsZXMKZ3JvdXAgPC0gYXMuY2hhcmFjdGVyKGdyb3VwKQp0eXBlIDwtIHNhcHBseShzdHJzcGxpdChncm91cCwgIi4iLCBmaXhlZD1UKSwgZnVuY3Rpb24oeCkgeFsxXSkKc3RhdHVzIDwtIHNhcHBseShzdHJzcGxpdChncm91cCwgIi4iLCBmaXhlZD1UKSwgZnVuY3Rpb24oeCkgeFsyXSkKIyBTcGVjaWZ5IGEgZGVzaWduIG1hdHJpeCB3aXRoIGFuIGludGVyY2VwdCB0ZXJtCmRlc2lnbiA8LSBtb2RlbC5tYXRyaXgofiB0eXBlICsgc3RhdHVzKQpkZXNpZ24KYGBgCgoKCiMjIyBEYXRhIGV4cGxvcmF0aW9uCkFuIE1EUyBwbG90IHNob3dzIGRpc3RhbmNlcywgaW4gdGVybXMgb2YgYmlvbG9naWNhbCBjb2VmZmljaWVudCBvZiB2YXJpYXRpb24gKEJDViksIGJldHdlZW4gc2FtcGxlcy4gV2hhdCBkbyB5b3UgdGhpbmsgb2YgdGhlIHF1YWxpdHkgb2YgdGhlIGRhdGE/IENhbiB5b3UgYW50aWNpcGF0ZSBpZiB0aGUgaW50ZXJhY3Rpb24gdGVybSB3aWxsIGJlIGltcG9ydGFudD8KYGBge3J9CnBsb3RNRFMoZGdlT2JqLCBsYWJlbHM9Z3JvdXAsIGNleD0wLjc1LCB4bGltPWMoLTQsIDUpKQpgYGAKIyMjIEVzdGltYXRpbmcgdGhlIGRpc3BlcnNpb24KClRoZSBjb21tb24gZGlzcGVyc2lvbiBlc3RpbWF0ZXMgdGhlIG92ZXJhbGwgQkNWIG9mIHRoZSBkYXRhc2V0LCBhdmVyYWdlZCBvdmVyIGFsbCBnZW5lczoKYGBge3J9CmRnZU9iaiA8LSBlc3RpbWF0ZUNvbW1vbkRpc3AoZGdlT2JqKQpgYGAKClRoZW4gd2UgZXN0aW1hdGUgZ2VuZS13aXNlIGRpc3BlcnNpb24gZXN0aW1hdGVzLCBhbGxvd2luZyBhIHBvc3NpYmxlIHRyZW5kIHdpdGggYXZlcmdlIGNvdW50IHNpemU6CmBgYHtyfQpkZ2VPYmogPC0gZXN0aW1hdGVHTE1UcmVuZGVkRGlzcChkZ2VPYmopCmRnZU9iaiA8LSBlc3RpbWF0ZVRhZ3dpc2VEaXNwKGRnZU9iaikKYGBgClBsb3QgdGhlIGVzdGltYXRlZCBkaXNwZXJzaW9uczoKYGBge3J9CnBsb3RCQ1YoZGdlT2JqKQpgYGAKCgojIyMgVGVzdGluZyBmb3IgZGlmZmVyZW50aWFsIGV4cHJlc3Npb24KCkZpcnN0LCB3ZSBmaXQgZ2VuZXdpc2UgZ2xtczoKCmBgYHtyfQojIEZpdCB0aGUgbGluZWFyIG1vZGVsCmZpdCA8LSBnbG1GaXQoZGdlT2JqLCBkZXNpZ24pCm5hbWVzKGZpdCkKaGVhZChjb2VmKGZpdCkpCmBgYApDb25kdWN0IGxpa2VsaWhvb2QgcmF0aW8gdGVzdHMgZm9yIGx1bWluYWwgdnMgYmFzYWwgYW5kIHNob3cgdGhlIHRvcCBnZW5lczoKYGBge3J9CmxydC5CdnNMIDwtIGdsbUxSVChmaXQsIGNvZWY9MikKdG9wVGFncyhscnQuQnZzTCkKCmBgYAoKCj4gIyMgQ2hhbGxlbmdlIHsuY2hhbGxlbmdlfQo+IENvbmR1Y3QgbGlrZWxpaG9vZCByYXRpbyB0ZXN0cyBmb3IgdmlyZ2luIHZzIGxhY3RhdGUgYW5kIHNob3cgdGhlIHRvcCBnZW5lcy4KCmBgYHtyfQoKYGBgCgoKCiMjIyBDb250cmFzdHMKClN1cHBvc2Ugd2Ugd2FudCB0byBmaW5kIGRpZmZlcmVudGlhbGx5IGV4cHJlc3NlZCBnZW5lcyBiZXR3ZWVuIHByZWduYW50IGFuZCB2aXJnaW4uIFdlIGRvbid0IGhhdmUgYSBwYXJhbWV0ZXIgdGhhdCBleHBsaWNpdGx5IHdpbGwgYWxsb3cgdXMgdG8gdGVzdCB0aGF0IGh5cG90aGVzaXMuIFdlIG5lZWQgdG8gYnVpbGQgYSBjb250cmFzdDoKCmBgYHtyfQpQdnNWIDwtIG1ha2VDb250cmFzdHMoc3RhdHVzcHJlZ25hbnQtc3RhdHVzdmlyZ2luLCBsZXZlbHM9ZGVzaWduKQpscnQucFZzViA8LSBnbG1MUlQoZml0LCBjb250cmFzdD1QdnNWKQp0b3BUYWdzKGxydC5wVnNWKQpgYGAKCgo+ICMjIENoYWxsZW5nZSB7LmNoYWxsZW5nZX0KPgo+IDEuRml0IGEgbW9kZWwgd2l0aCBpbnRlcmFjdGlvbjogV2hhdCBpcyB0aGUgcmF0aW9uYWxlIHRvIGluY2x1ZGUgdGhlIGludGVyYWN0aW9uIChXaGF0IGFzc3VtcHRpb24gYXJlIHlvdSByZWxheGluZz8pCj4gMi4gSXMgdGhlIG51bWJlciBvZiByZXBsaWNhdGVzIGdvb2QgZW5vdWdoIHRvIGluY2x1ZGUgdGhlIGludGVyYWN0aW9uPwo+IDMuIElzIHRoZSBpbnRlcmFjdGlvbiBuZWVkZWQgaW4gdGhlIG1vZGVsPwoKKipTb2x1dGlvbioqCmBgYHtyLGVjaG89RkFMU0V9CiMgU29sdXRpb24KYGBgCgoKIyMjIFRlc3RpbmcgcmVsYXRpdmUgdG8gYSB0aHJlc2hvbGQKClN0YXRpc3RpY2FsIHRlc3RzIGNhbiBzb21ldGltZXMgZmluZCBzbWFsbCBsb2ctZm9sZCBjaGFuZ2VzIHRvIGJlIGhpZ2hseSAKc2lnbmlmaWNhbnQuIEluIG1hbnkgYmlvbG9naWNhbCBhcHBsaWNhdGlvbnMsIHNtYWxsIGZvbGQtY2hhbmdlIGRpZmZlcmVuY2VzCm1heSBub3QgYmUgaW50ZXJlc3RpbmcsIGluIHdoaWNoIGNhc2Ugd2Ugd291bGQgbGlrZSB0byBmaW5kICJzaWduaWZpY2FudCIgREUKZ2VuZXMgdGhhdCBhbHNvIGhhdmUgYSBtaW5pbXVtIGZvbGQtY2hhbmdlLiBUaGUgYGdsbVRyZWF0YCBmdW5jdGlvbiB0byB0ZXN0IGZvciAKZGlmZmVyZW50aWFsIGV4cHJlc3Npb24gcmVsYXRpdmUgdG8gYSBmb2xkLWNoYW5nZSB0aHJlc2hvbGQgdGhhdCB3ZSBhcHBseS4KCkluIHRoaXMgZXhhbXBsZSwgd2Ugd2lsbCBzYXkgdGhhdCB3ZSBhcmUgbm90IGludGVyZXN0aW5nIGluIGFueSBnZW5lIHRoYXQgaGFzIGEKZm9sZC1jaGFuZ2UgbGVzcyB0aGFuIDEuNSwgc28gd2Ugc2V0IHRoZSB0aHJlc2hvbGQgdG8gYGxvZzIoMS41KWAuCgpgYGB7ciB0cmVhdH0KdHJ0LkJ2c0wgPC0gZ2xtVHJlYXQoZml0LCBjb2VmID0gMiwgbGZjID0gbG9nMigxLjUpKQp0b3BUYWdzKHRydC5CdnNMKQpgYGAKCkhlcmUsIHRoZSB0b3AgREUgZ2VuZXMgaGF2ZSBsYXJnZSBmb2xkLWNoYW5nZXMgYW55d2F5LCBidXQgb2JzZXJ2ZSB0aGUgCmRpZmZlcmVuY2UgaW4gcC12YWx1ZXMgYmV0d2VlbiB0aGlzIFRSRUFUIHRlc3QgYW5kIHRoZSBzdGFuZGFyZCBMUlQgYWJvdmUuCgo+ICMjIENoYWxsZW5nZSB7LmNoYWxsZW5nZX0KPgo+IDEuIEV4cGVyaW1lbnQgd2l0aCBkaWZmZXJlbnQgbGZjIHRocmVzaG9sZHMgaW4gZ2xtVHJlYXQuIEhvdyBkbyB0aGVzZSBjaGFuZ2VzIAphZmZlY3QgdGhlIHJlc3VsdHMgYW5kIGFueSBjb25jbHVzaW9ucyB5b3UgbWlnaHQgZHJhdz8KCioqU29sdXRpb24qKgpgYGB7cixlY2hvPUZBTFNFfQojIFNvbHV0aW9uCmBgYAoKCmBgYHtyfQpzYXZlKGxydC5CdnNMLCBkZ2VPYmosIGdyb3VwLGZpbGU9IlJvYmplY3RzL0RFLlJkYXRhIikKYGBgCgojIyBBbHRlcm5hdGl2ZTogYGxpbW1hYCBhbmFseXNpcwoKSWYgdGhlIHNlcXVlbmNpbmcgZGVwdGggaXMgcmVhc29uYWJseSBjb25zaXN0ZW50IGFjcm9zcyB0aGUgUk5BIHNhbXBsZXMsIHRoZW4gdGhlCnNpbXBsZXN0IGFuZCBtb3N0IHJvYnVzdCBhcHByb2FjaCB0byBkaWZmZXJlbnRpYWwgZXhpcyB0byB1c2UgbGltbWEtdHJlbmQuIFRoaXMgCmFwcHJvYWNoIHdpbGwgdXN1YWxseSB3b3JrIHdlbGwgaWYgdGhlIHJhdGlvIG9mIHRoZSBsYXJnZXN0IGxpYnJhcnkgc2l6ZSB0byB0aGUgCnNtYWxsZXN0IGlzIG5vdCBtb3JlIHRoYW4gYWJvdXQgMy1mb2xkLgoKIyMjIGxpbW1hLXRyZW5kCgpJbiB0aGUgbGltbWEtdHJlbmQgYXBwcm9hY2gsIHRoZSBjb3VudHMgYXJlIGNvbnZlcnRlZCB0byBsb2dDUE0gdmFsdWVzIHVzaW5nIGVkZ2VS4oCZcyBjcG0KZnVuY3Rpb246CgpgYGB7ciBsaW1tYS10cmVuZH0KbG9nQ1BNIDwtIGNwbShkZ2VPYmosIGxvZyA9IFRSVUUsIHByaW9yLmNvdW50ID0gMykKYGBgCgpUaGUgcHJpb3IgY291bnQgaXMgdXNlZCBoZXJlIHRvIGRhbXAgZG93biB0aGUgdmFyaWFuY2VzIG9mIGxvZ2FyaXRobXMgb2YgbG93IGNvdW50cy4KVGhlIGxvZ0NQTSB2YWx1ZXMgY2FuIHRoZW4gYmUgdXNlZCBpbiBhbnkgc3RhbmRhcmQgbGltbWEgcGlwZWxpbmUsIHVzaW5nIHRoZSB0cmVuZD1UUlVFCmFyZ3VtZW50IHdoZW4gcnVubmluZyBlQmF5ZXMuIEZvciBleGFtcGxlOgoKYGBge3IgbGltbWEtZml0fQpsbWZpdCA8LSBsbUZpdChsb2dDUE0sIGRlc2lnbikKbG1maXQgPC0gZUJheWVzKGxtZml0LCB0cmVuZD1UUlVFKQp0b3BUYWJsZShsbWZpdCwgY29lZiA9IDIpCmBgYAoKSG93IGRvZXMgdGhpcyBnZW5lIHJhbmtpbmcgY29tcGFyZSB0byB0aGF0IG9idGFpbmVkIHdpdGggZWRnZVI/CgojIyMgdm9vbQoKV2hlbiB0aGUgbGlicmFyeSBzaXplcyBhcmUgcXVpdGUgdmFyaWFibGUgYmV0d2VlbiBzYW1wbGVzLCB0aGVuIHRoZSB2b29tIGFwcHJvYWNoIGlzIHRoZW9yZXRpY2FsbHkgbW9yZSBwb3dlcmZ1bCB0aGFuIGxpbW1hLXRyZW5kLiBJbiB0aGlzIGFwcHJvYWNoLCB0aGUgdm9vbSB0cmFuc2Zvcm1hdGlvbiBpcyBhcHBsaWVkIHRvIHRoZSBub3JtYWxpemVkIGFuZCBmaWx0ZXJlZCBER0VMaXN0IG9iamVjdDoKCmBgYHtyIHZvb219CnYgPC0gdm9vbShkZ2VPYmosIGRlc2lnbiwgcGxvdCA9IFRSVUUpCmBgYAoKCkFmdGVyIHRoaXMsIHRoZSB1c3VhbCBsaW1tYSBwaXBlbGluZXMgZm9yIGRpZmZlcmVudGlhbCBleHByZXNzaW9uIGNhbiBiZSBhcHBsaWVkLCBmb3IgZXhhbXBsZToKCmBgYHtyIHZvb20tZml0fQp2Zml0IDwtIGxtRml0KHYsIGRlc2lnbikKdmZpdCA8LSBlQmF5ZXModmZpdCkKdG9wVGFibGUodmZpdCwgY29lZiA9IDIpCmBgYAoKT3IsIHRvIGdpdmUgbW9yZSB3ZWlnaHQgdG8gZm9sZC1jaGFuZ2VzIGluIHRoZSByYW5raW5nLCBvbmUgY291bGQgdXNlIGEgYHRyZWF0YAphcHByb2FjaCBhcyBpbnRyb2R1Y2VkIGVhcmxpZXI6CgpgYGB7ciBsaW1tYS10cmVhdH0KdHZmaXQgPC0gdHJlYXQodmZpdCwgbGZjID0gbG9nMigxLjUpKQp0b3BUcmVhdCh0dmZpdCwgY29lZiA9IDIpCmBgYAoKPiAjIyBDaGFsbGVuZ2Ugey5jaGFsbGVuZ2V9Cj4KPiAxLiBDb21wYXJlIHRoZSByZXN1bHRzIG9idGFpbmVkIHdpdGggYGVkZ2VSYCwgYHRyZWF0YCwgYGxpbW1hLXRyZW5kYCBhbmQgYGxpbW1hLXZvb21gLiBIb3cgZG8gdGhleSBkaWZmZXI/IEFyZSB0aGVzZSBkaWZmZXJlbmNlcyBpbXBvcnRhbnQ/CgoqKlNvbHV0aW9uKioKYGBge3IsZWNobz1GQUxTRX0KIyBTb2x1dGlvbgpgYGAKCgoKCgoK