Differential expression with edgeR
Now that we are happy that we have normalised the data and that the quality looks good, we can continue to testing for differentially expressed genes. There are a number of packages to analyse RNA-Seq data. Most people use DESEQ2 or edgeR. We will use edgeR for the rest of this practical.
**First make sure we have all the objects and libraries loaded*
library(edgeR)
library(limma)
library(Glimma)
library(gplots)
library(org.Mm.eg.db)
load("Robjects/preprocessing.Rdata")
Recap of pre-processing
The previous section walked-through the pre-processing and transformation of the count data. Here, for completeness, we list the minimal steps required to process the data prior to differential expression analysis.
## Read the counts from the downloaded data
seqdata <- read.delim("data/GSE60450_Lactation-GenewiseCounts.txt", stringsAsFactors = FALSE)
#
# Remove first two columns from seqdata
countdata <- seqdata[,-(1:2)]
# Store EntrezGeneID as rownames
rownames(countdata) <- seqdata[,1]
countdata
colnames(countdata) <- substr(colnames(countdata), 1, 7)
countdata
## Calculate the Counts Per Million measure
myCPM <- cpm(countdata)
## Identify genes with at least 0.5 cpm in at least 2 samples
thresh <- myCPM > 0.5
keep <- rowSums(thresh) >= 2
# Subset the rows of countdata to keep the more highly expressed genes
counts.keep <- countdata[keep,]
## Convert to an edgeR object
dgeObj <- DGEList(counts.keep)
## Perform TMM normalisation
dgeObj <- calcNormFactors(dgeObj)
## Obtain corrected sample information
sampleinfo <- read.delim("data/SampleInfo_Corrected.txt")
group <- paste(sampleinfo$CellType,sampleinfo$Status,sep=".")
group
Create the design matrix
First we need to create a design matrix for the groups, as we have seen in the linear models lecture. We have two variables, status and cell type. We will fit two models under two assumptions; no interaction and interaction of these two factors.
Let’s start with the model with only main effects, that is no interaction. The main assumption here is that the effect of the status is the same in both type of cells.
# Create the two variables
group <- as.character(group)
type <- sapply(strsplit(group, ".", fixed=T), function(x) x[1])
status <- sapply(strsplit(group, ".", fixed=T), function(x) x[2])
# Specify a design matrix with an intercept term
design <- model.matrix(~ type + status)
design
Data exploration
An MDS plot shows distances, in terms of biological coefficient of variation (BCV), between samples. What do you think of the quality of the data? Can you anticipate if the interaction term will be important?
plotMDS(dgeObj, labels=group, cex=0.75, xlim=c(-4, 5))
Estimating the dispersion
The common dispersion estimates the overall BCV of the dataset, averaged over all genes:
dgeObj <- estimateCommonDisp(dgeObj)
Then we estimate gene-wise dispersion estimates, allowing a possible trend with averge count size:
dgeObj <- estimateGLMTrendedDisp(dgeObj)
dgeObj <- estimateTagwiseDisp(dgeObj)
Plot the estimated dispersions:
plotBCV(dgeObj)
Testing for differential expression
First, we fit genewise glms:
# Fit the linear model
fit <- glmFit(dgeObj, design)
names(fit)
head(coef(fit))
Conduct likelihood ratio tests for luminal vs basal and show the top genes:
lrt.BvsL <- glmLRT(fit, coef=2)
topTags(lrt.BvsL)
Challenge
Conduct likelihood ratio tests for virgin vs lactate and show the top genes.
Contrasts
Suppose we want to find differentially expressed genes between pregnant and virgin. We don’t have a parameter that explicitly will allow us to test that hypothesis. We need to build a contrast:
PvsV <- makeContrasts(statuspregnant-statusvirgin, levels=design)
lrt.pVsV <- glmLRT(fit, contrast=PvsV)
topTags(lrt.pVsV)
Challenge
1.Fit a model with interaction: What is the rationale to include the interaction (What assumption are you relaxing?) 2. Is the number of replicates good enough to include the interaction? 3. Is the interaction needed in the model?
Solution
Testing relative to a threshold
Statistical tests can sometimes find small log-fold changes to be highly significant. In many biological applications, small fold-change differences may not be interesting, in which case we would like to find “significant” DE genes that also have a minimum fold-change. The glmTreat
function to test for differential expression relative to a fold-change threshold that we apply.
In this example, we will say that we are not interesting in any gene that has a fold-change less than 1.5, so we set the threshold to log2(1.5)
.
trt.BvsL <- glmTreat(fit, coef = 2, lfc = log2(1.5))
topTags(trt.BvsL)
Here, the top DE genes have large fold-changes anyway, but observe the difference in p-values between this TREAT test and the standard LRT above.
Challenge
- Experiment with different lfc thresholds in glmTreat. How do these changes affect the results and any conclusions you might draw?
Solution
save(lrt.BvsL, dgeObj, group,file="Robjects/DE.Rdata")
Alternative: limma
analysis
If the sequencing depth is reasonably consistent across the RNA samples, then the simplest and most robust approach to differential exis to use limma-trend. This approach will usually work well if the ratio of the largest library size to the smallest is not more than about 3-fold.
limma-trend
In the limma-trend approach, the counts are converted to logCPM values using edgeR’s cpm function:
logCPM <- cpm(dgeObj, log = TRUE, prior.count = 3)
The prior count is used here to damp down the variances of logarithms of low counts. The logCPM values can then be used in any standard limma pipeline, using the trend=TRUE argument when running eBayes. For example:
lmfit <- lmFit(logCPM, design)
lmfit <- eBayes(lmfit, trend=TRUE)
topTable(lmfit, coef = 2)
How does this gene ranking compare to that obtained with edgeR?
voom
When the library sizes are quite variable between samples, then the voom approach is theoretically more powerful than limma-trend. In this approach, the voom transformation is applied to the normalized and filtered DGEList object:
v <- voom(dgeObj, design, plot = TRUE)
After this, the usual limma pipelines for differential expression can be applied, for example:
vfit <- lmFit(v, design)
vfit <- eBayes(vfit)
topTable(vfit, coef = 2)
Or, to give more weight to fold-changes in the ranking, one could use a treat
approach as introduced earlier:
tvfit <- treat(vfit, lfc = log2(1.5))
topTreat(tvfit, coef = 2)
Challenge
- Compare the results obtained with
edgeR
, treat
, limma-trend
and limma-voom
. How do they differ? Are these differences important?
Solution
LS0tCnRpdGxlOiAiUk5BLXNlcSBkaWZmZXJlbnRpYWwgZXhwcmVzc2lvbiBhbmFseXNpcyBpbiBSIgphdXRob3I6ICJEYXZpcyBNY0NhcnRoeSwgU3RlcGhhbmUgQmFsbGVyZWF1LCBNYXJrIER1bm5pbmcsIE9zY2FyIFJ1ZWRhLCBBc2hsZXkgU2F3bGUiCmRhdGU6ICdgciBmb3JtYXQoU3lzLnRpbWUoKSwgIkxhc3QgbW9kaWZpZWQ6ICVkICViICVZIilgJwpvdXRwdXQ6CiAgaHRtbF9ub3RlYm9vazoKICAgIHRvYzogeWVzCiAgICB0b2NfZmxvYXQ6IHllcwogIGh0bWxfZG9jdW1lbnQ6CiAgICB0b2M6IHllcwogICAgdG9jX2Zsb2F0OiB5ZXMKbWludXRlczogMzAwCnN1YnRpdGxlOiBVc2luZyBlZGdlUiBhbmQgbGltbWEKbGF5b3V0OiBwYWdlCi0tLQoKYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9CmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gVFJVRSkKYGBgCgoqKk9yaWdpbmFsIEF1dGhvcnM6IEJlbGluZGEgUGhpcHNvbiwgQW5uYSBUcmlnb3MsIE1hdHQgUml0Y2hpZSwgTWFyaWEgRG95bGUsIEhhcnJpZXQgRGFzaG5vdywgQ2hhcml0eSBMYXcqKgpCYXNlZCBvbiB0aGUgY291cnNlIFtSTkFzZXEgYW5hbHlzaXMgaW4gUl0oaHR0cDovL2NvbWJpbmUtYXVzdHJhbGlhLmdpdGh1Yi5pby8yMDE2LTA1LTExLVJOQXNlcS8pIGRlbGl2ZXJlZCBvbiBNYXkgMTEvMTJ0aCAyMDE2CgojIyBSZXNvdXJjZXMgYW5kIGRhdGEgZmlsZXMKClRoaXMgbWF0ZXJpYWwgaGFzIGJlZW4gY3JlYXRlZCB1c2luZyB0aGUgZm9sbG93aW5nIHJlc291cmNlczogIApodHRwOi8vd3d3LnN0YXRzY2kub3JnL3NteXRoL3B1YnMvUUxlZGdlUlByZXByaW50LnBkZiBbQEx1bjIwMTZdICAKaHR0cDovL21vbmFzaGJpb2luZm9ybWF0aWNzcGxhdGZvcm0uZ2l0aHViLmlvL1JOQXNlcS1ERS1hbmFseXNpcy13aXRoLVIvOTktUk5Bc2VxX0RFX2FuYWx5c2lzX3dpdGhfUi5odG1sICAKCkRhdGEgZmlsZXMgZG93bmxvYWRlZCBmcm9tOiAgCmZ0cDovL2Z0cC5uY2JpLm5sbS5uaWguZ292L2dlby9zZXJpZXMvR1NFNjBubm4vR1NFNjA0NTAvc3VwcGwvR1NFNjA0NTBfTGFjdGF0aW9uLUdlbmV3aXNlQ291bnRzLnR4dC5negpodHRwOi8vYmlvaW5mLndlaGkuZWR1LmF1L3NvZnR3YXJlL01TaWdEQi9tb3VzZV9jMl92NS5yZGF0YQpodHRwOi8vYmlvaW5mLndlaGkuZWR1LmF1L3NvZnR3YXJlL01TaWdEQi9tb3VzZV9IX3Y1LnJkYXRhCgpEYXRhIGZpbGVzOiAgCnNhbXBsZWluZm8udHh0ICAKR1NFNjA0NTBfTGFjdGF0aW9uLUdlbmV3aXNlQ291bnRzLnR4dCAgCm1vdXNlX2MyX3Y1LnJkYXRhICAKbW91c2VfSF92NS5yZGF0YQoKRGF0YSBmaWxlcyBhdmFpbGFibGUgZnJvbTogW2h0dHBzOi8vZmlnc2hhcmUuY29tL3MvMWQ3ODhmZDM4NGQzM2U5MTNhMmFdKGh0dHBzOi8vZmlnc2hhcmUuY29tL3MvMWQ3ODhmZDM4NGQzM2U5MTNhMmEpCllvdSBzaG91bGQgZG93bmxvYWQgdGhlc2UgZmlsZXMgYW5kIHBsYWNlIHRoZW0gaW4geW91ciBgL2RhdGFgIGRpcmVjdG9yeS4KCiMjIERpZmZlcmVudGlhbCBleHByZXNzaW9uIHdpdGggZWRnZVIKCk5vdyB0aGF0IHdlIGFyZSBoYXBweSB0aGF0IHdlIGhhdmUgbm9ybWFsaXNlZCB0aGUgZGF0YSBhbmQgdGhhdCB0aGUgcXVhbGl0eSBsb29rcyBnb29kLCB3ZSBjYW4gY29udGludWUgdG8gdGVzdGluZyBmb3IgZGlmZmVyZW50aWFsbHkgZXhwcmVzc2VkIGdlbmVzLiBUaGVyZSBhcmUgYSBudW1iZXIgb2YgcGFja2FnZXMgdG8gYW5hbHlzZSBSTkEtU2VxIGRhdGEuIE1vc3QgcGVvcGxlIHVzZSBERVNFUTIgb3IgZWRnZVIuIFdlIHdpbGwgdXNlIGVkZ2VSIGZvciB0aGUgcmVzdCBvZiB0aGlzIHByYWN0aWNhbC4KCioqRmlyc3QgbWFrZSBzdXJlIHdlIGhhdmUgYWxsIHRoZSBvYmplY3RzIGFuZCBsaWJyYXJpZXMgbG9hZGVkKgoKYGBge3J9CmxpYnJhcnkoZWRnZVIpCmxpYnJhcnkobGltbWEpCmxpYnJhcnkoR2xpbW1hKQpsaWJyYXJ5KGdwbG90cykKbGlicmFyeShvcmcuTW0uZWcuZGIpCmxvYWQoIlJvYmplY3RzL3ByZXByb2Nlc3NpbmcuUmRhdGEiKQpgYGAKCgojIyMgUmVjYXAgb2YgcHJlLXByb2Nlc3NpbmcKClRoZSBwcmV2aW91cyBzZWN0aW9uIHdhbGtlZC10aHJvdWdoIHRoZSBwcmUtcHJvY2Vzc2luZyBhbmQgdHJhbnNmb3JtYXRpb24gb2YgdGhlIGNvdW50IGRhdGEuIEhlcmUsIGZvciBjb21wbGV0ZW5lc3MsIHdlIGxpc3QgdGhlIG1pbmltYWwgc3RlcHMgcmVxdWlyZWQgdG8gcHJvY2VzcyB0aGUgZGF0YSBwcmlvciB0byBkaWZmZXJlbnRpYWwgZXhwcmVzc2lvbiBhbmFseXNpcy4KCmBgYHtyIGV2YWw9RkFMU0V9CiMjIFJlYWQgdGhlIGNvdW50cyBmcm9tIHRoZSBkb3dubG9hZGVkIGRhdGEKc2VxZGF0YSA8LSByZWFkLmRlbGltKCJkYXRhL0dTRTYwNDUwX0xhY3RhdGlvbi1HZW5ld2lzZUNvdW50cy50eHQiLCBzdHJpbmdzQXNGYWN0b3JzID0gRkFMU0UpCiMKIyBSZW1vdmUgZmlyc3QgdHdvIGNvbHVtbnMgZnJvbSBzZXFkYXRhCgpjb3VudGRhdGEgPC0gc2VxZGF0YVssLSgxOjIpXQoKIyBTdG9yZSBFbnRyZXpHZW5lSUQgYXMgcm93bmFtZXMKcm93bmFtZXMoY291bnRkYXRhKSA8LSBzZXFkYXRhWywxXQpjb3VudGRhdGEKY29sbmFtZXMoY291bnRkYXRhKSA8LSBzdWJzdHIoY29sbmFtZXMoY291bnRkYXRhKSwgMSwgNykKY291bnRkYXRhCiMjIENhbGN1bGF0ZSB0aGUgQ291bnRzIFBlciBNaWxsaW9uIG1lYXN1cmUKbXlDUE0gPC0gY3BtKGNvdW50ZGF0YSkKIyMgSWRlbnRpZnkgZ2VuZXMgd2l0aCBhdCBsZWFzdCAwLjUgY3BtIGluIGF0IGxlYXN0IDIgc2FtcGxlcwp0aHJlc2ggPC0gbXlDUE0gPiAwLjUKa2VlcCA8LSByb3dTdW1zKHRocmVzaCkgPj0gMgojIFN1YnNldCB0aGUgcm93cyBvZiBjb3VudGRhdGEgdG8ga2VlcCB0aGUgbW9yZSBoaWdobHkgZXhwcmVzc2VkIGdlbmVzCmNvdW50cy5rZWVwIDwtIGNvdW50ZGF0YVtrZWVwLF0KIyMgQ29udmVydCB0byBhbiBlZGdlUiBvYmplY3QKZGdlT2JqIDwtIERHRUxpc3QoY291bnRzLmtlZXApCiMjIFBlcmZvcm0gVE1NIG5vcm1hbGlzYXRpb24KZGdlT2JqIDwtIGNhbGNOb3JtRmFjdG9ycyhkZ2VPYmopCiMjIE9idGFpbiBjb3JyZWN0ZWQgc2FtcGxlIGluZm9ybWF0aW9uCnNhbXBsZWluZm8gPC0gcmVhZC5kZWxpbSgiZGF0YS9TYW1wbGVJbmZvX0NvcnJlY3RlZC50eHQiKQpncm91cCA8LSBwYXN0ZShzYW1wbGVpbmZvJENlbGxUeXBlLHNhbXBsZWluZm8kU3RhdHVzLHNlcD0iLiIpCmdyb3VwCmBgYAoKCiMjIyBDcmVhdGUgdGhlIGRlc2lnbiBtYXRyaXgKCkZpcnN0IHdlIG5lZWQgdG8gY3JlYXRlIGEgZGVzaWduIG1hdHJpeCBmb3IgdGhlIGdyb3VwcywgYXMgd2UgaGF2ZSBzZWVuIGluIHRoZSBsaW5lYXIgbW9kZWxzIGxlY3R1cmUuIApXZSBoYXZlIHR3byB2YXJpYWJsZXMsIHN0YXR1cyBhbmQgY2VsbCB0eXBlLiBXZSB3aWxsIGZpdCB0d28gbW9kZWxzIHVuZGVyIHR3byBhc3N1bXB0aW9uczsgbm8gaW50ZXJhY3Rpb24gYW5kIGludGVyYWN0aW9uIG9mIHRoZXNlIHR3byBmYWN0b3JzLiAKCkxldCdzIHN0YXJ0IHdpdGggdGhlIG1vZGVsIHdpdGggb25seSBtYWluIGVmZmVjdHMsIHRoYXQgaXMgbm8gaW50ZXJhY3Rpb24uIFRoZSBtYWluIGFzc3VtcHRpb24gaGVyZSBpcyB0aGF0IHRoZSBlZmZlY3Qgb2YgdGhlIHN0YXR1cyBpcyB0aGUgc2FtZSBpbiBib3RoIHR5cGUgb2YgY2VsbHMuCgpgYGB7cn0KIyBDcmVhdGUgdGhlIHR3byB2YXJpYWJsZXMKZ3JvdXAgPC0gYXMuY2hhcmFjdGVyKGdyb3VwKQp0eXBlIDwtIHNhcHBseShzdHJzcGxpdChncm91cCwgIi4iLCBmaXhlZD1UKSwgZnVuY3Rpb24oeCkgeFsxXSkKc3RhdHVzIDwtIHNhcHBseShzdHJzcGxpdChncm91cCwgIi4iLCBmaXhlZD1UKSwgZnVuY3Rpb24oeCkgeFsyXSkKIyBTcGVjaWZ5IGEgZGVzaWduIG1hdHJpeCB3aXRoIGFuIGludGVyY2VwdCB0ZXJtCmRlc2lnbiA8LSBtb2RlbC5tYXRyaXgofiB0eXBlICsgc3RhdHVzKQpkZXNpZ24KYGBgCgoKCiMjIyBEYXRhIGV4cGxvcmF0aW9uCkFuIE1EUyBwbG90IHNob3dzIGRpc3RhbmNlcywgaW4gdGVybXMgb2YgYmlvbG9naWNhbCBjb2VmZmljaWVudCBvZiB2YXJpYXRpb24gKEJDViksIGJldHdlZW4gc2FtcGxlcy4gV2hhdCBkbyB5b3UgdGhpbmsgb2YgdGhlIHF1YWxpdHkgb2YgdGhlIGRhdGE/IENhbiB5b3UgYW50aWNpcGF0ZSBpZiB0aGUgaW50ZXJhY3Rpb24gdGVybSB3aWxsIGJlIGltcG9ydGFudD8KYGBge3J9CnBsb3RNRFMoZGdlT2JqLCBsYWJlbHM9Z3JvdXAsIGNleD0wLjc1LCB4bGltPWMoLTQsIDUpKQpgYGAKIyMjIEVzdGltYXRpbmcgdGhlIGRpc3BlcnNpb24KClRoZSBjb21tb24gZGlzcGVyc2lvbiBlc3RpbWF0ZXMgdGhlIG92ZXJhbGwgQkNWIG9mIHRoZSBkYXRhc2V0LCBhdmVyYWdlZCBvdmVyIGFsbCBnZW5lczoKYGBge3J9CmRnZU9iaiA8LSBlc3RpbWF0ZUNvbW1vbkRpc3AoZGdlT2JqKQpgYGAKClRoZW4gd2UgZXN0aW1hdGUgZ2VuZS13aXNlIGRpc3BlcnNpb24gZXN0aW1hdGVzLCBhbGxvd2luZyBhIHBvc3NpYmxlIHRyZW5kIHdpdGggYXZlcmdlIGNvdW50IHNpemU6CmBgYHtyfQpkZ2VPYmogPC0gZXN0aW1hdGVHTE1UcmVuZGVkRGlzcChkZ2VPYmopCmRnZU9iaiA8LSBlc3RpbWF0ZVRhZ3dpc2VEaXNwKGRnZU9iaikKYGBgClBsb3QgdGhlIGVzdGltYXRlZCBkaXNwZXJzaW9uczoKYGBge3J9CnBsb3RCQ1YoZGdlT2JqKQpgYGAKCgojIyMgVGVzdGluZyBmb3IgZGlmZmVyZW50aWFsIGV4cHJlc3Npb24KCkZpcnN0LCB3ZSBmaXQgZ2VuZXdpc2UgZ2xtczoKCmBgYHtyfQojIEZpdCB0aGUgbGluZWFyIG1vZGVsCmZpdCA8LSBnbG1GaXQoZGdlT2JqLCBkZXNpZ24pCm5hbWVzKGZpdCkKaGVhZChjb2VmKGZpdCkpCmBgYApDb25kdWN0IGxpa2VsaWhvb2QgcmF0aW8gdGVzdHMgZm9yIGx1bWluYWwgdnMgYmFzYWwgYW5kIHNob3cgdGhlIHRvcCBnZW5lczoKYGBge3J9CmxydC5CdnNMIDwtIGdsbUxSVChmaXQsIGNvZWY9MikKdG9wVGFncyhscnQuQnZzTCkKCmBgYAoKCj4gIyMgQ2hhbGxlbmdlIHsuY2hhbGxlbmdlfQo+IENvbmR1Y3QgbGlrZWxpaG9vZCByYXRpbyB0ZXN0cyBmb3IgdmlyZ2luIHZzIGxhY3RhdGUgYW5kIHNob3cgdGhlIHRvcCBnZW5lcy4KCmBgYHtyfQoKYGBgCgoKCiMjIyBDb250cmFzdHMKClN1cHBvc2Ugd2Ugd2FudCB0byBmaW5kIGRpZmZlcmVudGlhbGx5IGV4cHJlc3NlZCBnZW5lcyBiZXR3ZWVuIHByZWduYW50IGFuZCB2aXJnaW4uIFdlIGRvbid0IGhhdmUgYSBwYXJhbWV0ZXIgdGhhdCBleHBsaWNpdGx5IHdpbGwgYWxsb3cgdXMgdG8gdGVzdCB0aGF0IGh5cG90aGVzaXMuIFdlIG5lZWQgdG8gYnVpbGQgYSBjb250cmFzdDoKCmBgYHtyfQpQdnNWIDwtIG1ha2VDb250cmFzdHMoc3RhdHVzcHJlZ25hbnQtc3RhdHVzdmlyZ2luLCBsZXZlbHM9ZGVzaWduKQpscnQucFZzViA8LSBnbG1MUlQoZml0LCBjb250cmFzdD1QdnNWKQp0b3BUYWdzKGxydC5wVnNWKQpgYGAKCgo+ICMjIENoYWxsZW5nZSB7LmNoYWxsZW5nZX0KPgo+IDEuRml0IGEgbW9kZWwgd2l0aCBpbnRlcmFjdGlvbjogV2hhdCBpcyB0aGUgcmF0aW9uYWxlIHRvIGluY2x1ZGUgdGhlIGludGVyYWN0aW9uIChXaGF0IGFzc3VtcHRpb24gYXJlIHlvdSByZWxheGluZz8pCj4gMi4gSXMgdGhlIG51bWJlciBvZiByZXBsaWNhdGVzIGdvb2QgZW5vdWdoIHRvIGluY2x1ZGUgdGhlIGludGVyYWN0aW9uPwo+IDMuIElzIHRoZSBpbnRlcmFjdGlvbiBuZWVkZWQgaW4gdGhlIG1vZGVsPwoKKipTb2x1dGlvbioqCmBgYHtyLGVjaG89RkFMU0V9CiMgU29sdXRpb24KYGBgCgoKIyMjIFRlc3RpbmcgcmVsYXRpdmUgdG8gYSB0aHJlc2hvbGQKClN0YXRpc3RpY2FsIHRlc3RzIGNhbiBzb21ldGltZXMgZmluZCBzbWFsbCBsb2ctZm9sZCBjaGFuZ2VzIHRvIGJlIGhpZ2hseSAKc2lnbmlmaWNhbnQuIEluIG1hbnkgYmlvbG9naWNhbCBhcHBsaWNhdGlvbnMsIHNtYWxsIGZvbGQtY2hhbmdlIGRpZmZlcmVuY2VzCm1heSBub3QgYmUgaW50ZXJlc3RpbmcsIGluIHdoaWNoIGNhc2Ugd2Ugd291bGQgbGlrZSB0byBmaW5kICJzaWduaWZpY2FudCIgREUKZ2VuZXMgdGhhdCBhbHNvIGhhdmUgYSBtaW5pbXVtIGZvbGQtY2hhbmdlLiBUaGUgYGdsbVRyZWF0YCBmdW5jdGlvbiB0byB0ZXN0IGZvciAKZGlmZmVyZW50aWFsIGV4cHJlc3Npb24gcmVsYXRpdmUgdG8gYSBmb2xkLWNoYW5nZSB0aHJlc2hvbGQgdGhhdCB3ZSBhcHBseS4KCkluIHRoaXMgZXhhbXBsZSwgd2Ugd2lsbCBzYXkgdGhhdCB3ZSBhcmUgbm90IGludGVyZXN0aW5nIGluIGFueSBnZW5lIHRoYXQgaGFzIGEKZm9sZC1jaGFuZ2UgbGVzcyB0aGFuIDEuNSwgc28gd2Ugc2V0IHRoZSB0aHJlc2hvbGQgdG8gYGxvZzIoMS41KWAuCgpgYGB7ciB0cmVhdH0KdHJ0LkJ2c0wgPC0gZ2xtVHJlYXQoZml0LCBjb2VmID0gMiwgbGZjID0gbG9nMigxLjUpKQp0b3BUYWdzKHRydC5CdnNMKQpgYGAKCkhlcmUsIHRoZSB0b3AgREUgZ2VuZXMgaGF2ZSBsYXJnZSBmb2xkLWNoYW5nZXMgYW55d2F5LCBidXQgb2JzZXJ2ZSB0aGUgCmRpZmZlcmVuY2UgaW4gcC12YWx1ZXMgYmV0d2VlbiB0aGlzIFRSRUFUIHRlc3QgYW5kIHRoZSBzdGFuZGFyZCBMUlQgYWJvdmUuCgo+ICMjIENoYWxsZW5nZSB7LmNoYWxsZW5nZX0KPgo+IDEuIEV4cGVyaW1lbnQgd2l0aCBkaWZmZXJlbnQgbGZjIHRocmVzaG9sZHMgaW4gZ2xtVHJlYXQuIEhvdyBkbyB0aGVzZSBjaGFuZ2VzIAphZmZlY3QgdGhlIHJlc3VsdHMgYW5kIGFueSBjb25jbHVzaW9ucyB5b3UgbWlnaHQgZHJhdz8KCioqU29sdXRpb24qKgpgYGB7cixlY2hvPUZBTFNFfQojIFNvbHV0aW9uCmBgYAoKCmBgYHtyfQpzYXZlKGxydC5CdnNMLCBkZ2VPYmosIGdyb3VwLGZpbGU9IlJvYmplY3RzL0RFLlJkYXRhIikKYGBgCgojIyBBbHRlcm5hdGl2ZTogYGxpbW1hYCBhbmFseXNpcwoKSWYgdGhlIHNlcXVlbmNpbmcgZGVwdGggaXMgcmVhc29uYWJseSBjb25zaXN0ZW50IGFjcm9zcyB0aGUgUk5BIHNhbXBsZXMsIHRoZW4gdGhlCnNpbXBsZXN0IGFuZCBtb3N0IHJvYnVzdCBhcHByb2FjaCB0byBkaWZmZXJlbnRpYWwgZXhpcyB0byB1c2UgbGltbWEtdHJlbmQuIFRoaXMgCmFwcHJvYWNoIHdpbGwgdXN1YWxseSB3b3JrIHdlbGwgaWYgdGhlIHJhdGlvIG9mIHRoZSBsYXJnZXN0IGxpYnJhcnkgc2l6ZSB0byB0aGUgCnNtYWxsZXN0IGlzIG5vdCBtb3JlIHRoYW4gYWJvdXQgMy1mb2xkLgoKIyMjIGxpbW1hLXRyZW5kCgpJbiB0aGUgbGltbWEtdHJlbmQgYXBwcm9hY2gsIHRoZSBjb3VudHMgYXJlIGNvbnZlcnRlZCB0byBsb2dDUE0gdmFsdWVzIHVzaW5nIGVkZ2VS4oCZcyBjcG0KZnVuY3Rpb246CgpgYGB7ciBsaW1tYS10cmVuZH0KbG9nQ1BNIDwtIGNwbShkZ2VPYmosIGxvZyA9IFRSVUUsIHByaW9yLmNvdW50ID0gMykKYGBgCgpUaGUgcHJpb3IgY291bnQgaXMgdXNlZCBoZXJlIHRvIGRhbXAgZG93biB0aGUgdmFyaWFuY2VzIG9mIGxvZ2FyaXRobXMgb2YgbG93IGNvdW50cy4KVGhlIGxvZ0NQTSB2YWx1ZXMgY2FuIHRoZW4gYmUgdXNlZCBpbiBhbnkgc3RhbmRhcmQgbGltbWEgcGlwZWxpbmUsIHVzaW5nIHRoZSB0cmVuZD1UUlVFCmFyZ3VtZW50IHdoZW4gcnVubmluZyBlQmF5ZXMuIEZvciBleGFtcGxlOgoKYGBge3IgbGltbWEtZml0fQpsbWZpdCA8LSBsbUZpdChsb2dDUE0sIGRlc2lnbikKbG1maXQgPC0gZUJheWVzKGxtZml0LCB0cmVuZD1UUlVFKQp0b3BUYWJsZShsbWZpdCwgY29lZiA9IDIpCmBgYAoKSG93IGRvZXMgdGhpcyBnZW5lIHJhbmtpbmcgY29tcGFyZSB0byB0aGF0IG9idGFpbmVkIHdpdGggZWRnZVI/CgojIyMgdm9vbQoKV2hlbiB0aGUgbGlicmFyeSBzaXplcyBhcmUgcXVpdGUgdmFyaWFibGUgYmV0d2VlbiBzYW1wbGVzLCB0aGVuIHRoZSB2b29tIGFwcHJvYWNoIGlzIHRoZW9yZXRpY2FsbHkgbW9yZSBwb3dlcmZ1bCB0aGFuIGxpbW1hLXRyZW5kLiBJbiB0aGlzIGFwcHJvYWNoLCB0aGUgdm9vbSB0cmFuc2Zvcm1hdGlvbiBpcyBhcHBsaWVkIHRvIHRoZSBub3JtYWxpemVkIGFuZCBmaWx0ZXJlZCBER0VMaXN0IG9iamVjdDoKCmBgYHtyIHZvb219CnYgPC0gdm9vbShkZ2VPYmosIGRlc2lnbiwgcGxvdCA9IFRSVUUpCmBgYAoKCkFmdGVyIHRoaXMsIHRoZSB1c3VhbCBsaW1tYSBwaXBlbGluZXMgZm9yIGRpZmZlcmVudGlhbCBleHByZXNzaW9uIGNhbiBiZSBhcHBsaWVkLCBmb3IgZXhhbXBsZToKCmBgYHtyIHZvb20tZml0fQp2Zml0IDwtIGxtRml0KHYsIGRlc2lnbikKdmZpdCA8LSBlQmF5ZXModmZpdCkKdG9wVGFibGUodmZpdCwgY29lZiA9IDIpCmBgYAoKT3IsIHRvIGdpdmUgbW9yZSB3ZWlnaHQgdG8gZm9sZC1jaGFuZ2VzIGluIHRoZSByYW5raW5nLCBvbmUgY291bGQgdXNlIGEgYHRyZWF0YAphcHByb2FjaCBhcyBpbnRyb2R1Y2VkIGVhcmxpZXI6CgpgYGB7ciBsaW1tYS10cmVhdH0KdHZmaXQgPC0gdHJlYXQodmZpdCwgbGZjID0gbG9nMigxLjUpKQp0b3BUcmVhdCh0dmZpdCwgY29lZiA9IDIpCmBgYAoKPiAjIyBDaGFsbGVuZ2Ugey5jaGFsbGVuZ2V9Cj4KPiAxLiBDb21wYXJlIHRoZSByZXN1bHRzIG9idGFpbmVkIHdpdGggYGVkZ2VSYCwgYHRyZWF0YCwgYGxpbW1hLXRyZW5kYCBhbmQgYGxpbW1hLXZvb21gLiBIb3cgZG8gdGhleSBkaWZmZXI/IEFyZSB0aGVzZSBkaWZmZXJlbmNlcyBpbXBvcnRhbnQ/CgoqKlNvbHV0aW9uKioKYGBge3IsZWNobz1GQUxTRX0KIyBTb2x1dGlvbgpgYGAKCgoKCgoK