Image-based computational immune synapse analysisWe hypothesized that immune synapses between cells can be detected based on the preferential localization of proteins at interfaces where cells contact one another. While this concept is intuitive, it has been unknown whether current spatial proteomic profiling technologies, such as imaging mass cytometry, have sufficient resolution and precision to reveal such synapses. It also remains poorly understood what fraction of cell-cell contacts will have active synapses in a given tissue sample, as this quantity will depend on the cell types of interest, cell densities, the strength and persistence of synapses, and the tissue type. In addition, synapse quantification is affected by the accuracy of cell segmentation algorithms in identifying cell-cell interfaces in 2D spatial proteomic images. Given these uncertainties, a statistical approach that not only evaluates individual cell-cell contacts, but also integrates across an image, would be valuable to quantifying synaptic activity.To interrogate cell-cell interactions in the in situ immunological context, we defined an image-based T-cell immune synapse metric. T-cells are known to interact with APCs, which motivated us to focus on the behavior of TCR proteins at interfaces between T-cells and APCs; however, the approach we describe here is generalizable to any cell types and synapse markers. CISA defines the strength of an immune synapse by calculating the degree of TCR localization towards the cell-cell interface: A T-cell’s synapse strength σ is defined as the logarithm of the mean CD3 signal intensity (a proxy for TCR signal) at the membrane region in contact with an APC divided by the mean signal in the noncontact region (Fig. 1a, Methods). A positive synapse strength indicates T-cell synapse formation and potential functional interaction with another cell in the TME, modeling how the TCR aggregates at the synapse with an APC upon recognition of the presented antigen12. Because individual contacts are affected by high variability in measurement noise, we more generally consider phenotypic evaluations at the whole image level based on average synapse strength, which reduces the effect of measurement noise. The average synapse strength \(\bar{\sigma }\) can be considered for each tissue sample, by averaging the values of σ for all contacts between T-cells and APCs (or specified cell type) in the sample (Fig. 1b, c). We also implemented a null synapse model based on random sets of contiguous T-cell membrane pixels to account for baseline CD3 aggregation (see Methods). This CISA provides a way to investigate the functional importance of cell-cell contacts from images. CISA requires single channel protein images, cell segmentation masks, and cell classification as input, all of which are fundamental and essential in this image-based analysis workflow (Fig. 1c), as well as common in typical image analysis workflows.Fig. 1: Computational immune synapse analysis.a Individual synapse strength \(\sigma\) is the logarithm of the mean membrane signal intensity of CD3 at the T-cell-APC contact interface relative to the intensity outside the contact interface. b Image level synapse strength \(\bar{{\rm{\sigma }}}\) is the average over all neighboring cell-cell pairs in the image. Statistically, \(\bar{{\rm{\sigma }}}\) < 0 indicates a lack of interaction; \(\bar{{\rm{\sigma }}}\) > 0 indicates an active interaction. Showing an example diagram of \(\bar{{\rm{\sigma }}}\) > 0. c The general niche of CISA in an image-based analysis workflow. CISA takes three required inputs, namely single channel images, a raster cell segmentation mask image, and the cell classification. CISA outputs a table of \({\rm{\sigma }}\) at each T-neighbor interface, which can be then aggregated for sample level \(\bar{{\rm{\sigma }}}\) for statistical characterization.To test whether CISA is capable of quantifying immune synapses in high-resolution multiplexed images, we first applied it to a published melanoma IMC dataset from ref. 24 This dataset contains 30 pre-treatment patient tissue microarray samples from melanoma patients that received immune checkpoint inhibition therapy. We segmented cells in each image and mapped to the cell annotations from ref. 24. (see Methods). We then used CISA to compute σCD3(T-CD8 + , APC) for each T-cell:APC contact, i.e., the relative localization of CD3 to the contact region between a CD8 + T-cell and an adjacent APC. We also generalized CISA to compute σCD8(T-CD8 + , APC), the localization of CD8 toward the adjacent APC. We observed strong correlation (Fig. 2a) in the directional enrichment of CD3 and CD8 toward adjacent APCs, with r = 0.69 between σCD3(T-CD8 + , APC) and σCD8(T-CD8 + , APC), as would be expected from active T-cell: APC interactions. CD8 + T-cells can have a minor CD4 signal not expected to co-localize with CD3 in CD8+ cells, providing a control. Consistent with these expectations, the correlation of σCD3 and σCD4 (r = 0.17) in CD8+ cells is less than that between σCD3 and σCD8 (Fig. 2a). Likewise, for CD4 + T-cells, we observed a strong correlation of CD3 and CD4 toward adjacent APCs (σCD3(T-CD4 + , APC) and σCD4 (T-CD4 + , APC), r = 0.56), consistent with detection of TCR-mediated synapses (Fig. 2b). As expected, the correlation of σCD3(T-CD4 + , APC) and σCD8(T-CD4 + , APC) is lower (r = 0.26) than the CD3:CD4 correlation (Fig. 2b).Fig. 2: CISA robustly characterizes protein localization at cell-cell interfaces from IMC data.a Density plots showing correlation of σ(CD4) with σ(CD3) in CD8+ cells, and correlation of σ(CD8a) with σ(CD3) in CD8+ cells; (b) correlation of σ(CD4) with σ(CD3) in CD4+ cells, and correlation of σ(CD8a) with σ(CD3) in CD4+ cells. c Comparison of CD8/CD3 correlation in CD8+ cells at the pixel level to CISA. d Comparison of CD4/CD3 correlation in CD4+ cells to CISA. e–h Plots are analogous to (a–d) but for melanoma IMC data from (Hoch et al. 2022). Examples of positive (i) and negative (j) localization of CD3 and CD8a around the central T cell (dashed cell boundary) toward neighboring macrophages (solid white cell boundary). The “Combined” column shows the four channels listed as separate columns with corresponding colors, plus blue showing DNA. In the “Cell type” column, green marks macrophages and red marks T cells. In the CD3 and CD8a columns, numbers indicate synapse strengths of the shown marker between the central T cell and the indicated macrophage. Gray outlines: non-contacting or non-APC cells. Scale bar: 10 µm. k Correlations of σ(CD4), σ(CD8a) σ(CD7) and σ(ICOS) with σ(CD3) in CD4 + T cells. Each point is one TMA core. l Correlations of σ(CD4), σ(CD8a) σ(CD7) and σ(ICOS) with σ(CD3) in CD8 + T cells. m Average pixel intensity of markers within CD8 + T cells. ICOS (CD278), is significantly lower than CD4. Each point is the average in a TMA core. a–d, i, j are based on melanoma IMC spots from the (Moldoveanu et al. 2022)24 IMC dataset. e–h, k–m data are from the (Hoch et al. 2022)23 IMC data. Tc: CD8 + T cells; Th: CD4 + T cells. σ: interface-level synapse strength.While our results show that colocalization of TCR proteins toward APCs can be detected by CISA, some protein colocalization might occur regardless of an APC contact. To evaluate the contact-independent effect, we calculated pixel-wise correlations between CD3 and CD4 (or CD8) intensity within the complete T-cell areas in each spot. CISA scores exhibited stronger correlations than the contact-naïve pixel-wise methods (Fig. 2c, d). This was observed for correlations of CD3 and CD8 in CD8 + T cells (t-test, p = 5.8e-9), as well as for correlations of CD3 and CD4 in CD4 + T cells (t-test, p = 3.9e-7). These results indicate that CISA captures information on contact-dependent synapse activity.To verify the robustness of these results, we analyzed a second IMC dataset from ref. 23. It consists of 167 tissue microarray (TMA) spot images from melanoma patients. This dataset showed CISA behaviors consistent with the Moldoveanu et al. dataset24. CD3 and CD8 CISA correlations in CD8 + T cells were strong (r = 0.57, Fig. 2e) and higher than CD3-CD4 CISA correlations in the same cells (r = 0.09). CD3 and CD4 CISA correlations in CD4 + T cells were also strong (r = 0.43, Fig. 2f), and higher than CD3-CD8 CISA correlations in those cells (r = 0.14). As in the Moldoveanu et al. dataset, CISA correlations were stronger than pixel-wise correlations (Fig. 2g, h), both for CD3:CD8 in Tc cells (p = 1.8e-12) and for CD3:CD4 in Th cells (p = 3.8e-12). We also analyzed the correlations of T-cell:APC CISA scores for all pairs of proteins in the IMC data (Fig. S1). These showed a rich clustering structure reflecting behaviors such as TCR co-occurrence, T-cell specificity, and APC-specificity.To clarify the observed behaviors, example images of T-cell:APC contacts with positive or negative CISA scores are shown in Fig. 2i, j. Figure 2i shows a CD8 + T cell that has positive σCD3 and σCD8a with its neighboring macrophages, suggesting potential synaptic interactions between them; Fig. 2j, on the other hand, depicts a CD8 + T cell that has negative σCD3 and σCD8a with the neighboring macrophages, suggesting lack of synaptic interactions between them. These images show the variability in signal along membrane boundaries, supporting the importance of statistical approaches that integrate information along over entire cells and tissue images to provide robustness. For example, in addition to CD3, CD4 and CD8a, we were also able to evaluate correlations in CISA scores between CD3 and other T cell surface proteins, including ICOS (CD278) and CD7 (Fig. 2k, l). It is worth noting that ICOS has a significantly (p = 3e-9) lower expression level than CD4 in CD8 + T cells (Fig. 2m), but the correlation between σCD3(T-CD8 + , APC) and σCD278(T-CD8 + , APC) (Fig. 2l) is stronger than between CD3 and CD4 in those cells. CISA’s ability to detect colocalization of CD3 and CD278 despite the low expression level of the latter suggests CISA can detect polarization of cell membrane proteins even at a low expression level. Together, these results support CISA as a robust method to quantify immune synapses between T cells and neighboring APCs.Whole-slide images enable regional analysis of tumor microenvironmental interactionsHaving demonstrated that CISA is able to quantify immune synapses between T cells and APCs in IMC data, we were interested in whether it would be applicable to other types of imaging data. Different types of imaging techniques have bespoke sample acquisition protocols that affect the scale of regions assayed and the cell-cell contacts within them, which could impact CISA characterization statistics. To further verify the applicability of CISA to other types of image data, we generated and analyzed histocytometry27 whole slide images (WSI) of metastatic melanoma28. These WSIs are much larger than TMA spots and are based on fluorescence, providing substantial differences from IMC. We analyzed a cohort of 21 human metastatic melanoma samples from 20 patients (see Methods). A 6-marker panel, consisting of nuclei, melanoma antigens, Ki67, CD45, CD3, CD14, and CD19/138, was employed to characterize cell phenotypes across whole-slide sections, providing extensive data for identification of TME features and interactions. Because CD14 is a broad marker that can be expressed by a range of cells including macrophages, monocytes, and monocytic-DCs, the individual CD14+ cells from the 9 of the patients were harvested with laser capture microdissection for in situ transcriptomic profiling. The RNA data showed expression of CD14, CD68, CD163, and CD206, confirming a typical monocyte/macrophage phenotype for the CD14 protein positive cells (see also Martinek et al.28). The images in our cohort averaged 67 mm2 of tissue imaged at a resolution of 663 nm per pixel.We first analyzed regional cell type prevalence and spatial co-occurrence in these datasets, as cellular composition29 and APC expression28 could vary between the tumor stroma and tumor nests. The large size of WSI images makes them better than TMA spots for investigating potential region-specific T-cell interactions. After segmenting images into intratumor and stromal regions (see Methods, Fig. 3a, b), we observed that T-cells and CD14+ cells are both abundant in the stroma, but CD14+ cells make up the bulk of the intratumor immune infiltrate. Within the tumor, the cohort median density of T-cells is 86.3 cells per mm2, while the density of CD14+ cells is 252.6 cells per mm2 (CD14+ cell > T-cell: p = 2.4 × 10−4, Fig. 3c). CD14+ cells also exhibit differential melanoma antigen loading between the intratumoral and stromal regions, where loading is defined by the presence of melanoma antigen in the cytoplasm (Fig. 3d, see Methods). In the intratumoral region, the majority of CD14+ cells are loaded with unprocessed melanoma antigen (Fig. 3e, cohort median = 72.3%). In the stroma, a significantly lower fraction of CD14+ cells is loaded (cohort median = 7.7%, p = 9.5 × 10−7).Fig. 3: Histocytometry profiling of immune infiltrate in metastatic melanoma.a An example whole-slide histocytometry image with select markers shown. White borders denote tumor-stroma interface. Yellow box denotes the zoomed in region for (b). b Zoomed in view of tumor/stroma segmentation. c Comparison of immune infiltrate in tumor and stroma. Within the tumor, CD14+ cells are denser than T-cells, but the difference is not statistically significant in the stroma. * denotes p < 0.05. d An example of a melanoma antigen-loaded CD14+ cell (white bound cell) within the tumor. Note green fluorescence (melanoma antigen stain) within the cytoplasm. e Melanoma antigen-loaded CD14+ cells make up the bulk of intratumor CD14+ cell infiltrate (fraction loaded, median = 72.3%), but a significantly smaller fraction in the stroma (median = 7.7%, p = 9.5 × 10–7). CD3: T-cell. CD45: Leukocytes.T-cells colocalize with CD14+ cells in a region-dependent manner in melanomaBecause of the differential antigen-loading of CD14+ cells between the intratumoral and stromal regions, we hypothesized that cells of the TME interact in a region-specific manner. We first considered regional differences in cell-cell colocalization behavior, as these can impact CISA statistics. CISA statistics will be different in histocytometry compared to tissue microarray spots because the larger size of histocytometry images captures major variations in behavior across different types of tumor microenvironmental regions. We investigated this using a radial distribution function (RDF) analysis to identify spatial relationships between cell types in the TME (Fig. S2A, B). We applied RDF analysis to quantify the distance-dependent density of TME cells relative to T-cells (see Methods). We additionally devised a metric to quantify T-cell/TME cell colocalization relative to null expectations, ΔCDF, defined from the RDF curve as the excess of colocalization relative to a label-permuting null model—an approach that controls for variations in local cell density (Fig. S2C).The use of RDF analysis to reveal cell-cell spatial associations is illustrated in Fig. 4 for sample Mel-512, with T-cells as the reference cell. In the intratumoral region, a peak in the RDF for loaded CD14+ cells at approximately 12 μm indicates that T-cells and loaded CD14+ cells are often in close proximity (Fig. 4a, solid cyan line). The observed distribution is above the cell-permuted null expectation (Fig. 4a, dashed cyan line) indicating excess colocalization. T-cell colocalization with loaded CD14+ cells is significant when calculated over the full histocytometry cohort (\(\overline{\Delta {\rm{CDF}}}\) = 4.184, 1-sample t-test p = 9.6 × 10−10, Fig. 4b). Unloaded CD14+ cell colocalization with intratumoral T-cells is also observed (Fig. 4a), though the effect is weaker than for loaded CD14+ cells. Such colocalization is moderately significant across the cohort (\(\overline{\Delta {\rm{CDF}}}\) = 0.908, 1-sample t-test p = 0.026, Fig. 4b). Tumor cells have high absolute densities near intratumoral T-cells, but do not significantly colocalize (\(\overline{\Delta {\rm{CDF}}}\) = −1.311, 1-sample t-test p = 1.0, Fig. 4a, b). Intratumor T-cell colocalization with loaded CD14+ cells is significantly greater than with either unloaded CD14+ cells or tumor cells (Fig. 4b, p = 1.2 × 10−4 and 5.9 × 10−5 respectively).Fig. 4: The radial distribution function demonstrates region-specific T-cell interactions with CD14+ cells in whole slide images.a RDFs for sample Mel-512 with T-cells as the reference cell demonstrate that intratumor T-cells colocalize with both loaded CD14+ cells (cyan curve) and unloaded CD14+ cells (blue) above the expected distribution (dashed lines). Conversely, intratumor T-cells contact melanoma cells below null expectations (green). b The cohort distributions of ΔCDF demonstrate significant intratumor T-cell colocalization with loaded CD14+ cells, unloaded CD14+ cells, and other T-cells but not tumor cells. ΔCDF is significantly greater for loaded CD14+ cells compared to both unloaded CD14+ cells and melanoma cells. c In the stroma, T-cells colocalize with unloaded CD14+ cells (blue, right) but no colocalization with loaded CD14+ cells. d Cohort wide ΔCDF demonstrates significant intratumor T-cell colocalization with unloaded CD14+ cells and other T-cells but not loaded CD14+ cells. ΔCDF for stromal T-cells is significantly greater for unloaded CD14+ cells compared to loaded CD14+ cells. T-cells similarly show strongly positive ΔCDF. #: 1-sample t-test p < 0.05. *Wilcoxon rank-sum test p < 0.05.In the stromal regions, cell co-localization relationships differ from those within the tumor. For example, in the stroma of sample Mel-512, unloaded CD14+ cells have a small RDF peak with respect to T-cells at a distance of approximately 10 μm (Fig. 4c, blue). This colocalization exceeds the null expectation by a statistically significant but small margin (\(\overline{\Delta {\rm{CDF}}}\) = 2.642, 1-sample t-test p = 1.5 × 10−6, Fig. 4d). On the other hand, loaded CD14+ cells have lower colocalization with stromal T-cells than expected (Fig. 4c). Negative colocalization between these cell types is observed across the cohort (\(\overline{\Delta {\rm{CDF}}}\) = −3.567, 1-sample t-test p = 1.0, Fig. 4d). The unloaded CD14+ cell ΔCDF is significantly greater than for loaded CD14+ cells (p = 5.9 × 10−5). Meanwhile, T-cells exhibit strong colocalization to other T-cells both in the stroma and within the tumor (\(\overline{\Delta {\rm{CDF}}}\) = 4.616 and 4.713, 1-sample t-test p = 1.8 × 10−6 and 4.7 × 10−7, stroma and intratumor respectively Fig. 4b, d; S2B).CISA reveals region-specific T-cell/CD14+ cell synapses in whole slide imagesGiven the distinct cell co-localization behaviors in the intratumor and stromal regions, we hypothesized that CISA analysis would detect regional differences in immune synapse formation within whole slide histocytometry images. Indeed, CISA analysis of intratumoral regions showed that intratumor T-cells tend to form synapses to loaded CD14+ cells (\(\bar{\sigma }\) = 0.098, 1-sample t-test p = 3.9 × 10−3, Fig. 5a), consistent with the IMC results. However, this relationship was not observed in the stroma, and stromal T-cells instead tend to form synapses to unloaded CD14+ cells (\(\bar{\sigma }\) = 0.059, 1-sample t-test p = 5.0 × 10−3). Both synapse strengths are significantly greater than the null model (p = 2.4 × 10−5 and 2.9 × 10−6 respectively). T-cell synapses to loaded CD14+ cells are significantly stronger within the tumor than in the stroma, while T-cell synapses to unloaded CD14+ cells are significantly stronger in the stroma than in the tumor (Fig. 5b, p = 6.9 × 10−5 for both respectively). Within the tumor, T-cell synapse strength to loaded CD14+ cells is significantly stronger than to unloaded CD14+ cells (Fig. 5b, p = 3.7 × 10−4). In the stroma, T-cells have significantly stronger synapses to unloaded CD14+ cells than to loaded CD14+ cells (p = 1.3 × 10−3). In the tumor, T-cells showed no tendency to form synapses to melanoma cells (\(\bar{\sigma }\) = −0.101, 1-sample t-test p = 0.98), with synapse strength not significantly greater than the null model (p = 0.41, Fig. 5a). The greater T-cell synapse strength to loaded CD14+ cells than to melanoma cells is also statistically significant (p = 6.9 × 10−5).Fig. 5: Functional synapse interactions in metastatic melanoma.a Intratumor T-cell synapses with loaded CD14+ cells are significantly stronger than those with unloaded CD14+ cells and melanoma (p = 3.7 × 10–4 and p = 6.9 × 10–5 respectively) and are also stronger than expected from the intratumor null synapse model (denoted by *p = 2.4 × 10–5). Stromal T-cell synapses with unloaded CD14+ cells are significantly stronger than those with loaded CD14+ cells (p = 1.3 × 10–3) and the stromal null model (denoted by *p = 2.9 × 10–6). b Intratumor T-cell synapses with loaded CD14+ cells are stronger than stromal T-cell synapses to loaded CD14+ cells (p = 6.9 × 10–5). In contrast, T-cell synapses with unloaded CD14+ cells are stronger in the stroma than in the tumor (p = 1.3 × 10–3). c Super-resolution imaging of two stromal T-cells in contact with CD14+ cells in the TME. The T-cell on the right is in contact with the neighboring CD14+ cell’s dendrite, and CD3 is aggregating in the contact region. Scale bar = 4 μm. d Volumetric rendering of the right T-cell in C with ICAM-1 shown. The CD14+ cell’s ICAM-1 can be seen aggregating in the dendrite. Scale bar = 2 μm. APC antigen-presenting cell. \(\bar{{\rm{\sigma }}}\): sample-level mean synapse strength. Rendering was performed using the Imaris software.We employed synapse-focused super-resolution imaging to further verify T-cell synapses in the TME. Figure 5c shows two T-cell-CD14+ cell contacts with different CD3 aggregation behavior via stimulated emission depletion microscopy (Fig. 5c). The left T-cell exhibits relatively homogenous CD3 along its membrane without aggregation towards its neighboring CD14+ cell. Volumetric rendering suggests a CD3-containing T-cell protrusion towards the CD14+ cell which may signify early synapse formation or antigen sampling (Fig. S3A), though there is not a reciprocal concentration of ICAM-1 in the CD14+ cell that would demonstrate mature synapse formation (Fig. S3B). The right T-cell shows CD3 concentration towards the dendrite protrusion on the target CD14+ cell (Fig. 5C, yellow box). Volumetric rendering shows not only contact between the T-cell CD3 and the CD14+ cell dendrite but also reciprocal CD14+ cell ICAM-1 in the contact area, as expected from mature synapse formation and functional interaction (Fig. 5d, Fig. S3B).T-cell-CD14+ cell interactions are associated with T-cell proliferation in metastatic melanomaT-cells recognizing their cognate antigen in conjunction with pro-inflammatory signals undergo clonal expansion30 typically within the lymph node, but in vivo mouse data suggest this may occur in the tumor as well11. We therefore hypothesized that proliferating T-cells in the TME might have stronger synapse strengths than other T-cells. We extended CISA to address this by classifying T-cells as proliferating (Ki-67 + ) or non-proliferating (Ki-67 − ) from KI-67 imaging data. We found that intratumor Ki-67 + T-cells have significantly stronger synapses to loaded CD14+ cells than Ki-67 − T-cells (Fig. 6a, p = 4.2 × 10−4) and significantly greater than the Ki-67 + T-cell null model (p = 9.5 × 10−7). There was no significant difference in synapse strength between Ki-67+ and Ki-67− stromal T-cells, unlike in the intratumor region (Fig. 6a). Ki-67 + T-cell synapse strength to melanoma cells is also stronger than for Ki-67 − T-cells (p = 0.021), although not significantly greater than the null model. To further address the synapse-proliferation association, we compared the fraction of T-cells that are Ki-67+ in the synapse positive and non-positive groups. For intratumoral T-cells in contact with loaded CD14+ cells, we observed a significantly greater fraction of Ki-67 positivity in synapse-positive T-cells compared to non-positive synapse T-cells (p = 2.8 × 10−4, Fig. 6b). We observed a similar association between KI-67 and T-cell/melanoma synapses, albeit with lower statistical significance (p = 0.014).Fig. 6: Proliferative consequences of the T-cell-CD14+ cell interaction in metastatic melanoma.a Proliferative (Ki67+, orange boxes) T-cell synapses are significantly stronger than non-proliferative (Ki67-, green boxes) T-cell synapses, for both loaded CD14+ cells and melanoma cells (p = 4.2 × 10–4 and p = 0.021 respectively). Proliferative T-cell synapses with loaded CD14+ cells are stronger than those with unloaded CD14+ cells (#p = 6.0 × 10–5) and the null model (@, p = 9.5 × 10–7). In the stroma, there are no significant differences in synapses between proliferative and non-proliferative T-cells with either loaded or unloaded CD14+ cells. b Paired difference plot comparing the fraction of intratumor Ki67+ T-cells vs. synapse strength. The fraction of T-cells that are Ki67+ is significantly greater for T-cells with positive synapses compared to non-positive synapses, both for synapses to loaded CD14+ cells (p = 2.8 × 10–4) and to melanoma cells (p = 0.014). Black lines indicate a positive difference while red lines indicate a negative difference.\(\bar{{\rm{\sigma }}}\): sample-level mean synapse strength.On the other hand, T-cell synapse formation with unloaded CD14+ cells was not associated with a proliferative response. We observed no significant difference in average synapse strength between Ki-67+ and Ki-67 − T-cells in contact with unloaded CD14+ cells, either within the tumor or in the stroma (Fig. 6a). Additionally, intratumoral Ki-67 + T-cells have significantly higher average synapse strengths with loaded CD14+ cells than with unloaded CD14+ cells (Fig. 6a, p = 6.0 × 10−5). In the stroma, T-cell Ki-67 status is not associated with synapse strength to an adjacent CD14+ cell, regardless of whether the CD14+ cell has an antigen load.Synapse analysis of breast cancer imaging mass cytometry data reveal T-cell/B-cell interactionsTo test the applicability of the RDF and CISA approaches to other types of tumors, we applied these methods to a cohort of breast cancer IMC images of primary tumors from 281 patients20, of which 275 had associated survival data and clinical subtyping. Because these IMC images were derived from tissue microarrays, individual IMC spot images had far fewer cells than histocytometry images, resulting in noisy and uninformative sample-level T-cell RDF curves. Consequently, we aggregated all breast cancer IMC images to generate a single cohort RDF for each class of cell-cell co-localizations. We did not distinguish antigen-loaded from -unloaded macrophages in this analysis because loaded macrophages make up only 0.3% of macrophages in the stroma and only 11% of the total macrophage content.RDF analysis showed no distinguishable colocalization of T-cells with tumor cells within breast tumors (ΔCDF = − 0.465, Fig. S4), similar to T-cells within melanomas. This behavior was consistent across clinical subtypes, with HER2+ tumors showing the largest (negative) deviation from the null (ΔCDF = − 2.260, Fig. S5A–C). Intratumor T-cell/macrophage associations were close to null expectations (ΔCDF = 0.509). T-cell/B-cell interactions were below null expectations, though noisy. This is related to the fact that only 2.6% of B-cells reside within tumor nests, which hinders statistical assessment of colocalization. In the stromal region, T-cells show a co-localization with both macrophages and B-cells, and these effects are of comparable magnitude (ΔCDF = 1.261 and 1.567 respectively). The T-cell co-localization with B-cells is apparent despite the presence of B-cells in only 130 of 275 images, compared to 272 patient images containing macrophages. Stromal T-cell colocalization with B-cells and macrophages is consistent across clinical subtypes with some variation in effect size (Fig. S5A–C).We then applied CISA to the breast cancer IMC images to investigate T-cell synapses with macrophages and B-cells. Because some samples have very limited number of B cells, only samples with at least 5 contacts between T cells and the neighboring cell type are included in the following analysis to increase statistical stability. In the intratumoral region (Fig. 7a), T-cells do not have significant synapse formation to macrophages (mean \(\bar{\sigma }\) = 0.094, 1-sample t-test p = 0.21) or tumor cells (mean \(\bar{\sigma }\) = −0.108, 1-sample t test p = 1.0), consistent with the lack of T-cell/cancer synapses in melanoma. A noteworthy effect in the breast IMC images is that intratumor T-cells form strong synapses with B-cells (mean \(\bar{\sigma }\) = 0.837, 1-sample t-test p = 7.5 × 10−5). These effect sizes are larger than those for the T-cell/loaded macrophage interactions in melanoma (Figs. 5a, b, 6a). Remarkably, this synapse effect is highly significant despite the lack of frequent colocalization between B-cells and T-cells — intratumor T/B-cell contacts occur in only 45 of 275 (16%) images. In the stroma, T-cell synapse behaviors are consistent with the tendency of both macrophages and B-cells to co-localize with T-cells. Stromal T-cells have significant synapse formation with macrophages (mean \(\bar{\sigma }\,\)= 0.106, 1-sample t test p = 3.5 × 10−5, Fig. 7a), on par with the effect size seen in melanoma (Fig. 5a). Moreover, stromal T-cells form even stronger immune synapses with B-cells (mean \(\bar{\sigma }\,\)= 0.565, 1-sample t-test p = 2.6 × 10-21). Curiously, due to the paucity of B-cells, stromal T-/B-cell contacts are present in only 75 of 275 (27%) patients, compared to macrophage contacts being present in 182 patients. Synapse behaviors do not significantly differ across breast cancer subtypes (1-way ANOVA p > 0.05 for each target cell, Fig. S5D–F).Fig. 7: T-cell synapses to B-cells are associated with breast cancer disease-free survival.a Immune synapses of T-cells to B-cells are significantly >0 in tumor nests (denoted by *, mean = 0.837, N = 45, 1-sample t-test p = 7.5 × 10–5), and in the stroma (mean = 0.565, N = 75, 1-sample t-test p = 2.6 × 10–21). There is a significant but low effect size for T-cells to macrophages in the stroma (mean \(\bar{\sigma }\,\)= 0.106, N = 182, 1-sample t-test p = 3.5 × 10–5). b HR + HER2− patients with stromal T-cell synapses to B-cells have significantly improved disease-free survival (DFS) in Kaplan-Meier (KM) survival analysis (p = 0.016). c Patients with increased immune infiltration do not have significantly improved DFS (p = 0.073). d Cox proportional hazards (PH) shows that stromal synapses to B-cells have a significantly negative log hazard ratio (HR) in HR + HER2− patients when compared to other infiltrative and clinical factors (log HR = −1.45, p < 0.005) in determining DFS. e Triple-negative breast cancer patients with stromal T-cell synapses to B-cells have significantly improved DFS in Kaplan–Meier (KM) survival (p = 0.018). f Patients with increased immune infiltration do not have significantly improved DFS (p = 0.377). g Cox proportional hazards (PH) show the stromal synapses to B-cells have a significantly negative log HR in triple-negative patients when compared to other infiltrative and clinical factors (log HR = −2.45, p = 0.02). \(\bar{{\rm{\sigma }}}\): sample-level mean synapse strength. N: number of samples that have at least 5 cell-cell contact between T cells and the indicated neighbor cell type.Stromal T-cell synapses with B-cells are associated with improved breast cancer survivalB-cells can hinder or help tumor growth depending on the context, and prior studies have suggested T-cells may be necessary for the beneficial B-cell effects31. We therefore tested whether the T-cell synapses to B-cells were associated with clinical outcomes in breast cancer. We first analyzed the 173 hormone-receptor positive HER2 negative (HR + HER2 − ) patients. We divided these patients into two groups based on synapse formation to B-cells: (1) those with a stromal T-cell:B-cell σ > 0 (representing contact and synapse formation), vs. (2) those with either σ <= 0 (representing contact with no synapse formation) or no contact (and therefore no synapse formation) between T- and B-cells. We observed a significant difference in disease-free survival (DFS) between these groups by Kaplan-Meier (KM) estimation, with improved survival for group 1 (p = 0.016, Fig. 7B). This DFS benefit is specific to the stroma, as splitting by the intratumor σ does not result in a significant survival difference (p = 0.88, Fig. S6A), nor does splitting by the combined intratumor and stromal T-cell σ (p = 0.106, Fig. S6B). DFS differences cannot be explained simply by contact between stromal T- and B-cells: grouping patients by the presence or absence of T-cells in contact with B-cells regardless of synapse strength resulted in a non-significant DFS difference (p = 0.131, Fig. S6C). Additionally, we analyzed the association of T-cell/B-cell infiltration on survival. In some images there could be sparse lymphocytes which would not be expected to influence outcome. So to determine an appropriate threshold for separating patients into high and low infiltration groups, we fit a two-component Gaussian Mixture Model, which yielded a separating threshold of lymphocyte infiltration of 5.4% (Fig. S6D). T- plus B-cell infiltration, while qualitatively associated with improved outcome, was not sufficient to distinguish patient DFS (p = 0.073, Fig. 7C). We next integrated these factors into a multivariable survival regression model using Cox Proportional Hazards, regressing the aforementioned factors and patient metadata against DFS. Stromal T-cell synapse formation with B-cells exhibited a negative and significant hazard ratio (log HR = − 1.45, CI [ − 2.45, −0.45], p < 0.005, Fig. 7d) in distinction from other factors, including increased immune cell infiltration.We next investigated the association of DFS with T-cell/B-cell interactions in other breast cancer molecular subtypes. The Jackson et al. cohort20 has 48 TNBC (HR − HER2 − ) and 52 HER2+ (including 29 HR+ and 23 HR − ) patients. Splitting patients again by stromal T-cell σ to B-cells, the survival benefit in TNBC is significant by KM survival estimation (p = 0.018, Fig. 7e). No samples had a non-positive sample-average synapse strength in the TNBC cohort, either when considering only stromal or both stromal and intratumor synapses. As such, we could not test for additional survival benefits of T-cell-B-cell contact. Survival was not significantly different when splitting patients by quantity of T- and B-cell infiltration (p = 0.377, Fig. 7f). Multivariable survival regression demonstrated T-cell synapse formation to B-cells has a strong effect on DFS after controlling for infiltration factors (log HR = − 2.45, CI [ − 4.52, −0.38], p = 0.02, Fig. 7g). In the HER2+ cohort, KM survival estimation shows no DFS benefit for stromal T-cell synapse formation to B-cells (p = 0.727, Fig. S6E) or increased T- and B-cell infiltration (p = 0.515, Fig. S6F). Rather, the presence of hormone receptors estrogen receptor (ER) and progesterone receptor (PR) are the factors influencing survival in the Cox Proportional Hazards model with opposite effects (ER: log HR = 1.56, CI [0.59, 2.72], p = 0.01, PR: log HR = − 1.72, CI [−3.02, −0.41], p = 0.01, Fig. S6G). Thus, T-cell synapses to B-cells have strong survival benefits in TNBC but not HER2+ patients in this cohort.Different cell segmentation approaches can identify different sets of pixels as the cell boundary areas, which are involved in the calculation of CISA. We therefore investigated the sensitivity of CISA with respect to the segmentation method used, by comparing CISA results after applying two different cell segmentation methods to the same Jackson et al. breast cancer dataset20. This dataset was originally segmented using nuclear expansion as implemented in Ilastik32 and CellProfiler33. We additionally applied MESMER34 cell segmentation to the raw images and labeled the segmented cells according to the annotation of matched cells in the original segmentation. A visual inspection shows that cells segmented by MESMER tend to be larger and rounder, but fewer in number (Fig. S7A, B). Nevertheless, a comparison of CD3-based synapse strength shows that the strong synaptic signal between T cells and B cells in stroma regions which was observed with original segmentation (mean = 0.565, N = 75, 1-sample t-test p = 2.6 × 10−21), is also observed with MESMER segmentation (mean = 0.485, N = 55, 1-sample t-test p = 7.5 × 10−13) (Fig. S7C, D). Similarly, for the intratumoral T cell-B cell synapse observed in intratumoral regions using the original segmentation (mean = 0.837, N = 45, 1-sample t-test p = 7.5 × 10−5), comparable synapses were observed with MESMER segmentation (mean = 0.714, N = 30, 1-sample t-test p = 0.0012) (Fig. S7C, D), though the statistical significance is lower for the MESMER segmentation, likely because there are fewer neighboring T-B cell pairs due to decreased numbers of segmented cells.To further test the TNBC findings, we performed a similar CISA analysis on a TNBC IMC35 dataset generated by our team (referred to as JAX TNBC IMC, see Methods. Additional manuscript in preparation). As expected, in the CD8 + T cells, we observed strong correlation between σCD3(T-CD8 + , APC) and σCD8(T-CD8 + , APC), but weak correlation between σCD3(T-CD8 + , APC) and σCD4(T-CD8 + , APC) (Fig. S8A). Also as expected, in the CD4 + T cells, we observed strong correlation between σCD3(T-CD4 + , APC) and σCD4(T-CD4 + , APC), but weak correlation between σCD3(T-CD4 + , APC) and σCD8(T-CD4 + , APC) (Fig. S8B). The correlations indicate that CISA effectively characterizes the distribution of TCR molecules along the T cell membrane in this dataset.Interestingly, unlike the other breast cancer dataset, we do not detect a positive average synapse between T cells and neighboring macrophages, B cells, or tumor cells, either in the intratumoral or stroma regions, in the new dataset (Fig. S8C, D). We performed survival analysis on T-B synapse strength (Fig. S8E). Positive synapse patients do exhibit superior survival, but the effect is not statistically significant. However, notably none of the patients with a positive T-B synapse strength in the dataset were yet deceased. Using the same threshold of 5%, patients with increased T cell and B cell infiltration also had better survival but the effect was not statistically significantly (log ranked p = 0.158) (Fig. S8F). Thus, the two IMC datasets show qualitatively similar survival associations between each of synapse strength and lymphocyte infiltration, but with different statistical significance.Image preprocessing can enhance CISA performanceBecause CISA is based on the logarithm of intensity ratios, it is innately robust to linear noise. Still, denoising has the potential to enhance image data quality. To test the robustness of CISA to image denoising, we applied several alternative preprocessing workflows to the ref. 20 TNBC IMC dataset and the ref. 24 dataset and then reperformed CISA analysis.First, we compared CISA performance on the breast cancer IMC dataset using: raw images, IMC-Denoise36, and a simple thresholding-based denoising. Stronger T-B synapse strengths are observed with both denoising approaches, with IMC-denoise showing reduced variance and strengthened significance (Fig. S9A, B). Using thresholding denoising, in intratumoral regions immune synapses of T-cells to B are significantly > 0 (mean = 0.849, n = 45, one sample t test p = 0.00012); and in stroma regions immune synapses of T-cells to B are significantly > 0 (mean = 0.493, n = 75, one sample t-test p = 4e-16). With IMC-Denoise, in intratumoral regions immune synapses of T-cells to B are significantly > 0 (mean = 0.750, n = 45, one sample t test p = 4.5e-13); and in stroma regions immune synapses of T-cells to B are significantly > 0 (mean = 0.607, n = 75, one sample t test p = 5.2e-40).Because the breast cancer dataset lacks CD4 and CD8a in its marker panel, we used the ref. 24 dataset for correlation analysis. With IMC-denoising, the expected correlation pattern is still observed: CD3 distribution is strongly correlated with CD8 distribution but not CD4 distribution in CD8 + T cells; and CD3 distribution is strongly correlated with CD4 distribution but not CD8 distribution in CD4 + T cells (Fig. S9C, D). This implies that CISA can successfully detect membrane distribution of TCR markers in images processed with IMC-denoising.
