Focus : "Sound-image elevation used in combination with SRS to create a large sound image"--originally conceived for in-car listening, this effect is designed to lift the sound up near your head, something needed in a vehicle where speakers are or were generally placed at hip or ankle level. TruBass : "Psychoacoustic bass enhancement to enable deeper, natural bass of audio source material to be perceived over small speaker drivers. Essentially, TruBass uses acoustic trickery to make your brain believe that you are hearing bass that is actually in the song but that the hardware usually headphones is incapable of creating.
Pretty nifty, eh? Of course, technology can always be improved upon case in point: Have you ever seen a perfect 10 score on CNET? SRS Wow HD adds a few new tricks to its predecessor's audio suite, one of which I'm not allowed to tell you about yet. The other new addition is Definition, which brings out the high end, providing more clarity and detail, as well as a more realistic listening experience for live music.
SRS Labs also improves upon the TruBass function, adding user controls that allow the listener to select from eight speaker size settings. Also, given the linear speed of the polygon scanner, the acquired raw Raman spectrum is free of spectral channel distortion. For evaluation, we measured the spectral profiles of five chemicals and compared them with spontaneous Raman spectra Supplementary Fig. At concentrations as low as 0. To extract information from the high-speed yet noisy spectroscopic images, we apply a two-step processing approach that involves SNR recovery and chemical mapping.
We first generated pairs of spectroscopic SRS images as the training set, with high-speed, low-SNR images as the raw acquisition and a low-speed, high-SNR image through averaging of multiple raw acquisitions as the ground truth.
The framework of the network is shown in Supplementary Fig. Due to the large size of each spectroscopic image and the experimental difficulty of generating a large number of training images, the network is based on the U-net 38 encoder-decoder structure.
The up-sampling and skip-connection layers in the network improves the resolution of learned features and thus requires less training samples. Each convolution filter is detailed in Supplementary Fig. Since the memory cost for each filter is reduced, our network could incorporate six filters at each layer without exceeding the GPU memory limit. Finally, a residual learning scheme 40 is applied to facilitate the training of a deep network.
The overall U-net structure greatly reduced the need for the number of training samples. After SNR recovery, the spectroscopic image stack is linearly decomposed into chemical maps Fig.
The level of regularization can be fine-tuned such that the output can suppress crosstalks between different channels while avoiding artifacts. We applied the approach to unprecedented imaging conditions reaching high speed, high SNR, and high chemical specificity in the fingerprint region for a wide variety of biological samples. The applications include living cancer cells, whole mouse brain slice, and single bacteria, with a focus on chemicals that are difficult to study in the C—H region.
Lipid metabolism is a cellular process involving spatiotemporal dynamics of fatty acid and cholesterol. The distributions of different lipid species in the cell are tightly regulated to ensure proper cellular activities and function. Abnormal lipid metabolism is related to many human diseases, including aggressive cancer 8 , 9. Thus, quantitative imaging of lipids in living systems is of great interest. Unlike fluorescence imaging of lipophilic dyes, Raman spectroscopy provides high chemical specificity to differentiate lipid species, such as cholesterol and various fatty acids.
With enhanced signal levels, SRS is capable of quantitative imaging of specific lipid species. However, due to the limited signal levels in the fingerprint region, except in the abovementioned cases of excessive accumulation, it remains challenging to study cholesterol in single living cells or large-area tissues. For training, we first acquired a dataset consisting of pairs of raw and ground truth images of Mia PaCa-2 cells.
We used fixed Mia PaCa-2 cells to ensure that the ground truth images formulated by excessive averaging do not suffer from motion artifacts. After training, the performance of SNR recovery was validated using a set of previously unseen images.
The same validation dataset was processed by block-matching 4D filtering BM4D 44 , a state-of-the-art unsupervised 3D image denoising algorithm. Also, to compare the performance of spatial-spectral convolution, a U-net with 3D CNN was trained and tested on the same dataset.
The results Supplementary Fig. Both measurements suggest significant improvement of the image quality using SS-ResNet. The averaged spectral profiles from a small region of interest for the raw, ground truth, and recovered images Supplementary Fig.
To test whether the network recovery facilitates downstream spectral analysis, we selected a small region of interest from the validation set Fig.
The outputs from the network and the ground truth show similar spatial distributions and concentrations for all three components Fig.
In contrast, the results from the raw data failed to provide insights into the distributions of chemical species and were difficult to distinguish from the background noise Fig. The SSIM indices increased considerably after recovery, which proved that our approach did not introduce artifacts and provided reliable results on the subsequent chemical analysis.
GT and network vs. GT of the three chemical channels. The boxes show interquartile range IQR , the red line indicates medians, the black lines represent whiskers which extend to 1. SSIM, structural similarity index. Three significant motion artifacts are highlighted as circled regions.
To apply this high-speed, high-sensitivity method to the real-time mapping of lipid in living cells, we imaged living Mia PaCa-2 cells and recovered high-resolution images from the raw images taken at high speed by applying the same SS-ResNet trained on fixed cells. In living Mia PaCa-2 cells, lipid droplets are shown to be highly dynamic Live-cell imaging at the speed of 1. In contrast, we observed severe motion artifacts in the averaged image from the live-cell data Fig.
SS-ResNet recovered images from a single frame showed clear circular-shaped droplets within the cells, highlighting the importance of temporal resolution during live-cell imaging. The chemical maps of cholesterol and fatty acid Fig. We further asked whether this method could be used to track changes in cholesterol amount and distribution. These data collectively show that deep-learning high-speed fingerprint SRS imaging enables high-fidelity, real-time chemical mappings of chemical bonds in single living cells and facilitates the tracking of metabolite dynamics at subcellular levels.
Brain tissue is comprised of many cell types, and biomolecules in the tissue are highly heterogeneous among different brain areas. Chemical mapping of the whole brain is essential for studying the functionality of molecules in the brain. Previous label-free metabolic studies of mouse whole-brain slices were mainly based on multi-color SRS imaging in the C—H window, providing only protein and lipid information 7 , For the sake of maintaining sample conditions during the experiment, the total acquisition time of a mouse whole-brain slice is usually several hours.
Therefore, it remains challenging to perform spectroscopic SRS imaging in the fingerprint region to generate chemical maps of other biomolecules. Following the procedures in Fig.
Due to the much-complicated spatial features in the brain tissue, a total of 50 training image pairs were taken for training. Each raw image was taken at a speed of 3. After recovery, the SNR of the raw image improved significantly with the subcellular details preserved, reaching comparable image quality to the ground truth image Fig. The averaged spectral profiles from a selected region of interest for the raw, GT, and recovered images are shown in Supplementary Fig.
Taking advantage of the high imaging speed of our system and the ability to recover high SNR by SS-ResNet, we performed fingerprint SRS spectroscopic imaging of a mouse whole-brain slice. Different colors indicate different percentage concentrations from the three channels. Circled regions in the DG area include rare cells with high cholesterol content. The composite image of the three components shows significant heterogeneity among different cells and brain structures Fig.
To further characterize the distribution of the biomolecules, we focused on several brain regions and features Fig. Overall, the soma of mature neurons shows relatively lower concentrations of all three components compared to the surrounding tissue.
Surprisingly, we found abundant cholesterol-rich cells present near neurons in the LH and basal amygdaloid BM regions, which may represent different metabolic activities in this population of cells.
We also observed that nerve bundles in the ventral posterior nucleus VP and CPu are comprised of different ratios of cholesterol and fatty acid. Interestingly, there are a few rare cells that contain high cholesterol concentrations in the DG region Circled regions in Fig. In summary, large-area SRS imaging in the fingerprint region is a viable tool for label-free measurement of cellular cholesterol content, which could be used to address the relationship between cholesterol metabolic activity and a variety of brain diseases and disorders, including various neurodegenerating disorders and brain tumors.
Limonene and pinene are biofuel precursors that can be produced biosynthetically in microbes such as Escherichia coli E. Currently, quantitation of biochemical production levels mainly relies on gas chromatography-mass spectrometry GC—MS , which suffers from low throughput and requires extraction steps that destroy the sample. Strain engineering and optimization typically involve the construction of many variants, followed by screening, in a lengthy iterative process.
The limited throughput of GC—MS approaches hinders efficient optimization of design variables for biochemical synthesis. In addition, GC—MS only provides quantification of population-level production, ignoring the potential for genetic or phenotypic variation among cells 52 , Thus, a high-throughput quantification method that provides direct measurement of biofuel concentrations has the potential to improve the design, build, and test cycle necessary for improving production strains.
SRS is a promising approach to fulfill this requirement by detecting intrinsic vibrational signatures from the biofuels. Yet, due to the overwhelming SRS contributions from endogenous proteins and lipids, quantitative imaging of the production levels for certain biofuels i. High-throughput SRS imaging in the fingerprint region is expected to address this challenge by providing specific and well-separated Raman spectra for the biofuels.
We have applied our platform to perform high-throughput quantitative chemical imaging of chemical compounds produced by genetically engineered E. After training, SS-ResNet was applied to a validation set to test the recovery performance. Further quantitation of the reconstruction quality is depicted in Fig.
As depicted in Supplementary Fig. L-production, Limonene production. P-production, Pinene production. We compared this to limonene production 50 and pinene-production 51 strains of E. Based on the spectral profiles from pure chemicals, pixel-wise LASSO spectral analysis decomposed the network-recovered spectroscopic images of the strains into the maps of the three chemicals.
The chemical maps indicated that the wild-type strain Fig. We did not include fatty acid and cholesterol in the analysis due to the negligible contributions. An independent verification of the production level was performed by measuring the biofuel concentrations of the whole culture using GC—MS Supplementary Fig.
From the GC—MS results, limonene and pinene are clearly present in the limonene and pinene-production strains, respectively. Furthermore, GC—MS results represent the average chemical concentration of the entire culture, yet we noticed from the SRS results that the biofuels were highly aggregated as droplets in single cells, which result in a much higher local concentration that facilitates SRS detection.
Raman spectroscopic imaging of living systems has been a grand challenge due to limited speed in spectral acquisition. However, the standard implementation of spectral focusing relies on moving a delay stage mechanically. Because of the slow speed in this scheme, hyperspectral CARS or SRS imaging is commonly done in a frame-by-frame manner, and a hyperspectral cube would take a few minutes.
Such speed does not allow the study of a dynamic or living system without spectral distortion. For biological cells, fixation is needed, which may cause unwanted biochemical changes inside a cell. Compared with our previous implementation using a kHz resonant scanner 21 , the polygon scanning system not only improves the speed 5-fold but also achieves high-spectral linearity that increases reliability.
In addition, the high-speed delay scanning scheme can be applied to a broad range of modalities requiring a long delay scan, such as transient absorption spectroscopy and impulsive SRS imaging.
In this work, we trained a spatial-spectral residual net as a supervised denoiser that outperformed conventional unsupervised image restoration algorithms. The encoder-decoder structure alleviates the requirement for training data size, which is of great importance for biomedical imaging, given the high cost associated with acquiring training data.
Thus, on the one hand, when the images are very challenging to denoise, it can formulate a deeper network for better performance. On the other hand, if the image can be denoised equally well by the two methods, the reduced size of the model can always make room for more batches in the GPU memory for faster training. Our supervised denoiser can significantly increase the reliability of the subsequent chemical content analysis. Besides, for the task of denoising spectroscopic image stacks, due to the universal properties of noise under the same imaging conditions, a trained network can be quickly tweaked to denoise other samples by transfer learning.
As shown in Supplementary Fig. Direct application achieved high-SNR levels but sacrificed spatial resolution due to the differences between spatial features for the two datasets. By feeding in training data of the new samples, the network required less than half of the training epochs to converge and output high-resolution, high-SNR images, making it convenient to apply to different applications.
It is important to discuss how far the network can push the physical limit of SRS imaging. As discussed in the three demonstrations, the lowest SNR that a network can recover is dependent on the morphological and spectral structures of the samples.
In general, the network performs better for images with complex structures, such as cancer cells and tissue. Since SNR is proportional to the square root of imaging time, we select averages in our case. Further increasing the number of averages for the GT does not improve the denoising quality that much as the bottleneck has become the noise level of the input, whereas decreasing the number of averages will lead to poor quality of recovery since the network cannot learn the actual structures.
Thus, the deep-learning network used here can increase the imaging speed by roughly two orders of magnitude. Like most deep-learning-aided optical imaging applications, the most time-consuming part of the imaging and analysis pipeline is the training of the network. In addition, for similar experimental conditions, a pre-trained network can be quickly adjusted through transfer learning, which can greatly reduce the training time.
More importantly, SS-ResNet has better denoising performance to allow for a higher image acquisition throughput during the experiment, which is often more critical than the offline processing speed. To further improve the offline unmixing throughput for a large dataset, we can simply run multiple instances in parallel or even use several PCs since the spectral unmixing problem for each image is independent. An advantage of the SRL modality is that most laser power is on the Stokes beam, which has a longer wavelength and, consequently, a higher damage threshold.
Notably, in this work, we applied extensive pulse chirping for both beams, which much reduced the laser peak powers and diminished the nonlinear damage consequently. For the Mia PaCa-2 training and validation samples Fig. These spots were only found occasionally in fixed cells, which are likely the aggregates of cell debris formed during the fixation process. These aggregates were floating in the environment and could easily attach to the cells.
In comparison, we did not observe such bright spots in continuous imaging of live Mia PaCa-2 cells Supplementary Videos 1 — 5 , nor did we found cell membrane blebbing, a signature of cell membrane damage Before training, we performed image normalization using 0.
Therefore, these spots did not affect the actual performance of the network. It is important to discuss whether cell fixation alters the spectral profiles used in our hyperspectral SRS imaging. It is reported that after cross-linking, the amide I band showed a general peak intensity decrease, but the peak position had no shifting Comparing SRS images of fixes and live cells, we did not observe significant cellular intensity change to alter the SNR of the raw images.
Besides, since we used intensity-normalized spectral profiles as references, we anticipate that cross-linking of proteins does not affect the accuracy of spectral unmixing. It is important to know whether the grating can cause angular dispersion to the beam and subsequently induce spatial resolution degradation.
Thus, the spatial resolution of the polygon scanning system is not compromised. Due to the wavelength difference between pump and Stokes, it is a good practice to chirp them using the rods with different lengths.
For the fingerprint region data reported in this work, we used 5 SF57 rods cm each on the common path and added a cm rod on the Stokes path to compensate for its longer wavelength. For the CH region, the wavelength difference is more significant.
Yet, we note that an optimized chirping condition for the CH region will sacrifice the spectral resolution in the fingerprint region due to the over-chirping of Stokes. In conclusion, the combination of ultrafast tuning via a polygon scanner and SNR recovery via deep learning has enabled reliable fingerprint SRS imaging at microsecond spectral acquisition speed. The improved speed and spectral resolution by our polygon-based delay tuning of chirped pulses are essential for SRS imaging in the fingerprint region.
Meanwhile, the learning network allowed effective SNR enhancement by one order of magnitude. With such advances, we have demonstrated simultaneously imaging of various biomolecules that are difficult to identify in the high-wavenumber C—H window.
This technique has broad applications, as demonstrated in this study: from monitoring biofuel production levels in engineered bacteria to the metabolic study of cancer cells, up to large-area whole-brain tissue imaging. Collectively, our approach opens the door to a plethora of biomedical applications from tracking dynamics and interactions of metabolites in a single cell to the high-throughput compositional mapping of an unprocessed human tissue.
The Stokes beam was then directed to a polygon scanner Lincoln SA24, Cambridge Technology , which scanned the laser onto a blazed grating GR, Thorlabs positioned at Littrow configuration. The grating acted as a reflective wedge to reflect the Stokes beam along the same optical path.
Each scan by the polygon scanner thus introduces a continuous increase of light path for a few millimeters, resulting in a series of continuous temporal delays between the pump and the retroreflected Stokes beam. The maximum delay range of the system is determined by the length of the scan line and the blazed angle of the grating.
As shown in Fig. The beams were collinearly combined by a dichroic mirror DM, Chroma and were both broadened to picosecond by high dispersion glass rods SF To compensate for the chirping difference due to the wavelength of the two lasers, we used five cm glass rods on the common path and added one cm rod on the Stokes path.
The chirped beams were sent collinearly to an upright microscope, and a 2D galvo scanner set GVS, Thorlabs was used for scanning images.
After filtering the Stokes beam following the interaction with the sample, a photodiode S, Hamamatsu with a custom-built resonant circuit was used to collect signals. A max-pooling layer is applied at the end of each layer to reduce the dimensions. In the decoder phase, each layer first up samples the feature map and then concatenates it with the corresponding feature maps in the encoder phase. The same six SS-Conv layers are used at each layer. In addition, the prediction layer was added with the input layer such that the prediction value was the residual 40 with respect to the raw input image, which has been shown to predict higher resolution images.
The parameters were learned by minimizing a loss function that averages the mean squared error between the prediction and ground truth. To quantify the reconstruction error and compare it with the raw input, we first normalized the ground truth and the predicted image to the same dynamic range by the same method reported in the deep-learning image restoration work However, in practice, least-squares fitting alone generates chemical maps with severe crosstalks in complex biological samples where many biochemicals have overlapping spectral profiles.
To improve the performance, we observe that for each spatial pixel, only a few chemical components contribute significantly, which is equivalent to the sparsity of concentrations at each pixel. The method, known as the least absolute shrinkage and selection operator LASSO , has been widely used to solve problems in which the variable is sparse, e. With the use of pixel-wise LASSO unmixing, it is possible to resolve more chemicals in the same window since LASSO effectively stabilizes the solution and suppresses the crosstalks between different channels, especially for a complex living system when the spectral profiles of independent chemical components are similar.
The method does not have a strict constraint on the level of sparsity or the maximum number of independent chemical components to tolerate, it is a soft regularization method, and the levels of sparsity can tone down in the case of multiple mixtures.
The cells were then washed with and imaged in PBS buffer. The mouse brain slice was prepared from a mouse Jackson Lab at age 21 days. PBS was used for perfusion, after which formalin was perfused to fix the brain tissue. The E. The kanamycin resistance marker gene was removed from the Keio collection strain. For each demonstration, the SS-ResNet was independently trained three times with similar denoising and spectral unmixing results. For Fig.
For Supplementary Fig. Further information on research design is available in the Nature Research Reporting Summary linked to this article. All the data related to the work is available upon reasonable request to the corresponding author. Freudiger, C. Label-free biomedical imaging with high sensitivity by stimulated Raman scattering microscopy.
Science , — Nandakumar, P. Vibrational imaging based on stimulated Raman scattering microscopy. New J. Ploetz, E. Femtosecond stimulated Raman microscopy. B Lasers Opt. Ozeki, Y. Analysis and experimental assessment of the sensitivity of stimulated Raman scattering microscopy. Express 17 , — Slipchenko, M. High-speed vibrational imaging and spectral analysis of lipid bodies by compound Raman microscopy. B , — Lee, H. Label-free vibrational spectroscopic imaging of neuronal membrane potential.
Ji, M. Rapid, label-free detection of brain tumors with stimulated Raman scattering microscopy. Li, J. Lipid desaturation is a metabolic marker and therapeutic target of ovarian cancer stem cells. Cell Stem Cell 20 , — Yue, S. Cell Metab. Saar, B. Video-rate molecular imaging in vivo with stimulated Raman scattering.
High-speed molecular spectral imaging of tissue with stimulated Raman scattering. Photonics 6 , — Zhang, D. Quantitative vibrational imaging by hyperspectral stimulated Raman scattering microscopy and multivariate curve resolution analysis.
Broadband stimulated Raman scattering with Fourier-transform detection. Express 23 , Liao, C. Spectrometer-free vibrational imaging by retrieving stimulated Raman signal from highly scattered photons. Fu, D. Quantitative chemical imaging with multiplex stimulated Raman scattering microscopy. Berto, P.
0コメント