A pillar-based microfluidic chip for T-cells and B-cells isolation and detection with machine learning algorithm

Abstract

Absolute counting of total leukocytes and specific subset (such as T-cells and B-cells) within small amounts of whole blood is difficult due to the lack of techniques that enables separation of leukocytes from limited volume of whole blood. In this study, a microfluidic chip equipped with a size controlled micropillar array for highly separation of T-cells and B-cells from sub-microliter of whole blood was studied. Based on the difference in size and deformability, leukocytes were separated from other blood cells by micropillar arrays. However, the variability of cells in size, morphology and color intensity along with the spectrum crosstalk between fluorescence dyes make cell detection among pillars extremely difficult. In this paper, an support vector machine supervised machine learning classifier based on both Histogram of Oriented Gradients (HOG) and color distribution features was proposed to distinguish T-cells and B-cells fast and robustly. HOG features were utilized to detect cells from background and noise; color distribution features were employed to alleviate the effect of fluorescence spectrum crosstalk. Experiment showed we achieved average detection accuracy of 94% for detecting T-cells and B-cells from the background. Furthermore, we also got 96% accuracy with cross validation to detect T-cells from B-cells. Both theoretical analysis and experiments demonstrated the proposed method and system has high performance in T-cells and B-cells counting. And our microfluidic cell counting system has great potential as a tool for point-of-care leukocyte analysis system.

Introduction

Leukocytes and their subtypes population numbers change dramatically in the presence of infections, malignancies, autoimmune disorders, and chemical-induced hematotoxicity. The two categories of lymphocytes, T-cells and B-cells, are important constituents of the human immune system as their concentration, quantified by a count test, indicates the state of body’s immune response against infections [1]. Currently, fluorescence-activated cells sorting (FACS) is used as golden standard in biological research and clinical diagnostics to automatically determine the count or concentration for one or multiple types of cells. However, requirements of large volume sample (in the order of mL) and expensive reagent, bulky equipment size, sophisticated costly optical component and the requirement of technical personnel make conventional FACS prohibitive for point-of-care application. In addition, FACS is traditionally relied on non-imaging technique by laser scattering and fluorescence emission for cell identification and hence there is no image information of cells [2]. There is a tremendous need for an automated, portable point-of-care blood cell counter that could yield results in a matter of minutes from a drop of blood without any trained professionals to operate the instrument [3]. This technology could potentially have tremendous applications in hospitals, private clinics, retail clinics and the developing world.

In order to develop a compact leukocyte counting system, researchers are actively looking toward microfluidic devices as the platform for the next generation translatable cell sorter. Microfluidic based devices are a proven technology for cellular handling as they can offer precise spatial and temporal control in a greatly miniaturized platform [4, 5]. For cell isolation, there are several methods that utilize the physical characteristics of the cell [6]. Blood cells can be separated using non-inertial forces and concepts such as magnetic bead immunoassay [7], dielectric properties [8], optical force [9], magnetic [10], electrohydrodynamic [11], or acoustic [12, 13], as active separation strategies. The non-inertial forces may have some unexpected physiological effects on the blood component properties which are unsuitable for the downstream application of the separated products. These techniques add to the complexity of the microfluidic devices and normally run at very low flow rates. On the other hand, passive separation techniques exploit the difference in size and deformability of cells in which they do not involve external forces but rely purely on the interaction of the blood with internal geometries of microfluidic devices [14]. The pore-based or pillar-based passive devices have simple mechanical design and can function in wide range of flow rates [15, 16], WBCs are flexible and can deform to pass through 7 µm pores. The RBCs are more flexible and can fold or align to cross the filter opening of 3–4 µm [17]. A perfect blood microfilter can trap all WBCs and pass all RBCs continuously without loss of separation efficiency at different flow rates and cell concentration.

After isolation, captured cells are usually distinguished from microfluidic chip’s fluorescent microscopy images. Using different cell markers, we can detect, identify and count cells. However, cells imaged by fluorescence microscopy exhibit heterogeneous intensity levels and are often badly contrasted. The variability of cell size and morphology, differences in illumination over time and across whole microfluidic chip hamper the ability to specify a global set of parameters for cell detection algorithms over the whole experiment. Furthermore, there are serious spectrum crosstalk between fluorescence dyes when performing dual or multiple cells detection. The available algorithms for illumination correction and segmentation do not perform well enough to achieve satisfying results on many experiments [18]. Therefore, there is a strong motivation for the development of an automatic cell detection and counting method [19]. A machine learning algorithm, with an appropriate feature extractor for the application is the key to success. For this, Histogram of Oriented Gradients (HOG) represents a robust feature descriptor used in computer vision area for object detection [20]. One of the key advantages of HOG is being able to describe object orientation while showing invariance to geometric and photometric transformations because it operates in localized regions. In other words, HOG tends to be unaffected by changes in shapes and lighting, which appear in larger spatial regions. These advantages make HOG features fit our situation especially well due to the cells variability in intensity and shape on pillar based microfluidic chip. A support vector machine (SVM) can be used to classify HOG features. SVM is a supervised machine learning algorithm, which creates a model according to the training data, and then the test data can be classified according to said model [21].

In this study, we propose a pillar-based microfluidic chip and system with machine-learning for T-cells and B-cells isolation and detection. The design of micropillar array with gradually narrowed gaps were optimized to trap leukocytes while allowing other blood cells to flow through [22]. The size and deformability-based separation of leukocytes require micropillar of proper gap size and precise control of pressure applied to the array. Under optimized conditions, a sub-microliter of whole blood was suspended in a buffer and then directly introduced to the micropillar array without pre-treatment such as density gradient or RBCs lysis. All leukocyte types were individually trapped between micropillars, while other elements of blood passed through the micropillar array into a waste reservoir. To detect T-cells and B-cells from fluorescence microscopy images, a machine learning method based on both HOG and color features was studied.

Materials and methods

Peripheral blood sample preparation

Human blood samples were collected from healthy donors at National Hospital Organization Nagoya Medical Center. The study protocol was reviewed and approved by the institutional review board, and written informed consent was obtained from all participants. The samples were collected in a collection tube with EDTA to prevent coagulation and were used within 24 h. For T-cells and B-cells detection, blood sample was mixed with a two-color direct immunofluorescence reagent (BD Simultest TM CD3-FITC/CD19-PE, BD Bioscience, San Jose, CA) and incubated in room temperature for 15 min.

A pillar-based microfluidic chip

We designed a pillar-based microfluidic chip to isolate leukocytes from peripheral blood with high efficiency and without clogging [22]. Figure 1a shows the schematic layout of the proposed microfluidic filter. Blood is pumped into the device via inlet port, enters the filtration zone and leaves it at outlet port. Outlet port is for collecting the components of blood passing the filter. Proposed microfluidic chip has 12 different groups of pillar arrays [22]. Main difference between these pillar arrays is the spacing between each pillar, which we call gap size. The gap size for these 12 pillar arrays are gradually decreasing in size from the inlet port to the outlet port, 15–3 µm respectively. The smallest gap size, 3 µm, can prevent leukocytes flowing away while allow red blood cells or platelets to pass through. Larger and less deformable cells are captured by pillar arrays with bigger gaps while smaller cells flowed through, which makes this design suitable to prevent clogging. To further avoid clogging, escape routes without trap structure among pillars are designed, as Fig. 1a shows. The schematic and prototype of microfluidic chip was shown in Fig. 1b. The single inlet single outlet fabricated device was connected to a gastight syringe pump (1710 TLLX SYR, Hamilton, USA), which was manipulated by a high precision controller (QT-AMH2, Chuo Precision Industrial, Japan). The PTFE tubing was inserted into the Tygon tubes of the device fluidic interconnects and was used for pumping of sample to the inlet port of the device and transfer of separated components from the device outlet port to a collection tube. The whole experiments were monitored with an inverted optical microscope for detailed investigation of the filtration process. The images and videos were captured by a camera (ASI178 MC, Zhen Wang Optical Company, China, 3096 × 2080, pixel size 2.4 µm) installed on the eyepiece body tube of the microscope. The procedure of acquisition is semi-auto and the operator controls some camera parameters, such as, magnification, exposure time and focus. After sample introduction, the filtration zone on chip was scanned automatically frame by frame for counting trapped T-cells and B-cells. As described in sample preparation part, T-cells were stained by CD3-FITC (Ex 494 nm, Em 520 nm) excitation and B-cells were stained by CD19-PE (Ex 496 nm, Em 578 nm). In the next section, we would present a method for detecting T-cells and B-cells in the images characterized in this section.

Fig. 1

Overview of the proposed system. a Design of the microfluidic chip, blood is pumped into the device via inlet port. Due to the difference in size and deformability, leukocytes will be trapped in different zones. b Experiment setup. c Block diagram of cell detection framework using machine learning method

Microfluidic chip fabrication

Our microfluidic chip fabrication process is based on the soft-lithography technique. We used a Si-wafer to make a master mold. The microchannel part is made of poly-dimethylsiloxane (PDMS) by using a molding process. Then, the microchannel part is bonded to a glass plate to form the microfluidic chip. First, the photomask was prepared using laser lithography (DWL66FS, Heidelberg Instruments Mikrotechnik GmbH, Heidelberg, Germany). We then prepared the Si-wafer (thickness 500 μm, Matsuzaki Ltd., Japan) for spin coating with an epoxy-based photoresist (SU-8 3010, Microchem, MA, USA) to the required thickness (~ 10 μm) and cured it using a soft bake (95 °C, 30 min) process. Next, the Si-wafer was exposed to UV light with the photomask using a mask aligner (Suss MA6, SUSS MicroTec, Germany). The Si-wafer was developed and dry-etched by deep reactive ion etching (RIE-800, Samco, Japan). Plasma polymerization (RIE-800, Samco, Japan) of octafluorocyclobutane (C₄F₈) was used for passivation of the Si-mold to make the surface non-adhesive during repeated PDMS microfluidic chip demolding processes. The PDMS microchannel part was prepared using a PDMS pre-polymer (Silpot 184, Dow Corning Toray Co., Ltd., Japan) mixed in a 10:1 (w/w) ratio with a curing agent, then pouring onto the Si-mold and baking at 85 °C for 45 min. The cured PDMS was then demolded and the PDMS microchannel part was treated with an O₂ plasma (Femto Science Cute-MPR, South Korea) to complete the PDMS-glass bonding process.

Cell detection

The framework of object detection with HOG and color features is illustrated by the block diagram in Fig. 1c. It consists of separate training and testing phases. In the training phase, we trained two separate SVMs to detect cells from background and to identify the detected cells as T-cells or B-cells. First using the dataset that is generated from the dual dyed images, a linear-kernel SVM is trained to detect cells from background. Then, the images that is dyed with only singe dye is exhaustively searched using sliding window method, finding HOG for each window and classified by the first SVM. Found cells then used for their color information to train a Radial Basis Function (RBF)-kernel SVM [21] to identify T-cells from B-cells. Although RBF-SVM is timewise costly, it gives better accuracy. In the testing phase, unseen images are scanned with sliding window and HOG features are generated from the position of each window in the image. These features are classified as cell or background with the linear SVM classifier, then classified cells are further processed to get the color information and classified as T-cells or B-cells using the RBF SVM.

The HOG and color-based features extraction will be described in the next section.

Results

In order to get an idea about how well the T-cells and B-cells are detected, it is necessary to study both SVM classifier and performance of the system as a whole.

Cell isolation

Cell isolation experiment is performed for several times, to test the performance of our system and method. The experiment condition is decided empirically [22]. First, we introduced sheath liquid (PBS with 5 mM EDTA) into the microfluidic chip to remove the bubbles. We introduced 1 µL peripheral blood into chip and collected the fluid out of chip in a tube. After all blood sample flowed through the chip, 30 µL sheath liquid was introduced to remove non-trapped cells. The flow rates for peripheral blood and sheath liquid are both set to be 10 µL/min. Then, the whole filtration part on chip was scanned frame by frame with the control of motorized stage. When scanning, T-cells and B-cells are excited (488 nm). 420 frames were captured for the whole filtration zone. Figure 2 shows partial scanned images of the filtration zone. According to the design principle, larger cells were mostly trapped in wide pillar gaps, while smaller cells, such as T-cells and B-cells, were mainly trapped in narrow pillar gaps. As Fig. 2 shows, most T-cells (green) and B-cells (yellow) were captured in 5–8 µm gaps area.

Fig. 2

Partial scanned images of the filtration zone for T and B lymphocytes

Dataset and methodology

There is no standard dataset for fluorescent cell images, so we created a custom dataset to be used in our detection algorithms. To train and test the detection of cells we used the scan images from 10 experiments that is dyed with the dual dye. Seven experiments are used as training data which consists of 2940 images. Another three experiments which consists of 1320 images are reserved for test. A total of 6200 cell images and 35,000 background images are gathered semi-automatically from the training images. Figure 3 shows a sample from this dataset.

Fig. 3

Example images from the dataset. a Cell images centered and scaled. b Hard negative images, that is manually eliminated. c Background images

Cells come in many sizes and shapes since they are deformed by the pillars of the microfluidic chip. To simplify the problem we scaled each cell, so the detector only needs to be trained for the shape information. First a small set of cell images are cropped manually from the training images. This is used to train the initial detector which is then used to automatically detect and gather training images time effectively. Found cells are then cropped and scaled. Then all the cropped images are manually checked to eliminate the misdetections and noises, which are used as hard negatives as Fig. 3b shows. After this semi-automated process, 50 training images which are dense in cells are annotated manually, also fixing the scaling of the cells when necessary. Negative training samples are then automatically cropped in different scales from these annotated images. Using the positive and negative training data from this semi-automatic process the detector is trained, then annotated images are exhaustively searched for misdetections. Found hard negatives are added to the training dataset to re-train the final detector.

Crosstalk between the dyes, prevents us from manually annotating the datasets to be used as training data. Hence in order to train the second SVM which identifies T-cells from B-cells, we used the datasets which are only dyed with one color.

HOG descriptor and color distribution features

Preparing the data set appropriately is vital for achieving good performance. Cell images are centered and scaled. Since the size of the cells vary, we upscaled them to 80 pixels squares, with the scaling factor appropriate to the longest side of the cell bounding boxes. A margin of 16 pixels around the cell on all four sides are included, as this border provides a significant amount of context that helps detection. As Fig. 4 shows, HOG helps us define the shape context of the cells effectively, which in return makes it possible to detect all different shapes of cells deformed by the micropillars in our study.

Fig. 4

Original image 1(a), horizontal and vertical derivatives of image 1(b), orientations of gradients overlaid on each pixel 1(c), orientations gathered into histograms and block normalized 1(d). Histogram of gradient features overlaid on cells 2(a, b), noise 2(c) and background image 2(d)

We computed HOG descriptors for the prepared cell samples. A HOG feature vector is modeled for such an image. Figure 4 shows step by step computation of HOG descriptors. First, one the image gradient values are determined, representing directional changes in the intensity or color. The gradient vector is formed by combining the partial derivatives of image I in the x and y directions,

$$\nabla I = \sqrt {\left( {\frac{\partial I}{\partial x}} \right)^{2} + \left( {\frac{\partial I}{\partial y}} \right)^{2} }$$

The gradients in the two directions are computed by applying the 1D centered, point discrete derivative mask in the horizontal and vertical directions,

$$\frac{\partial I}{\partial x} = I \times \left[ {\begin{array}{*{20}c} { - 1} & 0 & 1 \\ \end{array} } \right],$$

$$\frac{\partial I}{\partial y} = I \times \left[ {\begin{array}{*{20}c} { - 1} \\ 0 \\ 1 \\ \end{array} } \right].$$

The gradient orientations are computed using the vertical and horizontal gradients as,

$$\uptheta = { \tan }^{ - 1} \left( {\frac{\partial I}{\partial x} \div \frac{\partial I}{\partial y}} \right)$$

The image I is then divided into cells, and for each cell, a local 1D histogram of gradient directions (orientations) is calculated over the pixels from that cell. We used nine bins for the local histogram. The histogram channels are evenly spread over 0°–180° in absolute values, so each histogram bin corresponds to a 20-degree orientation interval. The obtained cell histograms are then combined into a descriptor vector of the image. First, these cells locally contrast normalized, due to the variability of illumination and shadowing in the image. That requires grouping the cells together into larger, spatially-connected blocks. Once the normalization is performed, all the histograms can be concatenated in a single feature vector, representing the HOG descriptor. Depending on the cell size and block overlap, the amount of detail one could get changes. We used [8 × 8] cell size to calculate blocks of [16 × 16] pixels with overlaps using nine histogram channels. These setting can be thought as default choice in object detection. The feature vector of the image is computed as its HOG descriptor, with 2916 coefficients.

Because the long emission spectrum tail of dyes causes overlap like with the fluorophores FITC and PE, in our T-cells and B-cells counting experiment, cells are stained by a two color (CD3-FITC/CD19-PE) immunofluorescence dye. The color vectors for cells fluctuate in a big range. Cells can be sorted from background and noise through HOG features, but among the detected cells, it is difficult to decide the cell type when taking multicolor cell detection. To alleviate the influence of fluorescence emission spectrum crosstalk, we calibrated the color distribution of FITC and PE single dye stained cells and then classified dual stained cells by the model trained from single dye stained cells color features. We stained samples with single dye, FITC Mouse Anti-Human CD3 (BD Pharmingen™, BD Bioscience, San Jose, CA) for T-cells and PE Mouse Anti-Human CD19 (BD Pharmingen™, BD Bioscience, San Jose, CA) for B-cells. The single dye stained samples were introduced to the same isolation system and sorted under the same condition as dual dye stained sample. As stained by only one fluorescent dye, the detected cells’ type could be decided. We used previously trained HOG detector to detect the cells and extracted the average color in RGB space for every cell in FITC and PE. After the cells are detected, an adaptive thresholding using Otsu’s method is performed to find the cell segment [23]. Then average color in RGB is calculated using this thresholding as a mask. We trained an RBF kernel SVM using the color averages from single dye stained cells. This calibration needs to be performed only once. Through cross-validation, an average accuracy of 96% was achieved.

Detection results of T-cells and B-cells

We have created a data set to test the performance of the HOG detector. 500 positive cell images are cropped, centered and scaled from the annotated test images. 50 difficult examples containing noise and 450 randomly selected background examples are used as negative test set. To quantify the HOG detector performance, we plot Receiver Operating Characteristics (ROC’s), i.e. true positive rate $\left( {\frac{TruePos}{TruePos + FalseNeg}} \right)$ versus false positive rate $\left( {\frac{FalsePos}{FalsePos + TrueNeg}} \right)$. Using also this test set we calculated accuracy, specificity and sensitivity values as:

$${\text{Accuracy}} = \left( {\frac{TruePos + True Neg}{Total \;Population}} \right)$$

$${\text{Specificity}} = \left( {\frac{True Neg}{TrueNeg + FalsePos}} \right)$$

$${\text{Sensitivity}} = \left( {\frac{TruePos}{TruePos + FalseNeg}} \right)$$

In addition to this set of 1000 image patches, negative images from test data exhaustively searched and around 4,500,000 negative images are added to plot Detection Error Tradeoff (DET) curve on log–log scale i.e. miss rate $\left( {\frac{FalseNeg}{TruePos + FalseNeg}} \right)$ versus false positive per window $\left( {\frac{FalsePos}{Total Population}} \right)$. DET curves present the same information as ROC’s but small differences in probabilities are easier to distinguish. Figure 5 presents ROC’s and DET curves.

Fig. 5

ROC (a) and DET (b) curves for HOG detector

Different points in ROC and DET curves corresponds to different accuracy, specificity and sensitivity. We have achieved an accuracy of 94% and sensitivity of 90% when specificity was chosen to be 99%.

In one experiment, 420 frames are scanned for the filtration zone on the microfluidic chip. With the classifiers trained with HOG and color distribution feature, all the 420 images were processed in 67 s with our experiment PC (CPU i7-8700 K 3.7 GHz, Nvidia GTX 1080Ti, 64 GB RAM). We have used OpenCV libraries for HOG to perform GPU processing. Figure 6 shows the results of an example detection, although cells are of different morphology and size, all cells have been detected.

Fig. 6

Results of counting, a part of the image is cropped for better visibility. The detected T-cells were labeled with green rectangle and detected B-cells were labeled with orange rectangle

Discussion

Several methods exist for counting the cells in an image [24, 25]. The difficulty of performing the cell segmentation or detection task and choosing the correct algorithm depends much on the type of cells being targeted. If the cells are well separated from each other and have uniform intensity, simple thresholding or watershed algorithms are popular choices of approach. If the cells are packed together, algorithms which account for cell shape and size are preferred [26]. But in our application, the unavoidable uneven illumination over time and across the whole microfluidic chip makes the cells fluorescence intensity differs from image to image. And the cells’ shape and size were deformed by pillar structure. The intensity, shape and size features could not be utilized directly to detect cells in our pillar-based microfluidic chip system. Another approach, like the one used in this study, is to recognize cells using training-based machine learning methods. Examples of cells and background are shown to the detector, which then learns their most important characteristics. Furthermore, to detect both T-cells and B-cells in a single experiment, the emission spectrum crosstalk of these two cells’ dyes should also be overcome. In this work, cell detection from fluorescence microscope images is studied using both HOG and color distribution features with SVM learning classifier. HOG features were utilized to distinguish cells from background noise. But HOG features could only distinguish cells from background and noise, how to eliminate the crosstalk between FITC and PE emission spectrum is still a question. We evaluated the color distribution of cells solely stained by FITC or PE and trained the classifier based on the single stained cells’ color features. The classifier was used to determine the cell type when sample were stained by dual-color fluorescence (FITC and PE).

In our work, detection of T-cells and B-cells are important in terms of ratio in between them, hence it is more important to have small ratio of false positives than having small ratio of miss rates. Therefore, we set our detector to have less false positives. This is the reason we chose to prioritize having a high enough specificity such as 99% while sacrificing accuracy and sensitivity which were 94% and 90% respectively. For one single experiment, the total experiment time on microfluidic chip takes about 33 min, which was calculated as follows. Staining the sample requires 15 min, introducing 1 µL peripheral blood and 30 µL sheath liquid with 10 µL/min flow speed takes about 3 min, the whole chip scan costs 15 min, image processing can be simultaneously performed while scanning. FACS requires red blood cell lysis for sample preparation, therefore FACS needs time for the experiment. Compared to the time FACS need (> 1 h), the microfluidic chip requires less time to separate leukocytes directly from whole blood (no lysis). For specific cells types, such as total lymphocytes or T-cells and B-cells, the microfluidic counting system provided a fast and robust way for cell isolation and detection. Although, comparing to expensive FACS systems, there are still problems (clogging and spectrum overlap) in multi-type cells sorting for current microfluidic system.

Conclusion

A pillar-based microfluidic chip with machine learning algorithm for T-cells and B-cells isolation and detection were studied in this paper. A HOG and color features based SVM classifier was proposed to overcome the difficulties in cell detection process, such as fluorescence emission spectrum crosstalk, variability of cell size and morphology, differences in illumination and inconsistence fluorophores expressing level on living cells. We have performed numerous cell detection experiments using the detection technique provided here. The experiment tests performed on various image datasets, have produced satisfactory detection results that prove the effectiveness of our proposed approach. It has achieved a high accuracy of 94%, specificity of 99% and sensitivity of 90%. For T-cells B-cells detection we have achieved 96% cross-validation accuracy. The proposed method and system could also be applied to all other specific leukocytes using different stain agent. Compared to current commercial cell sorter, our system is low cost, small size and microfluidic chip is disposable, it has great potential as a tool for point-of-care cell sorting and analysis.