This data repository contains raw data, pre-prcessed data and evaluation code associated with the manuscript Noninvasive identification of carbon-based black pigments with pump-probe microscopy.

The logical order to evaluate these files would be the following: 
1) Run the Mathematica notebook "prepare_data.nb" to create pre-processed data
2) Run the python script "gridsearch_svm.py" which performs a grid search for the SVM
3) Run the script "train_and_classify.ipynb" which selects the optimal hyperparameter from the grid search and trains a final SVM classifier. This script can also be used to classify the pump-probe stacks of pigment mixtures. Note that within the jupyter lab notebook absolute paths are used and may need adjustment given the local environment.

Folder "data_raw":
Contains pairs of pump-probe images stacks (tiff data) and log files (txt data) for 20 pure pigment samples, 22 black - black pigment mixtures, and 36 ultramarine-blue - black mixtures. Specific pigment and mixing ratio are indicated with the filename. The corresponding logfiles contain experimental parameters.

Folder "data_pre_processed":
Contains a pre-processed version of all image stacks in folder data_raw. As the pre-processing is done with the proprietary software Wolfram Mathematica (Version 13.3), we decided to provide a pre-processed version of the data as well. The three files "classification_results_umb_run01.pickle", "classifier_umb_run01.pickle", and "gridsearch_umb_run01.pickle" are intermediate results saved in the python "pickle" format as binary. Note that content of these (initial) files results from test runs to validate if the evaluation procedure works. The remaining files represent classified pump-probe image stacks in form of abundance maps (png). Note that content of these (initial) images are results from test runs to validate if the evaluation procedure works.

Parent Directory containing evaluation scripts:

"prepare_data.nb": Wolfram Mathematica (Version 13.3) notebook for data pre-processing. The results from this notebook are saved in folder "data_pre_processed". Note that results from this notebook are already present in the folder in case Mathematica is not available to the user.

"pps.py": python library containing two classes PPS and ML which are used for data import, data export, file navigation, machine learning, and classification. Classes are used in the scripts "gridsearch_svm.py" and "train_and_classify.ipynb"

"gridsearch_svm.py" performs a grid search for a SVM for black pigment classification. The relevant function in the script is ML.grid_search(X, y, 0.25, 5, 5, "gridsearch_umb_run01.pickle", 5) with the first argument the input data, the second argument the data label, the third argument the ratio between training and test data, the fourth argument the number of repetition of the training and test procedure, the fifth argument the number of folds within each repetition, the sixth argument the filename in which the grid search results are saved, and the last argument the size of the grid.

"train_and_classify.ipynb": Jupyter lab notebook that is used to train the final SVM classifier and also to classify pump-probe image stacks.

Versions of used python packages:
numpy 1.26.4
matplotlib 3.8.0
scikit-learn 1.3.2
imbalanced-learn 0.11.0