This data repository contains raw data, pre-prcessed data and evaluation code associated with the manuscript Noninvasive identification of carbon-based black pigments with pump-probe microscopy.

The logical order to evaluate these files would be the following: 
1) Run the Mathematica notebook "prepare_data.nb" to create pre-processed data
2) Run the python script "gridsearch_svm.py" which performs a grid search for the SVM
3) Run the script "train_and_classify.ipynb" which selects the optimal hyperparameter from the grid search and trains a final SVM classifier. This script can also be used to classify the pump-probe stacks of pigment mixtures. Note that within the jupyter lab notebook absolute paths are used and may need adjustment given the local environment.

Folder "data_raw":
Contains pairs of pump-probe images stacks (tiff data) and log files (txt data) for 20 pure pigment samples, 22 black - black pigment mixtures, and 36 ultramarine-blue - black mixtures. Specific pigment and mixing ratio are indicated with the filename. The corresponding logfiles contain experimental parameters.

Folder "data_pre_processed":
Contains a pre-processed version of all image stacks in folder data_raw. As the pre-processing is done with the proprietary software Wolfram Mathematica (Version 13.3), we decided to provide a pre-processed version of the data as well. The three files "classification_results_umb_*.pickle", "classifier_umb_*.pickle", and "gridsearch_umb_*.pickle" are intermediate results saved in the python "pickle" format as binary. Note that content of these results stem from test runs to validate the code. The content of these files is not usefult for calssification of data. 

Parent Directory containing evaluation scripts:

"prepare_data.nb": Wolfram Mathematica (Version 13.3) notebook for data pre-processing. The results from this notebook are saved in folder "data_pre_processed". Note that results from this notebook are already present in the folder in case Mathematica is not available to the user.

"pps.py": python library containing three classes PPS, ML, and AM which are used for data import, data export, file navigation, machine learning, unmixing and classification. Classes are used in the scripts "gridsearch_svm.py", "train_and_classify.ipynb", and "unmixing.ipynb"

"gridsearch_svm.py" performs a grid search for a SVM for black pigment classification. The relevant function in the script is ML.grid_search(X, y, 0.25, 5, 5, "gridsearch_umb_run01.pickle", 15) with the first argument the input data, the second argument the data label, the third argument the ratio between training and test data, the fourth argument the number of repetition of the training and test procedure, the fifth argument the number of folds within each repetition, the sixth argument the filename in which the grid search results are saved, and the last argument the size of the grid. We used the arguments shown here for training, the arguments in the file might differ and stem from code testing.

"train_and_classify.ipynb": Jupyter lab notebook that is used to train the final SVM classifier and also to classify pump-probe image stacks.

"unmixing.ipynb": Jupyter lab notebook that is used for the unmixing approach.

Versions of used python packages:
numpy 1.26.4
matplotlib 3.8.0
scikit-learn 1.3.2
imbalanced-learn 0.11.0