AAV_PLSR

This repository has the notebooks for generating PLSR models to predict concentration and percent non-empty of spectral data.

Using GUI

Run GUI/AAV_gui.py
Click Choose CSV to upload input file. Then, click Run Prediction and Save CSV to save predictions in the location of your choice.
Input format:

label	wavenumber_1 (float)	wavenumber_2	...
string_1	absorbance (float)	absorbance	...
string_2	absorbance	absorbance	...
...

The default wavenumber input labels used by the trained models is:

**Click to see input labels**

989.379, 990.069, 990.759, 991.448, 992.138, 992.828, 993.517, 994.207, 994.897, 995.586, 996.276, 996.966, 997.655, 998.345, 999.034, 999.724, 1000.41, 1001.1, 1001.79, 1002.48, 1003.17, 1003.86, 1004.55, 1005.24, 1005.93, 1006.62, 1007.31, 1008, 1008.69, 1009.38, 1010.07, 1010.76, 1011.45, 1012.14, 1012.83, 1013.52, 1014.21, 1014.9, 1015.59, 1016.28, 1016.97, 1017.66, 1018.34, 1019.03, 1019.72, 1020.41, 1021.1, 1021.79, 1022.48, 1023.17, 1023.86, 1024.55, 1025.24, 1025.93, 1026.62, 1027.31, 1028, 1028.69, 1029.38, 1030.07, 1030.76, 1031.45, 1032.14, 1032.83, 1033.52, 1034.21, 1034.9, 1035.59, 1036.28, 1036.97, 1037.66, 1038.34, 1039.03, 1039.72, 1040.41, 1041.1, 1041.79, 1042.48, 1043.17, 1043.86, 1044.55, 1045.24, 1045.93, 1046.62, 1047.31, 1048, 1048.69, 1049.38, 1050.07, 1050.76, 1051.45, 1052.14, 1052.83, 1053.52, 1054.21, 1054.9, 1055.59, 1056.28, 1056.97, 1057.66, 1058.34, 1059.03, 1059.72, 1060.41, 1061.1, 1061.79, 1062.48, 1063.17, 1063.86, 1064.55, 1065.24, 1065.93, 1066.62, 1067.31, 1068, 1068.69, 1069.38, 1070.07, 1070.76, 1071.45, 1072.14, 1072.83, 1073.52, 1074.21, 1074.9, 1075.59, 1076.28, 1076.97, 1077.66, 1078.34, 1079.03, 1079.72, 1080.41, 1081.1, 1081.79, 1082.48, 1083.17, 1083.86, 1084.55, 1085.24, 1085.93, 1086.62, 1087.31, 1088, 1088.69, 1089.38, 1090.07, 1090.76, 1091.45, 1092.14, 1092.83, 1093.52, 1094.21, 1094.9, 1095.59, 1096.28, 1096.97, 1097.66, 1098.34, 1099.03, 1099.72, 1100.41, 1101.1, 1101.79, 1102.48, 1103.17, 1103.86, 1104.55, 1105.24, 1105.93, 1106.62, 1107.31, 1108, 1108.69, 1109.38, 1110.07, 1110.76, 1111.45, 1112.14, 1112.83, 1113.52, 1114.21, 1114.9, 1115.59, 1116.28, 1116.97, 1117.66, 1118.34, 1119.03, 1119.72, 1120.41, 1121.1, 1121.79, 1122.48, 1123.17, 1123.86, 1124.55, 1125.24, 1125.93, 1126.62, 1127.31, 1128, 1128.69, 1129.38, 1130.07, 1130.76, 1131.45, 1132.14, 1132.83, 1133.52, 1134.21, 1134.9, 1135.59, 1136.28, 1136.97, 1137.66, 1138.34, 1139.03, 1139.72, 1140.41, 1141.1, 1141.79, 1142.48, 1143.17, 1143.86, 1144.55, 1145.24, 1145.93, 1146.62, 1147.31, 1148, 1148.69, 1149.38, 1150.07, 1150.76, 1151.45, 1152.14, 1152.83, 1153.52, 1154.21, 1154.9, 1155.59, 1156.28, 1156.97, 1157.66, 1158.34, 1159.03, 1159.72, 1160.41, 1161.1, 1161.79, 1162.48, 1163.17, 1163.86, 1164.55, 1165.24, 1165.93, 1166.62, 1167.31, 1168, 1168.69, 1169.38, 1170.07, 1170.76, 1171.45, 1172.14, 1172.83, 1173.52, 1174.21, 1174.9, 1175.59, 1176.28, 1176.97, 1177.66, 1178.34, 1179.03, 1179.72, 1180.41, 1181.1, 1181.79, 1182.48, 1183.17, 1183.86, 1184.55, 1185.24, 1185.93, 1186.62, 1187.31, 1188, 1188.69, 1189.38, 1190.07, 1190.76, 1191.45, 1192.14, 1192.83, 1193.52, 1194.21, 1194.9, 1195.59, 1196.28, 1196.97, 1197.66, 1198.34, 1199.03, 1199.72, 1200.41, 1201.1, 1201.79, 1202.48, 1203.17, 1203.86, 1204.55, 1205.24, 1205.93, 1206.62, 1207.31, 1208, 1208.69, 1209.38, 1210.07, 1210.76, 1211.45, 1212.14, 1212.83, 1213.52, 1214.21, 1214.9, 1215.59, 1216.28, 1216.97, 1217.66, 1218.34, 1219.03, 1219.72, 1220.41, 1221.1, 1221.79, 1222.48, 1223.17, 1223.86, 1224.55, 1225.24, 1225.93, 1226.62, 1227.31, 1228, 1228.69, 1229.38, 1230.07, 1230.76, 1231.45, 1232.14, 1232.83, 1233.52, 1234.21, 1234.9, 1235.59, 1236.28, 1236.97, 1237.66, 1238.34, 1239.03, 1239.72, 1240.41, 1241.1, 1241.79, 1242.48, 1243.17, 1243.86, 1244.55, 1245.24, 1245.93, 1246.62, 1247.31, 1248, 1248.69, 1249.38, 1250.07, 1250.76, 1251.45, 1252.14, 1252.83, 1253.52, 1254.21, 1254.9, 1255.59, 1256.28, 1256.97, 1257.66, 1258.34, 1259.03, 1259.72, 1260.41, 1261.1, 1261.79, 1262.48, 1263.17, 1263.86, 1264.55, 1265.24, 1265.93, 1266.62, 1267.31, 1268, 1268.69, 1269.38, 1270.07, 1270.76, 1271.45, 1272.14, 1272.83, 1273.52, 1274.21, 1274.9, 1275.59, 1276.28, 1276.97, 1277.66, 1278.34, 1279.03, 1279.72, 1280.41, 1281.1, 1281.79, 1282.48, 1283.17, 1283.86, 1284.55, 1285.24, 1285.93, 1286.62, 1287.31, 1288, 1288.69, 1289.38, 1290.07, 1290.76, 1291.45, 1292.14, 1292.83, 1293.52, 1294.21, 1294.9, 1295.59, 1296.28, 1296.97, 1297.66, 1298.34, 1299.03, 1299.72, 1300.41, 1301.1, 1301.79, 1302.48, 1303.17, 1303.86, 1304.55, 1305.24, 1305.93, 1306.62, 1307.31, 1308, 1308.69, 1309.38, 1310.07, 1310.76, 1311.45, 1312.14, 1312.83, 1313.52, 1314.21, 1314.9, 1315.59, 1316.28, 1316.97, 1317.66, 1318.34, 1319.03, 1319.72, 1320.41, 1321.1, 1321.79, 1322.48, 1323.17, 1323.86, 1324.55, 1325.24, 1325.93, 1326.62, 1327.31, 1328, 1328.69, 1329.38, 1330.07, 1330.76, 1331.45, 1332.14, 1332.83, 1333.52, 1334.21, 1334.9, 1335.59, 1336.28, 1336.97, 1337.66, 1338.34, 1339.03, 1339.72, 1340.41, 1341.1, 1341.79, 1342.48, 1343.17, 1343.86, 1344.55, 1345.24, 1345.93, 1346.62, 1347.31, 1348, 1348.69, 1349.38, 1350.07, 1350.76, 1351.45, 1352.14, 1352.83, 1353.52, 1354.21, 1354.9, 1355.59, 1356.28, 1356.97, 1357.66, 1358.34, 1359.03, 1359.72, 1360.41, 1361.1, 1361.79, 1362.48, 1363.17, 1363.86, 1364.55, 1365.24, 1365.93, 1366.62, 1367.31, 1368, 1368.69, 1369.38, 1370.07, 1370.76, 1371.45, 1372.14, 1372.83, 1373.52, 1374.21, 1374.9, 1375.59, 1376.28, 1376.97, 1377.66, 1378.34, 1379.03, 1379.72, 1380.41, 1381.1, 1381.79, 1382.48, 1383.17, 1383.86, 1384.55, 1385.24, 1385.93, 1386.62, 1387.31, 1388, 1388.69, 1389.38, 1390.07, 1390.76, 1391.45, 1392.14, 1392.83, 1393.52, 1394.21, 1394.9, 1395.59, 1396.28, 1396.97, 1397.66, 1398.34, 1399.03, 1399.72, 1400.41, 1401.1, 1401.79, 1402.48, 1403.17, 1403.86, 1404.55, 1405.24, 1405.93, 1406.62, 1407.31, 1408, 1408.69, 1409.38, 1410.07, 1410.76, 1411.45, 1412.14, 1412.83, 1413.52, 1414.21, 1414.9, 1415.59, 1416.28, 1416.97, 1417.66, 1418.34, 1419.03, 1419.72, 1420.41, 1421.1, 1421.79, 1422.48, 1423.17, 1423.86, 1424.55, 1425.24, 1425.93, 1426.62, 1427.31, 1428, 1428.69, 1429.38, 1430.07, 1430.76, 1431.45, 1432.14, 1432.83, 1433.52, 1434.21, 1434.9, 1435.59, 1436.28, 1436.97, 1437.66, 1438.34, 1439.03, 1439.72, 1440.41, 1441.1, 1441.79, 1442.48, 1443.17, 1443.86

Training your own models

Input CSV format

wavenumber (cm-1)	concentration	percent_nonempty	wavenumber_1 (float)	wavenumber_2	...
label_string_1	concentration value (float)	percent non empty value (float)	absorbance (float)	absorbance	...
label_string_2	concentration value	percent non empty value	absorbance	absorbance	...
...

Generate synthetic replicates

Open 'synthetic_replicates.ipynb'
Modify variable input_path to the input path to your csv file for raw data.
Modify variable output_folder to the path to the folder where you want to save synthtic data.
Run all cells. Output files (synthtic data) will be in the folder you provided.
The file raw_data(name)_synthetic.csv should be used as input for all following programs.

Preprocessing

Open preprocess.ipynb
Modify variable input_path to the input path to your csv file.
Modify variable output_folder to the path to the folder where you want to save preprocessed data.
Run all cells. Output files (preprocessed data) will be in the folder you provided.
The file snv_normalized_synthetic.csv should be used as input for all following programs.

Training models

Using Leave-One-Out Cross Validation (LOOCV) to select the number of components and train PLSR models.

Open concentration_loocv.ipynb to train the concentration model or open percent_loocv.ipynb to train the percent model.
Modify input_path and output_folder to the path to input and the path to where output must be saved.
Run all cells.
Output will be saved in folder provided.
The onnx file in the output folder is the trained model.
The files ending with the following, either prefixed by concentration or percent are required for using the model
- _X_train_mean.csv
- _y_train_mean.csv

Using models

Copy the following files from the output folder of the training step to the model_files folder inside the GUI folder
- The onnx model file
- The _X_train_mean.csv file
- The _y_train_mean.csv file
Open GUI/AAV_gui.py
Update variables at the top of the file to point to the model files you want to use.
- Update these lines:

conc_model = "plsr_loocv_concentration_ncomp.onnx"
percent_model = "plsr_loocv_percent_ncomp.onnx"
conc_X_mean = "concentration_X_train_mean.csv"
conc_y_mean = "concentration_y_train_mean.csv"
percent_X_mean = "percent_X_train_mean.csv"
percent_y_mean = "percent_y_train_mean.csv"

Follow instruction in the Using GUI section to use the models.

Group K-Fold Validation Notebooks

concentration_groupkfold.ipynb: Train concentration model using Group K-Fold Cross Validation
percent_groupkfold.ipynb: Train percent non-empty model using Group K-Fold Cross Validation
Input and output format is same as described in the Training your own models section.
Modify input_path and output_folder variables in the respective notebooks before running.

VermaLab/AAV-PLSR

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Contributors

Uh oh!

Languages