DataGuide-AAPF-RGB

This document details the file system organization and navigation procedures for accessing both unprocessed RGB image data and the derived phenotypic measurements of the plants captured by the Ag Alumni Seed Phenotyping Facility (AAPF). The data can be categorized into two main parts:

Raw Measurement Data: This folder structure houses individual spreadsheets (.csv) and corresponding images (.png) for each measured phenotype.
Masterfile (.xlsx): This file contains a compilation of spreadsheets and a guide document explaining the data contents.

File System Structure
Filename Conventions
Image Products
How to Read Image Files Containing Morphological Measurement Info
How to Read Information in Spreadsheet files (.xlsx for masterfile or csv for individual measurements)

File System Structure

Example (does not contain all associated files):
Experiment_394 (/depot/smarterag/data/RGB/Experiment_394)
├── 394_SandSoybeanHigh
│   ├── VIS_R_2401323_240226103000057_all-all-all-APF_394_SandSoybeanHigh_240226105925431.apf (For internal reference only)
│   ├── VIS_R_2401323_240226103000057_FLUO-SideSmall-0-PNG_394_SandSoybeanHigh_240226105925431.png
│   ├── VIS_R_2401323_240226103000057_FLUO-SideSmall-30-PNG_394_SandSoybeanHigh_240226105925431.png
│   ├── VIS_R_2401323_240226103000057_FLUO-SideSmall-60-PNG_394_SandSoybeanHigh_240226105925431.png
│   ├── VIS_R_2401323_240226103000057_RGB-SideSmall-0-PNG_394_SandSoybeanHigh_240226105925431.png
│   ├── VIS_R_2401323_240226103000057_RGB-SideSmall-30-PNG_394_SandSoybeanHigh_240226105925431.png
│   ├── VIS_R_2401323_240226103000057_RGB-SideSmall-60-PNG_394_SandSoybeanHigh_240226105925431.png
│   ├── VIS_R_2401323_240226103000057_FLUO-Top-0-PNG_394_SandSoybeanHigh_240226105925431.png
│   ├── VIS_R_2401323_240226103000057_RGB-Top-0-PNG_394_SandSoybeanHigh_240226105925431.png
│   ├── VIS_V_2401323_240226103000057_RGB-SideSmall-0-MES_394_SandSoybeanHigh_240226105925431.png
│   ├── VIS_V_2401323_240226103000057_RGB-SideSmall-30-MES_394_SandSoybeanHigh_240226105925431.png
│   ├── VIS_V_2401323_240226103000057_RGB-SideSmall-60-MES_394_SandSoybeanHigh_240226105925431.png
│   ├── VIS_V_2401323_240226103000057_RGB-SideSmall-0-SEG_394_SandSoybeanHigh_240226105925431.png
│   ├── VIS_V_2401323_240226103000057_RGB-SideSmall-30-SEG_394_SandSoybeanHigh_240226105925431.png
│   ├── VIS_V_2401323_240226103000057_RGB-SideSmall-60-SEG_394_SandSoybeanHigh_240226105925431.png
│   ├── VIS_V_2401323_240226103000057_RGB-Top-0-MES_394_SandSoybeanHigh_240226105925431.png
│   ├── VIS_V_2401323_240226103000057_RGB-Top-0-SEG_394_SandSoybeanHigh_240226105925431.png
│   ├── VIS_P_2401323_240226103000057_Parameter_394_SandSoybeanHigh_240226105925431.csv 
|   └── VIS_S_2401323_240226103000057_Setting_394_SandSoybeanHigh_240226105925431.csv (For internal reference only)
├── 394_SandSoybeanMedium
│   └── (The contents of the directory adhere to the file type and format conventions outlined earlier) 
└── 394_SandSoybeanNo
    └── (The contents of the directory adhere to the file type and format conventions outlined earlier)

The data includes RGB and fluorescence images of plants along with derived morphological measurements and phenotypic information. Here's an example breakdown:

Parent directory
- Experiment_{Experiment_Nubmer}
- Example: Experiment_394
Subdirectories
- {Parent_directory}/{Experiment_Number}_{Treatment}
- Example: Experiment_394/394_SandSoybeanHigh

Filename Conventions

Individual files (interim data products) are located under subdirectories. Filenames follow a specific format that encodes information about the data:

VIS_<Data_Type>_<POT_BARCODE>_<Time_In>_<RGB_FLUO>-<View>-<Ang>-<Img_Product>_<Exp_No>_<Treatment>_<Time_Out>.{png|csv}

Example: VIS_V_2401323_240226103000057_RGB-SideSmall-30-MES_394_SandSoybeanHigh_240226105925431.png

Filename component	Description
`VIS`	Prefix indicating data from the visible light imaging chamber
`<Data_Type>`	R: raw image V: results derived from vegetation segments P: phenotypic parameters S: Sensor parameter (for internal reference only)
`<POT_BARCODE>`	Unique identifier for the plant pot
`<Time_In>`	Time (YYMMDDHHMMSS.SSS) when the pot entered the chamber
`<RGB_FLUO>`	Type of input sensor image (RGB or Fuorescence)
`<View>`	View from which the image was captured: Top: Top-down view Side{Bottom, Small, Top, Full}: height at which RGB sensor is located
`<Ang>`	Angle of the camera (0, 30, ..., 330 for side views, 0 for top-down view)
`<Img_Product>`.	Type of image product (see images below) PNG: RGB or flurescence image SEG: segmentation results MES: vegetation masking results augmented with morphological measurement info such as bounding box, convex hull
`<Exp_No>`	Experiment number within AAPF
`<Treatment>`	Treatment info
`<Time_Out>`	Time (YYMMDDHHMMSS.SSS) when the pot left the chamber

Image Products (example, resized)

PNG (RGB)	PNG (FLUO)	SEG	MES

How to Read Image Files Containing Morphological Measurement Info (image with MES in the filename)

These image files visually represent morphological measurements on the vegetation segments. Color overlays indicate different aspects:

Gray (pixels): vegetation pixels segmented
Green (Polyline): Smallest enclosing rectangle of the vegetation segments
Blue (Polyline): Convex hull (smallest enclosing polygon) of the vegetation segments
Red (Pollyline): Pixel-wise boundary of vegetation segments
Cyan (point): Center of mass coordinate

How to Read Information in Spreadsheet files (.xlsx for masterfile or csv for individual measurements)

Phenotypic measurements derived from RGB images and auxiliary data are stored in spreadsheets. The master data file (*.xlsx) is organized into three sections:

Top: This section contains phenotypic measurements acquired from a top-down view of the plants.
Side_average: This section provides the average phenotypic measurements calculated from data captured at various side-view angles (0°, 30°, 60°, ..., 330°).
Side_all: This section contains detailed phenotypic measurements for each individual side-view angle (0°, 30°, 60°, ..., 330°).

Both the master data file (.xlsx) and individual measurement files (.csv) share a common data headers. A detailed explanation of the column headers follows:

Column Head	Description
`Filename`	Input spreadsheet filename (.csv)
`EXP ID`	Experiment number within AAPF
`POT_BARCODE`	Unique identifier for the plant pot
`VARIETY`	Variety assigned in PPEW
`TREATMENT`	Treatment applied to plant
`SCAN_TIME`	Scan start time
`SCAN_DATE`	Scan start date
`DFP`	Age of plant in days from planting at time of imaging
`Angle`	Side view angle where the vegetation is most dispersed
`View`	TOP FRAME: Ignore this row TOP AVG: Data from the top view Side{Bottom, Small, Top, Full} FRAME 0 to 11: Data from various side view angles (0, 30, ..., 330 degrees) Side{Bottom, Small, Top, Full} AVG: Average of Side view FRAME data
`frame_nr`	Frame number from 0-11 for side view data
`Width`, `Height`	Dimensions of the smallest enclosing rectangle of the vegetation segments
`Surface`	Area of the vegetation within the pixel-wise boundary of vegetation segments
`Convex hull`	Area of the convex hull measured by the smallest enclosing polygon of the vegetation segments
`Roundness`	Roundness of the pixel-wise boundary of vegetation segments (❗ calculation method unclear)
`Center_of_mass_distance`	❗ Unclear meaning
`Center_of_mass_x`, `Center_of_mass_y`	Coordinates of the center of mass point
`Hue, Saturation`, `Intensity`	Average values for these color properties in the image
`Fluorescence`	Average fluorescence within the pixel-wise boundary of vegetation segments
`H###`, `S##`, `V##`	Frequency of pixels with specific hue (###: 0-359), saturation (##: 0-99), and value (##: 0-99) Check the definitions of hue, saturation, and value in this link
`F##`	Frequency of pixels with a specific fluorescence value (##: 0-99)