UAV Thermal Data Processing Pipeline (GRYFN+TeAx)

This user guide details the processing procedures for thermal imagery acquired by the TeAx sensor onboard the GRYFN platform. ❗ Process all sensor data except thermal using the most recent version of the GRYFN Processing Tool before proceeding.

Manual by Sungchan Oh (Sun😎)
First Created: August 31, 2023
Last Updated: May 21, 2024
Contact: oh231@purdue.edu

Here's a step-by-step walkthrough of the process.

❗ Process all non-thermal sensor data acquired by the GRYFN (+TeAx) platform using GRYFN Processing Tool.
Perform quality checks on the GRYFN Export Trajectory file (export*.txt).
- 📁 Locate the GRYFN Export Trajectory file using the provided shell script.
```
cd [your_flight_directory]
# Example: /depot/smarterag/fy2024_drone_data/yang1527/Field 58/2024_4_25_Field58
find ~+ -type f -name "export*.txt"
```
- For data consistency, verify the presence of specific columns in the GRYFN Export Trajectory file.
  - 🗒️ Open the GRYFN Export Trajectory file and navigate to line 27.
  - 🗒️ Look for the following column headers: EASTING(m), NORTHING(m), ELLIPSOID_HEIGTH(m).
    
    If any of these are present, you can assume all the listed headers
    (EASTING(m), NORTHING(m), ELLIPSOID_HEIGTH(m),
    EASTING_SD(m), NORTHING_SD(m), HEIGHT_SD(m),
    ROLL_SD(deg), PITCH_SD(deg), HEADING_SD(deg)) are included.
- 📁 Locate the directory containing raw thermal images (*.TMC).
Extract XYZ coordinates corresponding to each acquired thermal image.
- 🗒️ Open the Python script exportThermalXyzFromGryfnEvents.py located in your current directory.
- 🗒️ Based on the information from step 2, configure the following file paths:
  - GRYFN Export Trajectory file path (path_apx)
  - Directory containing raw TMC files (dir_tmc)
    
    Mac OSX or Linux: Use a forward slash (/) to separate directories, like this: path_apx = "/path/to/your/data/folder/file.txt" Replace /path/to/your/data/folder with the actual location of your file or directory.
    
    Windows: Use a backslash (\) to separate directories, like this: path_apx = "C:\Users\yourname\data\folder\file.txt" Replace C:\Users\yourname\data\folder with the actual location of your file or directory.
- (Optional) Visit the following link to check for the current leap second information. If the provided value is different from 18 seconds, update the leap_sec parameter in the script to match the current leap second value.
- Run exportThermalXyzFromGryfnEvents.py.
  
  ❕The exportThermalXyzFromGryfnEvents.py script ensures proper time synchronization between thermal images and their corresponding GPS data. Images with a time difference exceeding a specified threshold (max_timediff parameter) from the closest APX event (GPS signal reception) are considered unreliable and excluded from further processing. This filtering step helps maintain data integrity and avoid potential errors in subsequent processes.
- This dataset (*.TMC) comprises L0R data products.

Important

The exportThermalXyzFromGryfnEvents.py script generates a list of thermal images excluded from further processing due to exceeding the time synchronization threshold. These files are moved to nouse directory under dir_tmc.

Convert raw thermal data (TMC) to radiometrically corrected JPEG (RJPG) images (🪟Windows Only)

An RJPG file, also referred to as a radiometrically corrected JPG file, is essentially a JPEG image that has undergone additional processing to improve its accuracy in representing real-world temperatures.
- Launch ThermoViewer:
  - Ensure "Parse NMEA Data" is selected under Settings in the menu bar to process embedded GPS information within the thermal images.
- Configure Conversion Parameters:
  - Go to the menu bar and select "Display RAW Conversion Parameters dialog".
  - Under the "Color Adjustment > Color" menu, choose "Grayscale".
  - Under the "Temperatures" menu, select "TLinear high gain °C" next to the TLinear model.
  - Uncheck the "Show legend" option.
- Merge TMC Files:
  - From the menu bar, navigate to Edit > Merge Files.
  - Select TMC Files:
    - Click "Add File(s)" to include thermal data files.
    - Browse to the directory containing your thermal data (TMC files).
    - Select all TMC files.
    - Click "Merge" to combine the selected TMC files.
    - Save the merged file with a descriptive name (e.g., F42_20230731_Mohave.TMC).
- Open Merged File:
  - In the ThermoViewer left pane, navigate to the directory where you saved the merged file and open it.
  - (Optional) Review Data Quality:
    - Use the zoom button (3x) or arrow keys (⬅️ or ➡️) to visually inspect the data quality of the merged file.
- Export to RJPG format: File > Export frames
  - Set the "Target format" to RJPG.
  - Uncheck "Create meta data" as metadata was already generated in step 3.
  - Choose the desired output directory for the exported files.
  - Enter a prefix for the exported file names (e.g., F42_20230731).
  - Click "Export files" to begin the conversion process. ThermoViewer will convert your merged TMC data to individual RJPG images.
- This dataset comprises L0J data products.
Create Orthomosaic Image with Pix4D (🪟Windows Only):
- Launch Pix4D
- Create a New Project
- Import Images:
  - Add all the JPG images created in step 4 (converted thermal data).
    
    Note: You may encounter an error message after loading location information. This can be safely ignored.
- Set Coordinate System:
  - Define the coordinate system as "WGS 84 / UTM zone 16N" since you're working with data from Indiana (assuming this information is accurate).
- Geolocation and Orientation:
  - Under "Geolocation and Orientation", click "From File..." and load the metadata CSV file generated in step 3.
  - Verify that the data in columns X, Y, and Z is complete and represents the camera locations accurately.
- Confirm Coordinate System:
  - After verifying the metadata, proceed with the selected coordinate system (WGS 84 / UTM zone 16N).
- Select Processing Template:
  - Choose the "Standard - 3D Maps" processing template for optimal results.
- Start the Pix4D processing to generate the reflectance orthomosaic.
- The data under Pix4D project directory comprises L1P data products.
- The orthomosaic image under product directory under the Pix4D project directory comprises L2T data product.
Calibrate Non-Calibrated Thermal Images Using Ground-Truth Measurements
- Open field record (e.g., 2024_5_10_Field58.txt under /depot/smarterag/fy2024_drone_data/yang1527/Field 58/2024_5_10_Field58/) and check heat pad and cooling mat temperature.
- Manually identify temperature extremes in L2T Image.
  - Open the L2T image with QGIS.
  - Find the heat pad and cooling mat near the truck or pilot from the image.
  - Record the local maximum and minimum temperature values within the temperature reference.
- Find a conversion equation using Line through two points calculator.
  - input first point: (cooling mat temp in L2T, cooling mat temp in field record)
  - input second point: (heat pad temp in L2T, heat pad temp in field record)
- Use QGIS Raster Calculator to convert L2T image to temperature.
- The resultant image comprises L3T data products.
QGIS Georeferencing (Optional)
- ❕Importance of Georeferencing:
  
  While not strictly necessary, georeferencing your thermal image to the RGB orthomosaic is highly recommended. This process can improve the alignment between the two images, potentially reducing misalignments by several meters. This ensures your thermal data overlays accurately on the other orthomosaic images for further analysis.
- Workflow:
  - If you choose to georeference, follow these steps using QGIS:
    - Open QGIS software.
    - Load L3T image: Load both the thermal orthomosaic and the RGB orthomosaic into your QGIS project.
    - Georeferencing Tool: Navigate to Raster > Georeferencer to access the georeferencing functionality.
    - Detailed Instructions: Please refer to the official documentation for detailed instructions on using the QGIS georeferencer.
- The resultant image comprises L4T data products.
- Apply colormap (Inferno recommended) to L4T image to generate L4TH.

Pipeline-UAV-Thermal/README.md

UAV Thermal Data Processing Pipeline (GRYFN+TeAx)