Permalink
Cannot retrieve contributors at this time
Name already in use
A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Pipeline-UAV-Thermal/README.md
Go to fileThis commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
174 lines (115 sloc)
10.9 KB
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # UAV Thermal Data Processing Pipeline (GRYFN+TeAx) | |
|  | |
|  | |
|  | |
|  | |
| This user guide details the processing procedures for thermal imagery acquired by the [TeAx](https://thermalcapture.com/thermalcapture-2-0/) sensor onboard the [GRYFN](https://www.gryfn.io/) platform. ❗ Process all sensor data except thermal using the most recent version of the [GRYFN Processing Tool](https://www.gryfn.io/software) before proceeding. | |
| - Manual by Sungchan Oh (Sun😎) | |
| - First Created: August 31, 2023 | |
| - Last Updated: May 21, 2024 | |
| - Contact: oh231@purdue.edu | |
| Here's a step-by-step walkthrough of the process. | |
| --- | |
| 1. ❗ __Process all non-thermal sensor data acquired by the GRYFN (+TeAx) platform using GRYFN Processing Tool.__ | |
| 2. __Perform quality checks on the GRYFN Export Trajectory file (export*.txt).__ | |
| - 📁 Locate the GRYFN Export Trajectory file using the provided shell script. | |
| ```bash | |
| cd [your_flight_directory] | |
| # Example: /depot/smarterag/fy2024_drone_data/yang1527/Field 58/2024_4_25_Field58 | |
| find ~+ -type f -name "export*.txt" | |
| ``` | |
| - For data consistency, verify the presence of specific columns in the GRYFN Export Trajectory file. | |
| - 🗒️ Open the GRYFN Export Trajectory file and navigate to line 27. | |
| - 🗒️ Look for the following column headers: EASTING(m), NORTHING(m), ELLIPSOID_HEIGTH(m). | |
| > If any of these are present, you can assume all the listed headers <br> | |
| > (EASTING(m), NORTHING(m), ELLIPSOID_HEIGTH(m), <br> | |
| > EASTING_SD(m), NORTHING_SD(m), HEIGHT_SD(m), <br> | |
| > ROLL_SD(deg), PITCH_SD(deg), HEADING_SD(deg)) are included. | |
| - 📁 Locate the directory containing raw thermal images (*.TMC). | |
| 3. __Extract XYZ coordinates corresponding to each acquired thermal image.__ | |
| - 🗒️ Open the Python script [exportThermalXyzFromGryfnEvents.py](exportThermalXyzFromGryfnEvents.py) located in your current directory. | |
| - 🗒️ Based on the information from step 2, configure the following file paths: | |
| - GRYFN Export Trajectory file path (path_apx) | |
| - Directory containing raw TMC files (dir_tmc) | |
| > Mac OSX or Linux: | |
| > Use a forward slash (/) to separate directories, like this: | |
| > path_apx = "/path/to/your/data/folder/file.txt" | |
| > Replace /path/to/your/data/folder with the actual location of your file or directory. | |
| > Windows: | |
| > Use a backslash (\\) to separate directories, like this: | |
| > path_apx = "C:\\Users\\yourname\\data\\folder\\file.txt" | |
| > Replace C:\\Users\\yourname\\data\\folder with the actual location of your file or directory. | |
| - (Optional) Visit the following [link](http://en.racelogic.support/VBOX_Automotive/Knowledge_Base/What_are_GPS_Leap_Seconds%3F) to check for the current leap second information. If the provided value is different from 18 seconds, update the leap_sec parameter in the script to match the current leap second value. | |
| - Run [exportThermalXyzFromGryfnEvents.py](exportThermalXyzFromGryfnEvents.py). | |
| > ❕The exportThermalXyzFromGryfnEvents.py script ensures proper time synchronization between thermal images and their corresponding GPS data. Images with a time difference exceeding a specified threshold (max_timediff parameter) from the closest APX event (GPS signal reception) are considered unreliable and excluded from further processing. This filtering step helps maintain data integrity and avoid potential errors in subsequent processes. | |
| - This dataset (*.TMC) comprises [L0R](https://github.itap.purdue.edu/PlantScience/DataGuide-UAV-Thermal#data-product-level-definitions) data products. | |
| > [!IMPORTANT] | |
| > The exportThermalXyzFromGryfnEvents.py script generates a list of thermal images excluded from further processing due to exceeding the time synchronization threshold. These files are moved to nouse directory under dir_tmc. | |
| 4. __Convert raw thermal data (TMC) to radiometrically corrected JPEG ([RJPG](https://av3aerovisual.com/en/differences-between-images-in-tiff-jpg-and-rjpeg-formats/)) images (🪟Windows Only)__ | |
| > An RJPG file, also referred to as a radiometrically corrected JPG file, is essentially a JPEG image that has undergone additional processing to improve its accuracy in representing real-world temperatures. | |
| - Launch [ThermoViewer](https://thermalcapture.com/thermoviewer/): | |
| - Ensure "Parse NMEA Data" is selected under Settings in the menu bar to process embedded GPS information within the thermal images. | |
| - Configure Conversion Parameters: | |
| - Go to the menu bar and select "Display RAW Conversion Parameters dialog". | |
| - Under the "Color Adjustment > Color" menu, choose "Grayscale". | |
| - Under the "Temperatures" menu, select "TLinear high gain °C" next to the TLinear model. | |
| - Uncheck the "Show legend" option. | |
| - Merge TMC Files: | |
| - From the menu bar, navigate to Edit > Merge Files. | |
| - Select TMC Files: | |
| - Click "Add File(s)" to include thermal data files. | |
| - Browse to the directory containing your thermal data (TMC files). | |
| - Select all TMC files. | |
| - Click "Merge" to combine the selected TMC files. | |
| - Save the merged file with a descriptive name (e.g., F42_20230731_Mohave.TMC). | |
| - Open Merged File: | |
| - In the ThermoViewer left pane, navigate to the directory where you saved the merged file and open it. | |
| - (Optional) Review Data Quality: | |
| - Use the zoom button (3x) or arrow keys (⬅️ or ➡️) to visually inspect the data quality of the merged file. | |
| - Export to RJPG format: File > Export frames | |
| - Set the "Target format" to RJPG. | |
| - Uncheck "Create meta data" as metadata was already generated in step 3. | |
| - Choose the desired output directory for the exported files. | |
| - Enter a prefix for the exported file names (e.g., F42_20230731). | |
| - Click "Export files" to begin the conversion process. ThermoViewer will convert your merged TMC data to individual RJPG images. | |
| - This dataset comprises [L0J](https://github.itap.purdue.edu/PlantScience/DataGuide-UAV-Thermal#data-product-level-definitions) data products. | |
| 5. __Create Orthomosaic Image with Pix4D (🪟Windows Only):__ | |
| - Launch [Pix4D](https://www.pix4d.com/) | |
| - Create a New Project | |
| - Import Images: | |
| - Add all the JPG images created in step 4 (converted thermal data). | |
| > Note: You may encounter an error message after loading location information. This can be safely ignored. | |
| - Set Coordinate System: | |
| - Define the coordinate system as "WGS 84 / UTM zone 16N" since you're working with data from Indiana (assuming this information is accurate). | |
| - Geolocation and Orientation: | |
| - Under "Geolocation and Orientation", click "From File..." and load the metadata CSV file generated in step 3. | |
| - Verify that the data in columns X, Y, and Z is complete and represents the camera locations accurately. | |
| - Confirm Coordinate System: | |
| - After verifying the metadata, proceed with the selected coordinate system (WGS 84 / UTM zone 16N). | |
| - Select Processing Template: | |
| - Choose the "Standard - 3D Maps" processing template for optimal results. | |
| - Start the Pix4D processing to generate the reflectance orthomosaic. | |
| - The data under Pix4D project directory comprises [L1P](https://github.itap.purdue.edu/PlantScience/DataGuide-UAV-Thermal#data-product-level-definitions) data products. | |
| - The orthomosaic image under product directory under the Pix4D project directory comprises [L2T](https://github.itap.purdue.edu/PlantScience/DataGuide-UAV-Thermal#data-product-level-definitions) data product. | |
| 6. __Calibrate Non-Calibrated Thermal Images Using Ground-Truth Measurements__ | |
| - Open field record (e.g., 2024_5_10_Field58.txt under /depot/smarterag/fy2024_drone_data/yang1527/Field 58/2024_5_10_Field58/) and check heat pad and cooling mat temperature. | |
| - Manually identify temperature extremes in L2T Image. | |
| - Open the L2T image with [QGIS](https://www.qgis.org/en/site/). | |
| - Find the heat pad and cooling mat near the truck or pilot from the image. | |
| - Record the local maximum and minimum temperature values within the temperature reference. | |
| - Find a conversion equation using [Line through two points calculator](https://www.mathportal.org/calculators/analytic-geometry/two-point-form-calculator.php). | |
| - input first point: __(cooling mat temp in L2T, cooling mat temp in field record)__ | |
| - input second point: __(heat pad temp in L2T, heat pad temp in field record)__ | |
| - Use QGIS [Raster Calculator](https://docs.qgis.org/2.18/en/docs/user_manual/working_with_raster/raster_calculator.html) to convert L2T image to temperature. | |
| - The resultant image comprises [L3T](https://github.itap.purdue.edu/PlantScience/DataGuide-UAV-Thermal#data-product-level-definitions) data products. | |
| 7. __[QGIS](https://www.qgis.org/en/site/) Georeferencing (Optional)__ | |
| - ❕Importance of [Georeferencing](https://www.usgs.gov/faqs/what-does-georeferenced-mean): | |
| > While not strictly necessary, georeferencing your thermal image to the RGB orthomosaic is highly recommended. This process can improve the alignment between the two images, potentially reducing misalignments by several meters. This ensures your thermal data overlays accurately on the other orthomosaic images for further analysis. | |
| - Workflow: | |
| - If you choose to georeference, follow these steps using QGIS: | |
| - Open QGIS software. | |
| - Load [L3T](https://github.itap.purdue.edu/PlantScience/DataGuide-UAV-Thermal#data-product-level-definitions) image: Load both the thermal orthomosaic and the RGB orthomosaic into your QGIS project. | |
| - Georeferencing Tool: Navigate to Raster > Georeferencer to access the georeferencing functionality. | |
| - Detailed Instructions: Please refer to the [official documentation](https://docs.qgis.org/latest/en/docs/user_manual/working_with_raster/georeferencer.html) for detailed instructions on using the QGIS georeferencer. | |
| - The resultant image comprises [L4T](https://github.itap.purdue.edu/PlantScience/DataGuide-UAV-Thermal#data-product-level-definitions) data products. | |
| - Apply colormap (Inferno recommended) to [L4T](https://github.itap.purdue.edu/PlantScience/DataGuide-UAV-Thermal#data-product-level-definitions) image to generate [L4TH](https://github.itap.purdue.edu/PlantScience/DataGuide-UAV-Thermal#data-product-level-definitions). | |