added globus

README.md

@@ -19,26 +19,27 @@ The image below is a link to a YouTube video showing some of its design and feat
 
 ## Table of Contents
 
-- [Hardware](#hardware)
-- [Examples](#example-images)
-- [3D printer models](#3d-printer-models)
-- [Automated data analysis](#automated-data-analysis)
-- [Installation](#installation)
-  - [Enabling the Wi-Fi hotspot](#enabling-the-wi-fi-hotspot)
-- [Usage](#usage)
-  - [Working with SPIRO](#working-with-spiro)
-  - [Connecting to the web interface](#connecting-to-the-web-interface)
-  - [Setting up imaging](#setting-up-imaging)
-  - [Starting an experiment](#starting-an-experiment)
-  - [Downloading images](#downloading-images)
-- [Maintaining the system](#maintaining-the-system)
-  - [Restarting the software](#restarting-the-software)
-  - [Shutting down the system](#shutting-down-the-system)
-  - [Keeping software up to date](#keeping-software-up-to-date)
-- [Troubleshooting](#troubleshooting)
-  - [Viewing the software log](#viewing-the-software-log)
-  - [Testing the LED and motor](#testing-the-led-and-motor)
-- [Licensing](#licensing)
+* [Hardware](#hardware)
+* [Examples](#example-images)
+* [3D printer models](#3d-printer-models)
+* [Automated data analysis](#automated-data-analysis)
+* [Installation](#installation)
+  * [Enabling the Wi-Fi hotspot](#enabling-the-wi-fi-hotspot)
+* [Usage](#usage)
+  * [Working with SPIRO](#working-with-spiro)
+  * [Connecting to the web interface](#connecting-to-the-web-interface)
+  * [Setting up imaging](#setting-up-imaging)
+  * [Starting an experiment](#starting-an-experiment)
+  * [Globus transfer during experiments](#globus-transfer-during-experiments)
+  * [Downloading images](#downloading-images)
+* [Maintaining the system](#maintaining-the-system)
+  * [Restarting the software](#restarting-the-software)
+  * [Shutting down the system](#shutting-down-the-system)
+  * [Keeping software up to date](#keeping-software-up-to-date)
+* [Troubleshooting](#troubleshooting)
+  * [Viewing the software log](#viewing-the-software-log)
+  * [Testing the LED and motor](#testing-the-led-and-motor)
+* [Licensing](#licensing)
 
 ## Hardware
 
@@ -117,7 +118,6 @@ sudo raspi-config
 ```
 
 In the raspi-config interface, make the following changes:
-<<<<<<< Updated upstream
 * **Change the password**. The system will allow network access, and a weak password **will** compromise your network security **and** your experimental data.
 * After changing the password, connect the network cable (if you are using wired networking).
 * Under *Interfacing*, enable *I2C* and *SSH*. The camera is used through Bookworm's libcamera stack, so the old Legacy camera stack is not required.
@@ -126,17 +126,6 @@ In the raspi-config interface, make the following changes:
 * If needed, configure *Network* and *Localization* options here as well. Set a *Hostname* under Network if you plan on running several SPIROs.
 * Finally, select *Finish*, and choose to reboot the system when asked. 
 * After reboot, the system shows a message on the screen showing its IP address ("My IP address is: *a.b.c.d*"). Make a note of this address as you will need it to access the system over the network. Make sure that your network allows access to ports 8080 on this IP address. (Alternatively, see [Enabling the Wi-Fi hotspot](#enabling-the-wifi-hotspot))
-=======
-
-- **Change the password**. The system will allow network access, and a weak password **will** compromise your network security **and** your experimental data.
-- After changing the password, connect the network cable (if you are using wired networking).
-- Under _Interfacing_, enable _Camera_, _I2C_, and _SSH_.
-- In _Performance Options_, set _GPU Memory_ to 256.
-- Under _Localisation Options_, make sure to set the _Timezone_. Please note that a working network connection is required to maintain the correct date.
-- If needed, configure _Network_ and _Localization_ options here as well. Set a _Hostname_ under Network if you plan on running several SPIROs.
-- Finally, select _Finish_, and choose to reboot the system when asked.
-- After reboot, the system shows a message on the screen showing its IP address ("My IP address is: _a.b.c.d_"). Make a note of this address as you will need it to access the system over the network. Make sure that your network allows access to ports 8080 on this IP address. (Alternatively, see [Enabling the Wi-Fi hotspot](#enabling-the-wifi-hotspot))
->>>>>>> Stashed changes
 
 Next, ensure the system is connected to the internet. Update the operating system and install the required tools:
 
@@ -300,18 +289,44 @@ After logging in to the system, you are presented with the _Live view_. Here, yo
 
 Under _Day image settings_ and _Night image settings_, you can adjust the exposure time for day and night images in real time. For night images, make sure that representative conditions are used (i.e., turn off the lights in the growth chamber). When the image is captured according to your liking, choose _Update and save_.
 
-<<<<<<< Updated upstream
 For locating the imaging positions, SPIRO expects **two sensor events per position**. The first trigger is treated as the early-warning stop, telling the software that the final stop is approaching. The system then slows down, finds the second stop, and only after that applies the configured final offset (*calibration value*). That same final offset is used for all positions, not only the start position.
 
 Use the *Calibrate motor* view to tune the primary step delay, secondary step delay, secondary stop gap, stable sensor reads, and final offset. The page now also includes manual jog controls, live step counters, and repeatability tests so you can dial in settings without saving every trial.
-=======
-For locating the initial imaging position, the system turns the cube until a positional switch is activated. It then turns the cube a predefined amount of steps (_calibration value_). The check that the calibration value is correct, go to the _Start position calibration_ view. Here, you may try out the current value, as well as change it. Make sure to click _Save value_ if you change it.
->>>>>>> Stashed changes
 
 ### Starting an experiment
 
 When imaging parameters are set up to your liking, you are ready to start your experiments. In the _Experiment control_ view, choose a name for your experiment, as well as the duration and imaging frequency. After you choose _Start experiment_, the system will disable most of the functionality of the web interface, displaying a simple status window containing experiment parameters, as well as the last image captured.
 
+### Globus transfer during experiments
+
+SPIRO can transfer newly captured files to Globus while an experiment runs.
+
+On a Raspberry Pi running Raspberry Pi OS, install the local source endpoint software first:
+
+```bash
+sudo apt update
+sudo apt install -y wget python3-pip tcl tk
+python3 -m pip install --user globus-cli
+wget https://downloads.globus.org/globus-connect-personal/linux/stable/globusconnectpersonal-latest.tgz
+tar xzf globusconnectpersonal-latest.tgz
+mv globusconnectpersonal-* ~/.globusconnectpersonal
+```
+
+After installation:
+
+1. Open *System settings* and set your *Globus Client ID*.
+2. In *Local source endpoint*, use *Launch setup/login* to start Globus Connect Personal setup on the SPIRO device.
+3. If the Pi is headless, complete the first-time setup from a local shell by running `~/.globusconnectpersonal/globusconnectpersonal` or `~/.globusconnectpersonal/globusconnectpersonal -setup`.
+4. Use *Start local client* once Globus Connect Personal is configured.
+5. Use *Refresh local endpoint ID* to detect the SPIRO device's source endpoint via `globus endpoint local-id`.
+6. Click *Connect Globus* to authorize the SPIRO web app to submit transfer tasks.
+7. Copy the authorization code from Globus and paste it into *Complete Globus Login*.
+8. In *Experiment control*, enable transfer and provide only the destination endpoint ID and destination base path for this run.
+
+SPIRO now uses the local Globus Connect Personal endpoint on the device as the source endpoint automatically. The source path is set to the experiment directory for the active run.
+
+Transfers are queued once per imaging cycle and submitted in the background. If a transfer fails, SPIRO keeps imaging and retries automatically.
+
 ### Downloading images
 
 Images can be downloaded from the web interface under _File manager_. The File manager also allows deleting files to free up space on the SD card.
@@ -439,20 +454,12 @@ PY
 
 If the motor does not move correctly:
 
-<<<<<<< Updated upstream
 * Confirm the motor is connected to **M2** on the HAT.
 * Confirm the HAT's external motor power supply is connected and switched on.
 * Confirm the M2 DIP switches `D3-D5` are still at the factory default unless you intentionally recalibrated for another microstepping mode.
 * Confirm the motor wiring matches the HAT's `A3/A4/B3/B4` outputs.
 * Confirm the DRV8825 current limit is set appropriately for the motor.
 * If the stage moves in the wrong direction, set `motor_direction_active_high` in the SPIRO config instead of rewiring the whole system.
-=======
-If it doesn't respond to this command, this may indicate either miswiring, or that either the LED strip or the MOSFET is non-functional.
-
-Similarly, you can turn on and off the motor, by substituting the value _23_ for 17 in the above examples. When GPIO pin 23 is toggled on, the cube should be locked in position. If it is not, check that your wiring looks good, that the power supply is connected, and that the shaft coupler is firmly attached to both the cube and the motor.
-
-If the motor is moving jerkily during normal operation, there is likely a problem with the wiring of the coil pins (Ain1&2 and Bin1&2).
->>>>>>> Stashed changes
 
 ## Licensing
 

setup.py

@@ -32,7 +32,7 @@ def get_version_and_cmdclass(package_path):
       cmdclass = cmdclass,
       packages = find_packages(),
       scripts = ['bin/spiro'],
-    install_requires = ['picamera2==0.3.31', 'gpiozero>=2.0.1,<3', 'Flask==2.2.5', 'waitress==2.1.2', 'numpy==1.24.2', 'Werkzeug==2.2.3'],
+      install_requires = ['picamera2==0.3.31', 'RPi.GPIO==0.7.1', 'Flask==2.2.5', 'waitress==2.1.2', 'numpy==1.24.2', 'Werkzeug==2.2.3', 'globus-sdk>=3.39.0', 'cryptography>=42.0.0'],
       author = 'Kirby Kalbaugh',
       author_email = 'kkalbaug@purdue.edu',
       description = 'Control software for the SPIRO biological imaging system',

spiro/config.py

@@ -7,6 +7,7 @@
 import json
 import os
 import sys
+from cryptography.fernet import Fernet
 from importlib import metadata
 from ._version import __version__
 
@@ -89,7 +90,13 @@ class Config(object):
         'name': 'spiro',        # the name of this spiro instance
         'timezone': 'America/New_York',  # user display/save timezone; device clock remains UTC
         'debug': True,         # debug logging
-        'rotated_camera': True  # rotated camera house
+        'rotated_camera': True,  # rotated camera house
+        'globus_client_id': '',
+        'globus_token_encrypted': '',
+        'globus_gcp_install_dir': '~/.globusconnectpersonal',
+        'globus_local_endpoint_id': '',
+        'globus_source_endpoint': '',
+        'globus_source_base_path': '/'
     }
 
     config = {}
@@ -98,6 +105,7 @@ def __init__(self):
         """Initialize file locations, runtime version, and cached config state."""
         self.cfgdir = os.path.expanduser("~/.config/spiro")
         self.cfgfile = os.path.join(self.cfgdir, "spiro.conf")
+        self.keyfile = os.path.join(self.cfgdir, "token.key")
         self.version = _resolve_version()
         self.read()
         if os.path.exists(self.cfgfile):
@@ -124,9 +132,44 @@ def write(self):
             with open(self.cfgfile + ".tmp", 'w') as f:
                 json.dump(self.config, f, indent=4)
             os.replace(self.cfgfile + ".tmp", self.cfgfile)
+            os.chmod(self.cfgfile, 0o600)
         except OSError as e:
             log("Failed to write config file: " + e.strerror)
 
+
+    def _get_or_create_secret_key(self):
+        os.makedirs(self.cfgdir, exist_ok=True)
+        if os.path.exists(self.keyfile):
+            with open(self.keyfile, 'rb') as f:
+                key = f.read().strip()
+            if key:
+                return key
+
+        key = Fernet.generate_key()
+        with open(self.keyfile + '.tmp', 'wb') as f:
+            f.write(key)
+        os.replace(self.keyfile + '.tmp', self.keyfile)
+        os.chmod(self.keyfile, 0o600)
+        return key
+
+
+    def encrypt_secret(self, value):
+        if value in [None, '']:
+            return ''
+        cipher = Fernet(self._get_or_create_secret_key())
+        return cipher.encrypt(value.encode('utf-8')).decode('utf-8')
+
+
+    def decrypt_secret(self, value):
+        if value in [None, '']:
+            return ''
+        try:
+            cipher = Fernet(self._get_or_create_secret_key())
+            return cipher.decrypt(value.encode('utf-8')).decode('utf-8')
+        except Exception as e:
+            log('Failed to decrypt secret: ' + str(e))
+            return ''
+
     def get(self, key):
         """Return a config value, reloading from disk if another process updated it."""
         if os.path.exists(self.cfgfile):

spiro/experimenter.py

@@ -42,6 +42,7 @@ def __init__(self, hw=None, cam=None):
         self.preview_lock = threading.Lock()
         self.nshots = 0
         self.idlepos = 0
+        self.transfer_cycle_handler = None
         threading.Thread.__init__(self)
 
 
@@ -357,6 +358,7 @@ def runExperiment(self):
                 nextloop = time.time() + 60 * self.delay
                 if nextloop > self.endtime:
                     nextloop = self.endtime
+                cycle_files = []
 
                 self.hw.motorOn(True)
                 for i in range(self.position_count):
@@ -383,11 +385,18 @@ def runExperiment(self):
                     now = filename_timestamp(self.cfg.get('timezone'))
                     name = os.path.join("plate" + str(i + 1), "plate" + str(i + 1) + "-" + now)
                     self.takePicture(name, i)
+                    cycle_files.append(self.last_captured[i])
 
                 if self.stop_experiment:
                     self.hw.motorOn(False)
                     break
 
+                if self.transfer_cycle_handler and cycle_files:
+                    try:
+                        self.transfer_cycle_handler(cycle_files)
+                    except Exception as e:
+                        debug('Failed to queue transfer cycle files: ' + str(e))
+
                 self.nshots -= 1
                 self.hw.motorOn(False)
                 if self.status != "Stopping": self.status = "Waiting"