Integrate the RCAC computational pilot and separate physical from computational

.dockerignore

@@ -26,5 +26,10 @@ nanohub/config
 nanohub/data
 uploader/staging
 
+# Generated uploads and local mock databases are runtime data, not image assets.
+# Keep developer/test data out of API and worker builds.
+api/data
+api/local_data
+
 .DS_Store
 *.log

.gitignore

@@ -60,9 +60,16 @@ debug_*.py
 !api/tests/test_e2e_platform.py
 !api/tests/test_glance_db3_trace_import.py
 !api/tests/test_retrain.py
+!api/tests/test_monolayer.py
+!api/tests/test_rcac_hardening.py
 !api/tests/test_production_glance_ingestion.py
 !api/tests/test_glance_closed_loop.py
 !api/tests/test_shadow_evaluation.py
+!api/tests/test_experiment_run_endpoint.py
+!api/tests/test_proposal_objectives.py
+!api/tests/test_computation_cutoff_fidelity.py
+!api/tests/test_experiment_values_postgres_e2e.py
+!api/tests/test_pilot_review_permissions.py
 
 # Development files
 temp.py

.serena/.gitignore

@@ -0,0 +1,2 @@
+/cache
+/project.local.yml

.serena/project.yml

@@ -0,0 +1,169 @@
+# the name by which the project can be referenced within Serena/when chatting with the LLM.
+project_name: "dt-computational-integration"
+
+# list of language servers to start when using the LSP backend; choose from:
+#   ada                    al                     angular                ansible                bash
+#   bsl                    clojure                cpp                    cpp_ccls               crystal
+#   csharp                 csharp_omnisharp       cue                    dart                   deno
+#   elixir                 elm                    erlang                 fortran                fsharp
+#   gdscript               gleam                  go                     groovy                 haskell
+#   haxe                   hlsl                   html                   java                   json
+#   julia                  kotlin                 latex                  lean4                  lua
+#   luau                   markdown               matlab                 msl                    nextflow
+#   nix                    ocaml                  pascal                 perl                   php
+#   php_phpactor           php_phpantom           powershell             python                 python_basedpyright
+#   python_jedi            python_pyrefly         python_ty              qml                    r
+#   rego                   ruby                   ruby_solargraph        rust                   scala
+#   scss                   solidity               svelte                 swift                  systemverilog
+#   terraform              toml                   typescript             typescript_vts         vue
+#   wolfram                yaml                   zig
+#   (This list may be outdated; generated with scripts/print_language_list.py;
+#   For the current list, see values of the LanguageServerId enum here:
+#   https://github.com/oraios/serena/blob/main/src/solidlsp/ls_config.py)
+# For some languages, there are several alternative language servers, e.g. csharp_omnisharp, ruby_solargraph.)
+# Note:
+#   - For C, use cpp
+#   - For JavaScript, use typescript
+#   - For Angular projects, use angular (subsumes typescript+html; requires `npm install` in the project root)
+#   - For Svelte projects, use svelte (subsumes typescript/javascript for .svelte projects; requires npm)
+#   - For Deno projects, use deno (serves the same .ts/.js files as typescript; requires the deno CLI on PATH)
+#   - For SCSS / Sass / plain CSS, use scss (some-sass-language-server handles all three)
+#   - For Free Pascal/Lazarus, use pascal
+# Special requirements:
+#   Some language servers require additional setup/installations.
+#   See here for details: https://oraios.github.io/serena/01-about/020_programming-languages.html#language-servers
+# When using multiple language servers, the first language server that supports a given file will be used for that file.
+# The first language server is the default language and the respective language server will be used as a fallback.
+# Note that when using the JetBrains backend, language servers are not used and this list is correspondingly ignored.
+language_servers:
+- python
+
+# the encoding used by text files in the project
+# For a list of possible encodings, see https://docs.python.org/3.11/library/codecs.html#standard-encodings
+encoding: "utf-8"
+
+# optional shell command to run before the language backend (LSP or JetBrains) is initialised.
+# the command runs in the project root directory and is only executed if the project is trusted
+# (see trusted_project_path_patterns in the global configuration).
+# serena waits for the command to exit: a non-zero exit code is logged as an error but does not
+# abort activation. a per-project timeout (activation_command_timeout, default 180s) is the safety
+# backstop for non-terminating commands; on expiry the process is killed and activation continues.
+# example: activation_command: "npx nx run-many -t build"
+activation_command:
+
+# maximum time in seconds to wait for activation_command to complete before killing it (default 180s).
+# must be a positive number.
+activation_command_timeout: 180.0
+
+# line ending convention to use when writing source files.
+# Possible values: unset (use global setting), "lf", "crlf", or "native" (platform default)
+# This does not affect Serena's own files (e.g. memories and configuration files), which always use native line endings.
+line_ending:
+
+# The language backend to use for this project.
+# If not set, the global setting from serena_config.yml is used.
+# Valid values: LSP, JetBrains
+# Note: the backend is fixed at startup. If a project with a different backend
+# is activated post-init, an error will be returned.
+language_backend:
+
+# whether to use project's .gitignore files to ignore files
+ignore_all_files_in_gitignore: true
+
+# advanced configuration option allowing to configure language server-specific options.
+# Maps the language key to the options.
+# The settings are considered only if the project is trusted (see global configuration to define trusted projects).
+# See https://oraios.github.io/serena/02-usage/050_configuration.html#language-server-specific-settings
+ls_specific_settings: {}
+
+# list of workspace folder paths (LSP backend only).
+# These folders will be used to build up Serena's symbol index.
+# Paths must be within the project root and should thus be relative to the project root.
+# Furthermore, the paths should not be filtered by ignore settings.
+# Default setting: The entire project root folder (".") is considered.
+# In (large) monorepos, this can be used to index only subfolders of the project root, e.g.
+#   ls_workspace_folders:
+#     - "./subproject1"
+#     - "./subproject2"
+ls_workspace_folders:
+- "."
+
+# list of additional workspace folder paths for cross-package reference support.
+# Paths can be absolute or relative to the project root.
+# Each folder is registered as an LSP workspace folder, enabling language servers to discover
+# symbols and references across package boundaries, but these folders are not indexed by Serena,
+# i.e. the respective symbols will not be found using Serena's symbol search tools.
+# Example:
+#   additional_workspace_folders:
+#     - ../sibling-package
+#     - ../shared-lib
+ls_additional_workspace_folders: []
+
+# list of additional paths to ignore in this project.
+# Same syntax as gitignore, so you can use * and **.
+# Important: quote patterns that start with `*`, otherwise YAML treats them as aliases.
+# Example:
+#   ignored_paths:
+#     - "examples/**"
+#     - ".worktrees/**"
+#     - "**/bin/**"
+#     - "**/obj/**"
+# Note: global ignored_paths from serena_config.yml are also applied additively.
+ignored_paths: []
+
+# whether the project is in read-only mode
+# If set to true, all editing tools will be disabled and attempts to use them will result in an error
+# Added on 2025-04-18
+read_only: false
+
+# list of tool names to exclude.
+# This extends the existing exclusions (e.g. from the global configuration)
+# Find the list of tools here: https://oraios.github.io/serena/01-about/035_tools.html
+excluded_tools: []
+
+# list of tools to include that would otherwise be disabled (particularly optional tools that are disabled by default).
+# This extends the existing inclusions (e.g. from the global configuration).
+# Find the list of tools here: https://oraios.github.io/serena/01-about/035_tools.html
+included_optional_tools: []
+
+# fixed set of tools to use as the base tool set (if non-empty), replacing Serena's default set of tools.
+# This cannot be combined with non-empty excluded_tools or included_optional_tools.
+# Find the list of tools here: https://oraios.github.io/serena/01-about/035_tools.html
+fixed_tools: []
+
+# list of mode names that are to be activated by default, overriding the setting in the global configuration.
+# The full set of modes to be activated is base_modes (from global config) + default_modes + added_modes.
+# If the setting is undefined/empty, the default_modes from the global configuration (serena_config.yml) apply.
+# Otherwise, this overrides the setting from the global configuration (serena_config.yml).
+# Therefore, you can set this to [] if you do not want the default modes defined in the global config to apply
+# for this project.
+# This setting can, in turn, be overridden by CLI parameters (--mode).
+# See https://oraios.github.io/serena/02-usage/050_configuration.html#modes
+default_modes:
+
+# list of mode names to be activated additionally for this project, e.g. ["query-projects"]
+# The full set of modes to be activated is base_modes (from global config) + default_modes + added_modes.
+# See https://oraios.github.io/serena/02-usage/050_configuration.html#modes
+added_modes:
+
+# initial prompt for the project. It will always be given to the LLM upon activating the project
+# (contrary to the memories, which are loaded on demand).
+initial_prompt: ""
+
+# time budget (seconds) per tool call for the retrieval of additional symbol information
+# such as docstrings or parameter information.
+# This overrides the corresponding setting in the global configuration; see the documentation there.
+# If null or missing, use the setting from the global configuration.
+symbol_info_budget:
+
+# list of regex patterns which, when matched, mark a memory entry as read‑only.
+# Extends the list from the global configuration, merging the two lists.
+read_only_memory_patterns: []
+
+# list of regex patterns for memories to completely ignore.
+# Matching memories will not appear in list_memories or activate_project output
+# and cannot be accessed via read_memory or write_memory.
+# To access ignored memory files, use the read_file tool on the raw file path.
+# Extends the list from the global configuration, merging the two lists.
+# Example: ["_archive/.*", "_episodes/.*"]
+ignored_memory_patterns: []

COMPUTATIONAL_PILOT_GUIDE.md

@@ -0,0 +1,119 @@
+# Computational experiments: student pilot
+
+Implementation branch: `feature/sim2l-rcac-integration`.
+The existing production DT platform is unchanged. The instructions below apply
+to an explicitly enabled pilot environment; the public feature is not deployed.
+Start with the [Rushik/Justin handoff](COMPUTATIONAL_PILOT_HANDOFF.md),
+[September 28 deployment validation](geddes/k8s/pilots/computations/PILOT_VALIDATION_2026-09-28.md),
+and the [scientific review checklist](COMPUTATIONAL_SCIENTIFIC_REVIEW.md).
+The isolated API, web, database, and worker are deployed with real nanoHUB SAML.
+The validation report is the authority for current student access and readiness;
+September 23 notes describe an earlier preparation stage.
+See [live validation evidence](RCAC_E2E_VALIDATION_2026-09-22.md) and the
+[administrator provisioning guide](RCAC_SIM2L_INTEGRATION.md).
+
+## What is available
+
+| Registered experiment | Scientific task | Supported input |
+| --- | --- | --- |
+| CellRelaxDFT, `r99-dt2` | Three-dimensional cell relaxation | Materials Project structure and DFT parameters |
+| CellRelaxDFT 2D, `justin-dt1` | Extract and relax a monolayer with fixed vacuum | Justin's graphene example, `mp-48`; compatible layered structures subject to domain review |
+| ElasticDFT, `r27-dt2` | Three-dimensional, second-order, stress/strain elastic tensor | A converged **3D** CellRelaxDFT run in the same project |
+
+Both Negishi (QE 7.1) and Gautschi (QE 7.3.1) are supported. A versioned
+installation is registered as digital equipment; a scientific workflow is a unit
+experiment linked to compatible installations. Each run records which actual
+installation produced it. Existing physical-equipment routes are unchanged.
+
+This is not arbitrary Python/notebook/shell upload and execution. Only registered,
+reviewed workflows are executable. There is no Sim2L-library import or required
+Sim2L-repository merge. DT owns these adapters, scientific contracts, scheduling,
+and persistence. Original notebooks remain reference implementations.
+
+## Student workflow
+
+1. Sign in normally and open a project where you have write access.
+2. Find **Registered computational experiments**. Select a tool and cluster.
+3. For graphene, select **CellRelaxDFT 2D** and `mp-48`. Defaults are 180 Ry,
+   an 8×8×1 k-grid, 20 Å vacuum, PBEsol/PAW, and zero external pressure.
+   Common parameters have form fields; other supported settings are in the
+   advanced JSON form. The worker resolves the Materials Project ID using the
+   server-held key, never a key supplied in the notebook or stored in results.
+4. Submit once. The run panel shows its state, Slurm ID, cluster, requester, and
+   service-account execution identity. A cancel action is available for active
+   runs. A submission is not evidence of scientific convergence.
+5. When results are ingested, expand the run and inspect convergence, warnings,
+   inputs, structured outputs, and provenance. Download the JSON export.
+6. For **3D** elasticity, first complete a converged 3D CellRelaxDFT run, then
+   select it as the upstream result in ElasticDFT. Parameters and the relaxed
+   structure are inherited. A 2D relaxation is deliberately not offered as a
+   parent for the bulk elastic calculator.
+7. If result transport failed after Slurm ran, use **Retry result retrieval**.
+   This rechecks the same job and retrieves its output; it does not submit a
+   replacement calculation. Scientific failures require investigation, not this
+   transport-recovery action.
+
+The live silicon tests used a deliberately coarse 30 Ry/2×2×2 setup and 12 strain
+members to validate infrastructure cheaply. Those are **not recommended scientific
+production settings**. Justin/Rushik should supply convergence-tested settings.
+
+## Execution, identity, and isolation
+
+The browser calls the DT backend through its authenticated proxy. DT checks
+project permissions, validates registered inputs, and persists the request before
+the worker invokes the allow-listed Globus gateway. The endpoint stages inputs
+in scratch and submits `sbatch`; QE runs on allocated compute nodes, not login
+nodes. Slurm polling is centralized and batched at approximately 30-second
+intervals rather than performed by each browser.
+
+Jobs run as service account `dtwin0`, using `ncn / cpu / normal`. DT records the
+authenticated requesting student and adds `--comment=dt-run:<run UUID>` itself.
+This preserves attribution, **not individual-student Unix ownership or access to
+their other queues**. No student password, Duo response, or private SSH key is
+required by the runtime. Globus and Materials Project service secrets remain in
+the isolated namespace. Geddes etcd is not encrypted at rest; namespace access
+controls, secret rotation, and acceptance of that residual risk remain operational
+responsibilities. Switching to a service account does not encrypt its secrets.
+
+## Data and scientific meaning
+
+- Portable pymatgen structure dictionaries, numeric arrays, explicit units, and
+  JSON provenance are stored in the DT database, not Python pickles.
+- Runs retain inputs, upstream linkage, installation/modules, pseudopotential
+  hashes, job identity, and content hashes for archived artifacts.
+- Relaxation results retain energies, stresses, forces, and available cell/atomic
+  coordinate histories. The independent history arrays are not automatically a
+  frame-aligned ML training dataset.
+- Elastic results additionally retain each deformation's input structure and,
+  when present, final relaxed structure paired with final energy, forces, and
+  stress. `complete_structure_force_pair` is false when required information is
+  absent; missing coordinates are never silently replaced by unrelaxed ones.
+- 2D results record the original bulk structure, extracted site indices,
+  orientation/rotation, initial monolayer, vacuum, and final relaxed cell.
+  Supercell pressure/stress and volume include the vacuum normalization: they
+  are **not 2D elastic moduli**. The pilot uses periodic vacuum, not Coulomb
+  truncation, and rejects unsupported/mixed-dimensional input rather than guessing.
+- Raw selected QE inputs, output logs, scheduler logs, and the elastic input
+  archive are retained under `/depot/ncn/data/digital-twins/runs/<run UUID>/`.
+  Hashes and sizes are in the database. The branch now supports opt-in,
+  project-authorized downloads up to 8 MiB with checksum verification, enabled
+  in the isolated pilot. An authorized maintainer can retrieve larger files. Large `qe_tmp`
+  restart/wavefunction files are not part of this selected-artifact archive.
+
+## Boundaries before wider release
+
+Scientific approval of Justin's 2D adapter and convergence settings is still
+needed. The supplied reference CIF and metadata are not a full independently
+validated reference-output suite. The infrastructure tests do not prove the
+accuracy of an arbitrary material's result.
+
+2D elasticity, third-order tensors, an energy-fitting switch, automatic
+multi-cluster scheduling, and arbitrary script uploads are not implemented.
+Those need the actual scientific workflows and agreed contracts. No dummy tools
+or zero-filled tensor components have been registered in their place.
+
+The isolated pilot uses independent session/SAML credentials, a fully migrated
+private database, and the students' actual nanoHUB identities. Earlier temporary
+tests used signed test sessions; September 28 real sign-ins and Navid's confirmed
+logout/re-login cover the actual external login path. Production remains disabled
+until its separate rollout is authorized.