October 2026

leads

Oct 15, 2026 Offline Leads Meeting

Attending:  Dom Brailsford, Guiseppe Cerati, Erica Snider

Agenda and notes:

Erica:  LArSoft project   (slides for LArSoft report)

  • LArSoft work plan update
    • Start per-experiment planning meetings asap
    • Current state of work plan. Note that effort dedicated to LArSoft is now less than one FTE. Sufficient for reviewing PRs, routine maintenance. All major development tasks will rely on effort from project lead and immediate collaborators. Aside from Phlex adaptation, no large-scale architectural work will be possible as part of the next work plan.

Discussion

    • Will want to talk to DUNE ProtoDUNE people for 2027 work plan discussions

 

  • Spack migration
    • In production for LArSoft as of Aug 26
    • Will continue to create UPS-based and Spack-based releases until Spack is in production at all LArSoft experiments.
    • What is status at the experiments?

Discussion

    • DUNE:  FD is not ready. Still using UPS for production. ND is partly there (though not relevant to LArSoft decision tree).
    • ICARUS:  Recently regained ability to routinely cut releases off head of ‘develop’ branch. Currently catching up with pending PRs. Are able to create release with Spack..
    • SBND:  believe release team is ready to migrate to Spack, but are not yet producing Spack releases. Working closely with Patrick. They have declined help from Guiseppe, so seem to be in good shape.
    • GC commented on MicroBooNE based on ATT meeting this morning. MicroBooNE has a Spack environment for uboonecode. (See ATT meeting slides from this morning.) So may be close to production, but did not mention plans for Spack in production. 
      • [Spoke to Herb after the meeting. MicroBooNE is ready to use Spack for integration releases moving forward, and to drop UPS for those releases. So Spack is effectively in production at MicroBooNE]
  • art 3.15 migration
    • Allows updating several dependencies
      • Root → v6.30
      • cetmodules → v4
      • GENIE v3.08
    • Side effect:
      • Cannot build GENIE against Pythia 6
    • Without objection, plan to move ahead asap

Discussion

    • Clarify that Genie update to 3.08 is not mandated by this change. [Erica confirmed]
    • Also questions about impact on users / experiment code. I replied that we typically make all the changes that are needed [for code that is on develop].
  • Geant4 v4.11.4
    • Release candidate v10_20_09_04
    • Need sign-off from all experiments
    • What are validation plans? Can we move ahead?

Discussion

    • DUNE:  need to check in w ProtoDUNE. Otherwise, FD wants this asap.
    • SBN:  don’t have much to say. Don’t know if tested. Will check w John Plows et al. Will get back to us. [*** pending feedback from Guiseppe / SBN ***]
  • Phlex adaptation
    • Working closely with DUNE Phlex Adoption WG
    • Strategy
      • Factor algorithms from art framework and canvas interfaces
        • Includes factoring algorithms from art::Services and art::Tools
      • Move to framework independent repositories (LArSoftObj suite)
      • Create new repositories to interface to Phlex
    • Work areas (all assisted by AI coding agents)
      • Make LArSoftObj framework independent
        • Remove / replace art-affiliated product dependencies 
          • MessageFacility
          • fhiclcpp::ParameterSet
        • Remove / replace art-isms previously allowed
          • art::Ptr / art::Assns
        • Move art/root dictionary wrappers to a new repository
      • Factoring LArSoft algorithm code
        • Factor in place + validate
        • Perform LArSoftObj migrations to make framework independent
        • Move to LArSoftObj
      • Interfacing algorithms to Phlex
        • Requires data converters or native data sources or unit tests
        • Wrap algorithms into HOFs
        • Wrap Services and Tools into data layers
    • Status and plans
      • Working in parallel on framework independence of LArSoftObj and factoring algorithms from art interface code
      • Refactoring
        • Have demonstrated agentic workflow to assist (work by Gavin Davies, U Miss)
        • Changes will be validated and live in place until LArSoftObj is ready
      • LArSoftObj
        • All repositories have migrated from MessageFacility to spdlog (which is what Phlex will use)
          • Have AI workflow to convert
        • Will work on fhiclcpp next, followed by canvas dependence (art::Ptr, art::Assns)
      • As soon an algorithm is moved to LArSoftObj, work immediately to interface it to Phlex

Discussion and round table

  • DUNE: Dom
    • What about maintenance burden of supporting LArSoft for both Phlex and art?
      • That is a concern within the team. The art side of the house is primarily reviewing PRs, offering advice and guidance, providing software engineering expertise. That workload will be the same before / after adaptation (assuming the overall sharing scheme works)
      • The art maintenance per se is small now, so should continue to be small. The main workload will come from building art-based releases, which is non-trivial 
      • So will need to see how things go given the ever decreasing size of the support team.
    • Asked about the list of people on larsoft-coordination email list
      • Sent him a copy
      • Basically comes from the “informal list of experiment contacts” page in the LArSoft Wiki
      • Let me know when changes happen so that I can keep them up to date.
      • On quick review: add Laura (DUNE), Tracy (ICARUS), John Plows (SBND/SBN)

 

  • ICARUS /SBND: Giuseppe
    • Good progress on IaaS. Testing various models. Most productive (slowest) is DNN ROI within WireCell. Doing IaaS for that reduces both CPU and memory. It will benefit production processing quite a bit.
    • Need to define how to do in production. But promises benefits.
    • Q:  Where will production be run? 
      • Testing various configs. Local GPU. EAF, NERSC. 
      • If NERSC is used as Triton/GPU server, then can submit from Fermigrid. Need to see if network transfer time makes this usable
      • Can also run entire job at NERSC, but then need to transfer input files for things like overlay MC. MC production is the ideal situation – basically no input file. Two possible configurations at NERSC
        • Triton server for GPU, then run on CPU allocation
        • Run on GPU nodes directly with local Triton server
      • Had summer student apply to CVN (event-level interaction type classifier). (Same thing that DUNE FD is using)
    • Q:  at DUNE, have not found that inference needs to be accelerated. ?
      • Production bottleneck is WireCell ROI. Once on GPU to solve that, then makes sense to run all other models there
      • GPU is optional in all cases. Can run on CPU if desired.
    • Q: How do user-level jobs work?
      • Can go back to torch model. WireCell has torch plug-in
      • Can also start local triton server on grid node. Not as fast as running on GPU when that is available, but it works. 
      • Can also upload model to EAF triton server. That server is also accessible from Fermigrid. Works for jobs that are not very large scale
    • Dune has not seen need for the ROI yet. More of a need for … [missed…]

 

  • Erica:  Asked about someone at DUNE to work with Celeritas people on a use case that needs the more detailed information available (compared to the visibility service output) from a single-photon-tracking simulation. Celeritas developers eager to work with someone like this 
    • Dom:  There has been a lot of thought into getting better fidelity detector response simulation. But perhaps the interesting case [for single-photon tracking?] is in flash-matching
    • Semi-analytical model gets run on the fly for flash matching. Dom is thinking about other ways to utilize acceleration (??) [Erica    missed some critical point here. My be that semi-analytical is better than MC simulation for this, but acceleration still useful?? Need to clarify offline…]]
    • For Celeritas person, talk to ProtoDUNE sub-group (all detectors in same  group)
    • Will there be APEX ProtoDUNE? Unclear. There may be another run request.

 

  • DUNE:  Dom
    • Various things going on. Mostly production related. For FD, a significant push to understand VD in context of low-E program. Task force formed, joint Low-E and DAQ.
    • Low E simulation and trigger sim are the most complex elements. Radiologicals are complicated too.
    • Many geometries currently available. None are ideal, so they don’t meet current needs. The deficiencies all impact physics performance. No one is truly happy with the “workspace” provided by the smaller geometries (eg, 2x6x12). But full FD simulations are still too big.
      • So want to run everything in 10kT geometry, but only consider Edeps in a small portion.
      • Simulate radiologicals, get light yields, etc. 
    • Handling data in large geometries introduces complexities
      • The recent PR for regex getters was aimed at mitigating this problem in art. Necessary  for trigger simulations in large workspace geometries. Were hitting memory limits otherwise.
      • All downstream part of code […??…]
    • Erica:  Mentioned parallel geometry idea that came up in Celeritas discussions this week
      • Dom:  Large workspace idea caters to this parallel geometry idea
      • Only return E deps for selected regions. 
      • Charge simulate only those regions, but light simulation would be unrestricted.
      • Noted that semi-analytical model has ~10m light propagation limit (unclear what reason is). 
  • Discussed names in larsoft-coordination (noted above)

 

###