Overview
The NSF CC* CRISPIE Edge-DTN-STOR (Edge-DTN-STOR) is a high-performance research storage service built on Nextcloud and integrated with CRISPIE Data Transfer Nodes (DTN). It provides participating institutions with secure storage and multiple access methods for managing and transferring research data. Edge-DTN-STOR expands flexible storage resources across the Edge Network to accelerate cross-institutional computational and data-intensive research and STEM education.
Edge-DTN-STOR enhances the U.S. National Science Foundation (NSF) investments and successful regional collaborations, based on the NSF CC* Regional Networking: Connectivity through Regional Infrastructure for Scientific Partnerships, Innovation, and Education (CRISPIE, NSF #2346648)
What is a Data Transfer Node (DTN)
A Data Transfer Node (DTN) is a dedicated, high-performance computing environment optimized for moving large datasets over networks within a Science DMZ to bypass bottlenecks. DTNs provide high-speed, secure data movement for scientific instruments, supercomputers, and cloud storage, preventing data transfer tasks from slowing down user login nodes.
DTN Usage Examples
- Scientific Data Movement: Moving large genomic datasets or high-resolution images from instruments to analysis clusters.
- Cloud Storage Transfers: Utilizing tools like rclone or Globus to move data to/from the cloud or Google Drive.
- High-Performance Computing (HPC): Transferring datasets between supercomputing sites.
- Large File Transfers: Using SCP/SFTP for transferring large files on university grids.
For additional information about Data Transfer Nodes, please refer to ESNet Knowledge base.
What Edge-DTN-STOR provides
- 96 TB of storage for sharing among NSF CRISPIE Project partner institutions, optimized for streaming data into AI/ML algorithms and out of simulations and instruments
- Partner institutional access - available to NJEdge NSF CC* CRISPIE partner institutions: Brookdale Community College, Montclair State University, New Jersey Institute of Technology, Ramapo College, Rider University, Rowan University, The College of New Jersey.
- Researcher-friendly access: A web-based UI, and can mount for federally funded national resources, such as the National Research Platform (NRP), OSG, ACCESS, NAIRR Pilot.
Advanced Cyberinfrastructure Coordination Ecosystem (ACCESS)
ACCESS is an advanced computing and data resource program supported by the U.S. National Science Foundation (NSF). ACCESS offers allocations, training, and documentation to get started on national resources.
For additional information, please visit https://access-ci.org/
The National AI Research Resource (NAIRR) Pilot
The NAIRR Pilot aims to connect U.S. researchers and educators to computational, data, and training resources needed to advance AI research and research that employs AI. Federal agencies are collaborating with government-supported and non-governmental partners to implement the Pilot as a preparatory step toward an eventual full NAIRR implementation.
Additional information can be found at https://nairrpilot.org.
National Research Platform (NRP)
The NRP is a community-owned research and education platform connecting researchers and educators to foster collaboration, accelerate innovation, and share resources. Supported by over 50 institutions, including leadership from UC San Diego, the University of Nebraska-Lincoln, and the Massachusetts Green High Performance Computing Center, the NRP provides access to cutting-edge technologies in AI, high-performance computing, data storage, and networking. Open to all nonprofit higher education institutions, from community colleges to top research universities, the NRP advances learning and scientific breakthroughs with support from the U.S. National Science Foundation, Department of Energy, and Department of Defense among others.
Please visit https://nrp.ai/ for additional information.
Open Science Grid (OSG)
OSG is a consortium dedicated to the advancement of open science via the practice of distributed High Throughput Computing, and the advancement of its state of the art.
The OSG Consortium builds and operates a set of pools of shared computing and data capacity for distributed high-throughput computing (dHTC). Each pool is organized and operated to serve a particular research community (e.g. a campus, multi-institutional collaboration, etc.), using technologies and services provided by the core OSG Team. One of these pools, known as the Open Science Pool is operated for all of US-associated open science. The Consortium, thus, represents the totality of all researchers, resources, individuals and institutions that benefit from or contribute to any of the OSG Fabric of Services.
Please visit https://osg-htc.org/ for full set of services provided through OSG.
Who it serves
Researchers at NSF CRISPIE partner institutions and educators across the region and beyond, including communities in the National Research Platform (NRP), Open Science Grid (OSG), ACCESS programs, and more.
Why it matters
By advancing storage accessibility and performance, Edge-DTN-STOR supports AI/ML workflows, simulation, instrument data pipelines, and classroom integration, bringing advanced cyberinfrastructure capabilities to the NJ region and beyond.
Edge-DTN-STOR's storage capability enables researchers using AI, machine learning, and simulation to advance research and teaching across science and engineering disciplines.
Key merits
- Enhanced accessibility for compute and data workflows, including:
- ○ Mounting for high-throughput computing (HTC) jobs in the OSG and National Research Platform (NRP)
- ○ Web-based UI via Nextcloud
- Server-Site access allows for collaboration and sharing with NJ partner institutions
- Addresses unique challenges of performance and data sharing
- Advances CI in the New Jersey Region, a region ideal for developing advanced computing, data, and networking, especially for non-R1's including small liberal arts colleges and community colleges
Broader Impacts
Edge-DTN-STOR directly supports regional and national STEM strategies by:
- Building campus research capacity and innovation infrastructure
- Supporting industrial sectors critical for economic growth, including biopharmaceuticals, healthcare, and advanced materials
- Advancing HTC computing for research and education
Broadening participation
- Focus on non-R1 institutions across the region, including primarily undergraduate institutions, including community colleges
- Leverages existing relationships to provide in-person training and to connect researchers with regional peers
Dissemination and workforce development
- Online step-by-step guides
- Presentations and engagement at the local, regional, and national conferences, including EdgeCon events
- Educational components and development of technical and facilitation workforce capable of designing and implementing a modern high-performance research platform
- Student engagement in funded equipment and training activities
Edge CRISPIE Project team
The investigators are well qualified based on institutional partnerships of campus cyberinfrastructure experts. Co-PIs and collaborators have participated in other NSF projects, as well as Ecosystem for Research Networking (ERN), Campus Champions, the Campus Research Computing Consortium (CaRCC), and NRP. The team engages S&E faculty across the New Jersey region, including faculty from predominantly undergraduate institutions (PUIs).
PI and Co-PIs
Senior Members of the Project Team
Who can use Edge-DTN-STOR?
Edge-DTN-STOR is designed to be easy to access from campus and national CI ecosystems.
Researchers and educators associated with institutions in the NSF CC* CRISPIE project, as well as projects within NRP, OSG, and ACCESS programs (additional communities may be supported as policies evolve).
Access methods
- Globus (https://www.globus.org): For managed, high-performance file transfers
- Nextcloud: For web-based access, collaboration, and sharing
- Other possibilities include leveraging national resources such as:
- ○ OSG/NRP Mounts: For high-throughput computing (HTC) jobs
- ○ OSDF Origin Nodes: For publishing and serving datasets
Requesting access
Details of the access policies, allocation processes, and request forms will be published with the production service launch. In the interim:
- Talk to your campus CI lead or research computing support
- Identify your project, anticipated storage needs, and collaborators
- Note any integration requirements (OSG/NRP, Globus, ACCESS, NAIRR Pilot)
- Refer to The Edge-DTN-STOR Access Guide
Example workflows
- Stream instrument data to the high-speed tier for AI/ML preprocessing and analysis
- Land simulation outputs into the object tier; distribute curated results with Globus
- Mount Edge-DTN-STOR in OSG/NRP HTC jobs for distributed data processing
Getting Help
Have questions? We're here to help! Please or email us at crispiesupport@njedge.net.
This service supports open data only (does not satisfy for example HIPAA/Health data management requirements), and does not include data redundancy or backup. Users must maintain independent backups of all stored content.