May 2006
Designing and Supporting Science-Driven Infrastructure
Charlie Catlett, Pete Beckman, Dane Skow and Ian Foster, The Computation Institute, University of Chicago and Argonne National Laboratory

3. Integration: Building a National-Scale Grid Facility

The creation of a grid facility involves the integration of a set of resource providers. A coherent grid facility must leverage software infrastructure, as described above, to provide a set of common services, a framework that allows for exploitation of unique facilities, and the infrastructure needed to coordinate the efforts of the resource providers in support of users. Common services include operations centers, network connectivity, software architecture and support, planning, and verification and validation systems. Facility-wide infrastructure includes components such as web servers, collaboration systems, the framework for resource management policy and processes, operation coordination, training documentation and services, and software repositories. Underlying Grid middleware software provides common services and interfaces for such functions as authentication and authorization, job submission and execution, data movement, monitoring, discovery, resource brokering, and workflow.

For scientific computing, and in particular high-performance computing, the fact that a user can reliably expect the Unix operating system as the standard environment on almost all major shared resources has been a boon to scientists making persistent software investments. Internet connectivity and basic services such as SSH and FTP have similarly become standard offerings. A grid facility aims to provide services that allow for resources to be aggregated, such that applications hosted on various resources can be combined into a complex workflow. Such a set of services, operated within a single organization, would be merely complex. Providing these services across a collection of organizations adds policy, social, coordination, and other integration requirements that exceed the complexity of the grid middleware itself.

Addressing these requirements to create and manage a national-scale grid environment requires the creation and operation of both organizational and technical integration services. We do not attempt to prescribe organizational structures within which these functions reside. However, we do make several observations. First, a grid facility requires close collaboration and cooperation among all participating organizations, each of which provides one or more functions and services as part of the overall facility. Second, despite the fact that each participating resource provider shares the goal of creating and operating a high-quality grid facility, it is necessary to identify specific responsibilities for coordinating and providing common services. In most grid projects, this function is performed by a system integration team that coordinates and plans common services, providing these services directly and through partner organizations.

In the rest of this section, we use the TeraGrid to illustrate the specific functions and costs required to provide national cyberinfrastructure. For each functional section, we discuss the scope of work as well as the approximate staffing levels, both in the system integration team (the GIG) and at the resource provider facilities. We use units of “full time equivalents” or “FTE” to measure effort because most staff members are employed partially on TeraGrid funding and partially on other institutional funding.

Pages: 1 2 3 4 5 6 7

Reference this article
Catlett, C., Beckman, P., Skow, D., Foster, I. "Creating and Operating National-Scale Cyberinfrastructure Services," CTWatch Quarterly, Volume 2, Number 2, May 2006. http://www.ctwatch.org/quarterly/articles/2006/05/creating-and-operating-national-scale-cyberinfrastructure-services/

Any opinions expressed on this site belong to their respective authors and are not necessarily shared by the sponsoring institutions or the National Science Foundation (NSF).

Any trademarks or trade names, registered or otherwise, that appear on this site are the property of their respective owners and, unless noted, do not represent endorsement by the editors, publishers, sponsoring institutions, the National Science Foundation, or any other member of the CTWatch team.

No guarantee is granted by CTWatch that information appearing in articles published by the Quarterly or appearing in the Blog is complete or accurate. Information on this site is not intended for commercial purposes.