A New Tool for Science Gateways

I got an update from TeraGrid Science Gateways director Nancy Wilkins-Diehr and Stuart Martin on an important set of activities related to TeraGrid science gateways. During the past month or so the group has focused on testing a new GRAM audit service, which Stuart has been spearheading. From the relevant Globus Alliance bugzilla post describing the new feature:

An auditing mechanism for WS-GRAM and a proof-of-concept interface to compound audit/TeraGrid accounting database queries has been created using OGSA-DAI at the request of the TeraGrid infrastructure team. The next step is to actually deploy these components on TeraGrid to get a working example. This will provide a fully integrated proof of concept for the entire setup as well as allow TeraGrid people to use it and report back on how they would like to use it (i.e. what specific queries will they need). Additional campaigns may need to be created to add additional OGSA-DAI activities to support the desired query set.

What does this mean in practical terms, and why is it important to gateway providers?

TeraGrid is funded by the National Science Foundation as a service to researchers, who are allocated access based on peer review. That peer review process takes into account the scientific progress enabled by work associated with an allocation, or project. TeraGrid accounting systems keep track of usage for each individual job that has been executed, associating the usage with a specific allocation (project). Traditionally, a project will have a handful of users associated with it, and a principal investigator who keeps track of what science has been accomplished with that allocation. TeraGrid provides tools for the principal investigator to track usage, and the principal investigator works with his or her team to manage the allocation.

But science gateway principal investigators will use a community allocation to support a very large team of users - potentially hundreds! How does that gateway provider keep track of what is being accomplished with the allocation? Some science gateways may track usage on a per-user basis; others may track on a per-application basis (i.e. tracking usage by function, or application service rather than user, where an application, or function of the gateway, may be one of a small number of tools made available to the community through the gateway).

To track usage at this level, a gateway provider must be able to associate a grid job identifier (associated with the user or application service at the gateway) with the job entry in the TeraGrid accounting system. The current TeraGrid accounting systems report only the local job id on the TeraGrid resource, but have no information about the grid job id on the gateway end. Thus there is no way to correlate individual jobs in the TeraGrid accounting system with individual actions taken at the gateway (e.g. with users or applications).

The GRAM audit capability maintains this correlation in a database, allowing the gateway to retrieve usage information from the TeraGrid accounting system as well as mappings to individual users/applications from the audit database. This capability is in beta test today with some of the TeraGrid science gateways, and will be more tightly integrated with the accounting system, for example through the existing usage query mechanisms available to users.

This is a nice example of the symbiotic relationship TeraGrid has with the middleware community where an important capability (not just for TeraGrid but for other Grid projects) is created. Working with the open source Globus Alliance we were able to implement a new and necessary service in relatively short order, leveraging the UK eScience project led OGSA-DAI work that was standardized in the Open Grid Forum (previously Global Grid Forum).


Post a Comment

<< Home