TeraGrid Director's Whiteboard

Moving from blogspot to TypePad

2006-10-04T08:19:00.000-05:00

I'm moving this blog to TypePad, which isn't free like bolgspot, but has a nice feature that allows you to categorize posts. The tricky part will be to select the right categories, of course, but it makes a blog much easier to navigate. For example, maybe you are only interested in posts about user experience, or security, or even non-sequitur posts.

It was pretty painless to import all of the posts to date, including comments, and even to categorize all of the posts so it's already possible to filter on particular topics.

The new home for this blog is here:

http://teragrid.typepad.com/

CeC

European e-Infrastructures Roadmap

2006-10-04T06:37:00.000-05:00

I'm participating in a European Commission workshop at the Finnish IT Center for Science (CSC) in Helsinki, focused on the European e-Infrastructure Roadmap. (e-Infrastructure is the European analog to the NSF term Cyberinfrastructure.)

I gave a presentation on TeraGrid, including a discussion about cyberinfrastructure and related initatives at NSF and DOE.

Kyriakos Baxevanidis (from the European Commission) gave an overview of the new Seventh Framework (funding) Programme, FP7.

Timo Skyttä (from Nokia) gave an interesting talk about identity management and the Liberty Alliance. He points out that both Liberty and the Shibboleth project have adopted SAML 2.0, an XML-based set of specifications for exchange of authorization and auuthentication information. Skytta showed a number of case studies including the US E-Authentication initiative.

Klaus Ullman (from DANTE) gave an overview of the future of research and education networking in Europe. DANTE's current infrastructure is called GEANT2, a high-speed backbone network that interconnects 30+ national research and education networks (NRENs). There are some nice downloadable PDF maps at the DANTE website. Ullman gave a nice overview of their forecasts in terms of user requirements and how these shape the network deployment plans.

Presentations from this workshop can be downloaded from the workshop agenda page.

(CSC is actually in Espoo, just outside of Helsinki)

Non-sequiter of the week: Saving energy. Today I read in Argonne's daily newsletter that Argonne employees are encouraged to swap out incandesant light bulbs for compact flourescent lightbulbs (CFL's). We had already done this for many of the lights in our home, but the following statement caught my eye. If every household in the United States replaced one light bulb with an Energy Star qualified compact fluorescent light bulb (CFL), it would prevent enough pollution to equal removing one million cars from the road. Cool. Go for it - replace two!

Making TeraGrid More Accessible

2006-10-03T01:06:00.000-05:00

During the past year one of our priorities has been to give users better tools for learning about, and interacting with, TeraGrid resources and services. Diana Diehl and Tim Gumpto (SDSC) have led efforts to make the TeraGrid website easier to navigate while deepening the content. Eric Roberts and Maytal Dahan (TACC) are rolling out an updated version of the TeraGrid User Portal this week with some important new functions. And we have an experimental "crowdsourcing" site where we are inviting the TeraGrid community to collaborate on a next-generation TeraGrid website and watering hole.

The challenge for our team over the next few months will be to bring these three important efforts together into a coherent set of functions - this is a significant focus of our user services working group.

Cyberinfrastructure User Advisory Committee (CUAC)

2006-09-29T17:12:00.000-05:00

A few months ago the TeraGrid leadership team worked with NSF to create the Cyberinfrastructure user Advisory Committee, or CUAC. The CUAC is comprised of twelve end-users - consumers - of cyberinfrastructure. We were delighted to be able to find advisors from each and every one of NSF's science directorates, and we had our first meeting in June. We are beginning to pull together the first report from the CUAC, documenting a series of small-group discussions (the CUAC is loosely organized into three subgroups) that the CUAC has had over the past few months. Several themes strike me about this draft (which will eventually be posted at the CUAC website referenced above).

One of the topics that comes up repeatedly is the need for more training and education regarding how to use the individual TeraGrid resources as well as how to use them together, for example in a workflow. Having harnessed TeraGrid, many users also are looking for training and help in analysing and visualizing their data. Simply put, we need to look hard at how we can increase our training and education offerings.

CUAC members also recommended that we look carefully at the barriers faced by interested potential users, before they even become users. From the point of view of a scientist considering writing his or her first proposal for a TeraGrid allocation, it would be useful to understand what are the chances that their research can be accelerated by TeraGrid, and what are the chances that their proposal will result in an allocation. We do have development allocations, or DAC awards that are very straightforward to propose, and so much of this is also a matter of better communication with potential users. (actually the DAC process has been wildly successful, with nearly 250 awards granted already this calendar year!).

In a nutshell, communication, training and education are clearly high-priority items for TeraGrid to address in the coming months.

Non-sequiter of the week: Louisville Slugger Baseball Bats. After my daughter's cross country meet tomorrow we plan to head for the Louisville Slugger Museum and Factory which I'm told is a very fun tour, particularly if you are a baseball fan. Speaking of baseball - and it is that time of year - I'm pulling for a twenty-year anniversary world championship this year, which may give you an idea of my baseball leanings. :-)

Exploring VMs

2006-09-26T09:49:00.000-05:00

After posting a few days ago some thoughts on virtual machine technology, and spending lots of time thinking about how to leverage commercial services such as Amazon's EC2, I was talking with Kate Keahey from Argonne, who has been working in this area for a while. She gave me a very nice summary of work in the Globus Alliance that I thought would be worth sharing here:

One of the advantages of using virtual machines is the ability to easily and efficiently deploy desired software environments encapsulated in a VM image. This allows resource users to configure the virtual machine images themselves and deploy them on a VM-enabled platform made available by a resource provider. Another feature of interest is that VM tools offer capabilities allowing a resource provider to guarantee the delivery of specific resource quota (in terms of memory, CPU%, disk, bandwidth, etc.) to a VM -- this facilitates implementing sharing and accounting between different clients. The Globus Virtual Workspaces project leverages these capabilities to provide such controlled sharing and configuration independence (see a recent paper).
The configuration and performance isolation implemented by virtual machines enables a division of labor between resource provider and consumer which has the potential to significantly contribute to the growth and scalability of Grids.

The advantages of using virtual machines in Grid and generally distributed computing are still emerging as new hypervisor capabilities and new requirements emerge. The VTDC06 workshop, co-hosted with SC06 this year, brings together the virtualization and distributed computing communities to discuss the potential of virtualization in resource management, scheduling, security and service hosting.

Grid Interoperation (Now?)

2006-09-22T10:30:00.000-05:00

About a year ago many of us involved in major grid initiatives and facilities realized that there were many pair-wise discussions about interoperation, and a set of emerging "common themes" to these discussions. This quest for interoperation is driven by two strong needs. First, there are many research teams with collaborators located in different countries, and/or on different continents, with access to multiple grid facilities. How do we help them work together, which often involves use of grid resources in multiple grid facilities? A second driver here is a practical and technical desire to adopt working solutions from others rather than reinventing them.

Leaders from nine major Grid initiatives met in November 2005 to band together to drive interoperation (pardon the acronyms): TeraGrid (US), OSG (US), DEISA (Europe), NGS (UK), NAREGI (Japan), K*Grid (Korea), PRAGMA (Pacific Rim), APAC-Grid (Australia), and EGEE (Europe)

During a half-day discussion this group identified four areas where the current state of technology, with some coordination on our part, could begin to support interoperation. We formed several task-forces to develop interoperation plans in the areas of:

- Information services
- Job submission
- Data movement
- Authorization

The PRAGMA folks also took the lead in identifying several early-adopter applications to drive these four areas, and we set up an operations task force to capture that experience. Plans in these areas were presented at the Athens GGF meeting in February, and eleven more grid projects joined us (I won't try to list them here in this already acronym-rich post). A tremendous amount of work was done early this year, and we held updates on progress at the Tokyo (May) and Washington, DC (September) GGF meetings.

You can find details on this progress, constantly being updated and expanded as we move forward, at the Grid Interoperation Now (GIN) wiki hosted at the GGF site.

The next steps for this group involve expanding the applications effort to bring in at least another dozen science teams interested in testing what we have put in place and driving it forward. The GIN effort is completely open, and we are always looking for more people to help out- head over to the site and jump right in!

Predicting the (near) Future

2006-09-16T23:57:00.000-05:00

This past week we had a quarterly TeraGrid management meeting in Austin at the University of Texas, home of TACC. One of the discussions we had was regarding the growing number of computational resources available to users, and the need to help them to sort through the options. A key question for a user is "if I submit my job to this particular TeraGrid machine, and I'd like it to run in the next n minutes, what is the likelihood that it will run in that time?

It turns out that there are several tools that can give the user a prediction, albeit not with 100% certainty, based on the state and history of the queue in question. Rich Wolski (UC Santa Barbara) and his Network Weather Service project have been doing nice work in this area for quite a while. Rich's work with the VGrADS project has brought us a very nice tool that you can see demonstrated at his demo website.

We are working to get this capability embedded in the TeraGrid User Portal and after I sent email to our Science Gateways mailing list I found that many of them were already in the process of making this tool available. For a scientist trying to get work done, it will be a wonderful thing to be able to look across the now more than 20 major computational systems in TeraGrid and get a sense for where his or her job will run soonest!

Non Sequitur of the Week: Apple Widgets. Well this is really a non-sequitur if you're not a Mac person, but I am and I had been interested in what was involved in writing Dashboard widgets. I downloaded the Apple Developer kits and there were some nice examples in there. Since Rich Wolski sent me a tarball of the bqp (ok not a total non-sequitur, see above) command line utilities I decided to try to make a very simple widget, building on one of Apple's examples. It took about an hour to figure out the basics, and was kinda fun (I called it "AskRich"). It assumes you put the NWS command line tools in /usr/local/bin on your Mac and executes a hard-coded query, but it's a start. Perhaps next weekend I'll learn how to let the widget user select the options... (if you are interested in seeing the widget, send me email with the word widget in the subject line).

{Amazon, Google, eBay, Microsoft...}.EDU

2006-09-15T16:42:00.000-05:00

I've been having many discussions with people from the Research & Education community - TeraGrid Science Gateway providers, individual users, computer center directors, etc. - regarding the notion of taking advantage of some new and interesting storage and computing web services such as Amazon's S3 and EC2. Google, Microsoft, eBay, and others are surely going to provide new web servies in this space. Further, anyone paying moderate attention will also see that technology provider companies (IBM, EMC, Platform, Univa, etc.) are introducing powerful building blocks aimed at building service oriented systems (e.g. "Grids"). Some (especially end users!) respond with enthusiasm - and some folks have responded along the lines of "we can do it ourselves cheaper" or "performance isn't good enough."

I think these responses are true to some extent, but they also ignore some important factors. The first is Moore's Law. Today's price is irrelevant - prices based on technology (like disk or CPU or bandwidth) get cheaper, rapidly, over time. (Imagine if the $100,000 price tag on a visualization workstation twenty years ago had stopped us from developing imaging tools...) What we have typically done in this community is to ask what the computational environment of the future will look like, and we design and plan around the future - not the present. That's how you invent the future rather than just reacting to change as it hits you.

The second is mistaking oranges for apples, and thus doing an apples to oranges comparison. Take Amazon S3. It's way, way more expensive than buying a disk drive, especially if you already operate a large computing facility. But is it the same? Not if your computing server does not provide a web services interface! Does it matter? Only if your users want a web services interface, or if you want to develop a workflow, or other sophisticated capability with web services. Many users I've spoken with say they do!

Let's look at an example. If you don't already run a storage service, what's the best way to share something like a 5 TeraByte data collection with colleagues spread around the Internet? To set up a server with 5 TB of disk and a sensible backup system (if you care about that, otherwise the calculations change) you'll pay about the same as the storage cost for putting the data in S3 for three years. The open question is data transfer- if you're sharing the 5 TB with thousands of users you may be better off hosting it yourself due to the S3 I/O charges. But if you're sharing with a small community, with modest needs in terms of moving data out and in, then S3 is likely much cheaper than rolling your own- unless your system administration staff work for free.

I believe that TeraGrid and similar initiatives must seriously investigate what a partnership might look like with (web/grid) "service providers." While these services do not address the requirements of users who need multiple Teraflops of computing or tens of Terabytes of storage, they just may offer something for the many people who want to share smaller amounts of data, or have intermittent needs for rapidly accessible, modest computing power.

TeraGrid is focused, rightly, on providing for Petascale computational, storage, and data analysis services. For the Gigascale stuff, perhaps we should think about a new type of "resource provider" - Amazon.edu?

(at Austin)

Virtual Machines and Types of Service for TeraGrid Computing

2006-09-11T21:48:00.000-05:00

Foundational capabilities we provide in TeraGrid, such as "roaming" access and a "coordinated" software environment, open new possibilities in terms of more specialized services, or to allow the TeraGrid, as a system, to respond to supply and demand. For example, a resource provider might elect to increase the "price" of a queue in order to improve turnaround time by reducing demand, or decrease the price to increase demand (and thus utilization).

We also are looking at ways to support on-demand services for urgent computing, through projects like Pete Beckman's Spruce work. The tricky part is being able to service an on-demand job where it's not a viable option to keep supercomputers on hot standby! We have considered things like offering a 'preemptible' service on a particular resource, where the user is charged at a lower rate in exchange for knowing that his or her job may be killed to make room for an on-demand job.

It is worth considering the use of virtual machine technology for an even better 'preemptible' service, or even to support migration of jobs in the event of an on-demand service request. One might even consider migrating the jobs to a commercial service such as Amazon's EC2!

Many people have demonstrated moving virtual machine images around with virtually no disruption to the application. The TeraGyroid collaboration between TeraGrid and the UK Reality Grid project is an example, and at iGrid2005 Franco Travostino and others demonstrated job migration.

Of course the ideal applications to take advantage of a virtual machine service are those that involve ensembles of single-processor jobs without large data requirements. But we do have a large number of users whose applications fit this very profile, so it is worth investigating such a service. Further down the road we will want to be able to support message passing (multiple-processor parallel) jobs as well as data staging needs of applications that are data-intensive. Not being able to solve those issues just yet shouldn't prevent us from looking at services that solve simpler cases!

(at OGF/GlobusWorld)

A New Tool for Science Gateways

2006-09-06T10:37:00.000-05:00

I got an update from TeraGrid Science Gateways director Nancy Wilkins-Diehr and Stuart Martin on an important set of activities related to TeraGrid science gateways. During the past month or so the group has focused on testing a new GRAM audit service, which Stuart has been spearheading. From the relevant Globus Alliance bugzilla post describing the new feature:

An auditing mechanism for WS-GRAM and a proof-of-concept interface to compound audit/TeraGrid accounting database queries has been created using OGSA-DAI at the request of the TeraGrid infrastructure team. The next step is to actually deploy these components on TeraGrid to get a working example. This will provide a fully integrated proof of concept for the entire setup as well as allow TeraGrid people to use it and report back on how they would like to use it (i.e. what specific queries will they need). Additional campaigns may need to be created to add additional OGSA-DAI activities to support the desired query set.

What does this mean in practical terms, and why is it important to gateway providers?

TeraGrid is funded by the National Science Foundation as a service to researchers, who are allocated access based on peer review. That peer review process takes into account the scientific progress enabled by work associated with an allocation, or project. TeraGrid accounting systems keep track of usage for each individual job that has been executed, associating the usage with a specific allocation (project). Traditionally, a project will have a handful of users associated with it, and a principal investigator who keeps track of what science has been accomplished with that allocation. TeraGrid provides tools for the principal investigator to track usage, and the principal investigator works with his or her team to manage the allocation.

But science gateway principal investigators will use a community allocation to support a very large team of users - potentially hundreds! How does that gateway provider keep track of what is being accomplished with the allocation? Some science gateways may track usage on a per-user basis; others may track on a per-application basis (i.e. tracking usage by function, or application service rather than user, where an application, or function of the gateway, may be one of a small number of tools made available to the community through the gateway).

To track usage at this level, a gateway provider must be able to associate a grid job identifier (associated with the user or application service at the gateway) with the job entry in the TeraGrid accounting system. The current TeraGrid accounting systems report only the local job id on the TeraGrid resource, but have no information about the grid job id on the gateway end. Thus there is no way to correlate individual jobs in the TeraGrid accounting system with individual actions taken at the gateway (e.g. with users or applications).

The GRAM audit capability maintains this correlation in a database, allowing the gateway to retrieve usage information from the TeraGrid accounting system as well as mappings to individual users/applications from the audit database. This capability is in beta test today with some of the TeraGrid science gateways, and will be more tightly integrated with the accounting system, for example through the existing usage query mechanisms available to users.

This is a nice example of the symbiotic relationship TeraGrid has with the middleware community where an important capability (not just for TeraGrid but for other Grid projects) is created. Working with the open source Globus Alliance we were able to implement a new and necessary service in relatively short order, leveraging the UK eScience project led OGSA-DAI work that was standardized in the Open Grid Forum (previously Global Grid Forum).

TeraWikiPedia?

2006-09-02T21:58:00.000-05:00

I had a conference call with Scott Lathrop (director of TeraGrid's external communicaitons, education, outreach, and training) and a subgroup of the Cyberinfrastructure User Advisory Committee (CUAC) the other day. This subgroup is focusing on issues related to training, communications, education, and outreach.

We spent some time discussing strategies for expanding and improving on-line training for TeraGrid as well as the on-line documentation in general. Over the past year our external communications team has made tremendous improvements to the website, and they continue to do so. Is there a way to improve it even more, and make the information more fresh?

One approach we talked about was the use of technology such as is used for Wikipedia, allowing our team of experts and editors to be effectively expanded to include any member of the community. But can such an approach work for TeraGrid? Will the information be accurate?

Stanford's Roy Pea, one of our CUAC advisors, did an interesting experiment using the Wikipedia technology to engage a community of students to build a site for one of his graduate courses. He notes that common concerns to this approach include quality and accuracy, but these are challenges to address rather than fatal flaws to the approach.

Nature did a study comparing Wikipedia with the Encyclopedia Britanica in late 2005. Forty-two science articles - the Wikipedia version and the Britanica version - were sent to reviewers, who were not told which was which. Reviewers found on average 4 errors per article in the Wikipedia version and 3 errors per article in Britanica. In this Nature article, author Jim Giles writes:

Only eight serious errors, such as misinterpretations of important concepts, were detected in the pairs of articles reviewed, four from each encyclopaedia. But reviewers also found many factual errors, omissions or misleading statements: 162 and 123 in Wikipedia and Britannica, respectively.

So Wikipedia isn't quite up to the Britannica standard, but it's pretty close. My sense is that the Wikipedia approach to on-line training and documentation for cyberinfrastructure would give us much more up-to-date information, and would make the information more informative in many cases as domain experts contribute. At the same time, concerns from professional editors about quality have also been raised, and Wikipedia's founder, Jimmy Wales, has expressed the need to focus on quality at this year's Wikimania conference. I think for cyberinfrastructure, such as TeraGrid, the best approach will be to combine the strengths of our editors and writers with the input of the community. A "TeraWikiPedia" is likely to deepen and improve our documentation and online training much more rapidly - and allow it to adapt in near-real time. It will in fact mean we will rely even more heavily on our editors and writers to curate and polish the content.

I'd like to see us try the Wikipedia approach with a particular set of materials, such as our education, outreach, and training materials, to see how it goes. Based on our experience there we'll have a better idea of how best to harness the creativity of the community for all of our online content.

Non-sequitur of the week. Global Positioning System (GPS) - One of my favorite toys is my Garmin Legend Cx hand held GPS unit. Besides using it for navigation (thus never having to ask directions!), I "collect" two kinds of waypoints. One kind is what I'd call a souvenir waypoint - things that you don't need a GPS to find. For example, Tokyo Station. The second kind is much more useful - places I'd like to go back to, or point others to, that are not necessarily easy to find. My favorite coffee shop (41.89529N, 12.48019E), an excellent local artisan's pottery shop (48.68931N, 122.95795W), or a friend's office (35.27568S, 149.12085E). A fun thing to do with your waypoints is to mash them up into a Google map, which is an easy way to share them with friends. I use Mapbuilder to do this. In fact a couple of friends and I are using a shared map there to assemble our favorite waypoints (so we can harvest the benefits of one anothers' exploration!). Geeky? You betcha.

Improving Security and Usability....

2006-09-01T09:43:00.000-05:00

It's generally considered to be the case that security and usability (i.e. convenience) are mutually exclusive trade-offs. Anyone who has flown on a commercial flight in the past few weeks (years) has seen this in action. One place where usability and security tend to collide in a facility like TeraGrid is the process by which authorized users get authenticated and gain access to services and resources. We actually have an opportunity to move to an architecture that will both improve usability and security. Yes, it sounds too good to be true....

A few weeks ago I posted a note about attribute-based authorization and a pointer to a paper that Von Welch (from the GridShib project has been putting together (with myself, Ian Foster, Tom Scavo, and Frank Siebenlist), and a TeraGrid Authorization, Authentication, and Account Management workshop scheduled to take place at Argonne this week. Ian also recently wrote about attribute-based authorization in his blog with some good pointers.

The workshop concluded yesterday, and I spoke with Dane Skow (TeraGrid deputy director) this morning about how it went. Dane was one of the co-organizers of the workshop (along with Von and also PSC's Jim Marsteller, the head of the TeraGrid security working group). In addition to checking out the website for the workshop, where all of the notes and background information can be found, you might be interested in Dane's take on what was accomplished:

1) We figured out how to cut 1 week off the process of getting new users accounts in a pretty easy first step and identified a path to cutting the time to issue new accounts even further.
2) We identified a very small set of information (persistent unique identifier and (maybe) citizenship) as the required set for gatewayed users. [editor's note, the verb "to gateway" here refers to obtaining TeraGrid access via a Science Gateway... it is usually a good sign when the proper name for a project gets verbed]
3) We designed a testbed that would enable users to use their Shibboleth credentials from home institutions to generate credentials that would work on TeraGrid. They would not have to retain a persistent x509 environment on their workstations, though for some usage modes, they would have to use short-lived proxies put into a local Globus environment.

From my point of view it was tremendous to see about 35 participants working together from TeraGrid sites as well as partner organizations such as the Globus Alliance and the Internet2 Shibboleth project. We had experts in security, accounting, grid software development, and identity management constructively grappling with this important set of issues together. The event was a nice example of why you get on an airplane and travel to a workshop - to make progress about 50 times faster than exchanging email and position papers! Having made this investment, we are ready to take the next concrete steps to make this vision a reality.

Improving security and usability at the same time. How often do you get a chance to do that?

Trying out EC2

2006-08-28T19:09:00.000-05:00

About half-way through a Monday I decided to stretch my legs a bit and see how long it would take to get EC2 up and running (I got into the beta). Pete Beckman and I sat down in a cafe and found a very nice Exploring Amazon EC2 primer by Jesse Andrews which made it quite easy. The Amazon docs are also pretty good.

Once you sign up for the service (you have to already be an S3 customer, which I am) you generate your X.509 keypair and download it to your linux system (which in my case is a Mac). Amazon provides a set of command line API's that you download and unzip. You set up some environment variables and voila, you can use the command line utilities. The first thing we did was to query to find pre-made system images (rather than trying to build our own, which would take longer). You generate a keypair to control your "instance" (virtual machine), select one of these pre-made system images, and fire it up.

We found that within 3-4 minutes the instance was up and running, and we could SSH into it and begin to play. This is not particularly impressive as hacking code goes, but it seems to me that being able to sign up for a web services based computation service and get something up and running within an hour is quite nice!

So I just fired up an instance and I'm running a web server on it. It's costing me $2.40 a day so I'm not gonna leave it up for long, but if you are reading this post within a few days of the timestamp try out my Amazon EC2 virtual webserver! (too late- I left it up for about a month but it's gone now!)

Compute Services - Now we're talking!

2006-08-25T14:23:00.000-05:00

A while back IBM started an On Demand business that seems ideally tuned to companies that need a cluster (even a large one) periodically for surge capacity. It's aimed at businesses, and those kind of things involve a fair amount of initial investment of time and energy to get the paperwork set up, etc. To make this, and the cost, worth doing you need to have a fair amount of computing surge (and plenty of companies have this!).

More recently, Sun took this idea of a grid-based compute service further, launching what they called a "Utility Computing" service called Sun Grid. Nice. An individual can sign up and get an account, and in short order start uploading and running (Solaris, x64) applications from their portal. No lengthy setup. Currently Sun charges $1 per CPU hour.

Amazon's rolling out an even more flexible grid computing service. No lengthy setup process, and forget the portal- use web services. Yesterday Amazon announced a beta version of Elastic Compute Cloud, or EC2. Once available beyond beta, one will be able to just sign up and start computing via a SOAP API. It’s coupled with the S3 service, where you store your system images (which include your entire application software environment for your virtual machines) and data. Since it’s a virtual machine, you get root access and it's up to you what software you want to run. The virtual machines are the equivalent of a 1.75 GHz Xeon with 1.75 GB of memory, 160 GB of local disk, and network bandwidth of 250 Mbit/s. Amazon’s pricing - $0.10 per CPU hour.

As I mentioned on Monday (talking about Sharing Data) it seems to me that these commercial grid resources and services are worth examining in terms of support for scientific computing. I’m hoping the TeraGrid Science Gateway partners will jump on these and check them out!

Non-Sequitur of the Week: The World's Oldest Botanical Garden.
Earlier this summer we were in Italy on vacation and one of our stops was Padua, near Venice. Besids being the site of one of the worlds' oldest universities (University of Padova, founded in 1222), it's a nice base of operations to explore the area around and including Venice. Strolling around the city we chanced upon the Orto Botanico di Padova (Padua Botanical Gardens), which was magnificent. What we didn't know at the time was that it is the world's oldest, having been established in 1545! If you don't happen to be in Italy there are of course other fine botanical gardens we'd recommend, including the UC Berkeley Botanical Gardens, the Royal Botanical Gardens in Edinburgh, Scotland, and of course the Chicago Botanic Garden! (if you have recommendations please post them in comments)

Have you joined Facebook yet?

Social Networks...

2006-08-22T15:38:00.000-05:00

Last week I joined Facebook, which is an amazing and exploding social network for university and high-school students, staff, and faculty. I've been trying to figure out how to support and catalyze scientific communities for several years, including designing and launching the GridForge for the Global Grid Forum (now called Open Grid Forum, or OGF). GridForge uses the same underlying platform as Sourceforge.net, Slashdot, and a good number of other sites. But while this seems to be a great platform for supporting individual groups developing things (like software or documents) together, they are not geared toward finding and mapping social networks (i.e. finding colleagues or discovering potential collaborators with overlapping interests, etc.).

As I have explored Facebook I've kept the question in the back of my mind - could this be a useful tool for the scientific community as well? With some minor enhancements I think it very well could be. Already it is useful to see connections between people, find people, and even manage events.

I've searched for colleagues and I find that it seems perhaps many of us have become geezers with respect to new technology like this - most of the 7 million participants are college students or younger. I did find that some of us do not in fact believe that its hip to be square. I was happy to see Larry Smarr, Ian Foster, and Beth Plale were already in the mix.

It would be great if more people from our community (TeraGrid staff, users, collaborators...) joined Facebook to see if it might be a useful tool for our community.

See you there!

Sharing data

2006-08-21T11:59:00.000-05:00

I've been having lots of discussions with people about data. We typically focus on big data in TeraGrid but it's also important to note that many scientists have "small" volume and performance requirements (relative to supercomputers). Because their requirements are more closely matched to "commodity" solutions, we have more options with which we might help them. For example, users with only a small amount of data (measured in hundreds of MBytes, roughly speaking) could take advantage of a web services approach such as using Amazon's S3.

A web services approach also offers some advantage over solutions geared toward ultra-large data and ultra-high bandwidth, in that it becomes easier to share the data, which is increasingly important to scientific communities. Our science gateways partners, I believe, would be much more eager to leverage a web services storage resource than one that requires them to integrate with a traditional (in our circles) high-performance filesystem or storage system.

We certainly need to continue to provide, and enhance the high-performance solutions! (we are also pushing toward petaflops computing, which will stretch these systems considerably). Work we are doing, for instance, with the GPFS-WAN environment and with high-performance data movement tools such as GridFTP is very critical. But it's equally important to look at solutions that allow us to better integrate with "web 2.0" and emerging commercial services. As I've spoken over the past weeks to Jim Gray from Microsoft (8/16 post), Vint Cerf from Google (8/7 post), and today to Burton Smith from Microsoft it's quite clear that we should be collaborating in these areas with some amount of enthusiasm. These companies are moving fast, and in the right direction with respect to support for scientific communities. We have a tremendous amount of synergy with companies like Microsoft, Google, Amazon, and others where our customer base overlaps considerably with theirs.

On other fronts...

Ian Foster's blog notes that today is the 10th anniversay of the Globus software project in that on this date in 1996 DARPA awarded funding to develop the open source toolkit.

While we're being nostalgic.... Twenty years ago this year we deployed the NSFNET backbone, which used six supercomputer centers as connecting points for regional and campus networks. Five of those centers (NCSA, PSC, NCAR, SDSC, and Cornell Theory Center) are connected today with 1 or more 10 Gb/s links as part of the TeraGrid network. Much improved over the 56 Kbit/s links from 1986...

Science Gateways - Web Services We'd Like to See!

2006-08-18T15:20:00.000-05:00

I've been talking with Nancy Wilkins-Diehr, who heads up the TeraGrid Science Gateways initiative, about the common requirements we are seeing bubble to the surface as now well over 20 gateways (most of them could be called "eScience Portals") are working with us to "plug" in grid resources to these gateways. Nancy's list is very interesting, and it would be good to figure out which of these might already exist (and could be adopted), which we should commission to be developed, etc.

* Resource Status Service (both polling and pub/sub)
* Job Submission Interface (The gateways expect this to be provided by Globus WS-GRAM)
* Job Tracking Interface (Both polling and pub/sub)
* File/Data Staging Interface
* Retrieve Usage Information
* Retrieve Inca Info
* Advanced Reservation Interface
* Retrieve user information for a job
* Retrieve accounting information/statistics

These are the computational-centric services, but there are many other science gateway services, particularly collaboration and social networking tools, that many gateways are providing, and we would do well to look at what we could adopt from commercial providers of "Web 2.0" capabilities! (some of our gateway partners are doing this already)

Non-sequitur of the week:I just bought four tickets to see one of my favorite bands, Jars of Clay at the House of Blues in October. I especially like this band's leadership in solving real world problems though initiatives like the 1000 Wells project, which is in its second year with partners like Africare and has catalyzed lots of local efforts - for example at UC Davis.

Conversations about Data Analysis

2006-08-16T07:16:00.000-05:00

Monday I caught up with Jim Gray from Microsoft Bay Area Research Center to talk about web services, storage, and data. He's been working with the NVO and Sloan Digital Sky Survey groups, who obviously have very large data requirements. One angle they are looking at is providing analysis tools at the data archive, rather than the traditional mode of downloading data and analyzing it locally. An example of this kind of service is the Catalog Archive Server.

On a similar topic I met with the Interagency Modelling Analysis Group (IMAG) yesterday to talk about their requirements and how they might use TeraGrid. A message I heard from this group is that there are many scientists who are able to do their computational work locally, and may or may not need to scale their work to use TeraGrid. However, where TeraGrid could really help them would be in providing analysis and visualization services. I mentioned to them that we have a variety of these types of services at the TeraGrid resource provider sites and that the TACC and UC/ANL resource providers have deployed dedicated resources on which they are operating visualization services for the user community. As a next step we talked about having someone talk to the IMAG about TeraGrid visualizaion services and how these compare to the needs of this group of scientists.

CTSS V4 - Draft spec for the Workflow "kit"

2006-08-14T11:12:00.000-05:00

As mentioned earlier, we are moving toward an architecture where TeraGrid resources support (a) a core set of services, plus (b) one or more "kits" that provide specific functionality and services. One of the first kits we are discussing is the set of services and software necessary to support workflow on a resource. Lee Liming provides the following overview of the draft:

The purpose of defining this kit is not to redesign TeraGrid's capabilities in this area. Instead, the purpose is to make sure that we understand and document what the current capabilities are so that we have a solid baseline for subsequent changes. (Along the way, we can use it to improve the user documentation for the existing capabilities.) The "Future directions" section outlines what we currently know about likely subsequent changes. If there are any simple, non-controversial, beneficial, easy-to-implement enhancements that can be identified now, this would be a reasonable time to make them, so please speak up if you see something like this. Anything else (non-trivial, controversial, difficult to implement) will probably be put off until the next design cycle. HOWEVER, this would be a reasonable time to identify those things so that we can begin planning for them.

The draft is available for download from repo.teragrid.org and we are eager to see discussion as this moves forward! (note that this online draft is currently only available to TeraGrid participants, but I'd be happy to email to you if you're interested but not a participant)

...writing from

A Roadmap to Attribute-Based Authorization

2006-08-12T00:12:00.000-05:00

At the end of August we're holding an internal TeraGrid workshop to look at a number of important issues related to authorization, audit, accounting, and security. One of the things (I've mentioned in a few other posts) we will be talking about is attribute-based authorization. Von Welch (GridShib project) has been editing a paper, Scaling TeraGrid Access: A Roadmap for Attribute-based Authorization for a Large Cyberinfrastructure with several of us helping as co-authors. It is worth looking at even in draft format. Comments more than welcome!

CeC

week-ending at...

Non-sequitur of the week: One of the best places to rock-climb in the (OK, flat) midwest is Upper Limits. They've converted some old grain silos (it's the midwest...) in Bloomington, Illinois into a rock climbing gym (they have another gym in St. Louis). A few dozen 60 foot top-roped routes inside several silos, good bouldering inside and outside, and a couple of 100 foot routes up the side of the building outside. (I'd provide a Google Earth pointer but it's under a cloud....)

Dividing and Conquering - CTSS V4

2006-08-09T14:10:00.000-05:00

In the first instantiation of TeraGrid (nearly 5 years ago) we defined a set of software and services that would be run on all TeraGrid platforms. The purposes were to (a) provide users with a consistent experience on all TeraGrid systems and (b) integrate those systems with important functions such as single-signon, remote job submission, system-wide accounting, etc.

Affectionately named after the early Cray Time Sharing System (operating system), the Coordinated TeraGrid Software and Services (CTSS) v1 had many more components than was necessary, as we determined after users began to arrive, so CTSS v2 was more streamlined. A few months ago we moved to and even more streamlined CTSS v3 and we're still doing a post-mortem regarding how the transition went and how we can continue to improve on smooth evolution of the TeraGrid software environment. But the challenge of this software stack approach is moderate when you have a large number of resources, and significantly harder as those resources become more diverse.

As we have been developing plans for CTSS v4 we are evolving our approach. V4 will consist of a set of required core services (authorization, information services, verification and validation, and accounting/audit) along with an optional set of "kits" that provide specific functionality (workflow support, program development, job execution, etc.). Each TeraGrid resource provider will run the core components and will select the set of kits they will support on each of their systems. Some may elect only to support job execution and workflow. Others may also support the common program development environment. We are initially looking at a total of eight discrete kits, and these will be fleshed out over the next few months.

This approach moves us toward what I believe is an achievable, though challenging, goal. I'd like for CTSS V5 to be a set of service descriptions, leaving the implementation of those services at the discretion of the resource provider. Certainly we will continue to integrate and package software that can be used to implement TeraGrid CTSS, but the use of our implementations should not be necessary for a resource to interoperate within TeraGrid.

This is an approach that I think can work where resources reside in different Grid facilities as well. My view is that a science gateway user should be able to put together a workflow that harnesses resources in TeraGrid or other Grids, with the only key consideration being that the user is authorized to use those resources.

CTSS v4 will be an important move for the TeraGrid community, as the benefits to this modularization can only be harnessed by rigorous change management and a commitment to providing the tools that users and science gateway partners need in order to navigate resources with different combinations of services. We're starting with a careful evaluation of how the CTSS v3 transition has gone, and we will be engaging both resource providers and end users as we begin to specify the first set of kits.

If you're interested in helping out, pleaase contact Lee Liming, who is leading our CTSS v4 planning efforts.

Writing to you from...

Conversation with Vint Cerf

2006-08-07T19:49:00.000-05:00

I had breakfast with Vint Cerf this morning where we spent about 90 minutes talking about common interests Google and TeraGrid might have. There are many! We talked about Web Services and the Science Gateway initiative as well as how people are using TeraGrid today. We'll probably follow up with me giving a talk about TeraGrid out at Google and some introductions with the relevant technical people at Google to pursue further discussions.

Vint also gave the keynote today at the EDUCAUSE Snowmass meeting, His talks (which lately he is doing without slides) are fast-moving intellectual tours through the spaces where technology, policy, applications and humans mix. He covered a variety of very interesting topics, for example offering insight into issues including net neutrality, the need for improved security in applications and operating systems, and some of the challenges to effective search (both of Internet and other content). He closed, of course, with an update on the Interplanetary Internet. As always, a very fun and stimulating talk!

Writing to you from...

Campus Partnerships

2006-08-05T13:18:00.000-05:00

This morning I gave the opening talk at the Campus Cyberinfrastructure Workshop here in Snowmass, Colorado. It was a good opportunity to talk with leaders from the roughly 40 universities here about how TeraGrid can partner with campuses on key issues of Cyberinfrastructure. We've been looking at the following programs, each of which we want to flesh out with campus partners over the next several months:

- Cooperation in authorization infrastructure (wouldn't it be great for a campus user, assuming strong identity management and authentication on campus, to use his or her campus credentials to access TeraGrid...)

- Cooperative computational and storage/data management services (integration of capability resources such as TeraGrid with capacity resources on campuses)

- Federation of digital assets (data collections, etc.)

- An "affiliates" program for outreach beyond R1 institutions and for coordinating support for TeraGrid users, enabling campus staff to work with TeraGrid staff to support those users.

- Education and Training partnership to develop and harvest best-of-breed curricula and programs aimed at not only the next generation workforce (k-20) but continuing education for today's workforce.

Gary Bertoline (bertoline@purdue.edu) and Scott Lathrop (lathrop@mcs.anl.gov) are organizing these discussions - and always looking for more participants in those discussions.

(my location via Google Earth)

NSF draft Cyberinfrastructure Plans (v7)

2006-08-05T13:01:00.000-05:00

I was thrilled to see that NSF posted a new version of their Cyberinfrastructure Vision document. The last draft (v5) was posted in January, but this new draft (v7) is more complete and incorporates suggestions and changes from the community.

This is really worth downloading and reading, and it's still a draft- they are eager for input from the community so give it to them (they are listening!).

You can find the report at the NSF Office of Cyberinfrastructure website. The document is called "CI Vision" version 7.1. While you're at the OCI website you might also check out the "Report of Blue-Ribbon Advisory Panel on Cyberinfrastructure" (also called the Atkins report) which is still quite relevant 3.5 years after it was published.

(my location via Google Earth)

PKI and Multi-level Authentication

2006-08-04T20:10:00.000-05:00

I'm at an Educause confab in Snowmass this weekend. Today was the first day of a Public Key Infrastructure summit of the Identity Management Working Group. The PKI meeting today consisted of some interesting case studies from several universities and companies. Universities, particularly large ones (with 10's of thousands of students, faculty, and staff), have a significant challenge with scale (and thus cost!).

I was particularly impressed with the work done at the University of Wisconsin in evaluating various approaches to PKI and devices, such as USB keys, to hold credentials. (this was recently outlined in detail via an Educause online seminar. They looked at initial and 10-year costs of developing their own system from open source, developing their own system with commercial pieces, or partnering with key vendors to provide them with a solution. It turned out, when taking into account staff investment, that the vendor-partner solution was the most cost effective up front and over ten years.... and involved the shortest time to deploy a solution. The UW PKI Lab as well as collaborators at Dartmouth have done a good amount of investigation over the past few years in this area.

Given the difficulty in getting staff, faculty and students to buy into an extra device that they carry around, it was suggested the ideally the PKI vendors might consider developing toward the use of cel phones and iPods to hold credentials. (I thought this was a cool idea)

There were also overviews of Apple's approach to PKI and a presentation from Aladdin on the scale of the problem of identity theft in universities and labs. One of the innovative tools we are looking at within TeraGrid was discussed - the myVocs system from the UAB Advanced Technology Lab.

myVocs is a good example, along with Shibboleth and the Gridshib project, of technology that TeraGrid can leverage in a two-way partnership with campuses. This is, after all, where most of our users live and so we are seeking ways to lower barriers to their use of TeraGrid while (just as importantly) improving the security of our systems.

Tomorrow is a workshop where about 40 universities will be talking about how to work together to create cyberinfrastructure. More on that later this weekend.

(my location via Google Earth)

Outreach to Educators

2006-08-03T13:40:00.000-05:00

This week TeraGrid partnered with NCSA and the Shodor Education Foundation to hold a 5-day workshop at Chicago State University. There were about 30 faculty from Chicago State as well as local community colleges and local K-12 schools. The hands-on workshop exposed these faculty to interactive tools they can use to teach students about computational science. In addition to high-performance resources such as provided by TeraGrid, the participants learned about a range of computational science education resources available through the National Science Digital Library.

This is a neat example of the kind of workshops that are happening around the country with TeraGrid resource providers and other partners - as you can see at the TeraGrid Education, Outreach, and Training website. We are leveraging the annual high-performance computing SCxx conferences to further strengthen the nation's ability to prepare the next generation of our workforce. Purdue is leading the SC06 education program and TeraGrid will be coordinating the education programs at SC07, and SC08. Last week Scott Lathrop, TeraGrid's director of education, kicked off a planning meeting held at Argonne National Laboratory with 40 education leaders from almost as many organizations.

(my location via Google Earth)

Bootstrapping

2006-08-01T11:05:00.000-05:00

If you are involved in the National Science Foundation's TeraGrid project you know that we are just about a year into "bootstrapping" a virtual organization to operate and enhance the virtual facility we built over the past few years, serving a set of communities comprised of several thousand scientists and educators. We have many mechanisms for communication but most of it is rather one-way, which has its place of course.

Here I'd like to have a dialog about issues of urgency and/or importance to the TeraGrid community, including "internal" staff as well as our user community.

During TeraGrid'06 last month I talked about our overall mission and our strategies, and I'd like to note them here as a way of kicking off this blog.

What are we trying to accomplish with the TeraGrid facility? Our mission is to "create integrated, persistent, and pioneering computational resources that will significantly improve our nation’s ability and capacity to gain new insights into our most challenging research questions and societal problems. " In order to pursue this mission, we take an integrated approach to the scientific workflow including obtaining access, application development and execution, data analysis, collaboration and data management.

That integrated approach currently involves three organizing principles, or areas of focus:

DEEP- To ensure that scientists can exploit the enormous power of the TeraGrid resources (currently well over 100 TF in aggregate) as an integrated system we have staff at resource provider sites who work shoulder-to-shoulder with science teams. This program, "Advanced Support for TeraGrid Applications (ASTA), assigns 1/4 to 1/2 of a support person in a scientific application team for 4 to 12 months, working with that team to harness TeraGrid capabilities of particular importance to their scientific goals. The ASTA program supports roughly a dozen teams at a time, with a goal of 20-25 teams supported per year. We are exceeding this goal, and many of their stories are featured at the main TeraGrid website.

WIDE- For over 2 decades the NSF high performance computing program, including TeraGrid, has served several thousand users very effectively. However, NSF alone funds tens of thousands of scientists, most of whom have computational requirements that do not frequently require supercomputers. The TeraGrid Science Gateways program is a set of partnerships with discipline-specific teams who are providing computational infrastructure (in most cases either a specialized web portal or a community-organized grid) for their science communities. The partnerships involve integrating TeraGrid as a computational and data management “service provider” embedded in the science-community cyberinfrastructure. We have over twenty such gateway partners working with us today.

OPEN- TeraGrid began (in 2001) as an infrastructure involving four partner sites, grew to nine sites, and is currently organized as a set of resource providers and a core “grid infrastructure group” (GIG) that coordinates and provides common software services and support as well as planning, architecture, management and operations. TeraGrid software and services architecture is service-oriented, stressing open source standards such as are deployed with key software including the Globus Toolkit GT4, Condor, and other tools.

The TeraGrid organization is also open in that we anticipate more resource providers over time. In order to partner with the broader community of universities and other service and resource providers, we are working with colleagues in Educause, Internet2, EPIC, and other communities to design, together, a set of “campus partnerships” during 2006. The goal of these partnership programs is to work with campuses (where most TeraGrid users reside) to improve and streamline TeraGrid access from campus systems, as well as to work with campuses to develop a set of frameworks that can be used to create national-scale cyberinfrastructure for computation and for data federation. These programs will explicitly reach beyond the R1 institutions to include the broader R&E community.

That's a lot for one blog post... I'm interested in your comments on any of these topics and will be diving into them over the next few weeks in a more focused way, taking one or two projects at a time.

(my location via Google Earth)