Task Parallelism and Fortran

The task parallelism group at Argonne National Laboratory investigates applications, language constructs, compilation, and runtime system issues associated with the use of task parallelism, and mixed task/data parallelism. Much of this work is performed in the context of Fortran, but many of the techniques apply equally to other languages.

HPF/MPI: An HPF Binding for MPI

High Performance Fortran (HPF) provides a convenient syntax for specifying data-parallel execution, but does not at present allow programmers to write task-parallel programs in which different tasks perform different activities at the same time. As a solution to this problem, we advocate the development of an HPF binding for the Message Passing Interface (``HPF/MPI''). This provides a standard set of functions for coupling multiple HPF tasks to form task-parallel computations.

Working with Rakesh Krishnaiyer and Alok Choudhary at Syracuse University, we have developed a prototype implementation of an HPF/MPI library. This is a nontrivial exercise because (for example) sending and receiving tasks may employ different data distributions, and execute on different numbers of processors. Our prototype uses the HPF compiler produced by the Portland Group, Inc., and was demonstrated at the Supercomputing 95 conference.

In future work, we plan to use the HPF/MPI prototype to investigate algorithms for high-performance communication in mixed task/data-parallel computations. We also plan to investigate other libraries that implement other communication mechanisms, such as Fortran M channels, and to compare library-based and language-based approaches to task-parallel computation.

Four papers are available:

• Double Standards: Bringing Task Parallelism to HPF via the Message Passing Interface. I. Foster, D. Kohr, R. Krishnaiyer, and A. Choudhary, Proc. Supercomputing '96. Compressed Postscript and HTML. Motivates the HPF/MPI approach, describes the HPF/MPI implementation, and presents microbenchmark and application performance results.
• A Library-Based Approach to Task Parallelism in a Data-Parallel Language. I. Foster, D. Kohr, R. Krishnaiyer, and A. Choudhary, submitted to the Journal of Parallel and Distributed Computing. Compressed Postscript. This paper is an expanded, updated version of the Double Standards paper that was presented at SC96. It describes new implementation techniques and contains improved performance results that reflect the use of these techniques.
• MPI as a Coordination Layer for Communicating HPF Tasks. I. Foster, D. Kohr, R. Krishnaiyer, and A. Choudhary, Proc. 1996 MPI Developers Conference. Compressed Postscript and HTML. Describes the HPF/MPI implementation from an MPI perspective, presents microbenchmark and application performance results, and discusses how various MPI features can be realized in an HPF setting.
• Design and Optimization of Coordination Mechanisms for Data-parallel Tasks. D. Kohr, Master's thesis, Dept. of Computer Science, University of Illinois at Urbana-Champaign, Oct. 1996. Compressed Postscript. Discusses the design of HPF/MPI in depth, analyzes in detail communication performance measured using microbenchmarks, and presents an analytic model (with empirical validation) for predicting communication overhead among data-parallel tasks.

The HPF/MPI work is referenced in a survey of HPF tools.

Go to the HPF/MPI Developer's Page. (Only accessible to project members.)

Fortran M

Fortran~M is a small set of extensions to Fortran that supports a modular approach to the construction of sequential and parallel programs.  A portable Fortran~M compiler was written, based on the Nexus runtime system.

We are no longer developing or supporting Fortran M.  The source and documentation are still available for reference.

For More Information

• About FM
• FM Applications
• Integrating Task and Data Parallelism
• Obtaining Software and Documentation

Online Documentation

• The book, Designing and Building Parallel Programs
• Programming in Fortran M (manual)
• Slide Shows
• Papers and Reports

Contacts

• Staff
• Please send questions or comments to fortran-m@mcs.anl.gov .

Related Work

• CC++ has similar design goals and will be interoperable with Fortran M
• Nexus is used to implement Fortran M

FM is a joint project of Argonne National Laboratory and the NSF Center for Research on Parallel Computation