 |
|
|
|
David Levine
Argonne National Laboratory
Computational electromagnetics is a widely used technique in industrial, research, and defense applications. For the past twenty years most electromagnetic field computations have been done using programs based on the finite element method and run on sequential computers. However, limitations of accuracy, problem size, and solution time make further advances using this approach more and more difficult. We believe an approach that combines integral equation methods and massively parallel computers can address many of the current limitations and provide timely and accurate solutions to large, computationally demanding problems.
To extend the range of problems that may be solved by computational electromagnetics, we developed code based on integral equation methods. Integral methods offer several advantages. First, only the active regions need to be discretized. Therefore, in a problem with motion, there is no need to adjust the mesh in the region connecting the moving and stationary components, as that region is not meshed. This is a major advantage when the domain is not connected. Also, not meshing the air region saves considerable time in setting up the problem and eliminates many--often most--elements.
Second, far-field boundary conditions are automatically taken into account. There is no danger of error from taking the boundaries too close, nor is there a need to find how far away to place them by trial and error. Even a crude integral model will give a good estimate of the fringing field.
Third, fields in the air region show smooth and realistic variation. Variation is not determined by the order or geometry of the mesh, as it is for finite-element codes. This is a big advantage when the variation must be known very accurately, as in MRI or accelerator magnets.
Fourth, for eddy current problems there is no need to keep track of which elements are in motion and which are not, since the mesh is fixed to the region of interest. Conversely, with the FEM it is necessary to keep track of which elements connect moving and stationary elements--a task that can be quite difficult.
Finally, integral methods readily lend themselves to parallel processing. Both the evaluation of the different matrix elements and the determination of the field at different points after the problem is solved can be done completely in parallel. Indeed, several methods have already been developed to efficiently solve dense systems of linear equations on parallel computers.
The integral code we developed is called CORAL. CORAL runs on the IBM SP and can solve three-dimensional, nonlinear magnetostatic problems. A key component of CORAL is the use of the Chameleon and PSLES components of the PETSc (Portable and Extensible Tools for Scientific computing) library. These tools provide parallel portability and access to a variety of parallel linear system solvers to solve the large, dense systems of linear equations that are inherent in the use of integral equation methods.
Several special features of the linear systems arise in CORAL. First, the matrix is asymmetric. Second, each system of linear equations arises from an outer nonlinear problem and so may need only a relatively low accuracy solution. Third, the actual matrix, while dense, has many ``small'' elements. Finally, the size of the matrices to be solved varies significantly according to the mesh refinement and desired solution accuracy.
We compared direct methods for the solution of the linear systems with iterative methods. Two possible advantages of iterative methods are that to solve a nonlinear problem, one must solve a related sequence of linear systems. With iterative methods, we would be able to use the solution to one of the linear systems in the sequence as the starting solution to the next linear system in the sequence. This approach can considerably reduce the number of iterations required. Second, for some nonlinear problems it may not be necessary to solve the early linear systems in the sequence to high accuracy. Unlike direct methods, iterative methods allow an early exit from the solution procedure with an approximate solution.
Our sequential tests showed that iterative methods with appropriate preconditioners can outperform direct factorization, even when the underlying problem is a dense matrix. We found the generalized minimal residual (GMRES) iterative solver, using either block diagonal preconditioning (BDD) or a band matrix preconditioner, was more effective than LU factorization. Straighforward attempts to parallelize the BDD preconditioner using a block per processor, independent of the matrix size, however, do not scale well to large numbers of processors. We found it necessary to develop a parallel version of BDD where the number of BDD blocks is independent of the number of processors.
We tested the accuracy of CORAL's calculations on several problems from the standard TEAM (Testing Electromagnetic Analysis Methods) benchmark suite and on magnet designs for the Advanced Photon Source at Argonne National Laboratory. Our numerical results showed that CORAL can compute as good, or better solutions, than traditional methods. We have found, however, the dense equation matrix and the large amounts of data one has to store in the memory are inherent problems related to integral equations. For this reason, the IBM SP parallel computer with 128 MB of real memory and a 1 GB disk per node has been a key to our success.
Several significant conclusions result from our work:
Our results are of interest to researchers involved in electromagnetic field computation in a wide variety of fields including design of magnetic disks, semiconductors, loudspeakers, motors, and magnetic resonance imaging systems; nondestructive testing; oil exploration; and the development of maglev vehicles and particle accelerators.
Our research is evolving in two directions. First, based on the tests we have carried out, we find that in the case of static magnetic fields integral equations are competitive with traditional methods. However, our ultimate goal is time-dependent problems with moving objects. The fact that one need not discretize air and that exterior boundary conditions are automatically incorporated offers significant advantages compared to finite-element methods.
Second, we are exploring virtual reality visualization of the resulting magnetic fields using the CAVE (CAVE Automatic Virtual Environment) virtual reality environment. The CAVE environment allows us to explore and interact with the 3-D visualization of our results. This is particularly important in the three-dimensional case where visualizing the computational mesh and the results of the electromagnetic calculations is a difficult task.
Volume integral equations in nonlinear 3d magnetostatics by L. Kettunen, K. Forsman, D. Levine, and W. Gropp, accepted for publication in International Journal of Numerical Methods in Engineering.
Virtual reality visualization of accelerator magnets by M. Huang, L. Kettunen, D. Levine, M. Papka, L. Turner, and T. DeFanti, Proceedings of the High Performance Computing Multiconference, 1995.
Computational electromagnetics and parallel dense matrix computations by K. Forsman, W. Gropp, L. Kettunen, and D. Levine, accepted for Proceedings of the SIAM Parallel Processing for Scientific Computing conference, 1995.
Solutions of TEAM Problems 13 and 20 using a volume integral formulation by L. Kettunen, K. Forsman, D. Levine, and W. Gropp, Proceedings of Aix-les-Bains TEAM workshop, 1994.
Solutions of TEAM Problem 13 using integral equations in a sequential and parallel computing environment by L. Kettunen, K. Forsman, D. Levine, and W. Gropp, Proceedings of the Fourth International TEAM workshop, 1994.
Further description and illustrations
| [ Account Request | Quad | Denali | Yukon | Tundra | ADSM | Announcements | CCST | MCS ] | ||
|
