Scalable I/O: Applications

• Argonne National Labs has an SIO application suite which includes the following:
  • Astrophysics
  • Weather Modeling (MPMM)
  • Climate Modeling (PCCM2)
  • Parallel Rendering
  • Seismic
  • Shallow Water (PSTSWM)
  • Electronic Structure
  • Hartree-Fock
  • LU Factorization
  • CFS3D
• We currently have an FFT program capable of acting on a very large dataset.
  
  

Fast Fourier Transform (FFT)

We have developped a one dimensional FFT application. It is based on the algorithm descibed in "FFTs in External or Heirarchical Memory" by David H. Bailey. The algorithm consists of four steps. The data set is conceptual turned into a two dimensional n₁ by n₂ matrix. Each section of length n₁ becomes a new row.

1. Perform n₁ FFTs of length n₂ on the rows of the matrix.
2. Scale by multiplying each element A_j,k by e^2*pi*i*j*k/n.
3. Traspose the matrix.
4. Perform n₂ FFTs of length n₁ on the rows of the matrix.

An inverse FFT can be performed by substituting IFFT for the FFTs, and negating the exponent in step 2.

In this implementation, steps 2 and 3 have been merged to reduce the number of passes through the data. This procedure allows smaller sections of data to be operated on than the traditional FFT algorithms. It is applied recusively until the amount of memory necessary to perform each individual FFT in steps 1 and 4 is below a user defined limit.

Future development:

• Multi-processor version using a shared global queue and shared access to SIO Parallel File System.