When utilizing GPFS with Oracle, configuration and tuning considerations include:

  • When setting up your LUNS it is important to:
    1. Create the logical volumes such that they map one to one with a volume group and a volume group be one to one with a LUN which is a single RAID device.
    2. Not stripe logical volumes across multiple RAIDs for I/O parallelism when using raw logical volumes because:

    – GPFS puts each block of a file on different LUNs spread across all LUNs
    – Logical volume striping makes removing a bad RAID more difficult

  • For file systems holding large Oracle databases, set the GPFS file system block size through the mmcrfs command using the -B option, to a large value:
    1. 512 KB is generally suggested.
    2. 256 KB is suggested if there is activity other than Oracle using the file system and many small files exist which are not in the database.
    3. 1 MB is suggested for file systems 100 TB or larger.

    The large block size makes the allocation of space for the databases manageable and has no affect on performance when Oracle is using the Asynchronous I/O (AIO) and Direct I/O (DIO) features of AIX.

  • Set the GPFS worker threads through the mmchconfig – worker1Threads command to allow the maximum parallelism of the Oracle AIO threads:

On a 64-bit AIX kernel, the setting can be as large as 548.

  • The GPFS prefetch threads must be adjusted accordingly through the mmchconfig – prefetchThreads command as the sum of those two classes of threads must be 550 or less.On a 32-bit kernel, the setting can be as large as 162.
  • The GPFS prefetch threads must be adjusted accordingly through the mmchconfig – prefetchThreads command as the sum of those two classes of threads must be 164 or less.When requiring GPFS sequential I/O, set the prefetch threads between 50 and 100 (the default is 64), and set the worker threads to have the remainder.
    Note:
    These changes through the mmchconfig command take effect upon restart of the GPFS daemon.
  • The number of AIX AIO kprocs to create should be approximately the same as the GPFS worker1Threads setting.
  • The AIX AIO maxservers setting is the number of kprocs PER CPU. It is suggested to set is slightly larger than worker1Threads divided by the number of CPUs. For example if worker1Threads is set to 500 on a 32-way SMP, set maxservers to 20.
  • Set the Oracle database block size equal to the LUN segment size or a multiple of the LUN pdisk segment size.
  • Set the Oracle read-ahead value to prefetch one or two full GPFS blocks. For example, if your GPFS block size is 512 KB, set the Oracle blocks to either 32 or 64 16 KB blocks.
  • Do not use the dio option on the mount command as this forces DIO when accessing all files. Oracle automatically uses DIO to open database files on GPFS.
  • There are fixes in the GPFS PTF stream necessary for correct AIO/DIO operation for Oracle.
PrintFriendly

Viewers of this post also viewed...