How To Check if Asynchronous I/O is Working On Linux (Doc ID 237299.1)

 

Caveat for ASMLib

---

APPLIES TO:

Linux OS - Version Enterprise Linux 3.0 to Oracle Linux 6.5 with Unbreakable Enterprise Kernel [3.8.13] [Release RHEL3 to OL6U5]
Oracle Database - Enterprise Edition - Version 9.0.1.0 to 12.1.0.2 [Release 9.0.1 to 12.1]
Oracle Database Cloud Schema Service - Version N/A and later
Oracle Database Exadata Express Cloud Service - Version N/A and later
Gen 1 Exadata Cloud at Customer (Oracle Exadata Database Cloud Machine) - Version N/A and later
Linux x86
Linux x86-64
**Checked for relevance on 10-Nov-2010***

GOAL

In this document we are going to explain how to check that asynchronous I/O (AIO) is working. AIO can be enabled in Oracle 9i 9.2 and higher.

Many times there is a requirement to check if Asynchronous I/O is working on Linux Platform, so we can try to use it for our datafiles access inside database.

SOLUTION

The slabinfo maintains statistics about objects in memory. Some of the structs used by Asynchronous I/O are treated as objects in the virtual memory, so we can look for those structs on slabinfo. The ones related to AIO are named kio*.

$ cat /proc/slabinfo | grep kio

Since Oracle Linux provides different kernels (including RedHat Compatiable Kernel, Oracle UEK1/UEK2/UEK3), the output could be slightly different.

If Async I/O is enabled

$ cat /proc/slabinfo | grep kio
kioctx 270 270 128 9 9 1 : 252 126
kiocb 66080 66080 96 1652 1652 1 : 252 126
kiobuf 236 236 64 4 4 1 : 252 126

or

kioctx 518 670 384 10 1 : tunables 54 27 8 : slabdata 67 67 0

kiocb 595 720 256 15 1 : tunables 120 60 8 : slabdata 48 48 60 

and if Async I/O is disabled or not in using:

$ cat /proc/slabinfo | grep kio
kioctx 0 0 128 0 0 1 : 252 126
kiocb 0 0 96 0 0 1 : 252 126
kiobuf 0 0 64 0 0 1 : 252 126

or

kioctx 0 0 384 10 1 : tunables 54 27 8 : slabdata 0 0 0
kiocb 0 0 256 15 1 : tunables 120 60 8 : slabdata 0 0 0

In UEK4 the slabinfo entries for aio are no longer available. The caches used by the aio code are merged with other caches:

$ ls -l /sys/kernel/slab | grep kio
lrwxrwxrwx. 1 root root 0 Dec 15 15:53 aio_kiocb -> :t-0000128
lrwxrwxrwx. 1 root root 0 Dec 15 15:53 kioctx -> :t-0000640

In the SLAB allocator there are three different caches involved. The kioctx and kiocb are Async I/O data structures that are defined in aio.h header file. If it shows a non zero value that means async io is enabled.

If you have the source code loaded, you can review it at file aio.h. This file is located under:

/usr/src/linux-<version>/include/linux/aio.h

These data structures are using to track the I/O requests, and are allocated as part of the __init_aio_setup() call in aio.c.

Example strace of dbw0 process with AIO enabled (init.ora parameter filesystemio_options = asynch) shows:

...
io_submit(3071864832, 1, {{0xb7302e34, 0, 1, 0, 21}}) = 1
gettimeofday({1176916625, 58882}, NULL) = 0
io_getevents(-1223102464, 1, 1024, {{0xb7302e34, 0xb7302e34, 8192, 0}}, {600, 0}) = 1
...

Example strace of dbw0 process with AIO disabled (filesystemio_options = none):

...
pwrite64(21, "\6\242\0\0004\21\300\0\220B\243\0\0\0\1\6\207\357\0\0\1"..., 8192, 36077568) = 8192
times(NULL) = 1775653082
times(NULL) = 1775653082
pwrite64(21, "\6\242\0\0<\21\300\0\220B\243\0\0\0\1\6\254\0\0\0\2\0*"..., 8192, 36143104) = 8192
...

Caveat for ASMLib

If Oracle ASMLib (see http://oss.oracle.com/projects/oracleasm ) is deployed, the kiocb structs are not used. ASMLib does not use the POSIX aio_*() functions. You will never see any kioctx or kiocb structures from ASMLib. It is far lower level than that.

In fact, ASMLib does AIO or SyncIO depending on how the I/O is passed to it, It makes no decisions at all. This is entirely up to kfk and the layers above it, kfk is entirely controlled by the disk_asynch_io parameter. So, we can check whether ASMLib is doing AIO by SQL*Plus command "show param disk_asynch_io". (You can disable AIO by setting disk_asynch_io=false)

With ASMLib, AIO is done via ioctl() calls (2.4 kernel), or read() calls (2.6 kernel) on the ASM device. Whether ASMLib uses aio depends on whether oracle is configured to do aio, In oracle 10g, if ASMLib is in use, the i/o is asynchronous, because oracle 10g enables aio by default.

The strace when using ASMlib will show read calls that look like this:

...
read(16, "MSA\0\2\0\10\0P\0\0\0\222\377\377\377@\313\373\5\0\0\0"..., 80) = 80
...

The first 3 characters, byte-swapped, are ASM, indicating an ASMLib I/O command structure.

Note: Recommend "strace -c" which doesn't cause too much overhead.