The libioil library can be used to improve the performance of read and write
operations to DFuse from applications which use libc. The library bypasses the
kernel by intercepting POSIX read and write calls from the application and
servicing them directly in user-space.
Using libioil
To use the interception library, set LD_PRELOAD to point to the shared
library in the DAOS install directory:
LD_PRELOAD=/usr/lib64/libioil.so
For example:
$ dd if=/dev/zero of=./foo bs=1G count=20
20+0 records in
20+0 records out
21474836480 bytes (21 GB, 20 GiB) copied, 14.1946 s, 1.5 GB/s
$ LD_PRELOAD=/usr/lib64/libioil.so dd if=/dev/zero of=./bar bs=1G count=20
20+0 records in
20+0 records out
21474836480 bytes (21 GB, 20 GiB) copied, 5.0483 s, 4.3 GB/s
It is also possible to link applications with the interception library at
compile time with the -lioil flag.
Monitor activity
To enable activity reports for the interception library, set the D_IL_REPORT
environment variable. This returns a short summary of activity in the shared
library destructor, typically as a program exits.
The value assigned to D_IL_REPORT defines the number of read and write calls
to log to stderr. To log all calls, set the value to -1. To print the summary
at program exit only, set the value to 0.
For example:
$ D_IL_REPORT=1 LD_PRELOAD=/usr/lib64/libioil.so dd if=/dev/zero of=./bar bs=1G count=20
[libioil] Intercepting write of size 1073741824
20+0 records in
20+0 records out
21474836480 bytes (21 GB, 20 GiB) copied, 5.17297 s, 4.2 GB/s
$ D_IL_REPORT=3 LD_PRELOAD=/usr/lib64/libioil.so dd if=/dev/zero of=./bar bs=1G count=5
[libioil] Intercepting write of size 1073741824
[libioil] Intercepting write of size 1073741824
[libioil] Intercepting write of size 1073741824
5+0 records in
5+0 records out
5368709120 bytes (5.4 GB, 5.0 GiB) copied, 1.27362 s, 4.2 GB/s
$ D_IL_REPORT=-1 LD_PRELOAD=/usr/lib64/libioil.so dd if=/dev/zero of=./bar bs=1G count=5
[libioil] Intercepting write of size 1073741824
[libioil] Intercepting write of size 1073741824
[libioil] Intercepting write of size 1073741824
[libioil] Intercepting write of size 1073741824
[libioil] Intercepting write of size 1073741824
5+0 records in
5+0 records out
5368709120 bytes (5.4 GB, 5.0 GiB) copied, 1.29935 s, 4.1 GB/s
Some programs, including most GNU utilities from the coreutils package, have
a destructor function to close stderr on exit. This means that, while the
interception library works for many basic commands such as cp and cat, it's not
possible to see the summary generated by the interception library.