Using File Storage Parallel Tools | Parallel Unix Commands

The Parallel File Tools suite provides parallel versions of tar, rm, and cp. These tools can run requests on large file systems in parallel, maximizing performance for data protection operations.

The toolkit includes:

• partar: Use this command to create and extract tarballs in parallel.
   Note
  
  The partar tool supports the extraction of tar files created in the GNU basic tar POSIX 1003.1-1990 format. Files created in other archive formats, such as PAX, are not supported.
• parrm: You can use this command to recursively remove a directory in parallel.
• parcp: Use this command to recursively copy a directory in parallel.

Installing the Parallel File Tools

The tool suite is distributed as an RPM for Oracle Linux, Red Hat Enterprise Linux, and CentOS.

To install Parallel File Tools on Linux

To install Parallel File Tools on an Oracle Linux instance:

1. Open a terminal window on the destination instance.
2. Type the following command:
   sudo yum install -y fss-parallel-tools

To install Parallel File Tools on Oracle Linux 8

To install Parallel File Tools on an Oracle Linux 8 instance:

1. Open a terminal window on the destination instance.
2. Install the Oracle Linux developer repository, if needed, by using the following command:
   dnf install oraclelinux-developer-release-el8

3. Install the Parallel File Tools from the developer repository using the following command:
   dnf --enablerepo=ol8_developer install fss-parallel-tools

To install Parallel File Tools on CentOS and Red Hat 6.x

To install Parallel File Tools on CentOS and Red Hat 6.x:

1. Open a terminal window on the destination instance.
2. Type the following command:
   sudo wget http://yum.oracle.com/public-yum-ol6.repo -O /etc/yum.repos.d/public-yum-ol6.repo
   sudo wget http://yum.oracle.com/RPM-GPG-KEY-oracle-ol6 -O /etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
   sudo yum --enablerepo=ol6_developer install fss-parallel-tools

To install Parallel File Tools on CentOS and Red Hat 7.x

1. Open a terminal window on the destination instance.
2. Type the following command:
   sudo wget http://yum.oracle.com/public-yum-ol7.repo -O /etc/yum.repos.d/public-yum-ol7.repo
   sudo wget http://yum.oracle.com/RPM-GPG-KEY-oracle-ol7 -O /etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
   sudo yum --enablerepo=ol7_developer install fss-parallel-tools

Using the Tools - Basic Examples

Here are some simple examples of how the different tools are commonly used in Oracle Cloud Infrastructure File Storage.

To copy all files and folders from one directory to another

In this example, parcp is used to copy the directory "folder" in /source to /destination. The -P option is used to set the number of parallel threads you want to use.

$parcp -P 16 /source/folder /destination

In the following example, parcp is used to copy the contents of the directory "folder" in /source to /destination. The "folder" directory itself is not copied.

$parcp -P 16 /source/folder/. /destination

To create a .TAR archive of a directory

The following command creates a .tar archive of the contents of the specified directory, and stores it as a tarball in the directory. In the example below, the name of the directory that is being used to create the tarball is example.

$partar pcf example.tar example -P 16

You can also create a tarball and send it to a different directory. In the example below, the directory being used to create the tarball is example. The tarball is being created in the /test directory.

$partar pcf example.tar example -P 16 -C /test

Using the Tools - Advanced Examples

Here are some examples of how the different tools are used in more advanced scenarios.

To copy selected files or folders into a .TAR archive and exclude others

You can specify which files and folders are included when you create a .tar archive using partar. Let's say you have a directory that looks like this:

[opc@example sourcedir]$ ls -l
total 180
-rw-r-----.  1 opc opc          0 Apr 15 02:55 example2020-04-15_02-55-33_217107549.error
-rw-r-----.  1 opc opc         10 Apr 15 03:18 example2020-04-15_02-55-33_217107549.log
-rw-rw-r--.  1 opc opc         12 Apr 15 03:18 example2020-04-15_03-18-13_267771997.error
-rw-rw-r--.  1 opc opc         10 Apr 15 03:18 example2020-04-15_03-18-13_267771997.log
-rwxr-xr-x.  1 opc opc         37 Nov 30  2017 File1.txt
-rwxr-xr-x.  1 opc opc         15 Dec  1  2017 File2.txt
-rwxr-xr-x.  1 opc opc         39 Nov 30  2017 File3.txt
-rwxr-xr-x.  1 opc opc         57 Dec  1  2017 File4.txt

The following command creates a .tar archive that:

• Contains a mydir directory named as specified.
• Includes File1.txt, File2.txt, File3.txt, and File4.txt.
• Excludes all .log and .error files.
• Sends the .tar ball from /sourcedir to /mnt/destinationdir
• Extracts the .tar archive

[opc@example sourcedir]$ sudo partar cf - mydir --exclude '*.log*' --exclude '*.err*' | sudo partar xf - -C /mnt/destinationdir

Performing ls -l on /mnt/destinationdir/mytar shows that only the desired files have been copied.

[opc@example mytar]$ ls -l
total 148
-rwxr-xr-x.  1 opc opc         37 Nov 30  2017 File1.txt
-rwxr-xr-x.  1 opc opc         15 Dec  1  2017 File2.txt
-rwxr-xr-x.  1 opc opc         39 Nov 30  2017 File3.txt
-rwxr-xr-x.  1 opc opc         57 Dec  1  2017 File4.txt

When excluding a directory or file from the archive, provide only the name of the directory or file. The --exclude option does not support use of an absolute path. Using an absolute path in the --exclude option will not exclude the specified directory or files from the .tar archive. For example, if you need to exclude a directory called testing from the path of the source directory, you would specify that in a command like the following:

sudo partar pczf name_of_tar_file.tar.gz /<path_source_directory> --exclude=testing

 Note

All files or directories that match the --exclude pattern under the path of the source directory will be excluded from the partar archive.

To copy selected files or folders from one directory to another

You can specify which files and folders are included when you use parcp to copy from one directory to another. Let's say you have a directory that looks like this:

[opc@example sourcedir]$ ls -l
total 180
-rw-r-----.  1 opc opc          0 Apr 15 02:55 example2020-04-15_02-55-33_217107549.error
-rw-r-----.  1 opc opc         10 Apr 15 03:18 example2020-04-15_02-55-33_217107549.log
-rw-rw-r--.  1 opc opc         12 Apr 15 03:18 example2020-04-15_03-18-13_267771997.error
-rw-rw-r--.  1 opc opc         10 Apr 15 03:18 example2020-04-15_03-18-13_267771997.log
-rwxr-xr-x.  1 opc opc         37 Nov 30  2017 File1.txt
-rwxr-xr-x.  1 opc opc         15 Dec  1  2017 File2.txt
-rwxr-xr-x.  1 opc opc         39 Nov 30  2017 File3.txt
-rwxr-xr-x.  1 opc opc         57 Dec  1  2017 File4.txt

First, create a .txt file containing a list of files you want to exclude. In this example, it's /home/opc/list.txt.

The following command copies the contents from sourcedir to /mnt/destinationdir and:

• Copies File1.txt, File2.txt, and File3.txt.
• Excludes File4.txt and the .log and .error files, as listed in /home/opc/list.txt .

[opc@example ~]$ cat /home/opc/list.txt
File4.txt
*.log*
*.err*
[opc@example ~]$ date; time sudo parcp --exclude-from=/home/opc/list.txt -P 16 --restore /sourcedir /mnt/destinationdir;
date Mon Jun  1 15:58:30 GMT 2020

real 9m55.820s
user 0m3.602s
sys 1m5.441s

Mon Jun  1 16:08:25 GMT 2020

Performing ls -l on /mnt/destinationdir shows that only the desired files have been copied.

[opc@example destinationdir]$ ls -l
total 91
-rwxr-xr-x.  1 opc opc         37 Nov 30  2017 File1.txt
-rwxr-xr-x.  1 opc opc         15 Dec  1  2017 File2.txt
-rwxr-xr-x.  1 opc opc         39 Nov 30  2017 File3.txt

To use PARCP as an effective alternative for RSYNC in parallel

The --restore option in parcp is similar to using the -a -r -x and -H options in rsync. (See rsync(1)- Linux Man Page.) The -P option is used to set the number of parallel threads you want to use.

The restore option includes the following behavior:

• Recurse into directories
• Stop at file system boundaries
• Preserve hard links, symlinks, permissions, modification times, group, owners, and special files such as named sockets and fifo files

$parcp -P 16 --restore /source/folder/ /destination

You can use parcp with the --restore and --delete options to sync files between a source and target folder. This is a good substitute for using rsync in parallel. As files are added or removed from the source directory, you can run this command at regular intervals to add or remove the same files from the destination directory. You can automate syncing by using this command option in a cron job.

sudo parcp -P 32 --restore --delete /source/folder/ /destination

https://docs.oracle.com/en-us/iaas/Content/File/Tasks/using_file_storage_parallel_tools.htm

Database Options/Management Packs Usage Reporting for Oracle Databases 11.2 and later (Doc ID 1317265.1)

 Database Options/Management Packs Usage Report

You can determine whether an option is currently in use in a database by running options_packs_usage_statistics.sql, which replaces both former scripts option_usage.sql and used_options_details.sql.
You can manually run options_packs_usage_statistics.sql in an individual database or use Oracle Enterprise Manager Job System to automatically run options_packs_usage_statistics.sql on multiple databases.

For Container Databases (CDB):
 - when connected to CDB$ROOT container, the script lists data for all the open PDBs, properly detecting if Multitenant Option licensing is needed.
 - when connected to a PDB, the script lists only local data,  as there is no visibility to other PDBs, due to the isolation  provided by the Multitenant Architecture. For the same reason,  Multitenant Option usage (more than one PDB) cannot be detected.

Description

The report is divided into two main sections PRODUCT USAGE and FEATURE USAGE DETAILS which reports overview and detailed information for Database Options/Management Packs usage for Oracle Databases for 11.2 and later. You need the "SELECT ANY DICTIONARY" privilege to execute the script.
The report is based on the DBA_FEATURE_USAGE_STATISTICS view. Note that the feature usage data in the view is updated once a week, so it may take up to 7 days for the report to show recent usage of options and/or packs.
The output can be easily opened in a spreadsheet, if needed.

Please review the Description & Disclaimer section in the script before executing the script.

Kindly note the report generated is to be used for informational purposes only and this does not represent your license entitlement or requirement. for known issues with this check MOS DOC ID 1309070.1

 
PRODUCT USAGE

Description

This section provides an overview of each option/pack that has been used in the database and if they are currently in use. It will also provide information when the product was first used and when it was used last time.

FEATURE USAGE DETAILS

Description

This section provides a detailed report of features used that has triggered the usage of a product.
The report is based on the DBA_FEATURE_USAGE_STATISTICS view.
The CURRENTLY_USED column of DBA_FEATURE_USAGE_STATISTICS view  indicates if the feature in question was used during the last sampling interval by VERSION and DBID

 

Click here to download the options_packs_usage_statistics.sql script

How to generate the Database Options/Management Packs Reports?

sqlplus -s <UserName/Password> @options_packs_usage_statistics.sql

 

Who to contact for more information?

For any technical issues with this report or scripts or output of dba_feature_usage_statistics please create a service request.