I_MPI_ADJUST Family 

I_MPI_ADJUST_<opname> 

Control collective operation algorithm selection.

Syntax

I_MPI_ADJUST_<opname>=<algid>[:<conditions>][;<algid>:<conditions>[...]]

Arguments

<algid>

Algorithm identifier

>= 0

The default value of zero selects the reasonable settings

 

<conditions> 

A comma separated list of conditions. An empty list selects all message sizes and process combinations

<l>

Messages of size <l>

<l>-<m>

Messages of size from <l> to <m>, inclusive

<l>@<p>

Messages of size <l> and number of processes <p>

<l>-<m>@<p>-<q>

Messages of size from <l> to <m> and number of processes from <p> to <q>, inclusive

Description

Set this environment variable to select the desired algorithm(s) for the collective operation <opname> under particular conditions. Each collective operation has its own environment variable and algorithms.

Table 3.5-1 Environment Variables, Collective Operations, and Algorithms 

Environment Variable 

Collective Operation

Algorithms

I_MPI_ADJUST_ALLGATHER

MPI_Allgather

  1. Recursive doubling algorithm

  2. Bruck's algorithm

  3. Ring algorithm

  4. Topology aware Gatherv + Bcast algorithm

  5. Knomial algorithm

 

I_MPI_ADJUST_ALLGATHERV

MPI_Allgatherv

  1. Recursive doubling algorithm

  2. Bruck's algorithm

  3. Ring algorithm

  4. Topology aware Gatherv + Bcast algorithm

 

I_MPI_ADJUST_ALLREDUCE

MPI_Allreduce

  1. Recursive doubling algorithm

  2. Rabenseifner's algorithm

  3. Reduce + Bcast algorithm

  4. Topology aware Reduce + Bcast algorithm

  5. Binomial gather + scatter algorithm

  6. Topology aware binominal gather + scatter algorithm

  7. Shumilin's ring algorithm 

  8. Ring algorithm

  9. Knomial algorithm

 

I_MPI_ADJUST_ALLTOALL

MPI_Alltoall

  1. Bruck's algorithm

  2. Isend/Irecv + waitall algorithm

  3. Pair wise exchange algorithm

  4. Plum's algorithm

 

I_MPI_ADJUST_ALLTOALLV

MPI_Alltoallv

  1. Isend/Irecv + waitall algorithm

  2. Plum's algorithm 

 

I_MPI_ADJUST_ALLTOALLW

MPI_Alltoallw

Isend/Irecv + waitall algorithm

I_MPI_ADJUST_BARRIER

MPI_Barrier

  1. Dissemination algorithm

  2. Recursive doubling algorithm

  3. Topology aware dissemination algorithm

  4. Topology aware recursive doubling algorithm

  5. Binominal gather + scatter algorithm

  6. Topology aware binominal gather + scatter algorithm

 

I_MPI_ADJUST_BCAST

MPI_Bcast

  1. Binomial algorithm

  2. Recursive doubling algorithm

  3. Ring algorithm

  4. Topology aware binomial algorithm

  5. Topology aware recursive doubling algorithm

  6. Topology aware ring algorithm

  7. Shumilin's algorithm

  8. Knomial algorithm

 

I_MPI_ADJUST_EXSCAN

MPI_Exscan

  1. Partial results gathering algorithm

  2. Partial results gathering regarding algorithm layout of processes

 

I_MPI_ADJUST_GATHER

MPI_Gather

  1. Binomial algorithm

  2. Topology aware binomial algorithm

  3. Shumilin's algorithm 

 

I_MPI_ADJUST_GATHERV

MPI_Gatherv

  1. Linear algorithm

  2. Topology aware linear algorithm

 

I_MPI_ADJUST_REDUCE_SCATTER

MPI_Reduce_scatter

  1. Recursive having algorithm

  2. Pair wise exchange algorithm

  3. Recursive doubling algorithm

  4. Reduce + Scatterv algorithm

  5. Topology aware Reduce + Scatterv algorithm

 

I_MPI_ADJUST_REDUCE

MPI_Reduce

  1. Shumilin's algorithm

  2. Binomial algorithm

  3. Topology aware Shumilin's algorithm

  4. Topology aware binomial algorithm

  5. Rabenseifner's algorithm

  6.  Topology aware Rabenseifner's algorithm

  7. Knomial algorithm

 

I_MPI_ADJUST_SCAN

MPI_Scan

  1. Partial results gathering algorithm

  2. Topology aware partial results gathering algorithm

 

I_MPI_ADJUST_SCATTER

MPI_Scatter

  1. Binomial algorithm

  2. Topology aware binomial algorithm

  3. Shumilin's algorithm

 

I_MPI_ADJUST_SCATTERV

MPI_Scatterv

  1. Linear algorithm

  2. Topology aware linear algorithm

 

The message size calculation rules for the collective operations are described in the table. In the following table, "n/a" means that the corresponding interval <l>-<m> should be omitted.

Table 3.5-2 Message Collective Functions

Collective Function 

Message Size Formula

MPI_Allgather 

recv_count*recv_type_size

MPI_Allgatherv 

total_recv_count*recv_type_size

MPI_Allreduce 

count*type_size

MPI_Alltoall 

send_count*send_type_size

MPI_Alltoallv 

n/a 

MPI_Alltoallw 

n/a 

MPI_Barrier 

n/a

MPI_Bcast 

count*type_size

MPI_Exscan 

count*type_size

MPI_Gather 

recv_count*recv_type_size if MPI_IN_PLACE is used, otherwise send_count*send_type_size

MPI_Gatherv 

n/a

MPI_Reduce_scatter 

total_recv_count*type_size

MPI_Reduce 

count*type_size

MPI_Scan 

count*type_size

MPI_Scatter 

send_count*send_type_size if MPI_IN_PLACE is used, otherwise recv_count*recv_type_size

MPI_Scatterv 

n/a

Examples

Use the following settings to select the second algorithm for MPI_Reduce operation:
I_MPI_ADJUST_REDUCE=2

Use the following settings to define the algorithms for MPI_Reduce_scatter operation:
I_MPI_ADJUST_REDUCE_SCATTER=4:0-100,5001-10000;1:101-3200,2:3201-5000;3 

In this case. algorithm 4 is used for the message sizes between 0 and 100 bytes and from 5001 and 10000 bytes, algorithm 1 is used for the message sizes between 101 and 3200 bytes, algorithm 2 is used for the message sizes between 3201 and 5000 bytes, and algorithm 3 is used for all other messages.

I_MPI_ADJUST_REDUCE_SEGMENT

Syntax

I_MPI_ADJUST_REDUCE_SEGMENT=<block_size>|<algid>:<block_size>[,<algid>:<block_size>[...]]

Arguments

<algid>

Algorithm identifier

1

Shumilin’s algorithm

3

Topology aware Shumilin’s algorithm

<block_size>

Size in bytes of a message segment

> 0

The default value is 14000

Description:

Set an internal block size to control MPI_Reduce message segmentation for the specified algorithm. If the <algid> value is not set, the <block_size> value is applied for all the algorithms, where it is relevant.

Note:

This environment variable is relevant for Shumilin’s and topology aware Shumilin’s algorithms only (algorithm N1 and algorithm N3 correspondingly).

I_MPI_ADJUST_ALLGATHER_KN_RADIX

Syntax

I_MPI_ADJUST_ALLGATHER_KN_RADIX=<radix>

Arguments

<radix>

An integer that specifies a radix used by the Knomial MPI_Allgather algorithm to build a knomial communication tree

> 1

The default value is 2

Description:

Set this environment together with I_MPI_ADJUST_ALLGATHER=5 to select the knomial tree radix for the corresponding MPI_Allgather algorithm.

I_MPI_ADJUST_BCAST_KN_RADIX

Syntax

I_MPI_ADJUST_BCAST_KN_RADIX=<radix>

Arguments

<radix>

An integer that specifies a radix used by the Knomial MPI_Bcast algorithm to build a knomial communication tree

> 1

The default value is 4

Description:

Set this environment together with I_MPI_ADJUST_BCAST=8 to select the knomial tree radix for the corresponding MPI_Bcast algorithm.

I_MPI_ADJUST_ALLREDUCE_KN_RADIX

Syntax

I_MPI_ADJUST_ALLREDUCE_KN_RADIX=<radix>

Arguments

<radix>

An integer that specifies a radix used by the Knomial MPI_Allreduce algorithm to build a knomial communication tree

> 1

The default value is 4

Description:

Set this environment together with I_MPI_ADJUST_ALLREDUCE=9 to select the knomial tree radix for the corresponding MPI_Allreduce algorithm.

I_MPI_ADJUST_REDUCE_KN_RADIX

Syntax

I_MPI_ADJUST_REDUCE_KN_RADIX=<radix>

Arguments

<radix>

An integer that specifies a radix used by the Knomial MPI_Reduce algorithm to build a knomial communication tree

> 1

The default value is 4

Description:

Set this environment together with I_MPI_ADJUST_REDUCE=7 to select the knomial tree radix for the corresponding MPI_Reduce algorithm.