How to set configuration¶

In this notebook, we will provide details about how to set configurations for federated learning experiments.

Load default configuration¶

APPFL empolys OmegaConf package, a hierarchical configuration system, for FL training configurations. OmegaConf package allows users to create a hierarchical configuration in DictConfig type from a python @dataclass. For example, we can load the APPFL default configuration dataclass using OmegaConf.structured() as shown below

[1]:

from appfl.config import Config
from omegaconf import OmegaConf

cfg = OmegaConf.structured(Config)

The configuration cfg is initialized with the default values. Let’s check the configuration values.

[2]:

print(OmegaConf.to_yaml(cfg))

fed:
  type: federated
  servername: ServerFedAvg
  clientname: ClientOptim
  args:
    server_learning_rate: 0.01
    server_adapt_param: 0.001
    server_momentum_param_1: 0.9
    server_momentum_param_2: 0.99
    optim: SGD
    num_local_epochs: 10
    optim_args:
      lr: 0.001
    use_dp: false
    epsilon: 1
    clip_grad: false
    clip_value: 1
    clip_norm: 1
device: cpu
device_server: cpu
num_clients: 1
num_epochs: 2
num_workers: 0
batch_training: true
train_data_batch_size: 64
train_data_shuffle: true
validation: true
test_data_batch_size: 64
test_data_shuffle: false
data_sanity: false
reproduce: true
pca_dir: ''
params_start: 0
params_end: 49
ncomponents: 40
use_tensorboard: false
load_model: false
load_model_dirname: ''
load_model_filename: ''
save_model: false
save_model_dirname: ''
save_model_filename: ''
checkpoints_interval: 2
save_model_state_dict: false
send_final_model: false
output_dirname: output
output_filename: result
logginginfo: {}
summary_file: ''
personalization: false
p_layers: []
config_name: ''
max_message_size: 104857600
operator:
  id: 1
server:
  id: 1
  host: localhost
  port: 50051
  use_tls: false
  api_key: null
client:
  id: 1
enable_compression: false
lossy_compressor: SZ2
lossless_compressor: blosc
compressor_sz2_path: ../.compressor/SZ/build/sz/libSZ.dylib
compressor_sz3_path: ../.compressor/SZ3/build/tools/sz3c/libSZ3c.dylib
compressor_szx_path: ../.compressor/SZx-main/build/lib/libSZx.dylib
error_bounding_mode: ''
error_bound: 0.0
flat_model_dtype: np.float32
param_cutoff: 1024

Most variables are self-explanatory. Specifically,

Variable fed sets the choice of FL algorithm and the algorithm-related parameters, and it is also defined as a python @dataclass. We provide the definition of those dataclasses at appfl.config.fed.*. In details,
- appfl.config.fed.federated is a general dataclass for all synchronous FL algorithms, where you can specify the server algorithm name at servername, client algorithm name at clientname, and all related arguments and parameters at args.
- appfl.config.fed.fedasync is a general dataclass for all asynchronous FL algorithms, whose args contains commonly-used parameters in asynchronous FL.
- appfl.config.fed.iceadmm is a dataclass specifically wrote for the ICEADMM privacy-preserving FL algorithm, whose args contains all needed parameters for the ICEADMM algorithm.
- appfl.config.fed.iiadmm is a dataclass specifically wrote for the IIADMM privacy-preserving FL algorithm, whose args contains all needed parameters for the IIADMM algorithm.

Initialize configuration with arguments¶

We can also initialize the configuration with other values. For example, the following code is loading the configuration with the algorithm choice of IIADMM.

[3]:

from appfl.config.fed import IIADMM

cfg = OmegaConf.structured(Config(fed=IIADMM()))
print(OmegaConf.to_yaml(cfg.fed))

type: iiadmm
servername: IIADMMServer
clientname: IIADMMClient
args:
  num_local_epochs: 1
  accum_grad: true
  coeff_grad: false
  optim: SGD
  optim_args:
    lr: 0.01
  init_penalty: 100.0
  residual_balancing:
    res_on: false
    res_on_every_update: false
    tau: 1.1
    mu: 10
  use_dp: false
  epsilon: 1
  clip_grad: false
  clip_value: 1
  clip_norm: 1

Change configuration values¶

We can also change the configuration value after initialization. For example, we can change fed variable as follows:

[ ]:

from appfl.config.fed import FedAsync

cfg = OmegaConf.structured(Config)
my_fed = OmegaConf.structured(FedAsync)
cfg.fed = my_fed
print(OmegaConf.to_yaml(cfg.fed))

type: fedasync
servername: ServerFedAsynchronous
clientname: ClientOptim
args:
  server_learning_rate: 0.01
  server_adapt_param: 0.001
  server_momentum_param_1: 0.9
  server_momentum_param_2: 0.99
  optim: SGD
  num_local_epochs: 10
  optim_args:
    lr: 0.001
  use_dp: false
  epsilon: 1
  clip_grad: false
  clip_value: 1
  clip_norm: 1
  K: 3
  alpha: 0.9
  staleness_func:
    name: constant
    args:
      a: 0.5
      b: 4
  gradient_based: false

[ ]: