Tunable hyperparameters for kindling models

These parameters extend the dials framework to support hyperparameter tuning of neural networks built with the {kindling} package. They control network architecture, activation functions, optimization, and training behavior.

Usage

n_hlayers(range = c(1L, 5L), trans = NULL)

hidden_neurons(range = c(8L, 512L), trans = NULL)

activations(
  values = c("relu", "relu6", "elu", "selu", "celu", "leaky_relu", "gelu", "softplus",
    "softshrink", "softsign", "tanhshrink", "hardtanh", "hardshrink", "hardswish",
    "hardsigmoid", "silu", "mish", "logsigmoid")
)

output_activation(
  values = c("relu", "elu", "selu", "softplus", "softmax", "log_softmax", "logsigmoid",
    "hardtanh", "hardsigmoid", "silu")
)

optimizer(values = c("adam", "sgd", "rmsprop", "adamw"))

bias(values = c(TRUE, FALSE))

validation_split(range = c(0, 0.5), trans = NULL)

bidirectional(values = c(TRUE, FALSE))

Arguments

range

A two-element numeric vector with the default lower and upper bounds.

trans

An optional transformation; NULL for none.

values

Logical vector of possible values.

Architecture Strategy

Since tidymodels tuning works with independent parameters, we use a simplified approach where:

• hidden_neurons specifies a single value that will be used for ALL layers

• activations specifies a single activation that will be used for ALL layers

• n_hlayers controls the depth

For more complex architectures with different neurons/activations per layer, users should manually specify these after tuning or use custom tuning logic.

Parameters

n_hlayers

Number of hidden layers in the network.

hidden_neurons

Number of units per hidden layer (applied to all layers).

activation

Single activation function applied to all hidden layers.

output_activation

Activation function for the output layer.

optimizer

Optimizer algorithm.

bias

Whether to include bias terms in layers.

validation_split

Proportion of training data held out for validation.

bidirectional

Whether RNN layers are bidirectional.

Number of Hidden Layers

Controls the depth of the network. When tuning, this will determine how many layers are created, each with hidden_neurons units and activations function.

Hidden Units per Layer

Specifies the number of units per hidden layer.

Activation Function (Hidden Layers)

Activation functions for hidden layers.

Output Activation Function

Activation function applied to the output layer. Values must correspond to torch::nnf_* functions.

Optimizer Type

The optimization algorithm used during training.

Include Bias Terms

Whether layers should include bias parameters.

Validation Split Proportion

Fraction of the training data to use as a validation set during training.

Bidirectional RNN

Whether recurrent layers should process sequences in both directions.

Examples

if (FALSE) { # \dontrun{
library(dials)
library(tune)

# Create a tuning grid
grid = grid_regular(
    n_hlayers(range = c(1L, 4L)),
    hidden_neurons(range = c(32L, 256L)),
    activations(c('relu', 'elu', 'selu')),
    levels = c(4, 5, 3)
)

# Use in a model spec
mlp_spec = mlp_kindling(
    mode = "classification",
    hidden_neurons = tune(),
    activations = tune(),
    epochs = tune(),
    learn_rate = tune()
)
} # }