Configuring mlr
If you really know what you are doing you may think mlr is limiting you in certain ways. mlr is designed to make usage errors due to typos or invalid parameter values as unlikely as possible. But sometimes you want to break those barriers and get full access. For all available options, simply refer to the documentation of configureMlr.
Function configureMlr permits to set options globally for your current R session.
It is also possible to set options locally.
- All options referring to the behavior of learners (these are all options except
show.info
) can be set for an individual learner via theconfig
argument of makeLearner. The local precedes the global configuration. - Some functions like resample, benchmark, selectFeatures, tuneParams,
and tuneParamsMultiCrit have a
show.info
flag that controls if verbose messages are shown. The default value ofshow.info
can be set by configureMlr.
Example: Reducing the output on the console
You are bothered by all the output on the console like in this example?
rdesc = makeResampleDesc("Holdout")
r = resample("classif.multinom", iris.task, rdesc)
#> [Resample] holdout iter: 1
#> # weights: 18 (10 variable)
#> initial value 109.861229
#> iter 10 value 12.256619
#> iter 20 value 3.638740
#> iter 30 value 3.228628
#> iter 40 value 2.951100
#> iter 50 value 2.806521
#> iter 60 value 2.739076
#> iter 70 value 2.522206
#> iter 80 value 2.485225
#> iter 90 value 2.381397
#> iter 100 value 2.360602
#> final value 2.360602
#> stopped after 100 iterations
#> [Resample] Result: mmce.test.mean=0.02
You can suppress the output for this Learner and this resample call as follows:
lrn = makeLearner("classif.multinom", config = list(show.learner.output = FALSE))
r = resample(lrn, iris.task, rdesc, show.info = FALSE)
(Note that multinom has a trace
switch that can alternatively be used to turn off
the progress messages.)
To globally suppress the output for all subsequent learners and calls to resample, benchmark, etc. try the following:
configureMlr(show.learner.output = FALSE, show.info = FALSE)
r = resample("classif.multinom", iris.task, rdesc)
Accessing and resetting the configuration
Function getMlrOptions returns a list with the current configuration.
getMlrOptions()
#> $on.learner.error
#> [1] "stop"
#>
#> $on.learner.warning
#> [1] "warn"
#>
#> $on.par.out.of.bounds
#> [1] "stop"
#>
#> $on.par.without.desc
#> [1] "stop"
#>
#> $show.info
#> [1] FALSE
#>
#> $show.learner.output
#> [1] FALSE
To restore the default configuration call configureMlr with an empty argument list.
configureMlr()
getMlrOptions()
#> $on.learner.error
#> [1] "stop"
#>
#> $on.learner.warning
#> [1] "warn"
#>
#> $on.par.out.of.bounds
#> [1] "stop"
#>
#> $on.par.without.desc
#> [1] "stop"
#>
#> $show.info
#> [1] TRUE
#>
#> $show.learner.output
#> [1] TRUE
Example: Turning off parameter checking
It might happen that you want to access a new parameter of a Learner which is already available in mlr, but the parameter is not "registered" in the learner's parameter set yet. In this case you might want to contact us or open an issue as well! But until then you can turn off mlr's parameter checking. The parameter setting will then be passed to the underlying function without further ado.
## Support Vector Machine with linear kernel and new parameter 'newParam'
lrn = makeLearner("classif.ksvm", kernel = "vanilladot", newParam = 3)
#> Error in setHyperPars2.Learner(learner, insert(par.vals, args)): classif.ksvm: Setting parameter newParam without available description object!
#> You can switch off this check by using configureMlr!
## Turn off parameter checking completely
configureMlr(on.par.without.desc = "quiet")
lrn = makeLearner("classif.ksvm", kernel = "vanilladot", newParam = 3)
train(lrn, iris.task)
#> Setting default kernel parameters
#> Model for learner.id=classif.ksvm; learner.class=classif.ksvm
#> Trained on: task.id = iris-example; obs = 150; features = 4
#> Hyperparameters: fit=FALSE,kernel=vanilladot,newParam=<numeric>
## Option "quiet" also masks typos
lrn = makeLearner("classif.ksvm", kernl = "vanilladot")
train(lrn, iris.task)
#> Model for learner.id=classif.ksvm; learner.class=classif.ksvm
#> Trained on: task.id = iris-example; obs = 150; features = 4
#> Hyperparameters: fit=FALSE,kernl=<character>
## Alternatively turn off parameter checking, but still see warnings
configureMlr(on.par.without.desc = "warn")
lrn = makeLearner("classif.ksvm", kernl = "vanilladot", newParam = 3)
#> Warning in setHyperPars2.Learner(learner, insert(par.vals, args)): classif.ksvm: Setting parameter kernl without available description object!
#> You can switch off this check by using configureMlr!
#> Warning in setHyperPars2.Learner(learner, insert(par.vals, args)): classif.ksvm: Setting parameter newParam without available description object!
#> You can switch off this check by using configureMlr!
train(lrn, iris.task)
#> Model for learner.id=classif.ksvm; learner.class=classif.ksvm
#> Trained on: task.id = iris-example; obs = 150; features = 4
#> Hyperparameters: fit=FALSE,kernl=<character>,newParam=<numeric>
Example: Handling errors in an underlying learning method
If an underlying learning method throws an error the default behavior of mlr is to generate an exception as well. However, in some situations, for example if you conduct a benchmark study with multiple data sets and learners, you usually don't want the whole experiment stopped due to one error. The following example shows how to prevent this:
## This call gives an error caused by the low number of observations in class "virginica"
train("classif.qda", task = iris.task, subset = 1:104)
#> Error in qda.default(x, grouping, ...): some group is too small for 'qda'
#> Timing stopped at: 0.002 0 0.002
## Turn learner errors into warnings
configureMlr(on.learner.error = "warn")
mod = train("classif.qda", task = iris.task, subset = 1:104)
#> Warning in train("classif.qda", task = iris.task, subset = 1:104): Could not train learner classif.qda: Error in qda.default(x, grouping, ...) :
#> some group is too small for 'qda'
mod
#> Model for learner.id=classif.qda; learner.class=classif.qda
#> Trained on: task.id = iris-example; obs = 104; features = 4
#> Hyperparameters:
#> Training failed: Error in qda.default(x, grouping, ...) :
#> some group is too small for 'qda'
#>
#> Training failed: Error in qda.default(x, grouping, ...) :
#> some group is too small for 'qda'
## mod is an object of class FailureModel
isFailureModel(mod)
#> [1] TRUE
## Retrieve the error message
getFailureModelMsg(mod)
#> [1] "Error in qda.default(x, grouping, ...) : \n some group is too small for 'qda'\n"
## predict and performance return NA's
pred = predict(mod, iris.task)
pred
#> Prediction: 150 observations
#> predict.type: response
#> threshold:
#> time: NA
#> id truth response
#> 1 1 setosa <NA>
#> 2 2 setosa <NA>
#> 3 3 setosa <NA>
#> 4 4 setosa <NA>
#> 5 5 setosa <NA>
#> 6 6 setosa <NA>
performance(pred)
#> mmce
#> NA
Instead of an exception, a warning is issued and a FailureModel is created.
Function getFailureModelMsg extracts the error message.
All further steps like prediction and performance calculation work and return NA's
.