neuraxle.hyperparams.space

Hyperparameter Dictionary Conversions

Ways to convert from a nested dictionary of hyperparameters to a flat dictionary, and vice versa.

Here is a nested dictionary:

{
    "b": {
        "a": {
            "learning_rate": 7
        },
        "learning_rate": 9
    }
}

Here is an equivalent flat dictionary for the previous nested one:

{
    "b.a.learning_rate": 7,
    "b.learning_rate": 9
}

Notice that if you have a SKLearnWrapper on a sklearn Pipeline object, the hyperparameters past that point will use double underscores __ as a separator rather than a dot in flat dictionaries, and in nested dictionaries the sklearn params will appear as a flat past the sklearn wrapper, which is fine.

By default, hyperparameters are stored inside a HyperparameterSpace or inside a HyperparameterSamples object, which offers methods to do the conversions above, and also using ordered dicts (OrderedDict) to store parameters in-order.

A HyperparameterSpace can be sampled by calling the .rvs() method on it, which will recursively call .rvs() on all the HyperparameterSpace and HyperparameterDistribution that it contains. It will return a HyperparameterSamples object. A HyperparameterSpace can also be narrowed towards an better, finer subspace, which is itself a HyperparameterSpace. This can be done by calling the .narrow_space_from_best_guess method of the HyperparameterSpace which will also recursively apply the changes to its contained HyperparameterSpace and to all its contained HyperparameterDistribution.

The HyperparameterSamples contains sampled hyperparameter, that is, a valued point in the possible space. This is ready to be sent to an instance of the pipeline to try and score it, for example.

Functions

flat_to_nested_dict(flat_hyperparams[, …])

Convert a flat hyperparameter dictionary to a nested one.

nested_dict_to_flat(nested_hyperparams[, …])

Convert a nested hyperparameter dictionary to a flat one.

Classes

HyperparameterSamples

Wraps an hyperparameter nested dict or flat dict, and offer a few more functions.

HyperparameterSpace

Wraps an hyperparameter nested dict or flat dict, and offer a few more functions to process all contained HyperparameterDistribution.

class neuraxle.hyperparams.space.HyperparameterSamples[source]

Wraps an hyperparameter nested dict or flat dict, and offer a few more functions.

This can be set on a Pipeline with the method set_hyperparams.

HyperparameterSamples are often the result of calling .rvs() on an HyperparameterSpace.

clear() → None. Remove all items from od.
copy() → a shallow copy of od
fromkeys(S[, v]) → New ordered dictionary with keys from S.

If not specified, the value defaults to None.

get(k[, d]) → D[k] if k in D, else d. d defaults to None.
items() → a set-like object providing a view on D's items
keys() → a set-like object providing a view on D's keys
move_to_end()

Move an existing element to the end (or beginning if last==False).

Raises KeyError if the element does not exist. When last=True, acts like a fast version of self[key]=self.pop(key).

pop(k[, d]) → v, remove specified key and return the corresponding

value. If key is not found, d is returned if given, otherwise KeyError is raised.

popitem()

Remove and return a (key, value) pair from the dictionary.

Pairs are returned in LIFO order if last is true or FIFO order if false.

setdefault(k[, d]) → od.get(k,d), also set od[k]=d if k not in od
to_flat() → neuraxle.hyperparams.space.HyperparameterSamples[source]

Will create an equivalent flat HyperparameterSamples.

Returns

an HyperparameterSamples like self, flattened.

to_flat_as_dict_primitive() → dict[source]

Will create an equivalent flat HyperparameterSpace, as a dict.

Returns

an HyperparameterSpace like self, flattened.

to_flat_as_ordered_dict_primitive() → collections.OrderedDict[source]

Will create an equivalent flat HyperparameterSpace, as a dict.

Returns

an HyperparameterSpace like self, flattened.

to_nested_dict() → neuraxle.hyperparams.space.HyperparameterSamples[source]

Will create an equivalent nested dict HyperparameterSamples.

Returns

an HyperparameterSamples like self, as a nested dict.

to_nested_dict_as_dict_primitive() → dict[source]

Will create an equivalent nested dict HyperparameterSpace, as a dict.

Returns

a nested primitive dict type of self.

to_nested_dict_as_ordered_dict_primitive() → collections.OrderedDict[source]

Will create an equivalent nested dict HyperparameterSpace, as a dict.

Returns

a nested primitive dict type of self.

update([E, ]**F) → None. Update D from dict/iterable E and F.

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values() → an object providing a view on D's values
class neuraxle.hyperparams.space.HyperparameterSpace[source]

Wraps an hyperparameter nested dict or flat dict, and offer a few more functions to process all contained HyperparameterDistribution.

This can be set on a Pipeline with the method set_hyperparams_space.

Calling .rvs() on an HyperparameterSpace results in HyperparameterSamples.

clear() → None. Remove all items from od.
copy() → a shallow copy of od
fromkeys(S[, v]) → New ordered dictionary with keys from S.

If not specified, the value defaults to None.

get(k[, d]) → D[k] if k in D, else d. d defaults to None.
items() → a set-like object providing a view on D's items
keys() → a set-like object providing a view on D's keys
move_to_end()

Move an existing element to the end (or beginning if last==False).

Raises KeyError if the element does not exist. When last=True, acts like a fast version of self[key]=self.pop(key).

narrow_space_from_best_guess(best_guesses: neuraxle.hyperparams.space.HyperparameterSpace, kept_space_ratio: float = 0.5) → neuraxle.hyperparams.space.HyperparameterSpace[source]

Takes samples estimated to be the best ones of the space as of yet, and restrict the whole space towards that.

Parameters
  • best_guess – sampled HyperparameterSpace (the result of rvs on each parameter, but still stored as a HyperparameterSpace).

  • kept_space_ratio – what proportion of the space is kept. Should be between 0.0 and 1.0. Default is 0.5.

Returns

a new HyperparameterSpace containing the narrowed HyperparameterDistribution objects.

pop(k[, d]) → v, remove specified key and return the corresponding

value. If key is not found, d is returned if given, otherwise KeyError is raised.

popitem()

Remove and return a (key, value) pair from the dictionary.

Pairs are returned in LIFO order if last is true or FIFO order if false.

rvs() → neuraxle.hyperparams.space.HyperparameterSpace[source]

Sample the space of random variables.

Returns

a random HyperparameterSamples, sampled from a point of the present HyperparameterSpace.

setdefault(k[, d]) → od.get(k,d), also set od[k]=d if k not in od
to_flat() → neuraxle.hyperparams.space.HyperparameterSpace[source]

Will create an equivalent flat HyperparameterSpace.

Returns

an HyperparameterSpace like self, flattened.

to_flat_as_dict_primitive() → dict[source]

Will create an equivalent flat HyperparameterSpace, as a dict.

Returns

an HyperparameterSpace like self, flattened.

to_flat_as_ordered_dict_primitive() → collections.OrderedDict[source]

Will create an equivalent flat HyperparameterSpace, as a dict.

Returns

an HyperparameterSpace like self, flattened.

to_nested_dict() → neuraxle.hyperparams.space.HyperparameterSpace[source]

Will create an equivalent nested dict HyperparameterSpace.

Returns

an HyperparameterSpace like self, as a nested dict.

to_nested_dict_as_dict_primitive() → dict[source]

Will create an equivalent nested dict HyperparameterSpace, as a dict.

Returns

a nested primitive dict type of self.

to_nested_dict_as_ordered_dict_primitive() → collections.OrderedDict[source]

Will create an equivalent nested dict HyperparameterSpace, as a dict.

Returns

a nested primitive dict type of self.

unnarrow() → neuraxle.hyperparams.space.HyperparameterSpace[source]

Return the original space before narrowing of the distribution. If the distribution was never narrowed, the values in the dict will be copies.

Returns

the original HyperparameterSpace before narrowing.

update([E, ]**F) → None. Update D from dict/iterable E and F.

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values() → an object providing a view on D's values
neuraxle.hyperparams.space.flat_to_nested_dict(flat_hyperparams, dict_ctor=<class 'collections.OrderedDict'>)[source]

Convert a flat hyperparameter dictionary to a nested one.

Parameters
  • flat_hyperparams – a flat hyperparameter dictionary.

  • dict_ctorOrderedDict by default. Will use this as a class to create the new returned dict.

Returns

a nested hyperparameter dictionary.

neuraxle.hyperparams.space.nested_dict_to_flat(nested_hyperparams, dict_ctor=<class 'collections.OrderedDict'>)[source]

Convert a nested hyperparameter dictionary to a flat one.

Parameters
  • nested_hyperparams – a nested hyperparameter dictionary.

  • dict_ctorOrderedDict by default. Will use this as a class to create the new returned dict.

Returns

a flat hyperparameter dictionary.