neuraxle.hyperparams.distributions

Hyperparameter Distributions

Here you’ll find a few hyperparameter distributions. It’s also possible to create yours by inheriting from the base class. Each distribution must override the method rvs, which will return a sampled value from the distribution.

Classes

Boolean()

Get a random boolean hyperparameter.

Choice(choice_list)

Get a random value from a choice list of possible value for this hyperparameter.

FixedHyperparameter(value)

This is an hyperparameter that won’t change again, but that is still expressed as a distribution.

HyperparameterDistribution()

Base class for other hyperparameter distributions.

LogNormal(log2_space_mean, log2_space_std, …)

Get a LogNormal distribution.

LogUniform(min_included, max_included)

Get a LogUniform distribution.

Normal(mean, std, hard_clip_min, hard_clip_max)

Get a normal distribution.

PriorityChoice(choice_list)

Get a random value from a choice list of possible value for this hyperparameter.

Quantized(hd, hds)

A quantized wrapper for another distribution: will round() the rvs number.

RandInt(min_included, max_included)

Get a random integer within a range

Uniform(min_included, max_included)

Get a uniform distribution.

WrappedHyperparameterDistributions(hd, hds)

class neuraxle.hyperparams.distributions.Boolean[source]

Get a random boolean hyperparameter.

get_current_narrowing_value()[source]
narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.0) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Takes a value that is estimated to be the best one of the space, and restrict the space near that value. By default, this function will completely replace the returned value by the new guess if not overriden.

Parameters
  • best_guess – the value towards which we want to narrow down the space.

  • kept_space_ratio – what proportion of the space is kept. Should be between 0.0 and 1.0. Default is to keep only the best_guess (0.0).

Returns

a new HyperparameterDistribution object that has been narrowed down.

rvs()[source]

Get a random True or False.

Returns

True or False (random).

unnarrow() → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Return the original distribution before narrowing of the distribution. If the distribution was never narrowed, will return a copy of self.

Returns

the original HyperparameterDistribution before narrowing, or else self if the distribution is virgin.

was_narrowed_from(kept_space_ratio: float, original_hp: neuraxle.hyperparams.distributions.HyperparameterDistribution) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Keep track of the original distribution to restore it.

Parameters
  • kept_space_ratio – the ratio which made the current object narrower than the original_hp.

  • original_hp – The original HyperparameterDistribution, which will be kept in a private variable for an eventual restore.

Returns

self.

class neuraxle.hyperparams.distributions.Choice(choice_list: List)[source]

Get a random value from a choice list of possible value for this hyperparameter.

When narrowed, the choice will only collapse to a single element when narrowed enough. For example, if there are 4 items in the list, only at a narrowing value of 0.25 that the first item will be kept alone.

get_current_narrowing_value()[source]
narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.0) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Will narrow the space. If the cumulative kept_space_ratio gets to be under or equal to 1/len(choice_list), then the list is crunched to a single item as a FixedHyperparameter to reflect this narrowing. So once a small enough kept_space_ratio is reached, the list becomes a fixed unique item from the best guess. Otherwise, a deepcopy of self is returned.

Parameters
  • best_guess – the best item of the list to keep if truly narrowing.

  • kept_space_ratio – the ratio of the space to keep.

Returns

a deepcopy of self, or else a FixedHyperparameter of the best_guess.

rvs()[source]

Get one of the items randomly.

Returns

one of the items of the list.

unnarrow() → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Return the original distribution before narrowing of the distribution. If the distribution was never narrowed, will return a copy of self.

Returns

the original HyperparameterDistribution before narrowing, or else self if the distribution is virgin.

was_narrowed_from(kept_space_ratio: float, original_hp: neuraxle.hyperparams.distributions.HyperparameterDistribution) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Keep track of the original distribution to restore it.

Parameters
  • kept_space_ratio – the ratio which made the current object narrower than the original_hp.

  • original_hp – The original HyperparameterDistribution, which will be kept in a private variable for an eventual restore.

Returns

self.

class neuraxle.hyperparams.distributions.FixedHyperparameter(value)[source]

This is an hyperparameter that won’t change again, but that is still expressed as a distribution.

get_current_narrowing_value()[source]
narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.0) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Takes a value that is estimated to be the best one of the space, and restrict the space near that value. By default, this function will completely replace the returned value by the new guess if not overriden.

Parameters
  • best_guess – the value towards which we want to narrow down the space.

  • kept_space_ratio – what proportion of the space is kept. Should be between 0.0 and 1.0. Default is to keep only the best_guess (0.0).

Returns

a new HyperparameterDistribution object that has been narrowed down.

rvs()[source]

Sample the non-random anymore value.

Returns

the value given at creation.

unnarrow() → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Return the original distribution before narrowing of the distribution. If the distribution was never narrowed, will return a copy of self.

Returns

the original HyperparameterDistribution before narrowing, or else self if the distribution is virgin.

was_narrowed_from(kept_space_ratio: float, original_hp: neuraxle.hyperparams.distributions.HyperparameterDistribution) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Keep track of the original distribution to restore it.

Parameters
  • kept_space_ratio – the ratio which made the current object narrower than the original_hp.

  • original_hp – The original HyperparameterDistribution, which will be kept in a private variable for an eventual restore.

Returns

self.

class neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Base class for other hyperparameter distributions.

get_current_narrowing_value()[source]
narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.0) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Takes a value that is estimated to be the best one of the space, and restrict the space near that value. By default, this function will completely replace the returned value by the new guess if not overriden.

Parameters
  • best_guess – the value towards which we want to narrow down the space.

  • kept_space_ratio – what proportion of the space is kept. Should be between 0.0 and 1.0. Default is to keep only the best_guess (0.0).

Returns

a new HyperparameterDistribution object that has been narrowed down.

rvs()[source]

Sample the random variable.

Returns

The randomly sampled value.

unnarrow() → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Return the original distribution before narrowing of the distribution. If the distribution was never narrowed, will return a copy of self.

Returns

the original HyperparameterDistribution before narrowing, or else self if the distribution is virgin.

was_narrowed_from(kept_space_ratio: float, original_hp: neuraxle.hyperparams.distributions.HyperparameterDistribution) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Keep track of the original distribution to restore it.

Parameters
  • kept_space_ratio – the ratio which made the current object narrower than the original_hp.

  • original_hp – The original HyperparameterDistribution, which will be kept in a private variable for an eventual restore.

Returns

self.

class neuraxle.hyperparams.distributions.LogNormal(log2_space_mean: float, log2_space_std: float, hard_clip_min: float = None, hard_clip_max: float = None)[source]

Get a LogNormal distribution.

get_current_narrowing_value()[source]
narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.5) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Will narrow the distribution towards the new best_guess. The log2_space_mean (log space mean) will move, in log space, towards the new best guess, and the log2_space_std (log space standard deviation) will be multiplied by the kept_space_ratio.

Parameters
  • best_guess – the value towards which we want to narrow down the space’s mean. Should be between 0.0 and 1.0.

  • kept_space_ratio – what proportion of the space is kept. Default is to keep half the space (0.5).

Returns

a new HyperparameterDistribution that has been narrowed down.

rvs() → float[source]

Will return a float value in the specified range as specified at creation. Note: the range at creation was in log space. The return value is after taking an exponent.

Returns

a float.

unnarrow() → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Return the original distribution before narrowing of the distribution. If the distribution was never narrowed, will return a copy of self.

Returns

the original HyperparameterDistribution before narrowing, or else self if the distribution is virgin.

was_narrowed_from(kept_space_ratio: float, original_hp: neuraxle.hyperparams.distributions.HyperparameterDistribution) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Keep track of the original distribution to restore it.

Parameters
  • kept_space_ratio – the ratio which made the current object narrower than the original_hp.

  • original_hp – The original HyperparameterDistribution, which will be kept in a private variable for an eventual restore.

Returns

self.

class neuraxle.hyperparams.distributions.LogUniform(min_included: float, max_included: float)[source]

Get a LogUniform distribution.

For example, this is good for neural networks’ learning rates: that vary exponentially.

get_current_narrowing_value()[source]
narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.5) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Will narrow, in log space, the distribution towards the new best_guess.

Parameters
  • best_guess – the value towards which we want to narrow down the space. Should be between 0.0 and 1.0.

  • kept_space_ratio – what proportion of the space is kept. Default is to keep half the space (0.5).

Returns

a new HyperparameterDistribution that has been narrowed down.

rvs() → float[source]

Will return a float value in the specified range as specified at creation.

Returns

a float.

unnarrow() → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Return the original distribution before narrowing of the distribution. If the distribution was never narrowed, will return a copy of self.

Returns

the original HyperparameterDistribution before narrowing, or else self if the distribution is virgin.

was_narrowed_from(kept_space_ratio: float, original_hp: neuraxle.hyperparams.distributions.HyperparameterDistribution) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Keep track of the original distribution to restore it.

Parameters
  • kept_space_ratio – the ratio which made the current object narrower than the original_hp.

  • original_hp – The original HyperparameterDistribution, which will be kept in a private variable for an eventual restore.

Returns

self.

class neuraxle.hyperparams.distributions.Normal(mean: float, std: float, hard_clip_min: float = None, hard_clip_max: float = None)[source]

Get a normal distribution.

get_current_narrowing_value()[source]
narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.5) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Will narrow the distribution towards the new best_guess. The mean will move towards the new best guess, and the standard deviation will be multiplied by the kept_space_ratio. The hard clip limit is unchanged.

Parameters
  • best_guess – the value towards which we want to narrow down the space’s mean. Should be between 0.0 and 1.0.

  • kept_space_ratio – what proportion of the space is kept. Default is to keep half the space (0.5).

Returns

a new HyperparameterDistribution that has been narrowed down.

rvs() → float[source]

Will return a float value in the specified range as specified at creation.

Returns

a float.

unnarrow() → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Return the original distribution before narrowing of the distribution. If the distribution was never narrowed, will return a copy of self.

Returns

the original HyperparameterDistribution before narrowing, or else self if the distribution is virgin.

was_narrowed_from(kept_space_ratio: float, original_hp: neuraxle.hyperparams.distributions.HyperparameterDistribution) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Keep track of the original distribution to restore it.

Parameters
  • kept_space_ratio – the ratio which made the current object narrower than the original_hp.

  • original_hp – The original HyperparameterDistribution, which will be kept in a private variable for an eventual restore.

Returns

self.

class neuraxle.hyperparams.distributions.PriorityChoice(choice_list: List)[source]

Get a random value from a choice list of possible value for this hyperparameter.

The first parameters are kept until the end when the list is narrowed (it is narrowed progressively), unless there is a best guess that surpasses some of the top choices.

get_current_narrowing_value()[source]
narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.0) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Will narrow the space. If the cumulative kept_space_ratio gets to be under or equal to 1-1/len(choice_list), then the list is crunched to discard the last items to reflect this narrowing. After a few narrowing (or a big one), the list may become a FixedHyperparameter. Otherwise if the list is unchanged, a deepcopy of self is returned.

Parameters
  • best_guess – the best item of the list, which will be brought back as the first item.

  • kept_space_ratio – the ratio of the space to keep.

Returns

a deepcopy of self, or a subchoice of self, or else a FixedHyperparameter of the best_guess.

rvs()[source]

Get one of the items randomly.

Returns

one of the items of the list.

unnarrow() → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Return the original distribution before narrowing of the distribution. If the distribution was never narrowed, will return a copy of self.

Returns

the original HyperparameterDistribution before narrowing, or else self if the distribution is virgin.

was_narrowed_from(kept_space_ratio: float, original_hp: neuraxle.hyperparams.distributions.HyperparameterDistribution) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Keep track of the original distribution to restore it.

Parameters
  • kept_space_ratio – the ratio which made the current object narrower than the original_hp.

  • original_hp – The original HyperparameterDistribution, which will be kept in a private variable for an eventual restore.

Returns

self.

class neuraxle.hyperparams.distributions.Quantized(hd: neuraxle.hyperparams.distributions.HyperparameterDistribution = None, hds: List[neuraxle.hyperparams.distributions.HyperparameterDistribution] = None)[source]

A quantized wrapper for another distribution: will round() the rvs number.

get_current_narrowing_value()[source]
narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.5) → neuraxle.hyperparams.distributions.Quantized[source]

Will narrow the underlying distribution and re-wrap it under a Quantized.

Parameters
  • best_guess – the value towards which we want to narrow down the space.

  • kept_space_ratio – what proportion of the space is kept. Default is to keep half the space (0.5).

Returns

rvs() → int[source]

Will return an integer, rounded from the output of the previous distribution.

Returns

an integer.

unnarrow() → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Return the original distribution before narrowing of the distribution. If the distribution was never narrowed, will return a copy of self.

Returns

the original HyperparameterDistribution before narrowing, or else self if the distribution is virgin.

was_narrowed_from(kept_space_ratio: float, original_hp: neuraxle.hyperparams.distributions.HyperparameterDistribution) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Keep track of the original distribution to restore it.

Parameters
  • kept_space_ratio – the ratio which made the current object narrower than the original_hp.

  • original_hp – The original HyperparameterDistribution, which will be kept in a private variable for an eventual restore.

Returns

self.

class neuraxle.hyperparams.distributions.RandInt(min_included: int, max_included: int)[source]

Get a random integer within a range

get_current_narrowing_value()[source]
narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.5) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Will narrow the underlying distribution towards the best guess.

Parameters
  • best_guess – the value towards which we want to narrow down the space. Should be between 0.0 and 1.0.

  • kept_space_ratio – what proportion of the space is kept. Default is to keep half the space (0.5).

Returns

a new HyperparameterDistribution that has been narrowed down.

rvs() → int[source]

Will return an integer in the specified range as specified at creation.

Returns

an integer.

unnarrow() → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Return the original distribution before narrowing of the distribution. If the distribution was never narrowed, will return a copy of self.

Returns

the original HyperparameterDistribution before narrowing, or else self if the distribution is virgin.

was_narrowed_from(kept_space_ratio: float, original_hp: neuraxle.hyperparams.distributions.HyperparameterDistribution) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Keep track of the original distribution to restore it.

Parameters
  • kept_space_ratio – the ratio which made the current object narrower than the original_hp.

  • original_hp – The original HyperparameterDistribution, which will be kept in a private variable for an eventual restore.

Returns

self.

class neuraxle.hyperparams.distributions.Uniform(min_included: int, max_included: int)[source]

Get a uniform distribution.

get_current_narrowing_value()[source]
narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.5) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Will narrow the underlying distribution towards the best guess.

Parameters
  • best_guess – the value towards which we want to narrow down the space. Should be between 0.0 and 1.0.

  • kept_space_ratio – what proportion of the space is kept. Default is to keep half the space (0.5).

Returns

a new HyperparameterDistribution that has been narrowed down.

rvs() → float[source]

Will return a float value in the specified range as specified at creation.

Returns

a float.

unnarrow() → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Return the original distribution before narrowing of the distribution. If the distribution was never narrowed, will return a copy of self.

Returns

the original HyperparameterDistribution before narrowing, or else self if the distribution is virgin.

was_narrowed_from(kept_space_ratio: float, original_hp: neuraxle.hyperparams.distributions.HyperparameterDistribution) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Keep track of the original distribution to restore it.

Parameters
  • kept_space_ratio – the ratio which made the current object narrower than the original_hp.

  • original_hp – The original HyperparameterDistribution, which will be kept in a private variable for an eventual restore.

Returns

self.

class neuraxle.hyperparams.distributions.WrappedHyperparameterDistributions(hd: neuraxle.hyperparams.distributions.HyperparameterDistribution = None, hds: List[neuraxle.hyperparams.distributions.HyperparameterDistribution] = None)[source]
get_current_narrowing_value()[source]
narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.0) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Takes a value that is estimated to be the best one of the space, and restrict the space near that value. By default, this function will completely replace the returned value by the new guess if not overriden.

Parameters
  • best_guess – the value towards which we want to narrow down the space.

  • kept_space_ratio – what proportion of the space is kept. Should be between 0.0 and 1.0. Default is to keep only the best_guess (0.0).

Returns

a new HyperparameterDistribution object that has been narrowed down.

rvs()[source]

Sample the random variable.

Returns

The randomly sampled value.

unnarrow() → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Return the original distribution before narrowing of the distribution. If the distribution was never narrowed, will return a copy of self.

Returns

the original HyperparameterDistribution before narrowing, or else self if the distribution is virgin.

was_narrowed_from(kept_space_ratio: float, original_hp: neuraxle.hyperparams.distributions.HyperparameterDistribution) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Keep track of the original distribution to restore it.

Parameters
  • kept_space_ratio – the ratio which made the current object narrower than the original_hp.

  • original_hp – The original HyperparameterDistribution, which will be kept in a private variable for an eventual restore.

Returns

self.