Specify Your Parameter Transformations¶
The neural networks used by neurosynchro perform best when their input and output parameters are normalized. In order for neurosynchro to do this normalization, you must give it some hints about your input and output parameters.
These hints are stored in a configuration file that will go along with your
compiled neural network data. If you’re following along with the tutorial,
this file should already exist in the directory that your terminal is working
in; it was created by the neurosynchro init-nndir
command.
Before training the neural nets, however, we need to finalize the configuration file as described below.
Attention
If you’re following along with the tutorial, the final section gives important instructions you need to follow. Most of the rest of the material can be skimmed for now, though. Maybe read it while your nets are training?
How Neurosynchro transforms parameters¶
Before we go into the details of configuration, we need to describe the transformations that neurosynchro performs for its computations.
In the previous section, you transformed the output parameters of your training set. That first stage of transformation makes it so that the numerics don’t need to worry about obeying various physical invariants that apply to the Stokes-basis radiative transfer coefficients.
The numbers in the resulting output file (called transformed/all.txt
in
the example command) are referred to as physical values in the context of
neurosynchro. These are the numbers that you will eventually get back out of
the neural network.
Inside neurosynchro, physical values are transformed into what are called,
well, transformed values that are easier to work with. For instance, the
total emission coefficient j_I
should generally undergo a log transform
because its physical values span a huge dynamic range. At this stage, there is
also a question of how to deal physical inputs that are out of bounds: if you
trained your neural network on power-law indices between 1 and 3, and your
code asks neurosynchro to compute coefficients with a power-law index of
3.5, what should neurosynchro do? The “correct” answer to this question can
vary, so neurosynchro lets you configure its behavior.
Finally, for the innermost calculations the transformed values are converted to normalized values with a simple linear mapping:
Typically the mean is subtracted off and the standard deviation is divided out, so that the distribution of the normalized parameters should be approximately normal (i.e., Gaussian). In special circumstances, however, other transforms can be applied.
The nn_config.toml
configuration file¶
The neurosynchro init-nndir
initialization command will create a neural
network data directory and put one file in it: a configuration file named
nn_config.toml
. This file specifies how the input and output parameters
should be handled. The file is in TOML format, which is a simple scheme for
storing structured data. (TOML is like JSON but, in
the opinion of this writer, better.)
The default file consists of a series of stanzas each denoted by a pair of double square brackets. By default, the first stanza looks like this:
[[params]]
name = "s"
maptype = "log"
The delimiter [[params]]
indicates that this stanza defines an input
parameter. The following lines assign various settings as described below.
There are also stanzas marked with the delimiter [[results]]
, which give
configuration settings for output results. The two kinds of stanzas accept
almost identical kinds of configuration values; exceptions will be noted
below.
Allowed configuration values are as follows:
- name
- This isn’t really a configuration setting. It gives the name of the input or output parameter being configured. This should be one of the items that appears in the column headers of your training sample. Names corresponding to nonexistent columns will be ignored.
- maptype
- This setting specifies how the physical values given in your training set
are transformed. The most useful options are
direct
, which performs no transformation, andlog
, which takes the logarithm of the physical values. The full set of possible values is given below. - phys_bounds_mode
- As mentioned above, neurosynchro has to decide what to do if you ask it to
compute coefficients outside of the bounds spanned by its training set. This
parameter configures how those bounds are actually determined. There are
just two options:
empirical
andtheta
. Forempirical
, the bounds are determined by the empirical minimum and maximum physical parameter values seen in the training set. Fortheta
, the bounds are fixed to be the range between 0 and π/2. The default isempirical
, which is what makes sense in almost all cases. Thetheta
setting is recommended for thetheta
input parameter, so that calculations where θ got really close to π/2 don’t get rejected even if your training set didn’t get quite as close. - normalization_mode
- This setting specifies how to determine the parameters of the linear
transform used to obtain the final normalized values. If it is
gaussian
, the default, the mean and standard deviation of the transformed values (note: not the physical values) are used to yield an approximately normal distribution. Ifunit_interval
, the min and max of the transformed values will be used in such a way that the normalized values span the unit interval[0, 1]
. - out_of_sample
- This setting specifies what to do if a physical value lies beyond the range
of the training set. This can happen on either the input or the output
side. Your simulation might require input parameters that are beyond the
ones you trained on, but also the neural network approximator might yield
results that end up lying outside of training-set range. Possible values are
ignore
(the default),clip
, andnan
. Withignore
, the sample limits are ignored and the calculation plunges ahead recklessly. Withclip
, the input or output physical parameters are clipped to stay within the sampled physical range — note that means that you can get back results that just plain do not correspond to the parameters that you thought you were using! The neurosynchro driver code collects flags so that you can tell when this happens. Finally,nan
flags the affected calculations and causes the driver to return Not-a-Number values unconditionally. - trainer
- This setting only applies to output parameters. It specifies which scheme
will be used to train the neural network to compute this output. There is a
generic
trainer that generally does well; the list of all possibilities is given in the next section.
Map Types¶
Neurosynchro supports the following transformations between “physical” parameter values and internal “transformed” values:
- abs_log
- The transformed value is the logarithm of the absolute value of the physical value. This transform is not reversible on its own. It is used by the driver code for the rho_Q parameter, which both spans a large dynamic range and takes on both positive and negative values. The driver deals with this by splitting it into two components: an overall amplitude (using this mapping) and a sign term.
- direct
- The transformed value is the physical value. This is useful for parameters like power-law indices that do not span a large dynamic range.
- log
- The transformed value is the base-10 logarithm of the physical value. This is useful for parameters that span large dynamic ranges and are always positive.
- logit
The transformed value is the logit of the physical value:
\[t = \log(\frac{p}{1 - p})\]This maps a value in the range
(0, 1)
to the range(-∞, +∞)
, so to use this the physical value must be constrained to lie in the unit interval. This is the case for the “polarization share” parameters used in the transformed output parameters.- neg_log
- The transformed value is the base-10 logarithm of the negation of the physical value. This is useful for parameters that span large dynamic ranges and are always negative.
- ninth_root
- The transformed value is the ninth root of the physical value, preserving sign. This is adequate for parameters that span large-ish dynamic ranges and take on both positive and negative values. This mapping is no longer used in the recommended configuration.
- sign
- The transformed value is the signum of the physical value: either -1, 0, or 1 depending on physical value’s sign. This transform is not reversible on its own, but is used for the rho_Q parameter as described above.
Finalizing the Configuration File¶
The configuration file generated by the neurosynchro init-nndir
command
contains suggested defaults for the s and theta input parameters and the
suite of output parameters generated by the neurosynchro transform
step.
However, the command doesn’t (and can’t) know what other input parameters your
model uses, so you must edit the nn_config.toml
file to define them. Add
stanzas analogous to the example one used for the s input parameter. The
defaults are often useful, so you probably only need to ask yourself:
- Should this parameter have a
maptype
ofdirect
orlog
? - What do I want its
out_of_sample
behavior to be?
You may want to revisit this file to, for example, try a different neural network training scheme to improve neurosynchro’s performance for a certain model output parameter.
Finalizing Configuration for the Tutorial¶
For the purposes of the tutorial, here are the specific adjustments you should make to the default configuration file:
Add a stanza defining the power-law index parameter, after the stanza for the
theta
parameter:[[params]] name = "p" maptype = "direct"
Next: precalculate the domain and range of your training set.