Experiment Design
© Copyright 2004 Herbert J. Bernstein
The design of an experiment is a complex process. Traditionally
we think of an experiment as a tool to test a hypothesis that
has previously been formulated. In reality, the design of the
experiment can help in formulating the hypothesis to be tested,
and the outcome of the experiment can cause us to rethink and
redesign the experiment, the hypothesis or both.
- Define the Question
- Formulate a theory
- Observe a system
- Formulate a new/revised model
- Validate an new/existing/proposed model
- Set/adjust/refine the parameters of an existing model
- Pose hypothesis and null hypothesis
(see here
and here).
- Define the system and its boundaries
- Select the models (if any) to be used
- Identify the variables
- Independent variables
- Dependent variables
- Constants
- Correlated variables
- Extraneous variables (e.g. time, temperature, humidity)
- Transform the variables
- Decouple variables
- Identify ranges and domains
- Linearize (rectify) when possible
- Try to go through the origin (background removal)
- Dimensional analysis (see here)
- Error analysis (see here)
- Select your data collection protocol
- What data will you collect?
- How will you collect it?
- How precise will it be? (repeatable)
- How accurate will it be? (close to the right value)
- Will the collection interfere with the system? (observer effects)
- Will the collection interfere with itself? (dead time)
- How much data should you collect?
- Will the system change over time?
- How will you select the "correct" model?
- Select your model-fitting/refinement technique
- Taxonomies, data classification, partitioning
- Over-determined systems
- Linear least squares
- Non-linear least squares
- See here (Least Squares summary)
- See here or here (using
Excel)
- See here and
- See here
- Use of prior knowledge
- Weighting schemes, goodness of fit
- Under-determined systems
- Use of prior knowledge (see here)
- Maximum Entropy Methods (see here)
- Constrained/restrained least squares
- How will you validate your results?
- How will you decide if you are done?